Lectures & Talks

Statistical Learning Course

A 10-week class by Trevor Hastie and Rob Tibshirani (past offerings in 2014, 2015, 2016, now self-paced with certification option) This course is free to the public, and is based on our new book An Introduction to Statistical Learning, with Applications in R. Click on the image for further details.

Slides and videos for MOOC pointed to on Kevin Markham's data school website (originally posted by Kevin on R Bloggers website, but now out of date) Here is an additional link to the slides or a zipped version StatLearningSlides.zip Here is a post-MOOC lecture I gave, summarizing the experiences of making the course, the demographics and experiences of the first run.

JSM August 2021, Seattle Breiman Award Lecture

Some Comments on CV (pdf)
This talk is based on work with Stephen Bates and Rob Tibshirani. We address the problem of getting confidence intervals for prediction error based on CV. We show that the naive intervals can be severely biased, and propose a nested CV scheme to address this. We also tackle the question of what error CV estimates? i.e. conditional error or average error. The talk is based on our paper Cross-validation: what does it estimate and how well does it do it?
Here is a video link to a youtube video recording of the same talk, given remotely at DAGSTAT2022 in Hamburg

JSM August 2021, Seattle 50 Years of Ridge Regression

Ridge Regularization: An Essential Concept in Data Science (pdf)
This session celebrates the 50th anniversary of the 1970 papers on ridge regression by Hoerl and Kennard. My talk looks at the many uses of ridge in modern applied statistics. There were six excellent discussions by Jianqing Fan, Ed George, Liza Levina, Ming Yuan, Kerby Sheddon and Hui Zou. My paper and those of the discussants appear in a special issue of Technometrics in August 2020. Both this session and the special issue was organized by Roshan Joseph. I gave the same talk at ISI 2021 in a session names Data Science and Statistics organized by Elisabetta Carfagna.

ISCB 2020 Krakow, Poland

Predictive Models in Health Research (pdf)
Frank Harrel and I were paired plenary speakers. I spoke mostly in favor of using modern machine learning approaches (lasso, deep learning, random forests), as long as they demonstrably gave added benefit. Frank was more cautionary about power and over enthusiasm.
This was a remote conference, and my talk was pre-recorded using zoom. Here is video (77mb mp4)

Wald Lectures JSM 2019, Denver

Statistical Learning with Sparsity
Wald_I.pdf Lasso and glmnet, with applications in GWAS-scale prediction problems
Wald_II.pdf Matrix completion and softImpute, with applications in functional data and collaborative filtering
Wald_III.pdf Graphical model selection with applications in anomaly detection

Slides used at "Theory vs Practice" session at JSM2018

Two short examples. One on unhelpful vs helpful theory for understanding deep learning. The other on unifier theory for species-presence models in ecology.

Variable Selection at Scale, JSM 2017, Baltimore

Computer Age Statistical Inference session organized by Regina Liu, to celebrate the one-year anniversary of our new book. Brad Efron and me the speakers, Cun-Hui Zhang and Stefan Wager discussants, and Vijay Nair the chair.
A talk on the comparison of best-subset, forward-stepwise, lasso, and relaxed lasso for variable selection in the context of the linear model. Click on the image above for the pdf slides. The paper can be found at https://arxiv.org/abs/1707.08692. R package for reproducing all the figures and simulations at https://github.com/ryantibs/best-subset/.

Statistical Learning with Sparsity

Wasserstrom distinguished lecture, Northwestern University, Nov 1, 2016.
A talk on learning techniques that exploit sparsity in one form or another. Focus is on lasso, elastic net and coordinate descent, but time permitting, covers a lot of ground. Click on the image above for the pdf slides.

Statistical Learning with Big Data, Stanford, October 21, 2015

A talk on statistical learning intended for a general audience. This talk is part of the Data Science@Stanford seminar series, and this website has a link to the video of the talk. Click on the image above for the pdf slides.

25th birthday of GAMs - IFCS 2015, Bologna

A session at this meeting, organized by Prof. Angela Montanari, celebrated the 25 year anniversary of the publication of Generalized Additive Models (1990) by Rob Tibshirani and me. Rob's talk and my talk. Rob had to cancel at the last minute, and so I gave the slightly doctored version of his talk that is available here.

Big Data in Biomedicine - Stanford May 2014

A 15 minute video of a talk on data modeling given at the above conference. It was a "Ted-style" setup, where you had to talk within your time limit, and they cut you off if you went over!

Statistical Learning and Data Mining IV

A two-day course taught by Trevor Hastie and Rob Tibshirani. Click on the image for more details. What is the difference between the SLDMIII class and our online MOOC?

  • The MOOC takes 10 weeks, and is at a lower level than the two-day SLDMIII class.
  • In SLDMII you hear about the latest, cutting edge tools not covered in our MOOC.
  • SLDMIII is based on "Elements of Statistical Learning", which is a more advanced book than "An Introduction to Statistical Learning". It is an intensive two-day experience in which you get to interact with us and the other participants.
  • Participants get to ask questions during the lectures and the lunch and refreshment breaks

Sparse Linear Models, with demonstrations using GLMNET

Joint work with Jerome Friedman, Rob Tibshirani and Noah Simon.
Webinar given on May 3, 2013. Organized by Ray DiGiacomo and the Orange County R users group.
[Webinar slides - PDF 1.3Mb] plus some extras not covered.
[glmnet_webinar_Rsession.tgz 68Kb] gzipped archive of webinar R-session data and script (excluding tamayo data)
[tamayo.rda 22Mb] R data archive of tamayo data

Youtube video (the sound got slightly lagged wrt the video)

Keynote address, Baylearn 2013, Facebook Campus, Menlo Park [PDF 9Mb]

Invited talk at CASI 2013 (Conference on Applied Statistics in Ireland, Clane, Co. Kildare, May 15-17, 2013) [PDF 2.2Mb]

An Analysis of Approaches to Presence-Only Data

Joint work Will Fithian
Invited Talk, EcoStat 2013, Sydney [PDF 416Kb]
Invited Talk, Annual conference of Ecological Society of America (ESA), Portland, OR August 2012 [PDF 463Kb]
Invited talk JSM, San Diego, 2012 (given by Will Fithian) [PDF 463Kb]

Matrix Completion and Large-scale SVD Computations

joint work with Rahul Mazumder
Keynote address MBCII, Catania, Sicily, September 2012
Keynote address, Compstat 2012, Limassol, Cyprus, August 2012
[PDF 6.5Mb]
Keynote address, 43 Interface Symposium, Rice University, Houston, May 2012
Invited talk (footnote session), ISNPS (International Society for Nonparametric Statistics), Chalkidiki, Greece, June 2013
[PDF 6.5Mb]

Learning with Sparsity Constraints

Distinguished Lecture Series, SCI, Institute, University of Utah, February 2011
Invited talk, Ecole Normale Superieure, Paris December 2010
[PDF 6.8Mb]

Fast Regularization Paths via Coordinate Descent

Keynote lecture, useR09, Rennes 2009
Plenary talk ASMDA Yokahama 2008
Plenary talk (video), KDD 2008, Las Vegas.
Invited talk, ASA Conference, Denver 2008.
[PDF 730Kb]

Least Angle Regression

Tribute to Brad Efron on the occasion of his 70th birthday, May 2008.
[PDF 1.3Mb]

President's invited lecture, ISI meeting 2009, Durban, South Africa (updated).
Buehler-Martin lecture, University of Minnesota, March 9, 2009 (updated)
ICME Seminar, Stanford, November 13, 2006.
Keynote address, 1st South African Data Mining Conference, Stellenbosch, 2005
[PDF 609Kb]

Logistic Regression on Autopilot

Invited talk, Joint Statistical Meetings of ASA, Seattle, 2006.
[PDF 130Kb]

Regularization Paths

Plenary talk, Royal Statistical Society Annual Conference, Belfast, Ireland, 2006
Keynote Address, Nips 2005 Workshop on "Accuracy-Regularization Frontiers", Whistler, BC, December 2005.
[PDF 1.3Mb]

Prognostic Models for Cancer Progression using Gene Expression Signatures

Keynote Address, 5th Australian Microarray Conference, Barossa Valley, Australia, September 2005.
[PDF 4.1Mb]

Regularization and Variable Selection via the Elastic Net

Keynote Address, SIAM workshop on Variable Selection, Newport Beach, CA, April 23, 2005.
[PDF 189Kb]

The Entire Regularization Path for the Support Vector Machine

Statistics Department Seminar, Stanford University, April 27, 2004
[PDF 635Kb]


Invited special one-day tutorial on machine learning in Hamilton, New Zealand, Spring 2003
Invited tutorial at 50th anniversary of the South African Statitical Association, Gauteng Provence, November, 2003
[PDF 337Kb]    [PS 1.4MB]

Older Lectures & Talks

Support Vector Machine, Kernel Logistic Regression, and Boosting

Invited special one-day tutorial on machine learning in Hamilton, New Zealand, Spring 2003
Invited tutorial at 50th anniversary of the South African Statitical Association, Gauteng Provence, November, 2003
[PDF 237Kb]    [PS 830Kb]

Least Angle Regression, Forward Stagewise and the Lasso

Berkeley Biostatistics Seminar, Spring 2003
[PDF 169Kb]    [PS 1.5Mb]

Independent Component Analysis by Product Density Estimation

Berkeley-Stanford Colloguiem, Berkeley, Fall 2002
[PDF 495Kb]    [PS 6.2Mb]

Supervised Learning from Microarray data; Datamining with care

Keynote address, COMPSTAT 2002, Berlin
[PDF 2.2Mb]    [PS 7.3Mb]

Support Vector Machines, Kernel Logistic Regression, and Boosting

Keynote address, 2002 Spring Technical Conference, Ann Arbor, Michigan
[PDF 257Kb]    [PS 823Kb]

New Methods for the analysis of gene expression arrays

Invited address, 2000 Snowbird Learning conference, Utah
[PDF 5.4Mb]    [PS 5.2Mb]

Bayesian Backfitting

June 1998
[PDF 947Kb]    [PS 1.3Mb]

Flexible Discriminant and Mixture Models

April 1997
[PDF 351Kb]    [PS 711Kb]

Discriminant Adaptive Nearest Neighbor Classification

November 1995
[PDF 341Kb]    [PS 406Kb]

Confidence Sets in Nonparametric Regression

August 1990
[PDF 914Kb]