Lectures & Talks
Statistical Learning Course
A 10-week class by Trevor Hastie and Rob Tibshirani (past offerings in 2014, 2015, 2016, now self-paced with certification option) This course is free to the public, and is based on our new book An Introduction to Statistical Learning, with Applications in R. Click on the image for further details.
Slides and videos for MOOC pointed to on Kevin Markham's data school website (originally posted by Kevin on R Bloggers website, but now out of date) Here is an additional link to the slides or a zipped version StatLearningSlides.zip Here is a post-MOOC lecture I gave, summarizing the experiences of making the course, the demographics and experiences of the first run.
JSM August 2021, Seattle Breiman Award Lecture
Some Comments on CV (pdf)
This talk is based on work with Stephen Bates and Rob Tibshirani. We address the problem of getting confidence intervals for prediction error based on CV. We show that the naive intervals can be severely biased, and propose a nested CV scheme to address this.
We also tackle the question of what error CV estimates? i.e. conditional error or average error.
The talk is based on our paper Cross-validation: what does it estimate and how well does it do it?
Here is a video link to a youtube video recording of the same talk, given remotely at DAGSTAT2022 in Hamburg
JSM August 2021, Seattle 50 Years of Ridge Regression
Ridge Regularization: An Essential Concept in Data Science (pdf)
This session celebrates the 50th anniversary of the 1970 papers on ridge regression by Hoerl and Kennard.
My talk looks at the many uses of ridge in modern applied statistics. There were six excellent discussions by Jianqing Fan, Ed George, Liza Levina, Ming Yuan, Kerby Sheddon and Hui Zou. My
paper and those of the discussants appear in a special issue of Technometrics in August 2020. Both this session and the special issue was organized by Roshan Joseph.
I gave the same talk at ISI 2021 in a session names Data Science and Statistics organized by Elisabetta Carfagna.
ISCB 2020 Krakow, Poland
Predictive Models in Health Research (pdf)
Frank Harrel and I were paired plenary speakers. I spoke mostly in
favor of using modern machine learning approaches (lasso, deep learning, random forests), as long as they demonstrably gave added benefit.
Frank was more cautionary about power and over enthusiasm.
This was a remote conference, and my talk was pre-recorded using zoom. Here is video (77mb mp4)
Wald Lectures JSM 2019, Denver
Statistical Learning with Sparsity
Wald_I.pdf
Lasso and glmnet, with applications in GWAS-scale prediction problems
Wald_II.pdf
Matrix completion and softImpute, with applications in functional data and collaborative filtering
Wald_III.pdf
Graphical model selection with applications in anomaly detection
Slides used at "Theory vs Practice" session at JSM2018
hastie_panel_JSM_2018_Vancouver.pdf
Two short examples. One on unhelpful vs helpful theory for
understanding deep learning. The other on unifier theory for species-presence
models in ecology.
Variable Selection at Scale, JSM 2017, Baltimore
Computer Age Statistical Inference session organized by Regina Liu, to celebrate the one-year anniversary of our new book. Brad Efron and me the speakers, Cun-Hui Zhang and Stefan Wager
discussants, and Vijay Nair the chair.
A talk on the comparison of best-subset, forward-stepwise, lasso, and relaxed lasso for variable selection in the context of the linear
model. Click on the image above for the pdf slides. The paper can be found at https://arxiv.org/abs/1707.08692. R package for reproducing all the figures and simulations at
https://github.com/ryantibs/best-subset/.
Statistical Learning with Sparsity
Wasserstrom distinguished lecture, Northwestern University, Nov 1, 2016.
A talk on learning techniques that exploit sparsity in one form or
another. Focus is on lasso, elastic net and coordinate descent, but
time permitting, covers a lot of ground. Click on the image above for the pdf slides.
Statistical Learning with Big Data, Stanford, October 21, 2015
A talk on statistical learning intended for a general audience. This talk is part of the Data Science@Stanford seminar series, and this website has a
link to the video of the talk. Click on the image above for the pdf slides.
25th birthday of GAMs - IFCS 2015, Bologna
A session at this meeting, organized by Prof. Angela Montanari, celebrated the 25 year anniversary of the publication of Generalized Additive Models (1990) by Rob Tibshirani and me. Rob's
talk and my
talk. Rob had to cancel at the last minute, and so I gave the slightly doctored version of his talk that is available here.
Big Data in Biomedicine - Stanford May 2014
A 15 minute video of a talk on data modeling given at the above conference. It was a "Ted-style" setup, where you had to talk within your time limit, and they cut you off if you went over!
Statistical Learning and Data Mining IV
A two-day course taught by Trevor Hastie and Rob Tibshirani. Click on the image for more details.
What is the difference between the SLDMIII class and our online MOOC?
- The MOOC takes 10 weeks, and is at a lower level than the two-day SLDMIII class.
- In SLDMII you hear about the latest, cutting edge tools not covered in our MOOC.
- SLDMIII is based on "Elements of Statistical Learning", which is a more advanced book than "An Introduction to Statistical Learning". It is an intensive two-day experience in which you get to interact with us and the other participants.
- Participants get to ask questions during the lectures and the lunch and refreshment breaks
Sparse Linear Models, with demonstrations using GLMNET
Joint work with Jerome Friedman, Rob Tibshirani and Noah Simon.
Webinar given on May 3, 2013. Organized by Ray DiGiacomo and the Orange County R users group.
[Webinar slides - PDF 1.3Mb] plus some extras not covered.
[glmnet_webinar_Rsession.tgz 68Kb] gzipped archive of webinar R-session data and script (excluding tamayo data)
[tamayo.rda 22Mb] R data archive of tamayo data
Youtube video (the sound got slightly lagged wrt the video)
Keynote address, Baylearn 2013, Facebook Campus, Menlo Park [PDF 9Mb]
Invited talk at CASI 2013 (Conference on Applied Statistics in Ireland, Clane, Co. Kildare, May 15-17, 2013) [PDF 2.2Mb]
An Analysis of Approaches to Presence-Only Data
Joint work Will Fithian
Invited Talk, EcoStat 2013, Sydney [PDF 416Kb]
Invited Talk, Annual conference of Ecological Society of America (ESA), Portland, OR August 2012 [PDF 463Kb]
Invited talk JSM, San Diego, 2012 (given by Will Fithian) [PDF 463Kb]
Matrix Completion and Large-scale SVD Computations
joint work with Rahul Mazumder
Keynote address MBCII, Catania, Sicily, September 2012
Keynote address, Compstat 2012, Limassol, Cyprus, August 2012
[PDF 6.5Mb]
Keynote address, 43 Interface Symposium, Rice University, Houston, May 2012
Invited talk (footnote session), ISNPS (International Society for Nonparametric Statistics), Chalkidiki, Greece, June 2013
[PDF 6.5Mb]
Learning with Sparsity Constraints
Distinguished Lecture Series, SCI, Institute, University of Utah, February 2011
Invited talk, Ecole Normale Superieure, Paris December 2010
[PDF 6.8Mb]
Fast Regularization Paths via Coordinate Descent
Keynote lecture, useR09, Rennes 2009
Plenary talk ASMDA Yokahama 2008
Plenary talk (video), KDD 2008, Las Vegas.
Invited talk, ASA Conference, Denver 2008.
[PDF 730Kb]
Least Angle Regression
Tribute to Brad Efron on the occasion of his 70th birthday,
May 2008.
[PDF 1.3Mb]
Modern Trends in Data Mining
President's invited lecture, ISI meeting 2009, Durban, South Africa (updated).
Buehler-Martin lecture, University of Minnesota, March 9, 2009 (updated)
ICME Seminar, Stanford, November 13, 2006.
Keynote address, 1st South African Data Mining Conference,
Stellenbosch, 2005
[PDF 609Kb]
Logistic Regression on Autopilot
Invited talk, Joint Statistical Meetings of ASA, Seattle, 2006.
[PDF 130Kb]
Regularization Paths
Plenary talk, Royal Statistical Society Annual Conference,
Belfast, Ireland, 2006
Keynote Address, Nips 2005 Workshop on "Accuracy-Regularization
Frontiers", Whistler, BC, December 2005.
[PDF 1.3Mb]
Prognostic Models for Cancer Progression using Gene Expression Signatures
Keynote Address, 5th Australian Microarray Conference, Barossa
Valley, Australia, September 2005.
[PDF 4.1Mb]
Regularization and Variable Selection via the Elastic Net
Keynote Address, SIAM workshop on Variable Selection, Newport
Beach, CA, April 23, 2005.
[PDF 189Kb]
The Entire Regularization Path for the Support Vector Machine
Statistics Department Seminar, Stanford University, April 27, 2004
[PDF 635Kb]
Boosting
Older Lectures & Talks
Support Vector Machine, Kernel Logistic Regression, and Boosting
Invited special one-day tutorial on machine learning in
Hamilton, New Zealand, Spring 2003
Invited tutorial at 50th anniversary of the South African
Statitical Association, Gauteng Provence, November, 2003
[PDF 237Kb]
[PS 830Kb]
Least Angle Regression, Forward Stagewise and the Lasso
Independent Component Analysis by Product Density Estimation
Supervised Learning from Microarray data; Datamining with care
Support Vector Machines, Kernel Logistic Regression, and Boosting