Statistical Learning Course
A 10-week class by Trevor Hastie and Rob Tibshirani (past offerings in 2014, 2015, 2016, now self-paced with certification option) This course is free to the public, and is based on our new book An Introduction to Statistical Learning, with Applications in R. Click on the image for further details.

Slides and videos for MOOC pointed to on Kevin Markham's data school website (originally posted by Kevin on R Bloggers website, but now out of date)

Here is an additional link to the slides or a zipped version StatLearningSlides.zip

Here is a post-MOOC lecture I gave, summarizing the experiences of making the course, the demographics and experiences of the first run.

  JSM August 2021, Seattle
Breiman Award Lecture

Some Comments on CV (pdf)
This talk is based on work with Stephen Bates and Rob Tibshirani. We address the problem of getting confidence intervals for prediction error based on CV. We show that the naive intervals can be severely biased, and propose a nested CV scheme to address this. We also tackle the question of what error CV estimates? i.e. conditional error or average error. The talk is based on our paper Cross-validation: what does it estimate and how well does it do it?

  JSM August 2021, Seattle
50 Years of Ridge Regression

Ridge Regularization: An Essential Concept in Data Science (pdf)
This session celebrates the 50th anniversary of the 1970 papers on ridge regression by Hoerl and Kennard. My talk looks at the many uses of ridge in modern applied statistics. There were six excellent discussions by Jianqing Fan, Ed George, Liza Levina, Ming Yuan, Kerby Sheddon and Hui Zou. My paper and those of the discussants appear in a special issue of Technometrics in August 2020. Both this session and the special issue was organized by Roshan Joseph. I gave the same talk at ISI 2021 in a session names Data Science and Statistics organized by Elisabetta Carfagna.

  ISCB 2020 Krakow, Poland

Predictive Models in Health Research (pdf)
Frank Harrel and I were paired plenary speakers. I spoke mostly in favor of using modern machine learning approaches (lasso, deep learning, random forests), as long as they demonstrably gave added benefit. Frank was more cautionary about power and over enthusiasm.
This was a remote conference, and my talk was pre-recorded using zoom. Here is video (77mb mp4)

  Wald Lectures JSM 2019, Denver

Statistical Learning with Sparsity
Wald_I.pdf Lasso and glmnet, with applications in GWAS-scale prediction problems
Wald_II.pdf Matrix completion and softImpute, with applications in functional data and collaborative filtering
Wald_III.pdf Graphical model selection with applications in anomaly detection
  Slides used at "Theory vs Practice" session at JSM2018
Two short examples. One on unhelpful vs helpful theory for understanding deep learning. The other on unifier theory for species-presence models in ecology.

  Variable Selection at Scale, JSM 2017, Baltimore

Computer Age Statistical Inference session organized by Regina Liu, to celebrate the one-year anniversary of our new book. Brad Efron and me the speakers, Cun-Hui Zhang and Stefan Wager discussants, and Vijay Nair the chair.
A talk on the comparison of best-subset, forward-stepwise, lasso, and relaxed lasso for variable selection in the context of the linear model. Click on the image above for the pdf slides. The paper can be found at https://arxiv.org/abs/1707.08692. R package for reproducing all the figures and simulations at https://github.com/ryantibs/best-subset/.

  Statistical Learning with Sparsity.

Wasserstrom distinguished lecture, Northwestern University, Nov 1, 2016.
A talk on learning techniques that exploit sparsity in one form or another. Focus is on lasso, elastic net and coordinate descent, but time permitting, covers a lot of ground. Click on the image above for the pdf slides.

  Statistical Learning with Big Data, Stanford, October 21, 2015

A talk on statistical learning intended for a general audience. This talk is part of the Data Science@Stanford seminar series, and this website has a link to the video of the talk. Click on the image above for the pdf slides.

  25th birthday of GAMs - IFCS 2015, Bologna

A session at this meeting, organized by Prof. Angela Montanari, celebrated the 25 year anniversary of the publication of Generalized Additive Models (1990) by Rob Tibshirani and me. Rob's talk and my talk. Rob had to cancel at the last minute, and so I gave the slightly doctored version of his talk that is available here.

  Big Data in Biomedicine - Stanford May 2014

A 15 minute video of a talk on data modeling given at the above conference. It was a "Ted-style" setup, where you had to talk within your time limit, and they cut you off if you went over!

  Statistical Learning and Data Mining IV

A two-day course taught by Trevor Hastie and Rob Tibshirani. Click on the image for more details.

What is the difference between the SLDMIII class and our online MOOC?

  • The MOOC takes 10 weeks, and is at a lower level than the two-day SLDMIII class.
  • In SLDMII you hear about the latest, cutting edge tools not covered in our MOOC.
  • SLDMIII is based on "Elements of Statistical Learning", which is a more advanced book than "An Introduction to Statistical Learning". It is an intensive two-day experience in which you get to interact with us and the other participants.
  • Participants get to ask questions during the lectures and the lunch and refreshment breaks

  Sparse Linear Models, with demonstrations using GLMNET
Joint work with Jerome Friedman, Rob Tibshirani and Noah Simon.
Webinar given on May 3, 2013. Organized by Ray DiGiacomo and the Orange County R users group.
[Webinar slides - PDF 1.3Mb] plus some extras not covered.
[glmnet_webinar_Rsession.tgz 68Kb] gzipped archive of webinar R-session data and script (excluding tamayo data)
[tamayo.rda 22Mb] R data archive of tamayo data

Youtube video (the sound got slightly lagged wrt the video)

Keynote address, Baylearn 2013, Facebook Campus, Menlo Park [PDF 9Mb]

Invited talk at CASI 2013 (Conference on Applied Statistics in Ireland, Clane, Co. Kildare, May 15-17, 2013) [PDF 2.2Mb]

  An Analysis of Approaches to Presence-Only Data
Joint work Will Fithian
Invited Talk, EcoStat 2013, Sydney [PDF 416Kb]
Invited Talk, Annual conference of Ecological Society of America (ESA), Portland, OR August 2012 [PDF 463Kb]
Invited talk JSM, San Diego, 2012 (given by Will Fithian) [PDF 463Kb]

  Matrix Completion and Large-scale SVD Computations
joint work with Rahul Mazumder
Keynote address MBCII, Catania, Sicily, September 2012
Keynote address, Compstat 2012, Limassol, Cyprus, August 2012
[PDF 6.5Mb]
Keynote address, 43 Interface Symposium, Rice University, Houston, May 2012
Invited talk (footnote session), ISNPS (International Society for Nonparametric Statistics), Chalkidiki, Greece, June 2013
[PDF 6.5Mb]

  Learning with Sparsity Constraints
Distinguished Lecture Series, SCI, Institute, University of Utah, February 2011
Invited talk, Ecole Normale Superieure, Paris December 2010
[PDF 6.8Mb]

  Fast Regularization Paths via Coordinate Descent
Keynote lecture, useR09, Rennes 2009
Plenary talk ASMDA Yokahama 2008
Plenary talk (video), KDD 2008, Las Vegas.
Invited talk, ASA Conference, Denver 2008.
PDF [PDF 730Kb]

  Least Angle Regression
Tribute to Brad Efron on the occasion of his 70th birthday, May 2008.
PDF [PDF 1.3Mb]

  Modern Trends in Data Mining
President's invited lecture, ISI meeting 2009, Durban, South Africa (updated).
Buehler-Martin lecture, University of Minnesota, March 9, 2009 (updated)
ICME Seminar, Stanford, November 13, 2006.
Keynote address, 1st South African Data Mining Conference, Stellenbosch, 2005
PDF [PDF 609Kb]

  Logistic Regression on Autopilot
Invited talk, Joint Statistical Meetings of ASA, Seattle, 2006.
PDF [PDF 130Kb]

  Regularization Paths
Plenary talk, Royal Statistical Society Annual Conference, Belfast, Ireland, 2006
Keynote Address, Nips 2005 Workshop on "Accuracy-Regularization Frontiers", Whistler, BC, December 2005.
PDF [PDF 1.3Mb]

  Prognostic Models for Cancer Progression using Gene Expression Signatures
Keynote Address, 5th Australian Microarray Conference, Barossa Valley, Australia, September 2005.
PDF [PDF 4.1Mb]

  Regularization and Variable Selection via the Elastic Net
Keynote Address, SIAM workshop on Variable Selection, Newport Beach, CA, April 23, 2005.
PDF [PDF 189Kb]

  The Entire Regularization Path for the Support Vector Machine
Statistics Department Seminar, Stanford University, April 27, 2004
PDF [PDF 635Kb]

Invited special one-day tutorial on machine learning in Hamilton, New Zealand, Spring 2003
Invited tutorial at 50th anniversary of the South African Statitical Association, Gauteng Provence, November, 2003
PDF [PDF 337Kb]    [PS 1.4MB]



  Support Vector Machine, Kernel Logistic Regression, and Boosting
Invited special one-day tutorial on machine learning in Hamilton, New Zealand, Spring 2003
Invited tutorial at 50th anniversary of the South African Statitical Association, Gauteng Provence, November, 2003
PDF [PDF 237Kb]    [PS 830Kb]

  Least Angle Regression, Forward Stagewise and the Lasso
Berkeley Biostatistics Seminar, Spring 2003
PDF [PDF 169Kb]    [PS 1.5Mb]

  Independent Component Analysis by Product Density Estimation
Berkeley-Stanford Colloguiem, Berkeley, Fall 2002
PDF [PDF 495Kb]    [PS 6.2Mb]

  Supervised Learning from Microarray data; Datamining with care
Keynote address, COMPSTAT 2002, Berlin
PDF [PDF 2.2Mb]    [PS 7.3Mb]

  Support Vector Machines, Kernel Logistic Regression, and Boosting
Keynote address, 2002 Spring Technical Conference, Ann Arbor, Michigan
PDF [PDF 257Kb]    [PS 823Kb]

  New Methods for the analysis of gene expression arrays
Invited address, 2000 Snowbird Learning conference, Utah
PDF [PDF 5.4Mb]    [PS 5.2Mb]

  Bayesian Backfitting
June 1998
PDF [PDF 947Kb]    [PS 1.3Mb]

  Flexible Discriminant and Mixture Models
April 1997
PDF [PDF 351Kb]    [PS 711Kb]

  Discriminant Adaptive Nearest Neighbor Classification
November 1995
PDF [PDF 341Kb]    [PS 406Kb]

  Confidence Sets in Nonparametric Regression
August 1990
PDF [PDF 914Kb]   

 © copyright 2003 Trevor Hastie - All rights reserved.