My Publications
Books
by Gareth James, Daniela Witten, Trevor Hastie, Rob Tibshirani and Jonathan Taylor (July 2023). This book (ISLP) differs from the R book (ISLR2) in that the labs at the end of each chapter are implemented in Python.
Book Homepage and Resourcesby Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani (August 2021) 3 new chapters (+179 pages), including Deep Learning
Book Homepage and Resourcesby Bradley Efron and Trevor Hastie (August 2016)
Book Homepageby Trevor Hastie, Robert Tibshirani and Martin Wzainwright (May 2015)
Book Homepageby Trevor Hastie, Robert Tibshirani, and Jerome Friedman (2009)
Book Homepageby Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani (June 2013)
Book Homepageby Trevor Hastie, Robert Tibshirani, and Jerome Friedman (2001)
Book HomepagePapers
The research reported here was partially supported by grants from the National Science Foundation and the National Institutes of Health.
2023

Anav Sood and Trevor Hastie A Statistical View of Column Subset Selection. We consider the problem of selecting a small subset of representative variables from a large dataset. In the computer science literature, this dimensionality reduction problem is typically formalized as Column Subset Selection (CSS). Meanwhile, the typical statistical formalization is to find an informationmaximizing set of Principal Variables. This paper shows that these two approaches are equivalent, and moreover, both can be viewed as maximum likelihood estimation within a certain semiparametric model.
2022

Michael Greenacre, Patrick Groenen, Trevor Hastie, Alfonso Iodice d'Enza, Angelos Markos, and Elena Tuzhilina. Principal Component Analysis. A review article on PCA, to appear in Nature Reviews.

Samyak Rajnala, Stephen Bates, Trevor Hastie and Rob Tibshirani Confidence Intervals for the Generalisation Error of Random Forests. We use the OOB information to estimate jacknife standard errors for the OOB estimate of error.
2021

Added a new blog entry on Altered Priors. You build a classifier on some training data, but you would like to deploy it in a population where the class distribution (prior) is different. This comes up in casecontrol sampling, but also in other situations such as transfer learning.

Elena Tuzhilina and Trevor Hastie. Weighted Low Rank Matrix Approximation and Acceleration. We develop algorithms for computing an elementweighted lowrank matrix approximation (SVD) via projected gradient descent. We consider two acceleration schemes, Nesterov and Anderson, and discuss their implementation. We show how to scale these algorithms to highdimensional problems.

Zijun Gao and Trevor Hastie. LinCDE: Conditional density estimation via Lindsey's method. Lindsey's method allows for smooth density estimation by turning the density problem into a Poisson GLM. In particular, we represent an exponential tilt function in a basis of natural splines, and use discretization to deal with the normalization. In this paper we extend the method to conditional density estimation via trees and then gradient boosting with trees. JMLR 2022 R package installable from GITHUB: install_github("ZijunGao/LinCDE"); see LinCDE vignette for examples.

Yosuke Tanigawa, Junyang Qian, Guhan Venkataraman, Johanne Justesen, Ruilin Li, Robert Tibshirani, Trevor Hastie, Manuel Rivas. Significant Sparse Polygenic Risk Scores across 428 traits in UK Biobank. In this survey across the more than 1,600 traits in the UK Biobank, we report 428 strongly significant (p<2.5e5) sparse polygenic risk models, computed by the snpnet lasso model developed by this team.

Swarnadip Ghosh, Trevor Hastie and Art Owen. Scalable logistic regression with crossed random effects. We develop an approach for fitting crossed randomeffect logisitic regression models at massive scales, with applications in ecommerce. We adapt a procedure of Schall (1991) and backfitting algorithms to achieve O(n) algorithms. EJS 2022 doi.org/10.1214/22EJS2047

Zijun Gao and Trevor Hastie. DINA: Estimating Heterogenous Treatment Effects in Exponential Family and Cox Models. We extend the Rlearner framework to exponential families and the Cox model. Here we define the treatment effect to be the difference in natural parameter or DINA.

Stephen Bates, Trevor Hastie and Rob Tibshirani. Crossvalidation: what does it estimate and how well does it do it? Although CV is ubiquitous in data science, some of its properties are poorly understood. In this paper we argue that CV is better at estimating expected prediction error rather than the prediction error for the particular model fit to the training set. We also provide a method for computing the standard error of the CV estimate, which is bigger than the commonly used naive estimate which ignores the correlations in the folds. (to appear, JASA)

Elena Tuzhilina, Leonardo Tozzi and Trevor Hastie. Canonical Correlation Analysis in high dimensions with structured regularization.We develop structurally regularized versions of CCA for very highdimensional MRI images from neuroscience experiments. To appear, Statistical Modelling, 2021.

J. Kenneth Tay, Balasubramanian Narasimhan and Trevor Hastie. Elastic Net Regularization Paths for All Generalized Linear Models. This paper describes some of the substantial enhancements to the glmnet R package ver 4.1+. All programmed GLM families are accommodated through a family() argument. We also discuss relaxed fits, and facilities for modeling stop/start data and strata in survival models. To appear, Journal of Statistical Software.
2020

Benjamin HaibeKains et al. Transparency and reproducibility in artificial intelligence. A note in Matters arising, Nature, where we are critical of the lack of transparency in a highprofile paper on using AI for breastcancer screening.

Lukasz Kidzinski, Francis Hui, David Warton and Trevor Hastie. Generalized Matrix Factorization: efficient algorithms for fitting generalized linear latent variable models to large data arrays JMLR 23(291) 2022 Generalized Matrix Factorization We fit largescale generalized linear latent variable models (GLLVM) by using efficient methodology adapted from matrix factorization. Designed with large scale ecology studies in mind with tens of thousands of species and/or locations, these methods provide good approximations to the stateoftheart variational Bayes methods that do not scale gracefully to such sizes.

Swarnadip Ghosh, Trevor Hastie and Art Owen. Backfitting for large scale crossed random effects regressions Regression models with crossed randomeffects can be very expensive to compute in large scale applications (eg Ecommerce with millions of customers and many thousands of products). We adapt a backfitting algorithm to problems of this kind and achieve convergence (provably) linear in the number of observations. Annals of Statistics 2022

Elena Tuzhilina, Trevor Hastie and Mark Segal. Principal curve approaches for inferring 3D chromatin architecture We adapt metric scaling algorithms to approximate the 3D conformation of a chromosome based on contact maps. Biostatistics, Nov 2020.

Trevor Hastie. Ridge Regularization: an Essential Concept in Data Science This paper is written by request to celebrate the 50th anniversary of the first ridgeregression paper by Hoerl and Kennard (1070), Technometrics. In it I gather together some of my favorite aspects of ridge, and also touch on the "doubledescent" phenomenon. Technometrics online link. The R markdown source for some of the figures is figureSource.rmd

Kenneth Tay, Nima Aghaeepour, Trevor Hastie, Robert Tibshirani. Featureweighted elastic net: using "features of features" for better prediction We often have metadata that informs us about features to be used for supervised learning. For example, genomic features may belong to different pathways. Here we develop a method for exploiting this information to improve prediction. Statistica Sinica 2021.

Junyang Qian, Yosuke Tanigawa, Ruilin Li, Robert Tibshirani, Manuel A Rivas, Trevor Hastie. LargeScale Multivariate Sparse Regression with Applications to UK Biobank We develop methodology to build large sparse polygenic risk score models using multiple phenotypes (multitask learning). Our methodology builds on our early snpnet lasso models for single phenotypes, and includes missing phenotype imputation and adjustment for confounders.
Here is a link to the code and scripts used in the paper https://github.com/junyangq/scripts_multiSnpnet_paper (Annals of Applied Statistics, vol 16(3), 2022.)
2019

Trevor Hastie, Andrea Montanari, Saharon Rosset and Ryan Tibshirani. Surprises in HighDimensional Ridgeless Least Squares Interpolation. Interpolating fitting algorithms have attracted growing attention in machine learning, mainly because stateofthe art neural networks appear to be models of this type. In this paper, we study minimum L2norm ("ridgeless") interpolation in highdimensional least squares regression. We consider both a linear model and a version of a neural network. We recover several phenomena that have been observed in largescale neural networks and kernel machines, including the "double descent" behavior of the prediction risk, and the potential benefits of overparametrization. Annals of Statistics, 2022 50(2) pp 949986.

Didier Nibbering and Trevor Hastie Multiclasspenalized logistic regression We develop a model for clustering classes in multiclass logistic regression. Computational Statistics and Data Analysis, 2022

Zijun Gao, Trevor Hastie and Rob Tibshirani. Assessment of heterogeneous treatment effect estimation accuracy via matching We address the difficult problem of assessing the performance of an HTE estimator. Our approach has several novelties: a flexible matching metric based on randomforest proximity scores, an optimized matching algorithm, and a match then split crossvalidation scheme. Statistics in Medicine, April 2021
2018

Junyang Qian, Yosuke Tanigawa, Wenfei Du, Matthew Aguirre, Chris Chang, Robert Tibshirani, Manuel A. Rivas, Trevor Hastie. A Fast and Scalable Framework for Largescale and Ultrahighdimensional Sparse Regression with Application to the UK Biobank. PLOS Genetics October 2020. We develop a scalable lasso algorithm for fitting polygenic risk scores at GWAS scale. There is also a BiorXiv version. Our R package snpnet combines efficient batchwise strongrule screening with glmnet to fit lasso regularization paths on phenotypes in the UK Biobank data.
Here is a link to the code and scripts used in the paper https://github.com/junyangq/scripts_snpnet_paper 
Lukasz Kidzinski and Trevor Hastie Longitudinal data analysis using matrix completion We use a regularized form of matrix completion to fit functional principal component models, and extend these to other multivariate longitudinal regression models. We have an R package fcomplete which includes three vignettes demonstrating how it can be used.
2017

Trevor Hastie, Rob Tibshirani and Ryan Tibshirani Extended Comparisons of Best Subset Selection, Forward Stepwise Selection, and the Lasso This paper is a followup to "Best Subset Selection from a Modern Optimization Lens" by Bertsimas, King, and Mazumder (AoS, 2016). We compare these methods using a broad set of simulations that cover typical statistical applications. Our conclusions are that bestsubset selection is mainly needed in very high signaltonoise regimes, and the relaxed lasso is the overall winner. Published as Best Subset, Forward Stepwise or Lasso? Analysis and Recommendations Based on Extensive Comparisons. Statistical Science (2020) 35(4) 579592. The supplement shows a comprehensive set of comparisons.

Scott Powers, Junyang Qian, Kenneth Jung, Alejandro Schuler, Nigam Shah, Trevor Hastie and Robert Tibshirani. Some methods for heterogeneous treatment effect estimation in highdimensions We develop some new methods for estimating personalized treatment effects from observational data: causal boosting and causal MARS. Statistics in Medicine. (January 2018).
2016

Scott Powers, Trevor Hastie and Rob Tibshirani. Nuclear penalized multinomial regression with an application to predicting atbat outcomes in baseball. Here we use a convex formulation of the reducedrank multinomial model, in a novel application using a large dataset of baseball statistics. In special edition "Statistical Modelling for Sports Analytics", Statistical Modelling, vol. 18, 56: pp. 388410.

Qingyuan Zhao and Trevor Hastie Causal Interpretations of BlackBox Models. We draw connections between Friedman's partial dependence plot and Pearl's backdoor adjustment to explore the possibility of extracting causality statements after fitting complex models by machine learning algorithms. Finally published 2019 ASA/JBES, 39(1) 2019

Nicholas Boyd, Trevor Hastie, Stephen Boyd, Benjamin Recht, Michael Jordan. Saturating Splines and Feature Selection We use a convex framework based on TV (total variation) penalty norms for nonparametric regression. Saturation for degreetwo splines requires the solution extrapolates as a constant beyond the range of the data. This along with an additive model formulation leads to a convex path algorithm for variable selection and smoothing with generalized additive models. JMLR 18(197):132, 2018
2015

Scott Powers, Trevor Hastie and Robert Tibshirani Customized training with an application to mass spectrometric imaging of cancer tissue. Annals of Applied Statistics 9(4) (2015), 17091725.

Rakesh Achanta and Trevor Hastie Telugu OCR Framework using Deep Learning. We build an endtoend OCR system for Telugu script, that segments the text image, classifies the characters and extracts lines using a language model.The classification module, which is the most challenging task of the three, is a deep convolutional neural network.

Jingshu Wang, Qingyuan Zhao, Trevor Hastie and Art Owen. Confounder Adjustment in Multiple Hypotheses Testing. (accepted, Annals of Statistics, 2016). We present a unified framework for analysing different proposals for adjusting for confounders in multiple testing (e.g. in genomics). We also provide an R package cate on CRAN that implements these different approaches. The vignette shows some examples of how to use it.

Alexandra Chouldechova and Trevor Hastie Generalized Additive Model Selection A method for selecting terms in an additive model, with sticky selection between null, linear and nonlinear terms, as well as the amount of nonlinearity. The R package gamsel has been uploaded to CRAN.
2014

Matrix Completion and LowRank SVD via Fast Alternating Least Squares We develop a new method for matrix completion, that improves upon our earlier softImpute algorithm, as well as the popular ALS algorithm. JMLR 2015 16 33673402. We have also incorporated this method in our R package softImpute.

Sparse Quadratic Discriminant Analysis and Community Bayes We provide a framework for generalizing naive Bayes classification using sparse graphical models. Naive Bayes assumes conditional on the response, the variables are independent. We relax that, and assume their conditional dependence graph is shared and sparse.

Bias Correction in Species Distribution Models: Pooling Survey and Collection Data for Multiple Species We develop methods for teasing out observerbias in multispecies presenceonly data. (submitted, arXiv:1403.7274) ( Methods for Ecology and Evolution, October 10, 2014). Github link to the R package multispeciesPP

Local CaseControl Sampling: Efficient Subsampling in Imbalanced Data Sets (arXiv). Modern classification tasks are often extremely unbalanced  a situation where casecontrol sampling can be very helpful. This paper discusses the bias of CC sampling, and proposes a simple twostage subsampling procedure for removing this bias, and in cases dramatically improving the efficiency. Annals of Statistics 2014, Vol. 42, No. 5, 16931724. projecteuclid.org/euclid.aos/1410440622
2013

Lucas Janson, Will Fithian, and Trevor Hastie. Effective Degrees of Freedom: a Flawed Metaphor. The popular covariance formula for df gives some surprising results, like df>p in forward stepwise. May 2015, Biometrika

Stefan Wager, Trevor Hastie and Bradley Efron. Confidence Intervals for Random Forests: the Jackknife and the Infinitessimal Jackknife. We use ideas related to OOB errors to compute standard errors for bagging and random forests. Two approaches are presented, one based on the jacknife, the other on the infinitessimal jacknife. We study the bias of these estimates, as well as montecarlo errors. . (JMLR 2014, 15 16251651)

David Warton, Bill Shipley and Trevor Hastie. CATS regression  a modelbased approach to studying traitbased community assembly. We show how to use GLMs to fit community models, which are traditionally fit by maximum entropy. Apart from being a convenient platform for model fitting, a all the usual summaries, statistics and extensions of GLMs are available. ( Methods of Ecology and Evolution, September 29, 2014)

Hristo Paskov, Robert West, John Mitchell and Trevor Hastie. Compressive Feature Learning We use an unsupervised convex ducument compression algorithm to derive a sparse kgram representation for a corpus of documents. This same dictionary, in the spirit of "deep learning", is as good as the original kgram representation for document classification tasks. To appear, NIPS 2013.

My first blog post with Will Fithian. This post refers to concerns that were raised about crossvalidation.

Noah Simon, Jerome Friedman and Trevor Hastie. A Blockwise Descent Algorithm for Grouppenalized Multiresponse and Multinomial Regression. We use the group lasso in the context of multinomial and multiresponse regression. Each variable has multiple coefficients for the different responses, and they each get selected via a group lasso penalty. Our code is an efficient implementation of block coordinate descent, and is built into the glmnet package. (submitted, on arXiv).

Michael Lim and Trevor Hastie. Learning interactions via hierarchical grouplasso regularization. We use the overlap group lasso in the context of a linear model to test for interactions. Our methodology can handle qualitative as well as quantitative variables. Our R package glinternet can fit linear and logistic regression models. Optimized code can handle thousands of variables (our largest example had > 20K 3level factors). arXiv:1308.2719 (JCGS 2014, online access)

Michael Jordan et al. Frontiers in Massive Data Analysis. This 129 page document is the report produced by the Committee on the Analysis of Massive Data. This committee was established by the National Research Council of the National Academies, and met four times over 20112012 in Washington and California. Michael Jordan was the chair of the 18 member committee, made up of statisticians (5), computer scientists and mathematicians. I was a member of the committee, and was jointly responsible for Chapter 7 with David Madigan, although all committee members provided input to all chapters as well.

Trevor Hastie and Will Fithian. Inference from Presenceonly Data: the Ongoing Controversy. This short paper argues strongly against the use of rigid parametric logistic regression models to make inferences from presenceonly data in Ecology. Essentially the rigidity of the model manufactures information that is not present in the data. Ecography (2013, editors choice) Video interview with David Warton at Ecostats conference at UNSW in Sydney in July 2013. David was wearing his editor's hat for Methods in Ecology and Evolution, and the discussion centered on this paper.

Noah Simon, Jerome Friedman, Trevor Hastie and Rob Tibshirani. The Sparse Group Lasso By mixing L1 penalties with grouplasso L2 penalties, we achieve a sparse group lasso where some members of a group can end up being zero. JCGS, May 2013, 22(2), pages 231245.

Jianqiang Wang and Trevor Hastie. Boosted VaryingCoefficient Regression Models for Product Demand Prediction. We use the varying coefficient paradigm to fit a market segmented product demand model, with boosted regression trees as the nonparametric component. JCGS

Julia Viladomat, Rahul Mazumder, Alex McInturff, Douglas McCauley and Trevor Hastie Assessing the significance of global and local correlations under spatial autocorrelation; a nonparametric approach. Variables collected over a spatial domain often exhibit strong spatial autocorrelation. When such variables are used in a regression, pairwise correlation analysis, or in the popular geographically weighted regression, it can be difficult to assess significance. We propose a general approach based on randomization followed by smoothing to restore the spatial correlation structure. R code code used in the paper. (Biometrics, Jan 2014)
2012

Will Fithian and Trevor Hastie. Finitesample equivalence in statistical models for presenceonly data. We show that a lot of different approaches to presenceonly data are the same, in particular inhomogenous poisson proccesses, maxent, and naive logistic regression (when weighted appropriately). (AoAS 2013 7 (4) 19171939 ).

Jason Lee and Trevor Hastie. Learning Mixed Graphical Models. We use grouplasso regularized pseudo likelihood for learning the structure of a graphical model with mixed discrete and continuous variables. Our model respects the symmetry imposed by a Markov random field representation  each of the potentials gets a vote from a pair of regression models (Gaussian, logistic or multinomial), where each of the pair of variables is the response and predictor. (in Arxiv), JCGS 24(1) 2015. Go to Jason Lee's webpage for matlab code and a demo.

Rahul Mazumder, Jerome Friedman and Trevor Hastie. Sparsenet R package on CRAN. Fits sparse solution paths for linear models (squareerror loss) using coordinate descent with MC+ penalty family. Software is very fast, and can handle many thousands of variables. Functions for crossvalidation, prediction, plotting etc. Based on algorithms described in SparseNet : Coordinate Descent with NonConvex Penalties. JASA 2011, 106(495) 11251138.
2011

Rahul Mazumder and Trevor Hastie Exact Covariance Thresholding into Connected Components for largescale Graphical Lasso A block screening rule for graphical models that can dramatically speed up computations, and allow for distributed computing of models on a much larger scale. Original copy arXiv 8/18/2011 published JMLR March 2012 13, 723736.

Rahul Mazumder and Trevor Hastie The Graphical Lasso: New Insights and Alternatives (arXiv 11/23/2011, published November 2012) We examine the glasso algorithm for solving the graphical lasso problem. We show that it solves the dual problem, where the optimization variable is the covariance rather than the precision matrix. We propose a similar primal algorithm, which appears to be superior in speed and other propertiers. Electronic Journal of Statistics 20126 21252149. R package dpglasso

Gen Nowak; Trevor Hastie; Jonathan R. Pollack; Robert Tibshirani A fused lasso latent feature model for analyzing multisample aCGH data. Biostatistics (advanced access June 2011). A new approach for modeling multisample CGH data, that exploits the similarities in copynumber variation at the same locations in the genome. Here is the online link at the Biostatistics journal website. The R package FLLat is available from CRAN.

Noah Simon, Jerome Friedman, Trevor Hastie and Rob Tibshirani Regularization Paths for Cox's Proportional Hazards Model via Coordinate Descent. We develop the tools needed to include the Cox proprtional hazard's model in GLMNET. Journal of Statistical Software 39(5), 113 (2011).
2010

Rob Tibshirani, Jacob Bien, Jerome Friedman, Trevor Hastie, Noah Simon, Jonathan Taylor and Ryan Tibshirani: Strong rules for discarding predictors in lassotype problems. We develop rules for screening predictors for lasso and elasticnet penalized models. When p>>n, this can result in large computational savings, without any loss in accuracy. JRSS B (2012) 74

Jane Elith, Steven Phillips, Trevor Hastie, Miroslav Dudik, Yung En Chee and Colin Yates A statistical explanation of Maxent for Ecologists Maxent is a method for modeling species prevalence with presencebackground data. This paper explains maxent using the language of a statistical models. Diversity and Distributions November 2010

Jerome Friedman, Trevor Hastie and Rob Tibshirani: Applications of the lasso and grouped lasso to the estimation of sparse graphical models We develop efficient algorithms for fitting sparse, undirected graphical models. These are specially designed for the "large p" situation.

Jerome Friedman, Trevor Hastie and Robert Tibshirani: A Note on the Group Lasso and a Sparse Group Lasso. We develop a group lasso with both sparsity of groups and sparsity within groups. We also develop coordinatewise algorithms for fitting the both cases.
2009

Rahul Mazumder, Jerome Friedman and Trevor Hastie: SparseNet : Coordinate Descent with NonConvex Penalties. JASA 2011, 106(495) 11251138. Nonconvex penalties produce sparser models than the LASSO, but pose difficulties for optimization. We propose a structured algorithm using coordinate descent which finds good solutions with guaranteed convergence properties. Appendix for SparseNet paper with extra figures and some additional technical proofs.
sparsenet R package available from CRAN (Feb 2012) 
Rahul Mazumder, Trevor Hastie and Rob Tibshirani: Spectral Regularization Algorithms for Learning Large Incomplete Matrices. We develop an iterative algorithm for matrix completion using nuclearnorm regularization. JMLR 2010 11 22872322
MATLAB package SoftImpute for matrix completion (zip archive).
R package to appear soon. 
Daniela Witten, Rob Tibshirani and Trevor Hastie: A penalized matrix decomposition, with applications to sparse canonical correlation analysis and principal components Biostatistics 10(3)

Trevor Hastie, Robert Tibshirani and Jerome Friedman, Elements of Statistical Learning: Data Mining, Inference and Prediction (Second Edition). February, 2009. 745 pages in full color. SpringerVerlag, New York.
This second edition adds 4 new chapters: Random Forests, Ensemble Learning, Undirected Graphical Models, and High Dimensional Problems: p>>N. For more details see ESL book homepage.
In an agreement with Springer, we are able to offer for free the ESL book pdf (8.2M).
2008

Tong Tong Wu, Yi Fang Chen, Trevor Hastie, Eric Sobel and Kenneth Lange: Genomewide Association Analysis by Lasso Penalized Logistic Regression We develop efficient computational procedures for screening largescale genomewide association studies. Bioinformatics 25(6): 714721, 2009.

Ping Li, Ken Church and Trevor Hastie: One sketch for all: theory and application of conditional random sampling Nips08 proceedings.

Jerome Friedman, Trevor Hastie and Robert Tibshirani: Regularization Paths for Generalized Linear Models via Coordinate Descent. We use coordinate descent to develop regularization paths for linear, logistic and multinomial regression models. Our algorithms use the "elastic net" penalties of Zou and Hastie (2005), and create the path for a grid of values of the penalty parameter lambda. Journal of Statistical Software, 33(1), 2010
The R package glmnet is available from CRAN
A matlab wrapper for the glmnet fortran code, written by Hui Jiang. 
Jane Elith, John Leathwick and Trevor Hastie: A working guide to boosted regression trees (2008) Journal of Animal Ecology, 77 802813. Here are the online supplement materials, along with the associated zip file. In 2004 received award for most highly cited paper in any of the British Ecological Society journals in the past 5 years.

Line Clemmensen, Trevor Hastie, Daniela Witten and Bjarne Ersboll: Sparse Discriminant Analysis. We extend penalized linear and mixture discriminant analysis by incorporating a lasso penalty to encourage sparseness. Technometrics (2011)
53(4) 406413.

John Leathwick, Jane Elith, W. Chadderton, D. Rowe and Trevor Hastie: Dispersal, disturbance and the contrasting biogeophraphies of New Zealand's diadromous and nondiadronous fish species. An application of boosted regression trees in ecological mapping. J. Biogeography 2008, 35 14811497.

Debashis Paul, Eric Bair, Trevor Hastie and Robert Tibshirani: "Preconditioning" for feature selection and regression in highdimensional problems We show that supervised principal components followed by a variable selection procedure is an effective approach for variable selection in very high dimension. Annals of Statistics 36(4), 2008, 15951618.
2007

Jerome Friedman, Trevor Hastie and Robert Tibshirani, Sparse inverse covariance estimation with the lasso.

Trevor Hastie Comment on a paper in Statistical Science by Peter BÃ¼hlmann and Torsten Hothorn: Boosting Algorithms:Regularization, Prediction and Model Fitting (2007) 22(4),477522.

Jerome Friedman, Trevor Hastie and Robert Tibshirani, Discussion of "Evidence contrary to the statistical view of boosting (David Mease and Aaron Wyner)" Wyner and Mease show through examples some counterintuitive results with boosting, that appear to contradict our 2000 paper. We discount these claims by reversing their results using shrinkage along with boosting. JMLR9 (2008) 5964.

Jerome Friedman, Trevor Hastie, Holger Hoefling and Robert Tibshirani, Pathwise Coordinate Optimization. We show how coordinate descent algorithms can efficiently solve a number of popular regularized optimization problems, creating an entire path of solutions. We generalize this approach to derive an efficient algorithm for the fused lasso, both one and twodimensional. Annals of Applied Statistics (2007), 1(2), 302332.

Ping Li, Trevor Hastie and Kenneth Church. Nonlinear Estimators and Tail Bounds for Dimension Reduction in L1 using Cauchy Random Projections. We provide improved methods for approximating L1 distances in very high dimensions, based on maximumlikelihood estimation in the Cauchy family. JLMR 8, pp 24972532

Ping Li and Trevor Hastie. A Unified NearOptimal Estimator for Dimension Reduction in L_a (0< a <= 2) Using Stable Random Projections. NIPS2007 poster presentation

Brad Efron, Trevor Hastie and Rob Tibshirani, Discussion of the "Dantzig Selector" by Emmanuel Candes and Terrence Tao. Candes and Tao propose an alternative but similar procedure to the lasso. This discussion appears alongside the orginal article in the Annals of Statistics 35(6), pp23582364.

Trevor Hastie, Jonathan Taylor, Robert Tibshirani and Guenther Walther, Forward Stagewise Regression and the Monotone Lasso We characterize the incremental forward stagewise procedure as a monotone version of the lasso. Electronic Journal of Statistics 1 (2007).
2006

Trevor Hastie and Ji Zhu, Discussion of "Support Vector Machines with Applications" by Javier M. Moguerza and Alberto Munoz, Statistical Science 21(3) 352357.

Gill Ward, Trevor Hastie, Simon Barry, Jane Elith and John Leathwick, Presenceonly data and the EM algorithm. We develop a method for fitting the twoclass logistic regression model using labeled data from one class, a sample of unlabeled data, and knowledge of the class prevalences.Biometrics,65(2)554563, 2009.
Download the beta version of Gill Ward's R package ecogbm  ecogbm_1.01.tar.gz  for fitting boosted regression models for presenceonly data. See the student section of my homepage for Gill Ward's thesis.
A presentation by Gill Ward based on this work "Making the Best Use of Available Data: The PresenceOnly Problem in Ecology" won an honorable mention award at the 2007 Joint Statistical Meetings. 
John Leathwick, Jane Elith and Trevor Hastie, Comparative performance of generalized additive models and multivariate adaptive regression splines for statistical modelling of species distributions. (2006) Ecological Modelling 199 188196. This is a special issue of the journal devoted to the workshop on Advances in Predictive Species Distribution Models held in Riederalp,Switzerland, 2004.

Ping Li, Trevor Hastie and Kenneth Church, Very Sparse Random Projections. A method for approximating pairwise distances in very highdimensional spaces. Best student paper, KDD06, Philadelphia

MeeYoung Park and Trevor Hastie, Regularization Path Algorithms for Detecting Gene Interactions.We develop a path algorithm for fitting the "cosso" models of Yuan & Lin (2006) with logistic regression. This allows factors and interactions to enter the model in a smooth way.

MeeYoung Park and Trevor Hastie, Penalized Logistic Regression for Detecting Gene Interactions. A modified version of forwardstepwise logistic regression suitable for screening large numbers of genegene interactions.
stepPlr: R package for fitting PLR models. 
MeeYoung Park, Trevor Hastie and Rob Tibshirani, Averaged gene expressions for regression A regression method that combines the lasso with hierarchical clustering, intended for selecting groups of genes in microarray problems. Biostatistics (2007, 8, 212227). R Software for fitting these models.

Yaqian Guo, Trevor Hastie and Robert Tibshirani Regularized Discriminant Analysis and its Application in Microarrays. A method, similar to shrunken centroids, for classification and discrimination of microarrays, using regularized discriminant analysis with gene selection. Biostatistics (in press; epub)

MeeYoung Park and Trevor Hastie, An L1 Regularizationpath Algorithm for Generalized Linear Models. A generalization of the LARS algorithm for GLMs and the Cox proportional hazard model. Since the coefficient paths are piecewisenonlinear, approximations are made using the predictorcorrector algorithm of convext optimization. glmpath: R software package for fitting L1 regularized GLMs and Cox models. (JRSSB 2007 (69, part 4), pages 659677 )

Ping Li, Trevor Hastie and Kenneth Church, Improving Random Projections Using Marginal Information. Methods for speeding up document search and characterization. Accepted at Colt 2006 This paper draws on results in the following two technical reports:

Ping Li, Trevor Hastie, and Kenneth Church,
Marginconstrained Random Projections and Very Sparse Random
Projections.
Ping Li, Kenneth Church and Trevor Hastie, A Sketchbased Sampling Algorithm on Sparse Data 
Rob Tibshirani and Trevor Hastie, Margin Trees for Highdimensional Classification. A treestructured representation for a multiclass SVM classifier.

Hui Zou, Ji Zhu and Trevor Hastie, New Multicategory Boosting Algorithms Based on Multicategory FisherConsistent Losses. We provide some general requirements for multiclass marginbased classifiers. Annals of Applied Statistics 2(4) pp 12901306, 2008).
2005

Ji Zhu, Hui Zhou, Saharon Rosset and Trevor Hastie, Multiclass Adaboost. A multiclass generalization of the Adaboost algorithm, based on a generalization of the exponential loss.
Finally published in 2009 in Statistics and Its Interface Volume 2 (2009) 349360.
Code on Ji Zhu's website 
J. Leathwick, J. Elith, M. Francis, T. Hastie, P. Taylor. Variation in demersal fish species richness in the oceans surrounding New Zealand: an analysis using boosted regression trees.
(Marine Ecology Progress Series, published in 2006). A detailed analysis of species abundance using Poisson regression with boosted regression trees. All analysis done using the gbm package in R (Greg Ridgeway). 
J. Leathwick, D. Rowe, J. Richardson, J. Elith and T. Hastie, Using multivariate adaptive regression splines to predict the distributions of New Zealand's freshwater diadromous fish. Freshwater Biology 50 20342051.
Presenceabsence species data are modelled using a MARS along with GLM in R. 
MeeYoung Park and Trevor Hastie, Hierarchical Classification using Shrunken Centroids. A technique for classification when the number of classes is large. It produces an hierarchically structured classification rule, with the hardesttoseparate classes at the terminal nodes.

Hui Zou and Trevor Hastie. Regularization and Variable Selection via the Elastic Net (pdf). JRSSB (2005) 67(2) 301320. A compromise between ridge regression and the lasso, with the computational advantages of the lasso. The elastic net selects variables in correlated sets. An R package elasticnet is available from CRAN.
See interview on Essential Science Indicators.
Published minor correction.
2004

Hui Zou, Trevor Hastie, and Rob Tibshirani, On the "Degrees of Freedom" of the Lasso. A technical paper that establishes that the number of nonzero coefficients in a lasso model is unbiassed for the effective degrees of freedom. Published in Annals of Statistics (2007), 35, 5, 21732192.

Philip Beineke, Trevor Hastie and Shivakumar Vaithyanathan, The Sentimental Factor: Improving Review Classification via HumanProvided Information Proceedings ACL 2004, Barcelona. (ACL: Association of Computational Linguistics)

Eric Bair, Trevor Hastie, Debashis Paul, and Robert Tibshirani Prediction by Supervised Principal Components Published in JASA 2006 101 No 473, pp 11137.

NIPS2004  The following papers were accepted for NIPS 2004:

Trevor Hastie, Saharon Rosset, Rob Tibshirani and Ji Zhu,
The Entire Regularization Path for the Support Vector Machine (oral presentation)
Saharon Rosset, Hui Zou, Ji Zhu and Trevor Hastie, A Method for Inferring Label Sampling Mechanisms in SemiSupervised Learning (poster presentation)

Hui Zou, Trevor Hastie, and Rob Tibshirani. Sparse Principal Component Analysis. We present a new approach to principal component analysis, that allows us to use an L1 penalty to ensure sparseness of the loadings. Published in JCGS 2006 15(2): 262286. Software is available in R package elasticnet available from CRAN.

Trevor Hastie, Saharon Rosset, Rob Tibshirani and Ji Zhu. The Entire Regularization Path for the Support Vector Machine. JMLR, 5(Oct) 13911415. An algorithm for computing the twoclass SVM solution for all possible values of the regularization parameter C, at essentially the computational cost of a single SVM fit. Not only does this allow for efficient model selection, but it also exposes the role of regularization for SVMs. Several MPEG movies show the sequence of solutions for different examples. SvmPath software package for R.

Trevor Hastie and Robert Tibshirani. Efficient Quadratic Regularization for Expression Arrays. Biostatistics (2004), 5(3), pp 329340. Computational tricks for a large class of linear models fit by quadratic regularization.
2003

Jerome Friedman, Trevor Hastie, Saharon Rosset, Rob Tibshirani and Ji Zhu. Discussion of three Boosting papers Annals of Statistics, 2004, vol 32 (1) pp 102107. The three papers are by (1) Wenxin Jiang, (2) Gabor Lugosi and Nicolas Vayatis, and (3) Tong Zhang.

Saharon Rosset, Ji Zhu and Trevor Hastie. Margin Maximizing Loss Functions (accepted poster Nips 2003)

Ji Zhu, Saharon Rosset, Trevor Hastie and Rob Tibshirani. 1Norm Support Vector Machines (accepted spotlight poster Nips 2003)

Mu Zhu, Trevor Hastie and Guenther Walther. Constrained Ordination Analysis with Flexible Response Functions Constrained ordination via nonparametric discriminant analysis. Ecological Modelling (2005), 187(4), 524536.

Francesca Dominici, Aidan McDermott and Trevor Hastie. Improved Semiparametric Time Series Models of Air Pollution and Mortality [pdf Technical Report] JASA, December 2004, 99(468), 938948. [Sofware]

Ji Zhu and Trevor Hastie Classification of Gene Microarrays by Penalized Logistic Regression. Biostatistics 5(3):427443. R Code (from Ji Zhu's website)
2002

Trevor Hastie and Robert Tibshirani. Independent Component Analysis through Product Density Estimation (ps file). A direct statistical approach to ICA, using an attractive spline representation to model each of the marginal densities.A more recent (Nov 2002) talk (pdf)

Saharon Rosset, Ji Zhu and Trevor Hastie. Boosting as a Regularized Path to a Maximum Margin Classifier (pdf file) JMLR 5 (Aug 2004): 941973, 2004. We show that a version of boosting fits a model by optimizing a L1penalized loss function. This in turn shows that the corresponding versions of Adaboost and Logitboost converge to an "L1" optimal separating hyperplane.

Support Vector Machines, Kernel Logistic Regression and Boosting . Slides for talk given at Spring research conference in Michigan, MCS2002 in Sardinia, NPCONF2002 in Crete, and ASA2002 in New York.

Hongjuan Zhao, Trevor Hastie, Dr Michael L Whitfield, Prof AnneLise BorresenDale and Dr. Stefanie S Jeffrey
Optimization and evaluation of T7 based RNA linear amplification protocols for cDNA microarray analysis
BMC Genomics 2002, 3:31 (30 Oct 2002) Biomed central online 
Robert Tibshirani, Trevor Hastie, Balasubramanian Narasimhan, and Gilbert Chu. Class prediction by nearest shrunken centroids, with applications to DNA microarrays (ps file) (pdf file) This is a more statistical version of the PNAS paper below.

Rob Tibshirani, Trevor Hastie, B. Narashiman and Gilbert Chu: "Diagnosis of multiple cancer types by shrunken centroids of gene expression" (PNAS website). PNAS 2002 99:65676572 (May 14). See PAM website for software (available soon).

Bradley Efron, Trevor Hastie, Iain Johnstone and Robert Tibshirani, Least Angle Regression Annals of Statistics (with discussion) (2004) 32(2), 407499. A new method for variable subset selection, with the lasso and "epsilon" forward stagewise methods as special cases. LARS Software for R and Splus.

Antoine Guisan, Thomas Edwards and Trevor Hastie Generalized linear and generalized additive models in studies of species distributions: setting the scene. Ecological Modeling (2002) 157, 89100.
2001

Trevor Hastie, Robert Tibshirani and Jerome Friedman, "Elements of Statistical Learning: Data Mining, Inference and Prediction" SpringerVerlag, New York.

Mu Zhu and Trevor Hastie, "Feature extraction for nonparametric discriminant analysis" JCGS (2003, 12(1), pages 101120.

Ji Zhu and Trevor Hastie, "Kernel Logistic Regression and the Import Vector Machine", (NIPS, 2001; JCGS 2005). Copy of slides(pdf) presented by TH in Kyoto in December, 2001.
Ji Zhu's R code for fitting IVM models. 
Robert Tibshirani, Trevor Hastie, Balasubramanian Narasimhan, Michael Eisen, Gavin Sherlock, Pat Brown, and David Botstein
Exploratory screening of genes and clusters from microarray experiments (ps file) or pdf version. 
Therese Sorlie, Perou, C., Robert Tibshirani, Turid Aas, Stephanie Geisler, Hilde Johnsen, Trevor Hastie, Michael B. Eisen, Matt van de Rijn, Stefanie S. Jeffrey, Thor Thorsen, Hanne Quist, John C. Matese, Patrick O. Brown, David Botstein, Per Eystein Lonninngg, and AnneLise BorresenDale. Gene expression patterns of breast carcinomas distinguish tumor subclasses with clinical implications. PNAS 98: 1086910874. pdf version.
2000

Trevor Hastie, Robert Tibshirani, David Botstein and Pat Brown, "Supervised Harvesting of Expression Trees" (postscript) . Starting from a hierarchically clustered expression array, we build a predictive model for an outcome variable using cluster nodes as inputs.
(pdf version) Tech. report. August 2000. 
Olga Troyanskaya, Michael Cantor, Gavin Sherlock, Pat Brown, Trevor Hastie, Robert Tibshirani, David Botstein and Russ B. Altman, Missing value estimation methods for DNA microarrays BIOINFORMATICS Vol. 17 no. 6, 2001 Pages 520525

Eva Cantoni and Trevor Hastie "DegreesofFreedom Tests for Smoothing Splines." Tech Report, May 2000.
Published in Biometrika 2002, 89, 251263. A mixedeffects framework for smoothing splines and additive models allows for exact tests between nested models of different complexity. The complexity is calibrated via the effective degrees of freedom. 
Thomas Yee and Trevor Hastie. Reduced Rank Vector Generalized Linear Models (2003) Statistical Modeling, 3, pages 1541. Using the multinomial as a primary example, we propose reduced rank logit models for discrimination and classification. This is a conditional version of the reduced rank model of linear discriminant analysis.

Robert Tibshirani, Guenther Walther and Trevor Hastie.
"Estimating the number of clusters in a dataset via the Gap statistic". Journal of the Royal Statistical Society, B, 63:411423,2001. 
Stochastic Modeling and Tracking of Human Motion, a joint project with Dirk Ormoneit and Michael Black's group at Xerox Parc, with motion graphics demonstrations of learned walking characteristics.

Page 50 of "Generalized Additive Models" by Hastie and Tibshirani, 1990, Chapman and Hall. Some copies of the 1999 printing by CRC Press replaced page 50 with a page from a history text! page50.ps or page50.pdf

Trevor Hastie, Laura Bachrach, Balasubramanian Narasimhan and May Choo Wang. Flexible Statistical Models for Growth Fragments: a Study of Bone Mineral Acquisition Compare your own measurements using our online growth tables

Gareth James and Trevor Hastie Functional Linear Discriminant Analysis for Irregularly Sampled Curves (2001) Journal of the Royal Statistical Society, Series B JRSS B 63, 533550.

Trevor Hastie, Robert Tibshirani, Michael B Eisen, Ash Alizadeh, Ronald Levy, Louis Staudt, Wing C Chan, David Botstein, Patrick Brown. 'Gene shaving' as a method for identifying distinct sets of genes with similar expression patterns This is an online version of the paper, published in the online journal GenomeBiology.

Trevor Hastie, Robert Tibshirani, Michael Eisen, Pat Brown, Doug Ross, Uwe Scherf, John Weinstein, Ash Alizadeh, Louis Staudt, David Botstein "Gene Shaving: a New Class of Clustering Methods for Expression Arrays". Postscript (2.9mb) or Adobe pdf (5.4mb) Tech. report. Jan 2000.

James, G., Hastie, T., and Sugar, C. "A Principal Component Models for Sparse Functional Data". (2000, Biometrika, 87, 587602) (pdf). When the data are collections of sampled curves or images, functional principal components produce the principal modes of variation. Here we generalize these procedures to deal with the case when each curve is sparsely and irregularly sampled.
1999

Hastie, T., Tibshirani, R., Sherlock, G., Eisen, M., Brown, P. and Botstein, D. "Imputing Missing Data for Gene Expression Arrays". Technical report (1999), Stanford Statistics Department. pdf (145Kb) or postscript (450Kb)

Tibshirani, R., Hastie, T. Eisen, M., Ross, D. , Botstein, D. and Brown, P. "Clustering methods for the analysis of DNA microarray data". Postscript (4.8mb) or Compressed Postscript (1.8mb) Tech. report Oct. 1999.

D. Ormoneit and T. Hastie. Optimal kernel shapes for local linear regression. In S. A. Solla, T. K. Leen, and KR. Müller, editors, Advances in Neural Information Processing Systems 12. The MIT Press, 2000.

Tibshirani, R. and Lazzeroni, L. and Hastie, T. and Olshen, A. and Cox, D.R. "A Global Pairwise Approach to Radiation Hydrid Mapping". Technical Report January 1999. Using data of cooccurrence of hybridized markers after shattering, inference is made of the marker sequence in the chromosome.
1998

Friedman, J., Hastie, T. and Tibshirani, R. (Published version) Additive Logistic Regression: a Statistical View of Boosting Annals of Statistics 28(2), 337407. (with discussion)
We show that boosting fits an additive logistic regression model by stagewise optimization of a criterion very similar to the loglikelihood, and present likelihood based alternatives. We also propose a multilogit boosting procedure which appears to have advantages over other methods proposed so far. Here are the slides (2 per page) for my boosting talk. 
Crellin, N., Hastie, T. and Johnstone, I. "Statistical Models for Image Sequences" Technical report, submitted to "Human Brain Mapping". We study fMRI sequences of the human brain obtained from experiments involving repetitive neuronal activity. We investigate the function form of the hemodynamic response function, and provide evidence that the commonly adopted convolution model is inadequate.

Hastie, T. and Tibshirani, R. "Bayesian Backfitting" Stanford Technical report.
The Gibbs sampler looks and feels like the backfitting algorithm for fitting additive models. Indeed, a simple modification to backfitting turns it into a Gibbs sampler for spitting out samples from the "posterior" distribution for an additive fit.
Published Statistical Science 15, no. 3 (2000), 196223 
Wu, T.,Hastie, T., Schmidler, S. and Brutlag, D. "Regression Analysis of Multiple Protein Structures" Models for lining up and averaging groups of protein structures.
1997

Rubinstein, D. and Hastie, T. "Discriminative vs Informative Learning" A comparison of two frequently used but different paradigms for training classifiers.

Maes, S. and Hastie, T. "Dynamic Mixtures of Splines: a Model for Saliency Grouping in the Time Frequency Plane" This is an application of mixture modeling to speech data. We use a moving mixture of Gaussians to represent the formantfrequencies in speech data.

Hastie, T., and Tibshirani, R. and Buja, A. "Flexible Discriminant and Mixture Models" In edited proceedings of "Neural Networks and Statistics" conference, Edinburgh, 1995. J. Kay and D. Titterington, Eds. Oxford University Press.

Wu, T., Schmidler, S., Hastie, T., and Brutlag, D. "Modelling and superposition of multiple protein structures using affine transformations: analysis of the globins"

James, G., and Hastie, T. "Generalizations of the Bias/Variance Decomposition for Prediction Error". Several papers have recently appeared on this topic, and each have a different viewpoint and decomposition. We hope ours does not add to the confusion.

James, G., and Hastie, T. Error Coding and PaCTs. Gareth James' winning paper in the ASA student paper competition for the Statistical Computing Section. This is one of four winning papers.
1996

Hastie, T. "Neural Networks", to appear in Encyclopaedia of Biostatistics. A brief survey with some personal points of view.

Hastie, T., and Tibshirani, R. "Classification by Pairwise Coupling" We solve a multiclass classification problem by combining all the pairwise rules. This paper builds on ideas proposed by J. Friedman. An abbreviated version is published in Advances in Neural Information Processing Systems 10, M. I. Jordan, M. J. Kearns, S. A. Solla, eds., MIT Press, 1998.

Hastie, T. and Tibshirani, R. "Generalized Additive Models" to appear in "Encyclopaedia of Statistical Sciences". A survey paper on GAMs.
1995

Hastie, T. Pseudosplines. We develop an approximation to any smoother, but typically a smoothing spline. The idea is to develop an orthonormal basis and penalty sequence, which would then be used via a generalized ridge regularization. JRSSB(1996) 58(2), 379396.

Hastie, T. and Simard, P. "Models and Metrics for Handwritten Character Recognition". This paper gives a brief survey of techniques for handwritten digit recognition, and then goes into some particular technique based on invariant distance in some detail. Statistical Science 13(1) 1998, pp 5465. (published version)

Hastie, T. and Tibshirani, R. "Discriminant Adaptive Nearest Neighbor Classification." IEEE PAMI, 18, 607616, 1996.

Hastie, T. and Tibshirani, R. Generalized Additive Models for Medical Research. Encyclopaedia for Biostatistics 4 187196
1994

Hastie, T. and Tibshirani, R. "Discriminant Analysis by Gaussian Mixtures." JRSSB (Jan 1996). There is also a longer technical report of Feb 1994

Hastie, T. J., Buja, A., and Tibshirani, R. "Penalized Discriminant Analysis." (Bell Labs technical report; postscript). Pdf of published version, Annals of Statistics, 1995.

Hastie, T.J., Simard, P. Y., and Saeckinger, E. "Learning Prototype Models for Tangent Distance." NIPS proceedings, 1994.

Hastie, T.J and Tibshirani, R. "Handwritten Digit Recognition via Deformable Prototypes." AT&T Bell Laboratories Technical Report, 1994.
1993

Hastie, T. J., Tibshirani, R. and Buja, A. "Flexible Discriminant Analysis by Optimal Scoring." (Bell Labs Technical report; postscript). Pdf of published version, JASA, December 1994.

Roosen, C. B. and Hastie, T. "Automatic Smoothing Spline Projection Pursuit." AT&T Bell Labs Technical Report (Dec. 1993). Journal of Computational and Graphical Statistics 3(3) Sep 1994, 235248. Here is JSTOR link

Roosen, C. B. and Hastie, T. "Logistic Response Projection Pursuit." AT&T Bell Labs Technical Report (Aug. 1993).
1992

Hastie, T. J., Kishon, E., Clark, M., and Fan, J. "A Model for Signature Verification." AT&T Bell Laboratories Technical Report (Feb. 1992).
1990

Hastie, T. J., and Pregibon, D. "Shrinking Trees." AT&T Bell Laboratories Technical Report (March 1990). Unpublished manuscript. Thanks to Mu Zhu for turning the preweb technical memorandum into an online document.
1989

Buja, A. Hastie, T. J., Tibshirani, R. Linear Smoothers and Additive Models An overview of linear smoothing technology, including a proof of the convergence of the backfitting algorithm. Annals of Statistics (with discussion) 1989, 17(2) 453555.

Hastie, T. and Stuetzle, W. Principal Curves Journal of Amarican Statistical Association 1989, 84 502516

Hastie, T., Botha, J., Schnitzler, C. Regression with an Ordered Categorical Response Statistics in Medicine 1989, 8 785794
1987

Hastie, T. J., and Pregibon, D. "A new algorithm for matched casecontrol studies with applications to additive models." AT&T Bell Laboratories Technical Report (November 1987). Unpublished manuscript. A shorter version appeared in the proceedings of Compstat 1988

Hastie, T. and Little, F. Principal Profiles A nonlinear version of principal components for compositional data, that helps explain the horseshoe effect. Published in Proceedings of the Interface Meeting (Comp.Sci and Stat), 1987.

Greenacre, M. and Hastie, T. A Geometric Interpretation of Correspondence Analysis There are many ways to think of CA. This paper presents it as a form of PCA, or subspace approximation in a Chisquared metric. JASA (82) June 1987.

Hastie, T. A Closer Look at the Deviance The deviance for GLMs provides the analog in many ways for sumofsquares in linear regression. This little paper surveys the connections. The American Statistician 41(1), 1987, pp 1620
1986

Hastie, Trevor, and Tibshirani, R. "Generalized Additive Models (with discussion)" 1986 Statistical Science Vol 1, No 3, pages 297318
1984

Principal Curves and Surfaces SLAC (Stanford Linear Accelerator Center) has put up a pdf version of my Ph.D thesis.
Original principal curves and surfaces movie (youtube). 
Generalized Additive Models: the original technical report, written by two PhD students.
1980

The Stability of Ordinal Measures of Association in Contingency Tables My University of Cape Town MS thesis.