Errata since December 19, 2016 corrected online version of book.
Page 26, equation (2.26) (Rakesh Achanta): The least squares objective
should not be the squared norm, but just the norm. In this case may as
well do the same in (2.24), and should also remove the $1/2N$
terms. This is actually the "square-root" lasso, which is equivalent
to a lasso, but requires a bit more explanation.
Page 42, Figure 3.7 (Joe Suzuki): The orange and blue curves are
mislabeled; the colors should be switched.
Page 74, immdediately below (4.47) (Joe Suzuki): $\lambda=\tau^4/4$ should be $\lambda=\tau^2/4$.
(Joe: "I think that the derivation is due to the fact that a+b is no less
than twice the square root of ab")
Page 78, equation (4.61) (Paul Seidel): the L1 penalty should exclude the first
component of $\gamma$.
Page 102, Figure 5.4 (Xiaoyan Han): the following was intended:
1) The bottom dashed line should be labeled $f(\beta) + s <\nabla f(\beta), \Delta>$. (Without $\alpha$ since the line is the tangent)
2) The $s_0$ and $s_1$ in the labels of the top and bottom dashed lines should both be replaced with simply $s$ since it is $s$ that parameterizes the lines themselves (Otherwise, they would be a scalars?)
Page 106, equation (5.28) (Xiaoyan Han): It should it be $\|\beta^0 - \beta^\star\|$ on the RHS instead of $\|\beta^t - \beta^\star\|$
Page 108, Example 5.6 (Zachary Fisher): FISTA is "Fast Iterative
Shrinkage-Thresholding Algorithm"
Page 198, Ex 7.8 (Joe Suzuki): in (a), should be $1,\ldots,n$
Page 199, (7.47) (Joe Suzuki): drop the $\frac12$
Page 219, line 2 (Joe Suzuki): it should be $\frac12$ rather than $\frac1{2N}$
Page 236, Ex 8.10 (Joe Suzuki): (8.13) should be (8.18)
Page 238, Ex 8.16(c) (Joe Suzuki): it should be $N(1-\rho_1^2)$
Page 238, Ex 8.18 (Joe Suzuki): first penalty should be
$-\lambda_1\|\beta\|_1$ and last should be $-\lambda_2'\|\theta\|_2^2$
Page 131: Should be a reference in bibliography to "Least Angle Regression" paper: Efron,
Hastie, Johnstone and Tibshirani (2004) AoS
Page 286, (10.26) (Vladimir Vovk): the $d$ should be $N$
Page 301, Proof of bound (11.25b) (Murat Genc): last term in the displayed inequality should not be squared. Instead, it should be $\|\hat{\nu}\|_2$
December 19, 2016 online version of book has been corrected for errata below.
Page 112, equation (5.41): confusing to use summation index j; also
occurs later in text.
Page 165, equations (6.34) and (6.35): need a minus in front of the
first terms.
July 28, 2016 online version of book has been corrected for errata below.
Errata list (Mathias Drton, and students Kameron Harris, Chris Xie and Scott Roy). We give the list as received in this link. In this corrected version of SLS we have responded to most of the suggestions, and thank this team for their careful documentation.
Page 15, above (2.9) (Brandon Greenwell): replaced awkward wording regarding $z_i$.
Page 37, in (3.15) (Naoki Saito and students), $\{\beta_0, \beta_k\}_{k=1}^K$ should be replaced by $\{\beta_{0k}, \beta_k\}_{k=1}^K$
Page 39 (Naoki Saito and students), in the displayed equation after (3.18), the numerator of the last term should be $r_{i\ell} - \tilde{p}_\ell(x_i)$ instead of $y_{i\ell} - \tilde{p}_\ell(x_i)$
Page 63, Eq.(4.16a) (Naoki Saito and students): change $\theta_j \Longrightarrow \hat{\theta}_j$
Page 65, Eq.(4.23a), (4.23b) (Naoki Saito and students): change $\theta_j \Longrightarrow \hat{\theta}_j$
Page 84 (Naoki Saito and students), in the last line: $(0,\infty)$ should be replaced by $(1,\infty)$
Page 96, Figure 5.1 caption (Naoki Saito and students): $sf(\beta)+(1-s)f(\beta)$ and $f(s\beta+(1-s)\beta)$ should be replaced by $sf(\beta)+(1-s)f(\beta')$ and $f(s\beta+(1-s)\beta')$.
Page 100, three lines above the start of Subsection 5.3 (Naoki Saito and students): $j=1,\ldots,p \Longrightarrow j=1,\ldots,m.$
Page 101, line 11 (Naoki Saito and students): $-f(\beta^t) \Longrightarrow -\nabla f(\beta^t)$.
Page 105, Eq.(5.26) (Naoki Saito and students): The inner right parenthesis is not necessary.
Page 108, last paragraph (Naoki Saito and students): $t$ referred to as step-size, $t \in [0,0.5]$ should be $s^t$ instead (in two places); $t$ refers to number of steps/iterations, not the step size.
Page 109 (Naoki Saito and students), Figure 5.7, the titles of these two figures. The values of $\lambda$ should be swapped since $\lambda=0.05$ should correspond to the dense case whereas $\lambda=0.7$ should be the sparse case considering this regularization parameter for the $\ell^1$ norm constraint
Page 117, one line above Table 5.1 (Naoki Saito and students): Warms starts $\Longrightarrow$ Warm starts
Page 119, Algorithm 5.1 Step 3(b), the last line (Naoki Saito and students): $(\lambda_{k-1}-\lambda){\bf X}_\mathcal{A} \Delta \Longrightarrow (\lambda_{k-1}-\lambda){\bf X} \Delta$
Page 119, Algorithm 5.1 Step 3(d) (Naoki Saito and students): $\mathcal{A}=\mathcal{A} \cup j \Longrightarrow \mathcal{A}=\mathcal{A} \cup \{j\}$.
Page 119, second line below Algorithm 5.1 (Naoki Saito and students): $| \langle {\bf X}_\mathcal{A}, \boldsymbol{r}(\lambda) \rangle | = \lambda 1$ changed to $|\langle {\bf x}_j, \boldsymbol{r}(\lambda) \rangle | = \lambda,\; \forall j \in \mathcal{A}$
Page 124, line -6 (Naoki Saito and students): $\beta \in n \Longrightarrow \beta \in \mathbb{R}^n$.
Pages 127-128, equations (5.76) and (5.77) (Mohammad Hattab) - missing subscript in $\|{\bf x}\|_2$ - should be $\|{\bf x}_j\|_2$
Page 134, line 4 (Naoki Saito and students): Remove the extra $=$ from $=\leq \dots$
Page 144, last paragraph (Vladislavs Dovgalecs) Table 6.1 (not 6.2)
Page 169 (Vladislavs Dovgalecs) "If the columns of $\bf Z$ are centered then the right" (not left)
Page 294, equation (11.10) (TBM on Discuss) - should be $\nu^TX^TX\nu$.
January 4, 2016 online version of book has been corrected for errata below.
Page 2 (Maciej Lazarewicz) equations (1.2) and (1.3) $\beta_{ij}$ should be $\beta_j$
Page 32, eq (3.10) - there should not be a minus sign
Chapter 5, bib notes - Add reference to shooting algorithm of Fu (98)
Page 122 (Scott Powers) (5.66) - Three errors in this formula [sorry!]: