Preface Xi Part I Motivation And The Basics 1 Introduction 3 1. 1 Least - Squares And Regularized Regression 4 1. 2 Lasso: Survival Of The Bigger 6 1. 3 Thresholding The Sample Covariance Matrix 9 1. 4 Sparse Pca And Regression 10 1. 5 Graphical Models: Nodewise Regression 12 1. 6 Cholesky Decomposition And Regression 13 1. 7 The Bigger Picture: Latent Factor Models 14 1. 8 Further Reading 16 2 Data, Sparsity And Regularization 21 2. 1 Data Matrix: Examples 22 2. 2 Shrinking The Sample Covariance Matrix 26 2. 3 Distribution Of The Sample Eigenvalues 29 2. 4 Regularizing Covariances Like A Mean 30 2. 5 The Lasso Regression 32 2. 6 Lasso, Variable Selection And Prediction 36 2. 7 Lasso, Degrees Of Freedom And Bic 37 2. 8 Some Alternatives To The Lasso Penalty 38 3 Covariance Matrices 45 3. 1 Definition And Basic Properties 46 3. 2 The Spectral Decomposition 49 3. 3 Structured Covariance Matrices 52 3. 4 Functions Of A Covariance Matrix 55 3. 5 Pca: The Maximum Variance Property 59 3. 6 Modified Cholesky Decomposition 61 3. 7 Latent Factor Models 65 3. 8 Glm For Covariance Matrices 71 3. 9 Glm Via The Cholesky Decomposition 73 3. 10 The Glm For Incomplete Longitudinal Data 76 3. 11 A Data Example: Fruit Fly Mortality Rate 81 3. 12 Simulating Random Correlation Matrices 85 3. 13 Bayesian Analysis Of Covariance Matrices 88 Part Ii Covariance Estimation: Regularization 4 Regularizing The Eigenstructure 95 4. 1 Shrinking The Eigenvalues 96 4. 2 Regularizing The Eigenvectors 101 4. 3 A Duality Between Pca And Svd 103 4. 4 Implementing Sparse Pca: A Data Example 106 4. 5 Sparse Singular Value Decomposition (Ssvd) 108 4. 6 Consistency Of Pca 109 4. 7 Principal Subspace Estimation 113 4. 8 Further Reading 114 5 Sparse Gaussian Graphical Models 115 5. 1 Covariance Selection Models: Two Examples 116 5. 2 Regression Interpretation Of Entries Of - 1 118 5. 3 Penalized Likelihood And Graphical Lasso 120 5. 4 Penalized Quasi - Likelihood Formulation 126 5. 5 Penalizing The Cholesky Factor 127 5. 6 Consistency And Sparsistency 130 5. 7 Joint Graphical Models 130 5. 8 Further Reading 132 6 Banding, Tapering And Thresholding 135 6. 1 Banding The Sample Covariance Matrix 136 6. 2 Tapering The Sample Covariance Matrix 137 6. 3 Thresholding The Sample Covariance Matrix 138 6. 4 Low - Rank Plus Sparse Covariance Matrices 142 6. 5 Further Reading 143 7 Multivariate Regression: Accounting For Correlation 145 7. 1 Multivariate Regression ) 152 7. 5 Implementing Mrce: Data Examples 155 7. 5. 1 Intraday Electricity Prices 155 7. 5. 2 Predicting Asset Returns 158 7. 6 Further Reading 161 
