Lectures on High-Dimensional Statistical Inference
Tony Cai
Presented at ENSAE-CREST in Paris, March 22 - April 2, 2010
Abstract: The main focus of the lectures is to discuss new results and current research problems in high dimensional statistical inference, which is one of the most active research areas in statistics at the moment. These and other related problems have also attracted much recent interest in other fields including applied mathematics and electrical engineering.
To provide a strong background and foundation for the main topics, we shall begin with discussions on important results in nonparametric function estimation in the framework of the infinite dimensional Gaussian sequence model. Minimaxity, adaptive minimaxity, and oracle inequalities are covered in the context of the sequence model. In particular, Pinsker's results on linear minimaxity for estimation over an ellipsoid and the wavelet thresholding theory developed by Donoho and Johnstone will be discussed. We will then focus on current research problems in high dimensional inference including compressed sensing (large p, small n linear regression), detection of sparse signals, and estimation of large covariance matrices. We specifically cover in detail the constrained l1 minimization methods and present a unified and elementary analysis on sparse signal recovery in three settings: noiseless, bounded noise and Gaussian noise. In addition, new results on optimal estimation of large covariance matrices are presented. The analysis of the matrix estimation problems reveals new features that are quite different from those in the more conventional function/sequence estimation problems.
References:
- Bickel, P. J., Ritov, Y. and Tsybakov, A. B. (2009). Simultaneous Analysis of Lasso and Dantzig Selector. The Annals of Statistics 37, 1705-1732.
- Cai, T. (1999). Adaptive wavelet estimation: a block thresholding and oracle inequality approach. The Annals of Statistics 27, 898-924.
- Cai, T. (2008). On information pooling, adaptability and superefficiency in nonparametric function estimation. J. Multivariate Analysis 99, 412-436.
- Cai, T., Liu, W. & Luo, X. (2011). A constrained l1 minimization approach to sparse precision matrix estimation. J. American Statistical Association 106, 594-607.
- Cai, T., Wang, L. & Xu, G. (2010). Shifting inequality and recovery of sparse signals. IEEE Transactions on Signal Processing 58, 1300-1308.
- Cai, T., Wang, L. & Xu, G. (2010). Stable recovery of sparse signals and an oracle inequality. IEEE Transactions on Information Theory 56, 3516-3522.
- Cai, T., Zhang, C.-H. & Zhou, H. (2010). Optimal rates of convergence for covariance matrix estimation. The Annals of Statistics 38, 2118-2144.
- Candes, E. T. and Tao, T. (2007). The Dantzig Selector: Statistical Estimation when p is much larger than n (with discussion). The Annals of Statistics 35, 2313-2351.
- Johnstone, I. M. (1999). Function Estimation and Gaussian Sequence Models. Unpublished monograph. Available at http://www-stat.stanford.edu/~imj
- Tsybakov, A. B. (2009). Introduction to Nonparametric Function Estimation, Springer.