Differentially Private Functional Linear regression: Optimal Algorithms and Theoretical Guarantees
Yicheng Li and Tony Cai
-
Abstract:
We study functional linear regression under differential privacy in both fully observed and discretely observed settings. For the fully observed case, we propose a private algorithm that integrates private covariance estimation with noisy gradient descent, and establish its convergence rate. We further derive a matching minimax lower bound, demonstrating that the proposed estimator is minimax-optimal up to a logarithmic factor. In the discrete setting, we analyze the trade-offs among privacy, statistical accuracy, and discretization error, revealing a notable phase transition phenomenon. Simulations and real-data applications confirm the effectiveness of our approach in preserving privacy while maintaining statistical accuracy.
Differential privacy in functional linear regression presents unique challenges due to the infinite-dimensional structure of functional data, which complicates sensitivity analysis, noise calibration, and algorithm design. Standard privacy mechanisms developed for finite-dimensional settings are inadequate in this context. To address these difficulties, we use basis expansions to reduce the problem to high-dimensional linear regression and introduce a preconditioning strategy that ensures robustness, even under ill-conditioned covariance structures. This framework broadens the applicability of differential privacy to complex functional models and ill-posed settings, advancing privacy-preserving methods in infinite-dimensional data analysis.
- Paper: pdf file.