Differentially Private Neural Network Training Under the Hidden State Assumption

תקציר מקורי באנגליתarXiv:2407.08233v3 Announce Type: replace Abstract: Current differentially private learning paradigms face a severe utility bottleneck: DP-SGD degrades performance through noise accumulation over training steps, while aggregation-based approaches such as PATE suffer from data inefficiency due to disjoint data partitioning. We propose \textbf{Differentially Private Decoupled Training (DP-DT)}, a framework that decouples representation learning from privacy enforcement. DP-DT confines noise injection to the weight aggregation stage and employs auxiliary models, continuously synchronized with a global model, to perform noise-free feature extraction on private data shards. We prove that DP-DT converges globally to a limit point under non-convex objectives, via a Lyapunov potential analysis com