Reference Output

======================================================================================================================================

Standard answer:

Define

g(x) = x + ηLx^λ,
h = g^{-1}.

For any α > 1,

D_α( L(W_T) || L(W'_T) ) ≤ ε(α),

where

ε(α) ≤ min_{τ,β,a}
        [α / (2σ²)]
        Σ_{t=τ}^{T−1}
        [ (2ηK/n)² / β_t + a_t² / (1−β_t) ],

subject to

τ ∈ {0,1,...,T−1},
β_t ∈ [0,1],
a_t ≥ 0,

A_T = 0,
A_{t−1} = h(A_t + a_{t−1}),
A_τ ≥ D_τ,

D_0 = 0,

and

D_t = min {
    D_{t−1} + ηL D_{t−1}^λ + 2ηK/n,
    D_{t−1} + 2ηK,
    D
}.

======================================================================================================================================

Reference:

This is the full-batch result in Theorem 3.7 of:

Eli Chien and Pan Li,
“Convergent Privacy Loss of Noisy-SGD without Convexity and Smoothness,”
ICLR 2025.

The corresponding forward Wasserstein-distance recursion is given in Lemma 3.10.

======================================================================================================================================

Comparison with Claude Code:

Claude Code derived a looser bound. Its forward discrepancy recursion omitted the independent clipping-based branch

D_{t−1} + 2ηK,

and instead tracked only the Hölder-based branch together with the diameter bound. Since the omitted branch can be strictly smaller in relevant parameter regimes, the resulting optimized Rényi privacy bound is strictly looser than the reference bound.