cDPO (Conservative DPO)

Accounts for a known label-noise rate $\varepsilon \in (0, 0.5)$ in human/AI preference annotations by applying label smoothing directly to the DPO cross-entropy objective: $\mathcal{L}_{\text{cDPO}}(\theta) = -(1-\varepsilon)\log \sigma(\Delta_\theta) - \varepsilon \log \sigma(-\Delta_\theta)$