RainbowPO & MallowsPO (Unified Multi-Component Preference Optimization)

Integrates seven orthogonal DPO improvements—length normalization, margin calibration, reference-policy mixing ($\alpha \pi_\theta + (1-\alpha)\pi_{\text{ref}}$), WPO reweighting, and contextual scaling (Mallows dispersion)—into a single cohesive objective that outperforms individual DPO variants.