Task Arithmetic, TIES-Merging & DARE (Drop And REscale)

Operates on task vectors $\tau_m = \theta_m - \theta_{\text{base}}$:

• Task Arithmetic: $\theta_{\text{new}} = \theta_{\text{base}} + \sum_m \lambda_m \tau_m$.
• TIES-Merging (Trim, Elect Sign, Merge): Resolves parameter interference across multiple task vectors by (1) trimming the smallest $1-k\%$ magnitude deltas to zero, (2) electing the dominant sign $\text{sgn}(\sum_m \tau_m)$ per coordinate, and (3) averaging only coordinates matching the elected sign.
• DARE (Drop And REscale): Randomly drops $p \in [0.9, 0.99]$ of fine-tuned delta parameters $\tau_m$ using a Bernoulli mask and rescales the surviving $1-p$ deltas by $\frac{1}{1-p}$ before merging, eliminating cross-specialist interference.