Can isolating task-specific parameters prevent multi-task fine-tuning interference?
Explores whether identifying and protecting task-specific parameter regions can prevent the performance degradation that occurs when fine-tuning models on multiple tasks simultaneously. This matters because it could enable safe multi-task adaptation without sacrificing individual task performance.
Core Parameter Isolation Fine-Tuning (CPI-FT) starts from the hypothesis that the "seesaw effect" in multi-task fine-tuning — where improving one task degrades others — arises from parameter heterogeneity: distinct capabilities rely on specific, potentially overlapping parameter subsets, and uniform updates cause destructive interference.
The framework has five steps:
- Independent fine-tuning per task to identify each task's "core parameter region" (parameters with largest update magnitudes)
- Task clustering by core region overlap — tasks sharing similar core regions are grouped for joint training
- Parameter transplantation — core parameters from individually fine-tuned models are directly transplanted into a unified backbone
- SLERP fusion — non-core parameters are merged via Spherical Linear Interpolation, enabling geometry-aware blending that avoids abrupt transitions
- Pipeline SFT with core regions frozen to prevent catastrophic forgetting
The key finding: full multi-task SFT (uniform parameter updates across all tasks) consistently achieves the lowest performance across all tasks and configurations. Even heuristic multi-stage approaches (training tasks in sequence) only partially mitigate interference. CPI-FT consistently outperforms both — revealing that temporal task scheduling alone is insufficient without explicit structural parameter isolation.
This extends the sparse subnetwork finding from RL. Does reinforcement learning update only a small fraction of parameters? showed that RL naturally converges on sparse parameter subsets. CPI-FT shows that SFT exhibits the same structure — task-relevant changes are concentrated in specific regions — and that explicitly identifying and protecting these regions enables better multi-task adaptation.
The connection to catastrophic forgetting in safety alignment is direct: "Can model developers allow users to fine-tune their aligned models on custom datasets while retaining safety?" CPI-FT suggests yes — by freezing the core parameter regions associated with safety alignment during subsequent task-specific fine-tuning.
Source: Training Fine Tuning
Related concepts in this collection
-
Does reinforcement learning update only a small fraction of parameters?
Investigating whether RL algorithms consistently modify only 5–30% of model parameters across different LLMs and RL methods, and what structural properties those sparse updates possess.
same sparse-update structure in RL; CPI-FT exploits it for multi-task SFT
-
Why do specialized models fail outside their domain?
Deep domain optimization creates sharp performance cliffs at domain boundaries. Specialized models generate plausible-sounding but ungrounded responses when queries fall outside their training scope, and often fail to signal their own ignorance.
CPI-FT addresses this: task-specific optimization without domain cliff
-
Why does SFT-then-RL training follow a predictable three-phase pattern?
When expert data diverges from a model's learned patterns, SFT-then-RL training exhibits disruption, readaptation, and overfitting phases. Understanding this progression could improve how we combine imitation and reinforcement learning.
CPI-FT's pipeline training is a structured version of multi-stage training
-
Does training order reshape how models handle different task types?
Explores whether the sequence of multi-task RL training systematically affects model capabilities across structured and creative domains, and whether this ordering effect can be predicted and optimized.
complementary multi-task approaches: CPI-FT solves interference through spatial parameter isolation (identifying task-specific regions), while Omni-Thinker solves it through temporal task scheduling (BWT-guided ordering); CPI-FT's finding that temporal scheduling alone is insufficient suggests both spatial and temporal strategies may be needed together
Click a node to walk · click center to open · click Open full network for a force-directed map
Original note title
core parameter isolation prevents multi-task fine-tuning interference by identifying task-specific regions and merging non-core parameters geometrically