mSFT: Addressing Dataset Mixtures Overfitting Heterogeneously in Multi-task SFT
arXiv:2603.21606v4 Announce Type: replace Abstract: Current language model training commonly applies multi-task Supervised Fine-Tuning (SFT) using a homogeneous compute budget across all sub-datasets. This approach is fundamentally sub-optimal: heterogeneous learning dynamics cause faster-learning tasks to overfit early while slower ones…
