Mix, Don't Tune: Bilingual Pre-Training Outperforms Hyperparameter Search in Data-Constrained Settings
AI Classified by Officially
For most languages of the world, language model pre-training operates in a data-constrained regime where models must repeat their training data many times, degrading generalization. Two remedies exist: aggressive hyperparameter tuning such as high weight decay, and mixing in data from a high-resource auxiliary language to directly aid the low-resource target. …
This is an extract. The publication continues at the source.
Read the original at the source: https://machinelearning.apple.com/research/mix-dont-tune
Officially imported this from Apple Machine Learning Research’s own source and shows an extract. If you work there, claiming the profile and verifying the domain lets you choose to show the full text here.
Provenance
- Organization
- Apple Machine Learning Research — imported from official source
- Official source
- https://machinelearning.apple.com/rss.xml RSS
- Imported
- October 13, 2026 15:01
- Versions
- 1 recorded
- Identity
-
mix-dont-tune