Introducing AIMIP: The AI weather and climate model intercomparison project

A new generation of AI models can simulate aspects of Earth’s climate far more efficiently than traditional systems, but the field still needs rigorous, shared ways to test whether those models are accurate and reliable. 

To address that gap, we’ve been leading a community effort called AIMIP (AI Model Intercomparison Project) to support scientific understanding and open evaluation of AI models for climate forecasting. AIMIP brings together multiple modeling groups, including NVIDIA, Google Research, and others, around a shared benchmark experiment and dataset—making it easier to compare systems on common outputs and evaluation criteria and helping build confidence in how these models are assessed. 

As part of AIMIP Phase 1, we’re releasing a dataset of AI weather and climate model forecasts for the above-mentioned benchmark experiment, along with a report and evaluations showing that AI models are competitive on key climate metrics but continue to struggle in some areas.

Leveraging a revolution in weather and climate forecasts

AI climate models are relatively new, but they build on several years of rapid development in using AI to predict short-term weather patterns. Relying on a large dataset of historical weather observations spanning the entire atmosphere (called ERA5) as training data, AI-driven forecasts now regularly beat conventional weather models at key skill metrics for forecasts 1-10 days in the future as demonstrated on WeatherBench, a leaderboard for AI weather models. And they do so with extraordinary speed, using far less computational power than conventional models.