Why Weather Forecasting Is So Hard

Lately many people have probably seen the exchange between a certain page and the Meteorological Department about forecasting and early warning. Following on from my last post, I’d like to share a little about why weather forecasting is so hard, and what goes on behind the scenes of the Google forecast that some people say is the most accurate (roughly). The post may be a little long. The summary is at the end.

To frame the problem quickly, it’s similar to predicting earthquakes. Weather is extremely chaotic. But here are the details.

1. The butterfly effect

In 1961, Edward Lorenz, a meteorologist at MIT, was running a simple weather model on a computer and wanted to see the result again. So he typed in the starting values (such as the current temperature) from a printout, which had rounded 0.506127 down to 0.506, a difference only in the fourth decimal place. The result: at first the graph was exactly the same, but after a while the outputs came out completely different.

This was the beginning of chaos theory, and the origin of the term butterfly effect: a system where a tiny change in the starting values can change the future result enormously.

I tried simulating it with the Lorenz equations (1963). I ran two runs whose starting values differ by only 0.00000001. At first the curves are the same, but later on the results are very different. Weather forecasting is the same. There are small factors like this everywhere. Collect a slight error, and the forecast that comes out is no longer the same.

2. Errors grow exponentially

If you plot the distance between the results of the two runs on a log scale, you get a straight line. That means the error in the forecast grows exponentially. For example, if tomorrow’s forecast is off by 2 times, two days later it could be off by 4 times.

What I find most interesting is that if we measure the starting values 10 times more accurately, we gain only a little more forecasting time (about 2.6 units in this simulation), because the exponential growth of error dominates what can be predicted. To gain the same amount again, we’d need to be another 10 times more accurate, and so on.

In the real world, meteorologists estimate that the limit of daily weather forecasting is about 2 weeks, no matter how good our computers are, because this is the nature of the atmosphere itself, not a limit of technology. Forecast accuracy has been improving gradually over the years, roughly one more day of forecast range every 10 years.

3. Ensemble forecasts

Since we don’t know the starting values exactly, what forecasting centers do is run the model many times, each time with slightly different starting values. This is called an ensemble forecast. Then they look at how widely the results spread.

In my simulation of 50 runs with only slightly different starting values, at first all the results go the same way. As we go further out, the forecasts gradually spread out more and more.

This is why modern forecasts are stated as probabilities. A 60% chance of rain means roughly that 60% of all runs say it rains. As for the black line that is the average, far out it goes flat in the middle, which doesn’t match any single run.

4. Model resolution

Weather models divide the world into a grid, and each cell can hold only one value. Typical global models use cells of about 10 to 25 km, but the thunderstorm clouds that flood Bangkok are only a few kilometers wide.

In my simulation I modeled a rainstorm as small cells 2 to 4 km across in a 100 x 100 km area. In the real thing, some spots get 118 mm/hour. But viewed through a 25 km grid, the value gets averaged down to just 23 mm/hour, turning into moderate rain spread across the whole area. A 5 km grid can still see the heavy spots.

And Thailand is in the tropics. Most of our rain is convective: small clouds that form fast and disappear fast. That differs from temperate countries, where the air is driven by large frontal systems, which are easier to forecast. So this is especially hard for us.

5. We don’t even know what the atmosphere is like right now

The last point is one people don’t often think about. Before we can forecast the future, we need to know the present state. But the data we have comes from widely spaced weather stations, balloons, satellites, and radar, each measuring different variables at different resolutions, and these must be combined into a picture of the atmosphere of the whole globe. This step is called data assimilation, and it takes an enormous amount of computing.

The result of doing this for several decades back is a database called ERA5 from the European Centre for Medium-Range Weather Forecasts (ECMWF).

6. Where does AI help?

The traditional method (Numerical Weather Prediction) solves the physics equations of the atmosphere on a supercomputer, which is accurate but expensive and slow. Google DeepMind chose another way: take about 40 years of ERA5 history and let AI learn how the atmosphere changes from one state to the next, without solving any equations. Within the past 5 years, the trend of using AI to help forecast these phenomena has grown a lot.

• GraphCast (2023, published in Science) forecasts 10 days ahead in under a minute and is more accurate than ECMWF’s main model on more than 90% of the variables tested.
• GenCast (2024, Nature) changes from a single answer to an ensemble, using a diffusion model (the same family as image-generating AI), which lets it express uncertainty, matching point 3 above.
• WeatherNext 2 produces a 64-run ensemble much faster.
• WeatherNext 3 (launched on September 3 this year) is the most interesting, because it directly addresses points 4 and 5. It doesn’t learn only from ERA5, but takes in satellite imagery and weather station data directly. Its resolution goes down to 5 km for temperature and humidity and 10 km for other surface variables, and it forecasts anew every hour. Google claims rain forecasts are about 60% better than the previous version, and it is now being used in Google Search, Maps, and Gemini.

That means when we check the rain forecast on Google these days, what’s inside is this model.

7. Can AI solve chaos?

The short answer is no. Lorenz’s butterfly effect is still there, and the roughly two-week limit is still there, because it is the nature of the atmosphere. What AI can help with is:

1. It is much faster and cheaper, so we can run more ensemble members and forecast anew more often.
2. Higher resolution, so rain cells can be seen more clearly.
3. New observations can be brought in faster, making the starting values more accurate.

But there are caveats. AI learns from the past, and strange events that have almost never happened are what it sees least (similar to the problem with large-earthquake data that I described before). For example, if climate change suddenly caused snow in Bangkok, AI would have no way of forecasting it. And the data used to teach AI, like ERA5, still comes from physics models, so the two sides will probably need to go together for a long time.

In short

• The atmosphere is a chaotic system. Small errors grow exponentially, which gives a forecasting limit of about two weeks (and even that is still not 100% accurate).
• Modern forecasting speaks in probabilities from running many times (ensemble forecasts).
• Tropical rain cells are smaller than the grid used in the models, and we still don’t know the complete current state of the atmosphere.
• AI such as WeatherNext 3 makes things faster and finer and can use raw data, but the butterfly effect is still there either way.

In the next post I’ll continue with how GraphCast sees the whole world as a graph, and why that approach suits weather forecasting (which happens to match the course I teach at CMKL 😏).

Pai