Background

Forecasting plays a critical role in global decision-making. For example, forecasts about the spread of COVID-19 informed national lockdowns and economic forecasts influence how interest rates are set. These predictions generally rely on the careful judgment of human experts that must consider data from a variety of sources. Since AI systems are able to process large volumes of data, they can potentially be very useful in this domain.

We want to encourage the development of open source AI forecasting tools and methods because we think that the improvements they could make to decision-making would have far-reaching positive effects on the world.

The Challenge

The objective of the competition is to build a machine learning model that makes accurate and calibrated forecasts.

The questions are taken from forecasting tournaments such as Metaculus, Good Judgment Open, and CSET Foretell and can be true/false, multiple choice, or involve predicting a numerical quantity or date. They tend to be about topics of broad public interest and have clear resolution criteria. For example: “Who will win the 2022 presidential election in the Philippines?”

The competition will have multiple rounds: a warm-up round followed by future rounds.

Please refer to the official rules and the Autocast paper for more details.