Background

Competition Update: The Autocast competition will soon be closed. Due to the Collapse of the FTX Future Fund, we are unable to continue with the competition as initially planned. As such, we will be ending the Autocast competition by next Monday (4/17th). In case this notice is missed, late submissions can be sent to forecasting@mlsafety.org until the 27th. We will distribute a reduced prize and will reach out to the top three participants with further details on the prize distribution process.

Forecasting plays a critical role in global decision-making. For example, forecasts about the spread of COVID-19 informed national lockdowns and economic forecasts influence how interest rates are set. These predictions generally rely on the careful judgment of human experts that must consider data from a variety of sources. Since AI systems are able to process large volumes of data, they can potentially be very useful in this domain.

We want to encourage the development of open source AI forecasting tools and methods because we think that the improvements they could make to decision-making would have far-reaching positive effects on the world.

The Challenge

The objective of the competition is to build a machine learning model that makes accurate and calibrated forecasts.

The questions are taken from forecasting tournaments such as Metaculus, Good Judgment Open, and CSET Foretell and can be true/false, multiple choice, or involve predicting a numerical quantity or date. They tend to be about topics of broad public interest and have clear resolution criteria. For example: “Who will win the 2022 presidential election in the Philippines?”

The competition will have multiple rounds: a warm-up round followed by future rounds.

Please refer to the official rules and the Autocast paper for more details.

Important Dates

Warm-up round Submissions open: Sep 14th, 2022.

Warm-up round Submissions ends: Apr 17th, 2023

Future rounds: TBD