Forecasters vs. GPT-4: Who wins?

Forcasting is a method to give a best guess for what will happen in the future.

I recently had a client for autory.ai ask me about a tool that would help them forcast market fit for products.

Out of curiosity I built a version of this that can compete with other forcasters on websites like manifold and metaculus.

Inspired by the principles from the book "Superforecasting" by Tetlock (2015), which lays down ten rules for making better predictions, I worked on incorporating these into GPT-4 to hone its forecasting abilities.

The result? When faced with yes-or-no questions, it can correctly predict the outcome 50% of the time. It's almost like flipping a coin, but with a dash of high-tech!

Jokes aside, the next step might just be to bring in an actual coin to match this accuracy. 😄