- The study involved students and experienced banking executives — a total of 344 people.
- They went through a simulation based on gamification elements in which they had to make CEO decisions. The quality of these decisions was recorded using various metrics. The participants had to complete several rounds that built on each other in the form of business years. More than 500,000 decision combinations were possible per round.
- A digital twin of the US automotive industry was used as the data basis; it included information on car sales and pricing strategies as well as overarching factors such as economic trends and the effects of the COVID-19 pandemic.
- The goal: to maximize market capitalization, which results from a combination of sustainable growth rates and free cash flow (and not being fired from the virtual board by meeting various KPIs). “This goal served as a realistic benchmark to measure the actual performance of CEOs,” the scientists wrote.
- GPT-4o from OpenAI was then confronted with the same tasks and the results were compared with those of the best two human participants from both groups.
“The results were both surprising and provocative and challenged many of our assumptions about leadership, strategy and the potential of AI when it comes to high-level decision making,” the researchers reported. GPT-4o consistently outperformed the best human participants on almost all recorded metrics, designed products with surgical precision, and kept costs tightly under control. However, the researchers complain: “GPT-4o was dismissed from the virtual board faster than the students.”
They attribute this primarily to so-called “black swan” events: “We integrated these unpredictable shocks to simulate sudden price fluctuations, changes in consumer behavior and supply chain problems,” the scientists said. The top performers among the students approached these risks with caution. They focused primarily on remaining adaptable in uncertain times rather than pursuing short-term profits.
GPT-4o, on the other hand — like the best bank managers — took a different path, as the researchers note: “The AI adopted an optimization mindset and maximized growth and profitability regardless of losses — until it was thrown off course by a market shock.”
