Last year, Google got a silver medal at IMO using a “formal” system, meaning it required humans to translate problems into a machine-readable format. For this year, OpenAI and Google entered “informal” systems into the competition, which were able to ingest questions and generate proof-based answers in natural language. The companies are saying that their AI models correctly responded to four out of six questions on IMO’s test, scoring higher compared to most high school students and Google’s AI model from last year, without needing any human-machine translation.
TechCrunch conducted an interview with both research teams from OpenAI and Google’s IMO efforts, claiming that those efforts put in winning the gold-medal performances represent breakthroughs around AI reasoning models in non-verifiable domains.
More so, shortly after OpenAI announced its feat on Saturday morning, Google DeepMind’s CEO and researchers took to social media to slam OpenAI for announcing its gold medal prematurely – shortly after IMO announced which high schoolers had won, the competition on Friday night – and for not having theri medal’s test officially evaluated by IMO.
Luong also said that Google has been working with IMO’s organizers since last year in preparation for the test and wanted to have the IMO president’s blessing and official grinding before announcing its official results, which it did on Monday morning.
“The IMO organizers have their grading guideline,” Luong said. “So any evaluation that’s not based on that guideline could not make any claim about gold-medal level [performance].”
