OpenAI's latest model, Orion, is reportedly showing a slower rate of improvement compared to its predecessors, according to a report by The Information. This has sparked a debate in Silicon Valley about whether AI models are reaching a performance plateau, as the industry has seen rapid advancements in the past due to significant leaps in performance with each new release.
OpenAI CEO Sam Altman has previously spoken about the concept of "scaling laws," which suggest that AI models become smarter as they increase in size and have access to more data and computing power. However, The Information's report indicates that technical staff within the company are questioning these laws, as evidence mounts that leading AI models may be hitting a performance wall.
One of the key factors contributing to this potential plateau is the availability of data. Companies have been scraping vast amounts of human-created data to train their AI models, but the supply is finite. Additionally, the constraints of computing power are also a limiting factor in the continuous improvement of AI models.
Critics of the AI industry, such as NYU professor emeritus Gary Marcus, have argued that AI development is destined to hit a wall and that there are signs of "diminishing returns" in terms of performance improvements. However, there are still industry leaders, like Microsoft's Kevin Scott, who remain optimistic about the scaling potential of AI.
While OpenAI and other companies continue to explore ways to enhance AI performance, it remains to be seen whether future models will be able to deliver the same level of impressive advancements that have characterized the industry in recent years. The debate over the future of AI scaling laws and the potential limits of AI performance is ongoing in Silicon Valley.