A recent study published in the journal Science has revealed that OpenAI's new reasoning model, referred to as o1, is capable of diagnosing medical conditions at a level comparable to or surpassing that of trained physicians. Researchers tested the o1 model against OpenAI’s earlier model, GPT-4, as well as physicians and medical residents using previously unknown clinical cases. The results indicated that the o1 model achieved diagnostic accuracy over two-thirds of the time during initial triage, while expert attending physicians only reached correct diagnoses approximately half of the time.
Dr. Robert Wachter, a professor at the University of California, San Francisco, acknowledged the significance of these findings, stating that it is now "indisputable" that advanced AI can outperform both older models and human doctors in certain diagnostic scenarios. However, he cautioned that further research is necessary before AI can be fully integrated into clinical practice. Wachter pointed out that the study's limitations included its reliance on text-only inputs, lacking the visual and auditory cues that physicians typically utilize, such as patient demeanor and medical imaging.
The study's authors emphasized the need for additional research and prospective clinical trials to explore how AI could enhance clinical practice and patient outcomes. They noted that the rapid advancements in large language models (LLMs) carry significant implications for medicine. An accompanying article from Australian researchers echoed these sentiments, advocating for a collaborative model where AI supports, rather than replaces, human doctors, stressing the importance of accountability and safety in clinical applications of AI.