Can Artificial Intelligences Accurately Simulate Clinical Exams in Psychiatry?

Artificial intelligence tools capable of generating text are sparking growing interest in the field of medical training. A recent analysis evaluated their ability to respond to clinical situations in psychiatry, similar to those encountered during practical exams known as OSCEs. These exams test medical students’ skills through realistic scenarios, such as managing a patient after a suicide attempt or assessing an eating disorder.

The study shows that artificial intelligence can produce structured and medically relevant responses, provided the instructions given to it are clear and free of superfluous information. For example, when faced with a case of medication overdose, it is capable of asking the right questions about the amount ingested, the circumstances of intake, or associated risk factors. It also suggests appropriate management strategies, such as involving a senior physician or implementing psychological follow-up.

However, as soon as details unrelated to the medical situation are added to the instructions, the quality of the responses deteriorates. The suggestions become less precise, longer, and sometimes confusing. The tool can be distracted by anecdotal elements, such as the mention of an unusual object in the room or a patient’s hobby, which undermines the coherence of its responses. In some cases, it abruptly shifts from one topic to another without logical transition or adopts a less professional tone.

These observations highlight a major challenge: artificial intelligences struggle to filter relevant information when overwhelmed with extraneous data. Their operation is based on probabilities rather than a true understanding of context, making them vulnerable to errors when instructions lack clarity.

For medical students, these tools represent a useful resource for practice, especially in environments where access to simulations with real patients is limited. They allow students to practice clinical decision-making and the drafting of care plans. However, their use requires constant vigilance. Future physicians must learn to formulate precise queries and critically evaluate the generated responses, as blind trust could lead to errors in real-life situations.

The integration of these technologies into medical training must therefore be accompanied by rigorous oversight. They cannot replace human expertise, especially for essential skills such as empathy, communication, or clinical judgment. Their role is limited to supplementing traditional learning methods under the supervision of experienced instructors. As these tools evolve, further research will be necessary to ensure their reliability and suitability for the demands of medical practice.

Bibliography

Study Source

DOI: https://doi.org/10.57129/001c.159636

Title: Large language model used to simulate psychiatric OSCE scenarios: a medical student perspective

Journal: New Zealand Medical Student Journal

Publisher: New Zealand Medical Student Journal

Authors: Zhaochu Geng; Craig S. Webster; Yan Chen; Lillian Ng; Christian U. Krägeloh; Angel Li; Marcus A. Henning