Artificial intelligence bot ChatGPT made headlines earlier this year when it passed the U.S. Medical Licensing Exam; however, when given the American College of Gastroenterology's pre-board assessment, it failed to make the grade.
When researchers asked it to take the 2021 and 2022 multiple-choice self-assessment tests, ChatGPT scored 65.1 percent and 62.4 percent, compared to the 70 percent required to pass the exam, according to a May 22 press release from Lake Success, N.Y.-based Northwell Health's Feinstein Institutes for Medical Research.
Each test consists of 300 multiple choice questions with real-time feedback. Overall, ChatGPT answered 455 questions correctly. ChatGPT-3 answered 296 questions correctly, while ChatGPT-4 answered 284 questions correctly.