Worried that your job might one day be taken over by AI? That day could come sooner rather than later.
Two separate research papers have revealed that ChatGPT has what it takes to pass the U.S. Medical Licensing Exam and could potentially earn an MBA from an Ivy League business school.
Physicians and financiers beware — but lawyers can rest easy for now. Another study found that ChatGPT was only around 50 per cent accurate on the Multistate Bar Examination, a multiple-choice test that comprises one part of the legal licensing process in the U.S.
In a white paper titled “Would Chat GPT3 Get a Wharton MBA?,” University of Pennsylvania Prof. Christian Terwiesch details the results of putting the AI language model to the test. He gave it a final exam for an operations management course at the Wharton School of Business and found that the chatbot would have earned a B to B-.
“First, it does an amazing job at basic operations management and process analysis questions,” Terwiesch writes. “Not only are the answers correct, but the explanations are excellent.”
But Terwiesch found that the chatbot ran into trouble with performing even basic mathematics.
“Chat GPT3 at times makes surprising mistakes in relatively simple calculations at the level of 6th grade Math. These mistakes can be massive in magnitude,” the paper reads.
ChatGPT demonstrated “remarkable ability,” Terwiesch writes, noting it has the potential to automate skills performed by “highly compensated knowledge workers” like analysts, consultants and managers.
In a separate pre-print study, led by doctors from medical startup Ansible Health, researchers found that ChatGPT “performed at or near the passing threshold for all three exams” needed to be licensed as a doctor in the U.S.
The language model did this “without any specialized training or reinforcement,” and it “demonstrated a high level of concordance and insight in its explanations,” the paper noted. Its findings are still in the midst of the peer-review process.
As for ChatGPT’s performance at the bar, the chatbot earned a passing grade in the Evidence and Torts section of the Multistate Bar Exam, a December 2022 paper found. Its accuracy on all portions of the test was 50.3 per cent, though this score is still much higher than the 25 per cent baseline expected of random guesses.
The researchers noted that ChatGPT will likely be able to pass the U.S. bar “in the near future.”
Each study mentioned the future potential of integrating AI and language models into their respective fields, and ChatGPT has already begun to shake up how we approach education.
No one can say for sure to what degree AI will impact the future of work. What’s certain is that humans alone no longer have the market cornered on intelligence and creativity.