ChatGPT Bot Succeeds On The US Law School Exam

An internet-powered chatbot that wrote essays on subjects including constitutional law, taxation, and torts was able to pass tests at a US law school.

Using artificial intelligence (AI), ChatGPT from OpenAI, a US company that last week received a large funding injection from Microsoft, generates streams of text from simple prompts.

Because of the excellent results, educators have issued warnings that they might encourage widespread cheating or possibly spell the end of conventional classroom teaching methods.

Professor Jonathan Choi from the law school at Minnesota University administered the same 95 multiple-choice questions and 12 essay questions to ChatGPT.

He and his coauthors disclosed that the bot received an overall grade of C+ in a white paper titled ChatGPT goes to law school that was published on Monday.

While this was sufficient for a pass, the bot consistently scored in the bottom half of the class and "bombed" math multiple-choice tests.

The authors stated that ChatGPT "In writing essays, ChatGPT displayed a strong grasp of basic legal rules and had consistently solid organisation and composition,"

However, when given an open-ended prompt, a crucial ability on law school examinations, the bot "often struggled to spot issues when given an open-ended prompt, a core skill on law school exams".

The use of ChatGPT in schools has been banned in New York and other places, but Choi claimed it would be a valuable teaching tool.

He stated on Twitter that "Overall, ChatGPT wasn't a great law student acting alone," 

"But we expect that collaborating with humans, language models like ChatGPT would be very useful to law students taking exams and to practicing lawyers."

He also claimed, in response to another Twitter user, that two out of three markers had picked up on the document that had been authored by a bot, downplaying any chance of cheating.

"(They) had a hunch and their hunch was right, because ChatGPT had perfect grammar and was somewhat repetitive,"