Four ChatGPT Experiments That Failed Hilariously
People keep going on about half of humanity’s imminent plunge into technocratic destitution after A.I. takes our jobs, but we don’t have to brace for dystopia just yet. There’s still a lot A.I. isn’t good at — and sometimes hilariously bad at. For example…
It Can’t Play ‘Red Dead Redemption 2’
Even literal babies can play video games, but when a team of Chinese and Singaporean researchers tried to teach ChatGPT to play Red Dead Redemption 2, it got “quickly overwhelmed by the game’s diverse problem-solving missions.” Specifically, it sucked at figuring out how to move in space and making time-sensitive decisions. Fortunately, one job that A.I. won’t be taking is “fucking around with video games for science.”
It Can’t Do Your Taxes
The automation of frustrating number-crunching seems like one of the few tasks you’d be happy to hand over to our A.I. overlords, but don’t expect help from ChatGPT anytime soon. Accounting Today gave ChatGPT a practice CPA exam, and it “failed miserably” on all four sections. And unlike a real accountant, it’ll totally bail on you come audit time.
It Can’t Diagnose Your Illness
One of the scariest fields A.I. is being pushed into is medical diagnostics, especially considering it’s super bad at it. In one study published in JAMA Pediatrics, ChatGPT misdiagnosed 72 out of 100 cases, and in another 11 cases, its diagnosis was too broad to be considered accurate. That’s a 17 percent success rate. You could get a better result by throwing a dart at WebMD.
It Can’t Get a Medical License (in Some Countries!)
That’s just as well, because ChatGPT shouldn’t be practicing medicine at all, at least in some countries. It recently failed Taiwan’s Family Medicine Board Exam, though not necessarily because it’s bad at such exams. The researchers noted “the relatively weak database of traditional Chinese language resources” as one possible complication, which explains ChatGPT’s success with the U.S. Medical Licensing Examination.
To be fair, “a doctor who can’t diagnose you” is what most people’s insurance plans assign them anyway.
