Health - Resources and discussion for everything health-related

In the study, the researchers fed 60 medical scenarios to ChatGPT Health. The chatbot’s responses were compared with the responses of three physicians who also reviewed the scenarios and triaged each one based on medical guidelines and clinical expertise.

They should have included more physician opinions, because they can be highly variable, and, they should have done this blinded so the physicians didn't know which cases were in the study and they could have been taking more time and effort, skewing the data. The LLM will be more consistent that random MDs at the end of a 12 hour shift at 5am. I would have asked for more real world real time physician opinions versus Chat GPT Health.

Regardless, the genie is out of the bottle and all hospitals will eventually use LLMs to cross-check MD decisions. Certainly in pathology reports, automated scoring of imaging is far more accurate than even three MDs agreeing and pathology decisions are notoriously innaccurate from meatbags.

Here's a Harvard study where 83% of radiologists missed a gorilla pasted into images.

Pigeons are less biased in image anaysis.