this post was submitted on 01 Mar 2025
18 points (100.0% liked)
Technology
41021 readers
74 users here now
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
The passing score for USMLE is ~200 out of 300... how many "wrong times" is that?
https://www.kaptest.com/study/usmle/passing-scores/
When we look at passing scores, is there any way to quantitatively grade them for magnitude?
Not all bad advice is created equal.
The grading is a mess. It goes about qualitative, quantitative... and statistical corrections "to make it fair".
Anyway, there is ~30% margin on the scores for passing, so chances are that 9% is better than the worst doctor who still "passed".
I'd hope the bar for medical advice is higher than "better than the worst doctor".
Will be interesting to see where liability lies with this one. In the example given, following the advice could permanently worsen patients.
Given that the advice is proven to be wrong and goes against official medical guidance for doctors, that could potentially be material for a class action lawsuit.
It's like in the joke: "What do you call someone who barely finished medical school?... Doctor."
Every doctor is allowed to provide medical advice, even those who should better shut up. Liabilities are like what a friend got after a botched operation, when confronting her doctor: "Sue me, that's what my insurance is for".
I'd like to see the actual final assessment of an AI on these tests, but if it's just "9% vs 15% error rate", I'd take it.
My guess would be the AI might not be great at all kinds of assessments, but having a panel of specialized AIs, like we now have multiple specialist cooperating, sounds like a reasonable idea. Having a transcript of such meeting analyzed by a GP, could be even better.
I take your point. The version I heard of that joke is "the person who graduated at the bottom of their class in med school".
Still, at the moment we can try to avoid those doctors. I'm concerned about the popularizing and replication of bad advice beyond that.
The problem here is this tool is being marketed to GPs, not patients, so you wouldn't necessarily know where the opinion is coming from.
There is no realistic way of avoiding those doctors. I've been to a GP who, after looking at my medical history and the meds I was taking after a heart attack... slid me a business card for her homeopathic healing practice. 🙄
Still, I'd hope a majority of doctors would be more likely to be able to parse through an AI's advice, and take it into consideration, but not blindly depend on it, when giving their own advice.
Targeting it at GPs makes sense, since they're supposed to "know of everything", but no person is capable of doing that, definitely not of staying up to date on everything. Specialists have a narrower area of knowledge to keep up with, but could also benefit from some AI advice based on latest research.