19
AI-generated stories rated better quality than human-written ones, study finds
(www.theguardian.com)
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
Reading the journal paper this article referenced, I was a little disappointed that there weren't more variables accounted for regarding the participants and the texts. With the participants, factors I see commonly accounted for are levels of education, socioeconomic indicators (say income), or how many books the participants have read in the past month or something similar.
Regarding the text, some form of analysis to then account for or attempt to equalise the AI-generated texts so that they're similar to the author-written ones would have been nice - like average sentence lengths or average syllables per word.
However, this is still a data point (and a scary one) that it can be very hard to distinguish AI-generated content from human-generated ones..
I think it's even worse than that. quoting from the study's abstract:
hadn't heard of Profilic before, they describe themselves as:
"Prolific connects researchers, AI developers and product teams with real people who take part in studies online. It’s built for collecting high-quality, human data quickly and reliably, from training AI models to running behavioral research. Every participant is vetted, active and fairly paid, so you get trustworthy results without the usual noise."
so the common theme is that 100% of the study participants have opted-in to this "help evaluate AI as a paid side-hustle" thing. that is going to introduce a huge selection bias that is definitely not representative of the population as a whole.
Oof, I didn't even catch or look up the Prolific bit!
In this economy, I'm not going to sit down and do evaluation work for you to sell to another company unless you fucking compensate me for my time. Who is willing to help do this study for free?
AI text can be very hard or nearly impossible to detect from human generated, at times, depending in the model used.
It's rather surprising.
People are also way worse at distinguishing them than they think they are. And it's the same story with images.