In a study published in the journal Judgment and Decision Making, 1,682 adults were asked to read one of six short stories, three of which were written by humans and three by ChatGPT. Each AI story had a similar theme to one of the human-authored works.
The team told participants whether their story was written by a human or AI, but this information was not always correct. The researchers then asked participants to rate how absorbing and engaging they found the story, and its quality.
Participants who read an AI-generated story rated it as more absorbing and of higher quality than those who read a story written by a human. However, participants gave higher ratings to stories they had been told were written by people.
Dr Deena Skolnick Weisberg, a senior author of the research from Villanova University in Pennsylvania, said: “AI systems can already generate short stories that are seen as being at least as good as – if not better than – human-written stories. We should update our views of AI’s abilities accordingly.”
However, Weisberg, who is herself a creative writer, said that did not mean writing should be left to AI, noting that novels produced by tech were probably going to be different from those written by humans.
“We may need to make room for AI-generated novels, and for AI/human co-written novels, but that doesn’t mean that there’s no longer space for us to appreciate the process of human creativity,” she said.
Look, man, I have been in writing groups and classes with people of all ages and skill levels. I have read some truly horrendous stories. Is the infinite word predictive machine going to write a better story than the worst, least skillful writer? Easily. Is it going to produce a better story than some of the more skillful writers? Absolutely not.
Worth noting also, I’m not saying someone who is noted for selling a lot of books, or even someone in an MFA program for writing. I am saying, LLMs are unlikely to produce a better work than me, and I am just some guy who is pretty skilled and practiced at writing. Maybe the one or two most skilled in a group of hobbyists who meet and share feedback and try to get better.
AI is always going to shoot for the absolute middle in terms of quality, and can definitionally produce nothing new or innovative.
Reading the journal paper this article referenced, I was a little disappointed that there weren’t more variables accounted for regarding the participants and the texts. With the participants, factors I see commonly accounted for are levels of education, socioeconomic indicators (say income), or how many books the participants have read in the past month or something similar.
Regarding the text, some form of analysis to then account for or attempt to equalise the AI-generated texts so that they’re similar to the author-written ones would have been nice - like average sentence lengths or average syllables per word.
However, this is still a data point (and a scary one) that it can be very hard to distinguish AI-generated content from human-generated ones…
To be honest, this moment was always coming or is yet to come for anti-AI folks.
The real issue isn’t whether it’s good or bad. The reason it’s such a hot thing is because generally it’s good enough to get people interested. And that’s very unlikely to go away.
Too interested? Sure.
But machines that are better than people at something people value about themselves. That’s where we’re heading if we’re not there already.
Which unlocks a massive question: What kind of world do you want to live in and how much are you willing to do to maintain it? Just about everything else is no longer a given. Not our value or importance, not our position at the top of the food chain, not our democracy or wealth, or the continuous growth of capitalism, or even our continued existence.
We’re basically at the beginning of an AI apocalypse movie. You can’t presume that you know the ending. You have to figure out how you want it to end. And not just for you, but your children and grandchildren children. Cuz it’s all at stake now.
I read The machine stops short story and I really hope we don’t head that direction, but it seems we are.
What strikes me about the present moment is that it feels like particularly bad timing.
I’m not so sure that technological developments follow a well defined path and like to think about what alternative timelines would look like, especially with humanity being better prepared for them.
For instance, I’ve figured for a long time that the internet was a few decades earlier than it should have been for our social and political development. Probably TV too.
AI just feels 100s of years too early. Like we’re only just getting to grips with post-capitalism and climate change as ideas let alone problems we can solve.
I think this is actually advantageous to us in a way, because clearly AI is something of a system shock to many people right now, whereas in a hundred years it may have not been nearly as apparent as the danger it is (and I don’t mean LLMs, I mean AI in general, especially for surveillance and weapons).
Yea I hear. We’ll see I guess.
Personally, I’m seeing much more falling in line instincts than anything else and fear that any “shock” and reaction is confined to bubbles without any solid culture and infrastructure to break out into any sort of effective movement.
Luckily, revolutions have never required the majority of a populace to actively participate in to succeed, just a majority going “meh, sure”.
But yes, I am personally also very worried that we’re reaching a point with technology that state surveillance and violence will both be too easily applied and too pervasive for the revolution to overcome.
There’s only two choices:
- Learn how it works, adapt it to your lifestyle, use it as a weapon against the rich fucks that try to use it against you.
- Continue to act like luddites, bury your head in the sand, and pretend that it will go away. Which it won’t.
That’s it. Those are the choices. Most are sucking in that copeium for the second option.
Why are those two the only options that come to mind? You can’t imagine working against the rich fucks without using autocomplete?
And what is your plan exactly?
What’s yours? Pay Anthropic $2k/mo to generate fake C&D letters and send them out to random businesses?
Unless you have a plan that doesn’t rely on the service-based models, then I don’t see where you’re going with this. Sure, you can use self-hosted models, assuming you’re fine paying for the GPUs (Jensen Huang? Lisa Su? Lip-Bu Tan if you’re feeling special?) and power to run them of course. But you’ll be “behind” the big cloud-based models, endlessly chasing after them.
Or, hear me out, don’t use them and do it all yourself and you won’t have to pay these companies and make their execs richer.
Nice try fbi.
But the implication that a chatbot is needed to knock rich fucks down a peg is silly to me. Orcas didn’t need Claude to attack yachts, the ocean didn’t need Gemini to crush that dipshit and his sub, saint Luigi didn’t need grok when he was being a referee for my child’s soccer game
If a token predictor has removed your creative spark, then you’ve already lost. That’s unfortunate, but assuming everyone else has lost would make you wrong.
I don’t entirely disagree. I’d add that whatever other options there are, they involve skills, values and perspectives that are unpracticed and atrophied in modern western culture (and likely globally) to the point that that is its own problem.
I was a little disappointed that there weren’t more variables accounted for regarding the participants
I think it’s even worse than that. quoting from the study’s abstract:
In Study 1, participants (1,682 adults recruited from Prolific)
…
In Studies 2 and 3, participants (905 adults recruited from Prolific)
hadn’t heard of Profilic before, they describe themselves as:
so the common theme is that 100% of the study participants have opted-in to this “help evaluate AI as a paid side-hustle” thing. that is going to introduce a huge selection bias that is definitely not representative of the population as a whole.
Oof, I didn’t even catch or look up the Prolific bit!
This is like comparing landscape paintings to photographs of the same landscape from the same perspective and asking random people to judge which one they think is better. This is an insane comparison.
deleted by creator
People down voting a study. Hilarious.
Feels over reals. We’re fucking doomed.
A study isn’t valid simply because it’s a study, and headlines telling you what to think of their results without at least a vague sense of the numbers are propagandistic by nature.
Rated better quality? By who and how much? Pretty basic details to include.
From what the guardian reports on the details, this study looks like garbage. From the posted stories, the results are definitely garbage, and whether that’s the fault of the researcher or humanity will need a closer inspection.
I’ll tell you right now, coming to such a broad conclusion because some American sophomore students liked three AI stories better than three selected human written stories sounds pretty fucking stupid.
Are you kidding me? This was a University Prof, doing a proper study, published in the Cambridge University Press, and involving 1700 participants.
Did you not even bother to look up the source before jumping down my throat?
This is even further evidence of why we’re fucking doomed. Your feelings about this headline override any reasonability you have to the point where you won’t even look at the source.
I don’t feel that this matters much.
As something that uses statistics to generate its output, AI creates stuff that’s average, and so appealing to the masses. It’s not much different from mediocre books churned out by publishers chasing sales instead of art.
It’s the McDonald’s of art. Is it ok? Sure. Will it change your life? No fucking way.
At first I read the title as “AI-generated stores” and was like “well, yeah, Amazon’s design sucks, of course they’re better rated!”
