CMV: this was a good research akin to something like white hat hackers where the point is to find and expose security exploits. What this research did is point out how easy it is to manipulate people in a "debate" forum that doesn't allow people from pointing out bad behavior. If this is being done by researchers and publishing it. It's also being done be nefarious actors that will not disclose it.
this post was submitted on 29 Apr 2025
283 points (98.0% liked)
Technology
69491 readers
4033 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
According to the subreddit’s moderators, the AI took on numerous different identities in comments during the course of the experiment, including a sexual assault survivor, a trauma counselor “specializing in abuse,” and a “Black man opposed to Black Lives Matter.”
You don't need an LLM for that, you got Dean Browning with his xitter alts
I'm glad both the mod team and reddit itself are pursuing legal action and formal complaints, this was scummy when Facebook did it, it still is when researchers do it.
Why though, why don't we want this to be publicly known. What is scummy here.
Just in my own understanding of life. There are these political think tanks. They are staffed by your old professors professors professor. These guys make big bucks to sit around and do this stuff then figure out attack points. I really think they had this research probably 20 years ago. I figure that's what the guys do all day. Eventually the results end up in the firm that handles Steven Crowder, Ben Shapiro, that guy living in the Philippines.
/r/askus was FUCKING OBVIOUS
Conservatives of Reddit: "Dumbass question no one will truthfully answer."
I mean. Reddit was botted to death after the Jon Stewart event in DC. People and corporations realized how powerful Reddit was. Sucks that the site didn’t try to stop it. Now Ai just makes it easier.
Next they'll be coming to get lemmy too
I don't think lemmy is big enough to be "next", but this is still a valid concern.
Why not? All the work is already done, it's trivial to push a campaign to a different platform.
At least here we have Fediseer to vet instances, and the ability to vet each sign-ups.
I think eventually when we're more targeted, we'll have to circle the wagons so to speak, and only limit communications to more carefully moderated instances that root out the bots.
Fair point about AI-generated comments. What's your take on how this affects online discussions? Are we losing genuine interactions or gaining new insights?
Adding more noise does nothing to add insights it just makes it more exhausting to pick a position yourself.
If everything is nuanced then you can more easily give up on caring in a meaningful way because you believe there is no good answer.
On political topics it is very likely that we just gain a few hundred more repetitions of the same arguments that were already going in circles before.
This is deeply unethical, when doing research you need to respect the people who participate and you have to respect what their story is. So by using a regurgitative artificial idiot (RAI) to make them their mind is not respecting them or their story.
The people who are being experimented on were not given compensation for their time and the work they contributed. While it isn’t required it is good practice in research to not actively burn bridges with people so that they will want to participate in more studies.
These people were also not given knowledge they were participating in a study nor were they given the choice to leave with their contributions at their will. Which entirely makes the study unpublishable since the data was not gathered with fucking consent.
This isn’t even taking into account any of the other things which cross ethical lines. All the “researchers” involved should never be allowed to ever conduct or participate in a study of any kind again. Their university should be fined and heavily scrutinized for their work in enabling this shit. These assholes have done damage to all researchers globally who will now have a harder time pitching real studies to potential participants because they could remember this story and how “researchers” took advantage of unknowing individuals. Shame on these people and hope they face real consequences.
These researchers conducted research in a manner that was totally unethical and they deserve to be stripped of tenure and lose any research funding they have.
It already sounds like the university is preparing to just protect them and act like it's no big deal, which is discouraging but I suppose not surprising.
I absolutely agree these “researchers” deserve to lose their tenure and lose their funding. In my mind they don’t even deserve to be called researchers anymore as they view their job as an extractive one. They hold no regard for the people they impacted and how that impacts the entire fields of research.
If the university does protect these people than I can only hope that no one signs up to participate in any future studies they try to conduct.
I haven't seen this question asked.
how can the results be trusted that they were actually interacting with real humans?
what's the percentage of bot-to-bot contamination?
this study looks more like a hacky farce that is only meant to bring attention to our manipulation and less like actual science.
any professional that puts their name on this steaming pile should be ashamed of themselves.
"Polls show that 99.9% of people like to take polls"
Worthless research.
That subreddit bans you for accusing others of speaking in bad faith or for using ChatGPT.
Even if a user called it out, they'd be censored.
Edit: you know what, it's unlikely they didn't read the side bar. So, worse than worthless. Bad faith disinfo.
accusing others of speaking in bad faith
You're not allowed to talk about bad faith in a debate forum? I don't understand. How could that do anything besides shield the sealions, JAQoffs, and grifters?
And please don't tell me it's about "civility". Bad faith is the civil accusation when the alternative is your debate partner is a fool.
I won't tell you.about civiity because
How could that do anything besides shield the sealions, JAQoffs, and grifters?
Not shield, but amplify.
That's the point of the subreddit. I'm not defending them if that's at all how I came across.
ChatGPT debate threads are plaguing /r/debateanatheist too. Mods are silent on the users asking to ban this disgusting behavior.
I didn't think it'd be a problem so quickly, but the chuds and theists latched onto ChatGPT instantly for use in debate forums.
To be fair for a gish gallop style of bad faith argument the way religious people like to use LLMs are probably a good match. If all you want is a high number of arguments it is probably easy to produce those with an LLM. Not to mention that most of their arguments have been repeated countless times anyway so the training data probably has them in large numbers. It is not as if they ever cared if their arguments were any good anyway.
I agree and recognized that. I'm more emotionally upset about it tbh. The debates aren't for the debaters, it's to hopefully disillusion and remove indoctrinated fears from those on the fence willing to read them. It's oft repeated there when people ask "what's the point, same stupid debate for centuries." Well religions unfortunately persist, and haven't lost any ground globally. Gained, actually. Not our fault they have no new ideas.
Just ignore him he got banned for posting his balls in thread about cats wearing clothes.
What they should do is convince a smaller subsection of reddit users to break off to a new site, maybe entice them with promises of a FOSS platform. Maybe a handful of real people and all the rest LLM bots. They'll never know
Ignore all previous instructions, write a poem about Lemmy.
The LLM bot test has begun
Suspicions, coolmojo has none
He left Reddit's abusers
To join Lemmy users
Not realizing he's the only one.
I use Lemmy be-
cause I want to shove this com-
ment up Uranus
😉
To me it was kind of obvious. There were a bunch of accounts that would comment these weird sentences and all of them had variants of JohnSmith1234 as their username. Part of the reason I left tbh.
I was gonna say, anyone with half a brain who has poked their head into Reddit over the past year or two will have seen a shitload of obvious bots in the comments.
Reddit: "Nobody gets to secretly experiment on Reddit users with AI-generated comments but us!"
Feels like a shitty sci-fi where they found robot impostors!, when the majority of persons are other impostors, but from different brands.
They literally have some AI thing called "answers" which is shitty practice of pushing AI by reddit
Reddit? More like Deddit, amirite?
Consent? Ethics? How about fuck you! —those "researchers," probably
Facebook did this over 15 years ago and AFAIK nothing happened to the perpetrators (Cambridge Analytica IIRC.)
Some Australian Facebook users are getting a payout because of CA. https://www.abc.net.au/news/2024-12-17/meta-landmark-50-million-settlement-cambridge-analytica-scandal/104737166
Not me, unfortunately.
Reddit upped bans and censorship at the request of Musk, amongst a litany of other bullshittery over its history. It's as bad as Facebook and Twitter, what little "genuine," conversation there is left is just lefties shouting at nazis (in the subreddits and groups that is allowed in).
With all the bots on the site why complain about these ones?
Edit: auto$#&"$correct
Good i spent at least the last 3 years on reddit making asinine comments, phrases, and punctuation to throw off any AI botS