this post was submitted on 06 Sep 2024

218 points (98.7% liked)

News

23397 readers

3529 users here now

Welcome to the News community!

Rules:

1. Be civil

Attack the argument, not the person. No racism/sexism/bigotry. Good faith argumentation only. This includes accusing another user of being a bot or paid actor. Trolling is uncivil and is grounds for removal and/or a community ban. Do not respond to rule-breaking content; report it and move on.

2. All posts should contain a source (url) that is as reliable and unbiased as possible and must only contain one link.

Obvious right or left wing sources will be removed at the mods discretion. We have an actively updated blocklist, which you can see here: https://lemmy.world/post/2246130 if you feel like any website is missing, contact the mods. Supporting links can be added in comments or posted seperately but not to the post body.

3. No bots, spam or self-promotion.

Only approved bots, which follow the guidelines for bots set by the instance, are allowed.

4. Post titles should be the same as the article used as source.

Posts which titles don’t match the source won’t be removed, but the autoMod will notify you, and if your title misrepresents the original article, the post will be deleted. If the site changed their headline, the bot might still contact you, just ignore it, we won’t delete your post.

5. Only recent news is allowed.

Posts must be news from the most recent 30 days.

6. All posts must be news articles.

No opinion pieces, Listicles, editorials or celebrity gossip is allowed. All posts will be judged on a case-by-case basis.

7. No duplicate posts.

If a source you used was already posted by someone else, the autoMod will leave a message. Please remove your post if the autoMod is correct. If the post that matches your post is very old, we refer you to rule 5.

8. Misinformation is prohibited.

Misinformation / propaganda is strictly prohibited. Any comment or post containing or linking to misinformation will be removed. If you feel that your post has been removed in error, credible sources must be provided.

9. No link shorteners.

The auto mod will contact you if a link shortener is detected, please delete your post if they are right.

10. Don't copy entire article in your post body

For copyright reasons, you are not allowed to copy an entire article into your post body. This is an instance wide rule, that is strictly enforced in this community.

founded 1 year ago

MODERATORS

JonsJava@lemmy.world

rjc@lemmy.world

Thekingoflorda@lemmy.world

Tenthrow@lemmy.world

gedaliyah@lemmy.world

little_cow@lemmy.world

jeffw@lemmy.world

enu@lemmy.world

218

Cops lure pedophiles with AI pics of teen girl. Ethical triumph or new disaster? (arstechnica.com)

submitted 2 months ago by some_guy@lemmy.sdf.org to c/news@lemmy.world

74 comments fedilink hide all child comments

New Mexico is seeking an injunction to permanently block Snap from practices allegedly harming kids. That includes a halt on advertising Snapchat as "more private" or "less permanent" due to the alleged "core design problem" and "inherent danger" of Snap's disappearing messages. The state's complaint noted that the FBI has said that "Snapchat is the preferred app by criminals because its design features provide a false sense of security to the victim that their photos will disappear and not be screenshotted."

top 50 comments

sorted by: hot top controversial new old

[–] Chozo@fedia.io 83 points 2 months ago (2 children)

"Heather" also tested out Snapchat's search tool, finding that "even though she used no sexually explicit language, the algorithm must have determined that she was looking for CSAM" when she searched for other teen users.

But literally in just the previous paragraph:

Posing as "Sexy14Heather," the investigator swapped messages with adult accounts, including users who "sent inappropriate messages and explicit photos.

Gee, I wonder how the algorithm could've possibly suggested these users. What a mystery.

I'm not defending Snapchat here - they're a scumbag company with a scumbag product and they should be held responsible for enabling the sharing of CSAM on their platform - but it doesn't just match you with random predators out of thin air. They went in with specific keywords in their username and a pattern of account engagement.

[–] some_guy@lemmy.sdf.org 27 points 2 months ago

That's nuance that I hadn't considered and I appreciate you pointing it out. I'm not on any of these sharing platforms so I have no idea what they're like and that made it easy to overlook this detail which is probably pretty relevant.

[–] shani66@ani.social 8 points 2 months ago (1 children)

They're also trying to claim screen shots are a major security issue, even though everyone knows they can just do that and the real benefit of deleted messages is that no corpo, cop, or cracker can snoop on them later.

They are completely incompetent and tech illiterate, is what I'm saying

[–] JustEnoughDucks@feddit.nl 3 points 2 months ago* (last edited 2 months ago) (1 children)

You know that on Snapchat the messages aren't deleted anymore right? They are stored on your phone and the Snapchat servers. That is how "memories" works, and there used to be screenshot workaround to grab the photo back after the fact, but I am not sure if they have hid them better by now.

Snapchat is not even close to a secure messaging program...

[–] shani66@ani.social 2 points 2 months ago (1 children)

Wow that's fucked. That's the entire reason snapchat is even still a thing.

[–] prettybunnys@sh.itjust.works 3 points 2 months ago

The messages never disappeared really.

There have been alternate apps and even jailbreak workarounds for Snapchat since the beginning to copy and keep the messages.

The benefit of Snapchat was the theater of it

[–] Gork@lemm.ee 78 points 2 months ago (2 children)

Despite Snapchat setting the fake minor's profile to private and the account not adding any followers, "Heather" was soon recommended widely to "dangerous accounts, including ones named 'child.rape' and 'pedo_lover10,' in addition to others that are even more explicit," the New Mexico DOJ said in a press release.

wtf

[–] Cryophilia@lemmy.world 17 points 2 months ago (1 children)

How the fuck do you get even more explicit than that

[–] INHALE_VEGETABLES@aussie.zone 2 points 2 months ago (1 children)

Yo I can give you a more explicit username if you want one.

[–] Spacehooks@reddthat.com 13 points 2 months ago

it's more how was it allowed.

load more comments (1 replies)

[–] Sterile_Technique@lemmy.world 28 points 2 months ago

Ethical triumph or new disaster?

Yes.

[–] ravhall@discuss.online 23 points 2 months ago (3 children)

Tough call. If you put out bait, you’re gonna get someone. But would that person have done the same thing if they had not seen your bait? Chicken and the egg. On one hand, it looks like entrapment.

[–] TheTetrapod@lemmy.world 16 points 2 months ago (1 children)

I mean, that part isn't really at issue here. It's fundamentally the same technique that's been used since the 90's, famously on To Catch a Predator. Seemingly, the "entrapment" angle has been settled.

[–] ravhall@discuss.online 8 points 2 months ago (7 children)

But now they can argue that they aren’t sexually attracted to children, just AI artwork, which is technically not an image of a child. And unless I missed it, they were not trying to meet the girl.

The problem is going to be that images that aren’t real of a crime aren’t a crime. Of the opposite was true, images of murder would be illegal. Can’t just cherry pick.

If I draw a stick figure and label it “naked girl,” does it become child porn? What if I’m a really good artist?

[–] Erasmus@lemmy.world 7 points 2 months ago (3 children)

I believe that cartoon images depicting sex of underage kids is still illegal. At least in the US.

Feel free to correct me if I am wrong but seems like I remember this from a news article a while back. Maybe it was just a specific state.

I am not going to Google that one though to find out though.

[–] VelvetStorm@lemmy.world 6 points 2 months ago (1 children)

https://cellebrite.com/en/ai-and-csam-a-look-at-real-cases/

Best I could find about this.

Imo as long as the ai was not trained on actual CSAM and the product is not depicting real people, then it shouldn't be illegal as it is not hurting anyone which is why we have laws against CSAM in the first place.

[–] fine_sandy_bottom@lemmy.federate.cc 3 points 2 months ago (1 children)

What if it normalises CSAM and some people don't discerne between real and AI?

[–] VelvetStorm@lemmy.world 9 points 2 months ago (2 children)

And what if video games, movies, and books normalize killing? There is no evidence to show that it does or that it will.

[–] shani66@ani.social 4 points 2 months ago* (last edited 2 months ago)

While I'm not going to have this specific topic in my search history, sexually violent porn very likely does nothing to encourage actual sexual violence. Most studies show that it has no effect on sexual violence at all, some show it decreases it, and only a few studies show it increases it (and those ones tend to have smarter people than me saying they are flawed).

While media can have psychological effects, normalizing extreme behavior doesn't seem to be one of them. That said, I wouldn't trust an ai bro or their ai to handle something like that. At best they don't know what goes into their training sets, at worst they would probably deliberately include csam.

load more comments (1 replies)

[–] ravhall@discuss.online 3 points 2 months ago (1 children)

Yeah don’t Google it hahaha

It what makes it a child? There’s some creepy anime girls who definitely fall into that questionable category. And if I label a stick figure with an age… does that make it illegal? What about an ai image with bubble text that says “I’m not real. I’m 18, I have a magical curse on me etc etc” now it’s fiction?

Since it isn’t actually real… what is the line, and how can that line be measured? Since this is just going to keep being a problem, this awkward conversation needs to happen in a logical, calm manner.

[–] VelvetStorm@lemmy.world 4 points 2 months ago (1 children)

https://cellebrite.com/en/ai-and-csam-a-look-at-real-cases/

Best I could find about this.

[–] ravhall@discuss.online 8 points 2 months ago (1 children)

I definitely don’t want to sound as if I’m promoting this material, but I agree. Fake things are fake and real things are real. Yeah, it makes a lot of people uncomfortable to think about it and I totally understand.

Fake images of murder seem to be perfectly fine! And that’s arguably the worst crime possible. We show that shit to our kids.

[–] VelvetStorm@lemmy.world 7 points 2 months ago (2 children)

Isn't it sad that we even have to say we don't promote it before we say anything else?

[–] ravhall@discuss.online 5 points 2 months ago

Yeah. But you know how it is here, you’re either against it or you’re one of them. Make a logical comparison between two nearly identical things and you’re whatabouting. I appreciate you recognizing the difference.

[–] shani66@ani.social 2 points 2 months ago

Probably not as bad on lemmy, at least.

[–] bdonvr@thelemmy.club 3 points 2 months ago (1 children)

I think the bar is whether it could be reasonably mistaken for a real child. Which makes quite a lot of disgusting content legal.

[–] VelvetStorm@lemmy.world 7 points 2 months ago (1 children)

I also find it to be repugnant, but if the images are not based on real people and the ai was not trained on real csam(good luck proving this either way), then it shouldn't be illegal. The laws were made to protect kids, and drawings of purly fictional characters are not hurting the kids.

[–] Cryophilia@lemmy.world 3 points 2 months ago (10 children)

Pretty much every law ever made in the history of humanity that was ostensibly to protect children is actually about control of the population.

load more comments (10 replies)

load more comments (6 replies)

[–] fine_sandy_bottom@lemmy.federate.cc 11 points 2 months ago (1 children)

That's not what this article is about though.

They're not saying "this user looked at our image so they're a pedo and must go to jail".

They're saying snapchat is full of pedos, and using the proliferation of this account as evidence supporting that claim.

[–] ravhall@discuss.online 3 points 2 months ago (1 children)

I agree completely with you that Snapchat is an unmonitored disaster that gives the user the impression that the person they are sharing their nudes with cannot save the content. A good portion of the videos on porn sites have that little Snapchat progress wheel on them and are clearly screencaps.

Aside: I think there is a big topic that conveniently gets overlooked because it’s so much easier to blame “social media” or the “predator,” and that is—where are the fucking parents?

[–] zbyte64@awful.systems 3 points 2 months ago

The parents are out buying an AR-15 for their son.

On a more serious note, parents always get judged, just not always fairly.

[–] would_be_appreciated@lemmy.ml 7 points 2 months ago (1 children)

IANAL, but my understanding is entrapment is when they convince you to do something you might not otherwise have done. So if the cops create an account of a minor and message an adult asking if they want to fuck, and the answer is like "uh no, absolutely not," and then the cops follow up by repeatedly sexting, and the adult blocks their account, but the cops relentlessly keep sexting from burner accounts, and plant people in the adult's work and social environments who keep talking about how normal it is to fuck minors who sext you out of the blue, and then the adult is finally like "oh whatever, fine" - that's entrapment.

Now, most people still are literally never going to take the minor up on the offer, no matter how relentless they are or how normalized it is in their environment. That's true about most crimes. The question is how many people wouldn't have committed that crime unless this very specific police-created situation came up, and that difference is what falls into entrapment.

I'd argue this isn't even close to entrapment, because all they did was set up an account much like all the others that exist, and waited for others to find them. It's no different from leaving a bike unlocked, then catching somebody who steals it. There are unlocked bikes everywhere, and people don't suddenly decide to steal the only bike of their life because they happened to find that unlocked bike.

Of course, they could also be spending this time and money getting to the root of societal issues and fixing the core problems instead of catching a small percentage of active pedophiles and letting the rest of them continue to cause irreparable harm.

[–] ravhall@discuss.online 4 points 2 months ago

The last paragraph is the big issue. Fix society. We can argue all day long about what line does “artwork” cross before it becomes illegal, but that’s not actually preventing anyone from getting abused.

And imo, it seems a little sick to say, “we made a bunch of kiddie porn that didn’t exist previously, and I’m going to distribute it to catch criminals—using tax dollars” … tf?

[–] Melody@lemmy.one 22 points 2 months ago

In general I think using AI imagery, and catfishing in general, is basically entrapment. In most civilized countries; that's illegal for police to do.

Now if they begin to actually trade in actually legitimate forbidden materials...sure; by all means arrest for that charge alone. That wouldn't be unjustified. But provoking someone who might then turn around and harm a real child, seems wrong.

[–] Cocodapuf@lemmy.world 15 points 2 months ago (1 children)

Maybe we should just make those AI tools public and free and leave it at that. If you monsters want kiddie porn, have this AI generated stuff. It's sort of like having needle exchanges or safe using centers for addicts. It doesn't solve the problem, but it makes it safer for everyone.

[–] zbyte64@awful.systems 8 points 2 months ago

If we're talking about it from a harm reduction or public health view point then it is the circulation of the content, not the personal generation that poses the greatest risk. Once the material gets circulated, it becomes harder to know it's origins and it might look like someone IRL and expose them to risks.

[–] Grandwolf319@sh.itjust.works 12 points 2 months ago (8 children)

I don’t understand how this would be fine but pedophiles generating them at home without distributing them would be illegal.

Cause the cops are creating CSAM and putting it out there in some shape or form, which you could argue would encourage pedophiles same way circulating ai images would as well.

Either generating under age sensitive material is always wrong or it’s not.

[–] some_guy@lemmy.sdf.org 2 points 2 months ago (1 children)

The article didn't say the cops generated AI CSAM. It said they created a profile pic, which was shown in the article.

[–] Grandwolf319@sh.itjust.works 4 points 2 months ago (1 children)

So if someone generates a minor’s image and it’s not nude, is that not CSAM?

I’m genuinely asking, I always thought it was about sexualizing children, not whether they are nude or not.

[–] some_guy@lemmy.sdf.org 2 points 2 months ago (1 children)

I don't think so. People keep throwing that acronym around but I suspect they didn't read the article and find out that it was one normal picture of a high school-aged girl.

load more comments (1 replies)

load more comments (7 replies)

[–] daydrinkingchickadee@lemmy.ml 10 points 2 months ago

In an alternate universe:

Cops help pedophiles with AI pics of teen girl. Ethical triumph or new disaster?

[–] DrFuggles 7 points 2 months ago

I feel like there's two stories here:

Snap is a pedo grooming ground
Police are generating fake CSAM
is, well, yeah . Like any major network you have creeps, Nazis and pedos and it's good to root them out. It's weird that themselves don't do that.
is ... difficult. I feel like the cops that prompt GenAIs to produce (sexualized) images of children risk severe trauma. It must be punishing to ask a a machine to produce fake images of morally reprehensible material. That's gotta take a toll on you. What's even more important is that those people could be spending their time taking down actual child abuse images.

I can't wrap my head around that.

[–] originalucifer@moist.catsweat.com 5 points 2 months ago

already been a movie along these lines

the artifice girl

[–] PrincessLeiasCat@sh.itjust.works 2 points 2 months ago

What does Chris Hansen have to say?

load more comments