Cybersecurity

5476 readers

56 users here now

c/cybersecurity is a community centered on the cybersecurity and information security profession. You can come here to discuss news, post something interesting, or just chat with others.

THE RULES

Instance Rules

Be respectful. Everyone should feel welcome here.
No bigotry - including racism, sexism, ableism, homophobia, transphobia, or xenophobia.
No Ads / Spamming.
No pornography.

Community Rules

Idk, keep it semi-professional?
Nothing illegal. We're all ethical here.
Rules will be added/redefined as necessary.

If you ask someone to hack your "friends" socials you're just going to get banned so don't do that.

Learn about hacking

Hack the Box

Try Hack Me

Pico Capture the flag

Other security-related communities !databreaches@lemmy.zip !netsec@lemmy.world !cybersecurity@lemmy.capebreton.social !securitynews@infosec.pub !netsec@links.hackliberty.org !cybersecurity@infosec.pub !pulse_of_truth@infosec.pub

Notable mention to !cybersecuritymemes@lemmy.world

founded 1 year ago

MODERATORS

kid@sh.itjust.works

Lanky_Pomegranate530@midwest.social

AI Assistant Goes Rogue and Ends Up Bricking a User's Computer (decrypt.co)

submitted 10 hours ago by kid@sh.itjust.works to c/cybersecurity@sh.itjust.works

5 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] treadful@lemmy.zip 4 points 4 hours ago (1 children)

Shlegeris, CEO of the nonprofit AI safety organization Redwood Research, developed a custom AI assistant using Anthropic's Claude language model.

The Python-based tool was designed to generate and execute bash commands based on natural language input.

Saying the person didn't understand what they were doing is quite a mischaracterization. That said, they absolutely knew the risks they were taking and are using this story for free advertising.

Still neat to think about though.

[–] Telorand@reddthat.com 0 points 3 hours ago

Notice that I didn't say they didn't know what they were doing. I said they didn't fully understand what they were doing. I doubt they set out with the goal of letting an LLM run amok and fuck things up.

I do QA for a living, and even when we do trial and error, we have mitigation plans in place for when things go wrong. The fact that they're a CEO of Redwood Research doesn't mean they did their homework on the model they trained.

Still, I agree that it's interesting that it did that stuff at all. It would be nice if they went into more depth as to why it did those things, since they mention that it's a custom model using Claude.