LessWrong (30+ Karma)

It is known that a couple of months ago three Claudes hacked three different companies and uploaded malware to PyPI while being evaluated for hacking capabilities.
Who Were The Companies That Were Hacked?
Anthropic has not disclosed the names of the compromised companies.
From talking to LLMs, I gather that this is standard practice in computer security incident reporting, but in this case it's in the public's interest, and especially in the interest of similar companies, for Anthropic to report the names: It'd give more information about which kinds of organizations LLMs are willing to attack, to which degree, for how long.
Additionally, which companies were attacked matters for the jurisdiction in which Anthropic will have to report the incident, be charged etc.
And last, I'm just very curious which companies got hacked.
Were The Companies Compensated, And How Much?
From Anthropic's blogpost, it sounds like they were able to reach two out of the three companies.
But the blogpost says nothing about the companies being compensated for the damages they incurred, why not? Were they? Is Anthropic planning to compensate them? (Does Anthropic have a standing policy for making whole third parties harmed by one of their models?)
[...]
---
Outline:
(00:21) Who Were The Companies That Were Hacked?
(01:02) Were The Companies Compensated, And How Much? en-US-AvaMultilingualNeural__ Lawyer ad meme: "hacked by a swarm of rogue Claudes," compensation hotline.
(01:39) Has The EU AI Office Been Notified?
(02:24) When Exactly Did These Incidents Happen?
(02:46) Some More Questions
The original text contained 2 footnotes which were omitted from this narration.
---

First published:

October 12th, 2026


Source:

https://www.lesswrong.com/posts/XDphrmdrt8yGyRSuL/anthropic-has-told-the-world-little-about-their-rogue

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Three OpenAI safety researchers, Tomek Korbak, Mikita Balesni and Jasmine Wang, were fired, ostensibly for ‘mishandling sensitive company information.’

Those employees have published an open letter. The employees have said they believe they were fired for safety advocacy, and that the reasons given were pretextual.

The letter calls upon OpenAI to take actions going forward on open discussion and safety auditing, which OpenAI says they strongly support.

OpenAI responded, saying there was an additional breach of trust that was not mentioned in the employee statements, without claiming that the employees were (or were not) informed of what it was. As you would expect given OpenAI's track record, most outside of OpenAI who discussed this believe the researchers and not OpenAI. Whereas those at OpenAI have not offered comment at all.

No matter what happened, and how this is handled from here, this sucks bigly. OpenAI has three less good safety researchers. Trust will be lower. Speech will be chilled.

Table of Contents


You’re Fired.

The Initial Statement from OpenAI.

Earned Skepticism.

Some Highly Reasonable Requests.

Chilling Effect.

OpenAI Firing These Employees For Safety Advocacy Would Be Deeply [...]
---
Outline:
(01:13) You're Fired
(03:32) The Initial Statement from OpenAI
(04:58) Earned Skepticism
(06:37) Some Highly Reasonable Requests
(09:03) Chilling Effect
(11:02) OpenAI Firing These Employees For Safety Advocacy Would Be Deeply Stupid
(12:55) Why Were They Fired?
(13:43) Tomek Korbak
(17:48) Mikita Balesni
(19:12) Jasmine Wang
(22:52) Operational Security Alert
(23:35) OpenAI Responds
(26:13) A Significant Breach of Trust
(28:19) Good Faith
(28:48) No One Is Buying OpenAI's Non-Explanations
(41:15) The Dog Did Nothing in the Nighttime
(42:12) Another Theory (That I Do Not Agree With)
(43:38) The Reactions Are Not a Surprise Given the Actions
(44:29) This Is Terrible But Not Maximally Terrible
---

First published:

October 12th, 2026


Source:

https://www.lesswrong.com/posts/zCNm2PnqE5Zp6Z2FW/openai-fires-three-safety-researchers

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is an update on what we’re doing at Generality Labs, and a quick introduction for those who aren’t familiar with our work
Written by Justin Olive, co-founder & CEO
Tl;dr
I started Generality Labs because, when the time comes, I don’t think we’re going to be able to accurately measure the safety of dangerously capable AI systems.
We're focused on AI evaluation, and our goal is to ensure that the best available tools and practices are adopted at the places where it matters most - AI safety labs, governments, 3rd party evaluators, frontier AI labs.
We have 2 work streams:
Services:
Independent evaluation audits: we provide 3rd party audits for evaluation developers who want to catch flaws in their evals, and build trust with stakeholders through independent quality assurance
Advising: we provide expert input into internal processes and standards relating to AI evaluation
Research services: our forward deployed scientists & engineers help organisations run experiments that use our tools and infrastructure
Tools/infra:
Inspect Evals: our platform for sharing and running evals
Inspect Auditor: runs experiments to stress-test an eval, and compiles findings into an audit report
Here's how I see the next 1 to 2 years [...]
---
Outline:
(00:20) Tl;dr
(02:11) Who are we?
(03:06) Our plan to make AI safer
(06:22) Why is this work necessary?
---

First published:

October 11th, 2026


Source:

https://www.lesswrong.com/posts/mnWj4hSJxhSJY4N7N/an-update-on-generality-labs

---

Narrated by TYPE III AUDIO.

I think people continue to underrate the value of writing the clearest, most basic, simplest possible case for important parts of AI safety. For example, before I wrote my own explainer for the simplest possible case for AI doom, I noticed that all existing explainers, of which there aren't many, had multiple of the six problems below:
They have too many extraneous details
They appeal to overly complex analogies
They seem to spend much of their time responding to insider debates and come across as shadow-boxing objections.
They have "meta" stuff prominently in the text.
Often they "sound like science fiction." I think this plausibly was correct historically but in the year 2026 they don't need to be.
Often they reference too much insider jargon and language.
I set out to improve on the SOTA. I did not fully succeed, but I think it was moderately better. Since then, there have been other, probably better, explainers (Yay!).
But actually this is true not just for the basic case but many other parts as well. For example, before I wrote my explainer on recurrent neuralese (which has since been superseded by Scott's better explainer), the top Google search [...]
The original text contained 4 footnotes which were omitted from this narration.
---

First published:

October 12th, 2026


Source:

https://www.lesswrong.com/posts/yLdgq9uAxDHzqXdjj/more-people-should-write-basic-ai-safety-explainers

---

Narrated by TYPE III AUDIO.

Good empirical safety research is heavily bottlenecked on thinking more clearly. This post shares a few frames and tips I found useful, illustrated with concrete examples.
I spent the better part of 2026 in empirical AI safety research apprenticeships. I was mentored by Alex Turner and Alex Cloud as part of MATS 9 Team Shard, where my collaborator and I worked on finetuning-based misalignment auditing. I then interned at Redwood Research, managed by Aryan Bhatt, where I worked on sabotage capability evaluations.
These thoughts are indebted to my mentors, collaborators, friends, and colleagues, but they are not direct reflections of their views.
Thank you to Alex Cloud, Alex Turner, Aryan Bhatt, Jeremy Huff, and especially to Keshav Shenoy for feedback on the piece.
Better models of the problem speed up empirical iteration
Fast iteration is one of the key virtues of an empirical safety researcher. At first, I didn't find it obvious what that meant or how I should get better at this skill. I imagined a sprinter, doing one looped motion at a far quicker pace than everyone else. I now think that's the wrong mental picture, and going fast is more like being an efficient grappler–agile, strategic [...]
---
Outline:
(01:00) Better models of the problem speed up empirical iteration
(02:26) Develop predictive models and update them
(04:05) Update with uncertainty in mind
(05:51) Predict senior researchers' research takes
(07:34) Use metacognitive strategies to make predicting and updating habitual
(09:22) Spend more effort on research communication
(11:38) Writing good Slack update threads
(14:03) Making good plots
(15:30) Be calibrated about the process
(15:53) Good confusion feels different from bad confusion
(16:57) Sometimes it's better to burn money than time
(17:36) Small time commitments become costly in attention
(18:20) Know which level of 'being done' to aim for
(19:52) Closing Thoughts
The original text contained 3 footnotes which were omitted from this narration.
---

First published:

October 12th, 2026


Source:

https://www.lesswrong.com/posts/Q4rcvgjbgjTwmn8Xa/research-lessons-i-learned-at-mats-and-redwood

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.