A Ten Percent Chance of Catastrophe Sounds Low. Here's Why It Isn't.

Forget the scary headline number for a second and look at the plain math of how many times people actually try.

Somewhere in the AI safety debate this year, someone finally put an actual number on it. Something like a one in ten chance that this technology contributes to a genuine catastrophe within the next decade. Ten percent. Read quickly, that almost sounds reassuring. Ninety percent chance it is fine, right?

Hold on though, why does a small percentage matter at all if the far more likely outcome is still nothing happens?

Because you are not rolling that dice once. You are rolling it every single time somebody with bad intentions sits down and tries something. And people are already trying. Earlier this year one of the major AI labs published a plain accounting of what it had actually caught happening in the real world over about eight months, not hypothetical scenarios, actual attempts.

  • People probing the models for help with biological weapons research.
  • Attempts to plan and support large scale cyberattacks.
  • Spyware operators trying to automate surveillance for hire.
  • Propaganda networks trying to scale up disinformation campaigns.

Every one of those was caught before real damage landed. That is the reassuring half. The unsettling half is that all of it was real, not theoretical, and it happened in under a year.

This is where the math turns ugly

Even if the odds of any single attempt actually succeeding are absurdly small, say one in ten thousand, once you have millions of people poking at the same tool, that tiny number stops behaving like a long shot and starts behaving like a near certainty. It is the same logic that keeps a casino profitable over enough hands even with a razor thin house edge, and the same logic that makes a spam campaign worth sending to a billion inboxes when only a sliver of a percent needs to click.

So the ten percent figure was never really the headline. Volume is. As these tools get cheaper, more capable and more widely available, the number of attempts keeps climbing, and the only thing standing between mostly fine and eventually something gets through is whether detection can keep pace with how many people are trying.