The Machine That Trusted Its Software Too Much
A Signal Casefile reconstruction of the Therac-25 accidents, where software race conditions, removed hardware safeguards, and cryptic error handling turned a medical machine into a landmark case in engineering ethics.
The Therac-25 looked like progress: a radiation therapy machine controlled by software instead of heavy physical interlocks. Hospitals trusted it because earlier machines had worked, and because the screen looked precise.
But inside the system, speed, race conditions, and removed hardware safeguards created a hidden failure path. Operators entered commands quickly, corrected settings, and sometimes triggered a state the machine did not safely understand.
Patients received radiation doses far beyond what treatment required. The first reports sounded impossible, because the machine displayed errors that looked minor, cryptic, and routine.
That is what made the case so dangerous: the evidence appeared as a message nobody knew how to fear. The company initially struggled to reproduce the failures, and hospitals continued using a device that should have been treated as evidence.
The old hardware locks had been replaced by confidence in code. But software is not a safety system just because it is written near one.
Each incident exposed the same deeper truth: automation can hide responsibility until the damage has already happened. Therac-25 became a landmark in engineering ethics because it turned invisible assumptions into documented harm.
The casefile is not anti-technology. It is anti-magic.
A machine that can injure at the speed of electricity needs proof, not trust. The most dangerous button is the one everyone believes has already been made safe.
The Signal Brief
One sourced idea worth your attention, in your inbox. No noise.