Context: Vasiliy Arkhipov was a staff officer on a Russian submarine during the Cuban Missile Crisis. After several days of no communication from Moscow, the submarine's three senior officers voted to launch a nuclear torpedo at American ships, assuming that war had already begun and Moscow had fallen. Vasiliy voted against it and since they could not form a unanimous vote the torpedo was not launched. It's widely speculated that if the torpedo were to be launched, thermonuclear war was extremely likely to follow.
Which is why humans should always be in the loop on decisions that can end the world. Software has already been given the power to intentionally kill people, and has already been making mistakes.
- me, a software developer