Kenneth Glad
Technology now quietly manages countless choices across the modern world. But as automated tools take over increasingly critical tasks, society faces a stark question about where software power should end.
For more than ten years, governments have gathered in Geneva to draft rules governing autonomous weapons. In September, delegates from 128 nations met to establish a binding treaty on automated arms, according to former Human Rights Watch director Kenneth Roth writing in The Guardian.
Yet representatives sent by Donald Trump and Vladimir Putin worked to weaken key safeguards. Crucially, they diluted a provision requiring human operators to review military targets created by software before launching strikes.
While global debates center on futuristic killer robots, software automation is already selecting who lives and dies in current conflicts.
In Gaza, military forces deployed an algorithm named Lavender to track phone usage patterns and identify suspected militants, Roth says.. Another program called Where’s Daddy then tracked those individuals to their family homes, leaving relatives as collateral damage.
A rubber stamp
In previous military operations, target selections underwent thorough review before commanders authorized a strike. But following the October 7 Hamas attack, an overwhelming demand for targets swept those traditional safeguards aside. Computer systems picked candidates and routed them to intelligence officers, who then passed them directly to field units launching the attacks.
This high-speed workflow reduced human oversight to a quick formality. Intelligence officers routinely approved computer recommendations without second-guessing them.
One officer recalled handling dozens of files every day, spending a mere twenty seconds on each selection. “I had zero added-value as a human, apart from being a stamp of approval,” the officer explained, noting that the automated setup saved massive amounts of time.
Some personnel defended the practice, arguing that software prevents human grief and anger from clouding decisions during war. As one soldier put it, “I have much more trust in a statistical mechanism than a soldier who lost a friend two days ago.”
Fading human control
Yet computer programs inevitably reflect the assumptions and rules set by their human creators. Programmers decide how much evidence is needed to classify someone as a target and what margin of error is tolerated. Even military estimates conceded that roughly 10 percent of targeted individuals were not combatants at all, according to Roth’s column.
Commanders also showed a willingness to accept heavy collateral damage, permitting up to 20 civilian deaths for a single low-level suspect. Legal experts consider such ratios unlawful under international humanitarian rules of proportionality. Israeli President Isaac Herzog reflected this mindset when he claimed that “an entire nation out there” was responsible.
This climate encourages what the International Committee of the Red Cross calls “automation bias”, where operators trust computer outputs without sufficient questioning. The authoritative precision of software lulls personnel into complacency, evoking what Roth described as the “banality of evil” as ordinary soldiers blindly trust “the system”.
Without intense questioning and real commander accountability, automated systems risk turning lethal military choices into a cold, mechanical routine.
Source: The Guardian
