Three-year-old Reem was in her bedroom beside her brother Tariq when a 2,000lb bomb exploded metres away, collapsing the building. Her mother in the room next door was trapped under the concrete of her ceiling and left powerless as her daughter screamed for her.
Reem and her brother, who was five, died before they could be dug out from the rubble. Their grandfather Khaled was left to bury their bodies, fastening the golden earring of Reem on his shirt collar in memory of her. By late November 2023, more than 15,000 Palestinians had reportedly been killed during Israel’s military operations in Gaza, according to figures cited by the United Nations.
AI is sold to militaries as being able to strike the enemy faster and at greater scale, but civilians from Gaza to Yemen to Iran are paying the price for innovation. The target of the bomb that killed Reem and Tariq was reportedly a Hamas fighter living near them in the densely populated Al Nuseirat refugee camp in central Gaza.

Data on Reem would have been collected and processed by the IDF for its artificial intelligence decision-support systems (AI-DSS), which are known by names like “the Gospel” and “Lavender”. These systems draw on invasive surveillance, social media data, images, phone use and locations to generate thousands of targets on the likely location of Hamas fighters through an algorithm.
Whereas it used to take human military analysts a year to identify 50 military targets, intelligence officers say AI can generate 100 per day. For instance, the Gospel was used to generate more than 12,000 targets in the first 27 days of the war in Gaza, according to the IDF.
The strike that killed Reem and Tariq appears to have been one of these thousands of targets. A human operator would then have approved the airstrike using a precise laser-guided weapon, accurate within five metres.

The Insights section is committed to high-quality longform journalism. Our editors work with academics from many different backgrounds who are tackling a wide range of societal and scientific challenges.
It is unclear if any Hamas members were killed in that attack, but the scale of civilians killed through the escalation of force through AI has seen such attacks labelled by the United Nations as war crimes and contributing to systematic destruction that amounts to genocide.
Just 13 months after the bombing, an Israeli tank-shell hit the same area and killed Khaled, metres from where his grandchildren had died.
In my new book, Algorithms of War, I investigate the potential and the dangers of AI in warfare. I find that this technology is changing how other human beings are seen, with civilians often bearing the cost of innovation with their lives. So what does a better way forward look like?
The Maven programme
Israel has not been the only country to use AI to increase the scale and speed of its military operations. The United States has also heavily invested, including in the Maven programme. It operates similarly to the Gospel and Lavender systems, bringing together a range of data and intelligence to label and indicate likely military targets and enemy fighters.
The US has been using Maven in strikes in its wars in the Middle East, including Yemen and Iran. The impact on civilians has been devastating. In Yemen in April 2025, the US launched strikes against the Ras Isa oil port on the Red Sea in response to rebel Houthi attacks on shipping in the region. According to Airwars, between 84 and 90 civilians were killed, including at least three children, and 171 people were injured. The US military subsequently acknowledged killing 80 civilians and injuring 171 others. The casualties included truck driver Saleh Mohamed al-Zoudi and his young son, both of whom were killed in the first strikes. Further strikes hit the port while rescuers were treating and evacuating casualties, reportedly killing emergency responders.
This was part of Operation Rough Rider (March-May 2025), where in 53 days over 339 strikes were carried out, killing at least 238 civilians. This was a big increase on US-UK Operation Poseidon Archer (January 2024-January 2025), which saw 931 strikes over 358 days and 21 civilians killed. According to investigative journalist Katrina Manson, it is likely that Maven made the difference with over ten times the number of civilians killed in less than 8% of the time.
This pattern of Maven enabling large-scale bombardment that leads to significant civilian casualties has continued in recent months. On February 28 2026, the opening day of the US war on Iran, two Tomahawk missiles struck the Shajarah Tayyebeh Elementary School in Minab, killing more than 150 people, including at least 123 children, according to a UN report. The school stood on land that had once belonged to a military compound.

According to Pentagon investigators, the site was still catalogued as an Islamic Revolutionary Guard Corps facility because of outdated data. This was fed into Maven alongside other target candidates and emerged as a recommended day-one target, even though the school had its own website and appeared on Google Maps. It appeared that some personnel had expected Maven to flag stale records or errors, though it had not been designed to perform that function.
Palantir, the tech company which makes the Maven system, has said that human error was probably to blame for what happened, pending the final report from the Pentagon into the atrocity. It’s likely, however, that the AI system’s speed was part of the problem.
It helped generate hundreds of strike coordinates in the first 24 hours of the war, enabling more than 1,000 targets to be hit. At that tempo, there would have been little room for anyone to ask whether a name on a list still matched what stood on the ground. AI providers and militaries indicate that they learn and improve their systems after such tragedies, but that’s small comfort to the families of civilians killed in the meantime.
AI-assisted targeting technology similar to Maven has also reportedly been deployed by US personnel working with Saudi Arabia in its military campaign against the Houthis. According to the World Health Organization, this has seen over 1,700 killed and 8,000 injured since August 2026, though it’s not known what proportion are civilians.
The limits of AI
If you have used AI, whether it be large-language models (LLMs) like Claude or ChatGPT or real-time traffic navigation with the likes of Google Maps, you will be familiar with their limits, including confident mistakes (hallucinations) and missing context. With military AI, the situation is no different.
Fundamentally, AI is limited to its dataset and training. It struggles with understanding a context or environment that is different from what it has been trained on. Early versions of Maven tested in the US in 2024 were reported to struggle in telling apart a tank from a tree, as much of its data was based on drone footage from the Middle East. A change in weather was also enough to halve its accuracy.
Integration of newer algorithms to pull together sensors and satellite data can overcome some of these challenges, yet the system still creates a dangerous illusion of battlefield certainty – giving operators false confidence in information that lacks any real human insight. Battlefields are sites of data conflict, because the enemy hides, moves and deceives, so the conditions a system was trained on rarely hold for long.
And as we’ve seen, the AI is only as capable as the information it’s given. As well as the risk of situations like the one at the school in Minab, there’s also a growing risk that data can be “poisoned” by an enemy: this is where they corrupt training data through a cyber-attack so that the information coming out of the AI is misleading.
An error in one algorithm or AI system can cascade into another. Recently US forces nearly boarded a Chinese ship in the Middle East on the basis of fake intelligence generated by an AI chatbot, which identified the boat as carrying nuclear weapon parts. This underscores another risk introduced by AI, namely escalating conflict by mistake.

Companies like Palantir have said they have added features to allow such inconsistencies and inaccuracies to be flagged. Yet the potential for AI to make the battlefield more transparent and accurate to avoid civilian harm remains elusive, perhaps illusionary.
There is clearly a human side to these errors and biases. People tend to over-trust fast, confident AI output. Apart from the risk of relying on wrong information, another concern is that it’s leading to less scrutiny in the decision-making process around which targets to attack.
Despite all these issues, the US military and others are increasing their investment in AI: by one estimate, military AI spending has risen about 25% to US$11.7 billion (£8.9 billion) in the past two years and is on course to reach nearly US$20 billion by 2030. With increased spending comes increased reliance: the UK has recently indicated it is looking to introduce AI “autonomous targeting by default, human by exception”, for instance.
There are several reasons why states might want to reconsider such courses of action. One is the public relations perspective. As US commander of coalition forces in Afghanistan, General Stanley McChrystal said in 2009: “We must avoid the trap of winning tactical victories, but suffering strategic defeats, by causing civilian casualties or excessive damage and thus alienating the people.”
The wars in Gaza, Iran and Yemen have shown that using AI to carry out mass bombardments does not lead to decisive victories. Instead the massive civilian harm that appears to have been caused has attracted widespread condemnation. When things go wrong, it is the militaries who are using these systems that get the blame, not necessarily those profiting from their sale.
There’s also the international legal dimension. Any new weapons or methods of warfare are strictly regulated by the laws of war, such as the Geneva Conventions. Autonomous weapons risk violating rules requiring forces to distinguish enemy soldiers from civilians, avoid excessive civilian harm, and avoid attacking wounded and surrendering soldiers.
These rules came out of the horrors of the two world wars and other conflicts to restrain humans’ worst excesses. Will nations be willing to cooperate with or trust allies who use these systems, both risking their reputation and potentially exposing them to liability for war crimes?
Drone warfare
The same kinds of concerns can be raised about drones. In 25 years they’ve evolved from being used by the US military in Middle East surveillance to being involved in the offensive strategies of most militaries, as well as over 65 rebel groups.
We’ve seen tens of thousands of drones used every day in Ukraine, as well as heavy use in recent months in Myanmar, Colombia and Sudan. It’s true that AI and drones have enabled Ukraine and Iran to fight far larger militaries without the same manpower, but the increased use of drones has also seen numerous civilians killed as a result. According to a recent UN communication: “Drone warfare has become a leading cause of civilian death in some armed conflicts.”

On July 6, something new happened: a small Russian drone exploded in the city of Zaporizhzhia in eastern Ukraine. According to the New York Times, it killed 19-year-old college student Tetiana Bubynets, 41-year-old Oleksii Svirin and 48-year-old Roman Karpii. This was a new type of drone – a “killer robot” that can navigate by itself and select its own targets, also known as a lethal autonomous weapons system.
This new technology has emerged because of challenges with prior generations of drones, which are guided by a human pilot connected by a satellite data-link or wireless radio frequency. Both sides in the Ukraine war have found they can make these crash by using electronic jamming to break the pilot’s control by blocking the connection.
Ukrainian and Russian forces have tried to overcome this by attaching kilometres-long fibre optic cable to drones, or using AI to lock on to a target for the final few hundred metres to neutralise electronic jamming (known as terminal guidance).
The Russian drone used on the July 6 strike was instead following a pre-programmed flight path to the target zone and was trained on images of propane tanks to strike. Once in the target zone its onboard vision software was left to navigate, hunt for these objects, and detonate entirely on its own. It was not trained to target civilians or humans, but exploded as it turned a corner.
Whether the system misidentified a civilian object or simply glitched and crashed, it’s a harbinger of what is becoming the new norm. Again, there are questions over these devices’ object recognition: they can’t necessarily understand the context, intent or risk to nearby civilians or other protected persons.
Their vision capabilities are at least improving, and companies and states are working to test and verify algorithms that make it still more reliable, especially in drone interceptors. The manufacturers also talk about other improvements, such as real-time abort systems for when drones go off course or have a system failure. Yet there’s no independent way to verify or test such claims, due to military secrecy and national security concerns around such technology.
Deploying such unpredictable autonomy into an inhabited street is again unlawful. Under the Geneva Conventions, it’s illegal to launch within the vicinity of a civilian population an “indiscriminate attack”, meaning one that can’t be directed at a specific military objective – seeking out generic objects used both by civilians and military personnel would not fall under that exception.
With a view to a more explicit law, states and civil society organisations have for a decade been calling for an international treaty to ban drones that cannot tell a civilian from a military target. Between November 16 and 20, states have been gathering at a UN conference in Geneva which aims to clear the way to begin formal negotiations on an international agreement to prohibit these machines.
This would be done through the Convention on Certain Conventional Weapons (CCW), which is the most direct route to regulating autonomous drones. The convention already restricts the likes of landmines, incendiary weapons and blinding lasers, and Article 8 makes it possible to extend the rules to new kinds of weapons. Campaigners and the International Committee of the Red Cross want to achieve this through a new protocol on autonomous weapons.
After three years of work, an expert group advising the conference reached consensus in early September on a report outlining which autonomous weapons should be banned and which regulated. Human Rights Watch counts 76 states now pushing to turn that framework into formal negotiations for a legally binding treaty.
The difficulty of the CCW process is that it needs all states to agree. The goal of the Geneva conference is to decide on future steps, including whether to launch negotiations on a legally binding instrument.
There have been previous situations in which a push to extend the CCW failed to reach a consensus, only for countries to devise a different treaty that at least banned certain practices between them. In 1997 in Ottawa, over 130 countries agreed to ban anti-personnel mines that at the time were killing or injuring over 2,000 people a month. Today, 162 countries have banned these mines and reduced the casualties from over 25,000 per year to around 6,000. In other words, in 30 years this treaty has potentially saved over half a million people from mines.
If the move to extend the CCW to autonomous drones fails, supporters may turn to the UN General Assembly. An October resolution at the assembly called for legally binding rules on autonomous weapons, passing with 163 votes in favour and five against (the United States, Israel, North Korea, Belarus and Russia). Of course, even if an agreement is reached over extending the CCW, the pervasive use of drones and AI in targeting decisions remains a threat to civilians around the world that will require wider efforts to make nations accountable.
A more dangerous tomorrow?
The other major question is how AI and robotics changes warfare in years to come. Threats of killer robots are not likely to see a Terminator uprising that wipes out humanity – autonomous humanoids on battlefields are still some way away.
More likely in the near term is the increasing use of millions of drones on land, sea, air and space. Such mass-scale deployment will see them face off in large numbers against enemy drones, as already is starting to occur in Ukraine, where AI-guided interceptor drones are being used to take out their enemy counterparts.

Air and sea drones are already delivering smaller drones to the battlefield so they can strike the enemy at greater distances. AI will probably enable the coordination of drone swarms, with a human commander releasing such weapons on a designated enemy position.
Some will argue that increasing the scale of such drone use will reduce their risks, but the danger is that we’re just exposing civilians to greater harm. It is also increasingly likely that terrorist organisations will carry out mass-casualty attacks using drones. There are already reports of groups using AI LLMs like Claude to create biological weapons, better missiles and improvised mines to upscale their violence. Militaries are spending billions on making AI more lethal, but where is the investment in AI to help protect civilians and friendly forces?
AI could be a way to minimise the horrors of war. AI is being used to identify mass graves, locate landmines and coordinate humanitarian aid.
Some leading tech companies providing AI to the military, such as Palantir and Anthropic, are starting to pay attention to responsible design with protection of civilians in mind. This is welcome but it should not just be a public relations exercise. It needs to be subject to verification and regulation to continue the basic principle that we protect civilians from warfare.
The urgent need for clear international rules for AI on the battlefield is obvious: we need to ensure that everyone is on the same page on how best to use this technology and how to minimise its risks. People like Reem, Saleh and Tatiana are the first victims of AI warfare, and will be joined by thousands of others if we do not restrain the use of this technology on the battlefield.
This article features references to books that have been included for editorial reasons, and may contain links to bookshop.org. If you click on one of the links and go on to buy something, The Conversation UK may earn a commission.