Anthropic’s Misguided Moral Stance

Anthropic pulled out of its contract with the US military because it could violate two of Anthropic’s moral principles:

1. No fully autonomous weapons

2. No large-scale domestic surveillance

While much of the Silicon Valley crowd lauded Anthropic’s stance, it is actually confused due to a lack of thought on the situation and what is actually being forbidden. As we will see, Anthropic should have had a more thoughtful, nuanced position, rather than blanket pronouncements that overstepped their intentions. Then Anthropic would have found a way to serve America to save lives.

Autonomous AI agents

The major problem with Anthropic’s autonomous weapons position is that it completely ignores the critical role AI plays in defending our troops’ lives. Anthropic assumes that AI is used to target people milling around on the ground, but this is only a small part of its role. Its bigger role is in identifying and reacting to incoming attacks on US soldiers.

Until recently, a pilot flying in a warzone needed to identify threats with their eyes or the plane’s radar, interpret that threat, and make a response decision. This wasted precious time and led to many pilots being killed, or shot down and captured, due to not being able to respond in time or appropriately. The pilot needed to determine if the blip on their radar screen was a missile, plane, or something else. They also needed to determine whether it was heading at the plane or something else. Even after determining what it was, the pilot had to choose the correct response. Plus, if the threat was identified by another plane’s radar, that plane’s pilot had to verbally convey the threat to the targeted pilot, wasting more time. The first pilot may not even know about the location of the second pilot, and thus not know that the second pilot is in danger. Pilots may only have seconds to make the right decision in response to an incoming missile, and many died due to not being able to perform all of the steps in time.

AI changed all this. Now pilots fly within a networked hive that can interpret information from a variety of sources and make decisions about the proper response within seconds. Imagine that four F-35 fighter/bombers are on a mission through enemy territory. They are accompanied by drones with different jobs. Some identify threats, while others carry defensive missiles, while others stay with the piloted planes to fly into a threat that other systems missed.

The group works as a single AI hive brain. If one of the drones is shot down, its duties are reassigned to other drones by the hive. Imagine that one of the drones detects an incoming threat. The entire hive can focus its radars on the threat to determine what it is and where it is going. The system determines the target and best response. One drone might detect the threat, while a crewed plane is determined to be the target, and still another is in the best position to respond and is instructed to launch its missile, thus saving the pilot’s life. Importantly, the chain does not have time for human interpretation and decision-making, it must react more quickly than humans can to save lives.

This is not science fiction, it is the current reality of warfare, where time periods are compressed due to modern weapons. Would Anthropic prefer that a pilot died instead of being saved by AI? Hopefully not, which is why Anthropic’s position forbids too much.

Moreover, Anthropic stakes its position on the grounds that AI can make mistakes. Of course, it can, but so can humans—the alternative. The question is not whether AI can make mistakes, but rather whether it is more or less likely to do so than a human. Anthropic assumes human infallibility, which is wrong.

Take the example of self-driving cars. Many people say that they would not want self-driving cars because the AI can make mistakes. But around 50,000 Americans a year die in road accidents, usually due to driver error. If self-driving cars are safer than human-driven cars, then people should want to live in a world of self-driven cars for their own safety. In fact, studies suggest that self-driving cars are in fact safer than human-driven cars, as AI does not get drunk, tired, text, etc.

Similarly, why does Anthropic assume that human targeting decisions are infallible? Take the example of the US mistakenly hitting an Iranian school, apparently because it was part of a military base and converted to a school only recently. Imagine, hypothetically, that a human decides to target it based on the available data, but an AI system notices movement around the building that suggests children playing, and warns against the strike. But the human making the decision trusts the targeting information and goes ahead with the strike. Here, human intervention caused harm that AI could have avoided. Why does Anthropic not consider this possibility?

Instead of just making a blanket pronouncement that sounded noble, Anthropic should have worked with the US military on these issues and helped guide the development of AI systems, but instead, it just walked away.

Domestic Surveillance

Anthropic’s blanket pronouncement against AI use for domestic surveillance suffers from the same lack of foresight. AI is just a tool, and like any tool, and whether it is good or bad depends on how it is used.

For instance, I happen to know an FBI agent who once told me that he was watching a bank with binoculars because of information that a robber would show up. He is using binoculars as a tool of surveillance, but who would suggest that we should thereby forbid FBI agents from using binoculars? If he were instead watching a woman undress in front of her fifth-floor window, we would say that he is using the tool wrongly, not that the tool itself is at fault.

This shows that the real question is: How is AI being used? The military said that it would only be used in legal ways, but Anthropic did not worry about how it would be used. In reality, AI allows agencies to find things that humans would have missed. Obama noted that intelligence agencies had information suggesting the 9/11 attack, but did not “connect the dots” between the information.

This is precisely what AI can do. AI can connect the dots between disparate bits of information from different agencies to stop terrorist attacks. Would Anthropic prefer that over 3,000 Americans die rather than AI being used to identify a terrorist attack? Presumably not, but its blanket pronouncement forbids everything. Anthropic needs to think about what it is really against in order to help craft the use of AI for good. Unfortunately, it had a chance to do that and blew it.