Anthropic’s fight with the Pentagon highlights the risks of autonomous warfare — but obscures just how close it is.
The Convention on Certain Conventional Weapons, an international forum that focuses on lethal autonomous systems, is hosted twice a year at the United Nations in Geneva. When Branka Marijan attended in November 2017, she thought the five-day sessions — which dealt largely in hypotheticals, speculating on a world where warfare was fought with killer robots — would be business as usual. After all, this was technology some thought might never be developed, and likely never deployed. That year, she quickly realized, was different. That distant, imagined future was suddenly closer and realer than ever.
On the first day, some attendees watched a short film called Slaughterbots, put together by the Future of Life Institute. The video featured a fictional defense contractor pitching an AI-powered drone that could kill unassisted with precision strikes. “They used to say guns don’t kill people, people do,” its CEO tells the audience. “But people don’t. They get emotional, disobey orders, aim high. Let’s watch the weapons make the decisions.” The mood in the room, Marijan recalls, suddenly turned apprehensive. The most frightening part wasn’t the premise — it was that the Pentagon was already developing a version of this technology.
That meeting was the first one held after the start of Project Maven, a US Department of Defense initiative using AI to analyze drone surveillance footage. And by late 2017, Maven had a major tech company on board: Google. “The systems we were talking about were not futuristic,” said Marijan, who is a senior researcher at Project Ploughshares, a peace-focused independent research institute. “They were existing platforms that had degrees of autonomy in them, or the capability to select and engage targets based on sensor data and sensor input.”
The world had already seen drone warfare — deadly machines directed by humans. Now, it was looking at a future where humans may be removed from the loop entirely. “These were not these Terminator-like figures that we were concerned about, but really what was happening with the enablement of autonomy,” said Marijan.
The US military has backed AI development for decades, and in turn, AI has transformed warfare
Nearly a decade later, militaries haven’t yet developed fully autonomous lethal weapons. But these systems sit squarely in the center of a recent high-stakes battle between the US government and AI startup Anthropic. Anthropic is seeking to preserve two “red lines”: bans on domestic mass surveillance and on weapons that can identify, track, and kill targets with zero human involvement. Since the start of the year, it’s emerged as the only military AI contractor to place meaningful limits on what experts call one of the final frontiers of AI warfare.
Anthropic’s fight with the Pentagon highlights the risks of autonomous warfare — but obscures just how close it is.
The Convention on Certain Conventional Weapons, an international forum that focuses on lethal autonomous systems, is hosted twice a year at the United Nations in Geneva. When Branka Marijan attended in November 2017, she thought the five-day sessions — which dealt largely in hypotheticals, speculating on a world where warfare was fought with killer robots — would be business as usual. After all, this was technology some thought might never be developed, and likely never deployed. That year, she quickly realized, was different. That distant, imagined future was suddenly closer and realer than ever.
On the first day, some attendees watched a short film called Slaughterbots, put together by the Future of Life Institute. The video featured a fictional defense contractor pitching an AI-powered drone that could kill unassisted with precision strikes. “They used to say guns don’t kill people, people do,” its CEO tells the audience. “But people don’t. They get emotional, disobey orders, aim high. Let’s watch the weapons make the decisions.” The mood in the room, Marijan recalls, suddenly turned apprehensive. The most frightening part wasn’t the premise — it was that the Pentagon was already developing a version of this technology.
But amid shifting alliances, lawsuits, and melodrama, it’s easy to lose sight of the larger context — that AI is, and long has been, deeply embedded in the military. Seventy years ago, a summer meeting between scientists in New Hampshire made the Department of Defense sit up and take notice of AI’s potential for war. Since then, its influence has grown exponentially every decade. In recent years in particular, AI has enabled more and faster killings than ever before.
Even Anthropic seems to think its red lines won’t hold for long. After all, history has proven otherwise.
The US military has backed AI development for decades, and in turn, AI has transformed warfare. In the 2000s, the technology became capable of parsing unprecedented amounts of data collected worldwide, creating a surveillance revolution. And the late 2010s saw the development of advanced facial recognition and other sophisticated machine vision systems.
The fight between Anthropic and the Pentagon has drawn attention to the growing power of these systems. It began in January of 2026, when Defense Secretary Pete Hegseth demanded a renegotiation of the DOD’s existing AI contracts. The terms did away with any gray areas or previously agreed-upon terms, allowing the Pentagon to use the companies’ technology within the vague and expansive limits of “any lawful use.” Anthropic — the only AI company approved to deploy its tech on the Pentagon’s classified networks at that point — objected.
Even Anthropic seems to think its red lines won’t hold for long
A contractor like Anthropic setting limits on specific uses of its tech is unusual. “It’s not government-created technology in the way that the Manhattan Project was,” nor a conventional military supplier like Northrop Grumman, said Andrew Reddie, an associate research professor of public policy at the University of California, Berkeley. “This is one of the pain points that’s made clear when you’ve got this startup ecosystem engaging with the Pentagon directly.” Even within Silicon Valley, he said, there’s “a lot of disagreement” over when limits should be set.
It’s also unclear who will win. As a negotiation tactic, the DOD designated Anthropic a military supply chain risk in March, and President Donald Trump declared he was banning all government agencies from using its Claude system. The relationship has apparently warmed somewhat since then, with the release of Anthropic’s cybersecurity-focused model Mythos, but a court battle is still playing out. Anthropic declined to provide a comment for this story.
Either way, the debate has brought “fully autonomous weapons” into the public lexicon in a new way. But AI’s creeping influence in military operations hasn’t slowed down in decades.
“We’ve kind of crossed the rubicon while we pretend that we haven’t,” Reddie said.
At the center of the debates is DOD Directive 3000.09, one of the only policies governing the use of lethal autonomous weapons. Originally written in 2012, it defines such a system as one that, “once activated, can select and engage targets without further intervention by an operator.” And it decrees that both fully autonomous and semi-autonomous weapons be designed to allow humans to “exercise appropriate levels” of judgment over the use of force.
The directive set up the “first policy on the use of autonomy in warfare,” said Hamza Chaudhry, who leads AI and national security at the Future of Life Institute.
“Even without full autonomy, AI compresses kill chains to mere seconds.”
Depending on how you interpret the definition, however, certain missile defense programs may have crossed that line decades ago. Take the Phalanx CIWS, for instance. It’s an automated weapon system resembling a very large gun, built to defend naval vessels from incoming missile attacks. That type of system wouldn’t work if there were a human in the loop, since it has to respond in milliseconds.
The difference, some experts say, is that these systems operate solely in a defense-only, fixed environment. They’re engaging, this interpretation goes, but not deciding — just reacting to an incoming threat. “The ‘and’ is doing a lot of work inside of that statute — we have systems that can decide and systems that can engage but you can’t have a system that does both,” Reddie said.
“Even without full autonomy, AI compresses kill chains to mere seconds so that humans are not actually making the assessments that international humanitarian law requires to prevent civilian harm,” said Maddy Batt, legal fellow at Tech Justice Law. “When humans’ failure to do that results in civilian death, that is a war crime.”
The definition between offensive and defensive, too, is fuzzy. “One of my favorite exercises with my students is putting up a military technology and then asking, ‘Is this defensive or offensive?’” Reddie said, pointing to the example of a nuclear weapon in a silo, which some would argue is defense because it’s being used to deter, and some would argue is offense because it’s designed to hit foreign targets. “Just because its primary function … is defensive in nature doesn’t mean the technology itself is defensive in nature.”
Certain missile defense systems may have crossed the line of autonomous response decades ago
Sorin Adam Matei, a professor at Purdue University, was blunter: “You cannot fight a war only in defense.”
In 2023, the government’s Chief Digital and Artificial Intelligence Office (CDAO) — which is the central hub of all the DOD’s AI operations — issued an update to DOD Directive 3000.09. But it didn’t resolve the document’s core ambiguities. In 2024, the Biden administration published a memorandum on AI and national security, setting up rules for how AI can be used in certain national security scenarios — and for now, even under Trump, that policy is in force. But the Pentagon has seen significant upheaval. The CDAO is currently undergoing a significant restructuring that makes it more isolated from the rest of the DOD, and the office now reports to Emil Michael, who is both the DOD’s undersecretary of research and engineering and the department’s CTO.
International efforts, such as the Convention on Certain Conventional Weapons and the Campaign to Stop Killer Robots, haven’t been able to make much progress either. Marijan told The Verge that though the CCW has greatly helped smaller countries to understand the landscape around AI warfare, progress overall has “been very slow and we haven’t seen concrete agreement, particularly among the major countries and the more sophisticated militaries.” Though some countries have expressed interest in a ban on lethal autonomous weapons, there’s not even an official international definition of the term. The situation “often results in a lot of people talking past each other … and some countries find the lack of a binding instrument to be to their advantage,” said Sarah Shoker, a senior research scholar at the University of California, Berkeley and former lead of OpenAI’s geopolitics team.
Some countries have expressed interest in a ban on lethal autonomous weapons, but there’s not even an official international definition
“I think most people — policymakers, civil society members … who attend these meetings are likely tired,” Shoker added. “It’s been over a decade, and there is really no agreement.”
Whatever the status of autonomous weapons before 2017, Maven accelerated their timeline dramatically — and changed the terms of debate. The program marked the first time the idyllic, “don’t be evil” Silicon Valley of the 2010s was obviously involved in warfare, forcing Google employees and the public to react. “That was the first national conversation we had about the role of AI technologies in military operations,” said Matei.
Project Maven began with a memo from Robert Work, then deputy defense secretary. He established an “Algorithmic Warfare Cross-Functional Team” tasked with training AI algorithms that could be deployed to war zones, maybe even by year-end. Google was soon awarded the contract.
Internal resistance grew quickly. In April 2018, about 4,000 employees signed a petition demanding that Google remove itself from the “business of war.” The letter said a Google executive had assured Googlers their tech “will not ‘operate or fly drones’ and ‘will not be used to launch weapons.’” But “while this eliminates a narrow set of direct applications, the technology is being built for the military, and once it’s delivered it could easily be used to assist in these tasks,” the employees said. “We cannot outsource the moral responsibility of our technologies to third parties.” Google did not respond to a request for comment for this story.
Recently, MSS has reportedly been integral to both the capture of Venezuelan President Nicolás Maduro and the US strikes on Iran, which have reportedly killed thousands to date in the region.
“You cannot fight a war only in defense.”
Google employees argued their company should take a stand — and it did, choosing not to renew its contract amid the controversy in mid-2018. But Amazon and Microsoft quickly swooped in to pick up tens of millions of dollars in contracts for the same work. Palantir soon took over, and Project Maven became the Maven Smart System (MSS), which not only allows for object detection and tracking but also analyzing surveillance data on a large scale.
Claude is intertwined with MSS too, albeit in a more limited capacity. After Anthropic began allowing military use of its technology in 2024, its Claude chatbot was added to the user interface, helping analysts query the system about certain geographical areas and types of intelligence or targets. Even this relatively minor addition, some experts say, has made the system more efficient. As a result, Claude enabled targeting, potentially striking many more individuals.
The sheer volume of targets could make any meaningful human supervision difficult, said Shoker. “What we know about MSS is that it reduces the number of human beings in the targeting cycle — and that’s actually by design.”
While Anthropic might have been all right reducing human intervention, it’s pushed back against setting it to zero. As Google found with Project Maven, though, competitors are more than willing to fill the gap.
Other AI companies have jumped at the chance to work with the Pentagon. Even before its fight with Anthropic began, several major AI labs had loosened their operating guidelines or tweaked their mission statements to allow for military deals — including Google, OpenAI, and arguably Anthropic itself. OpenAI quickly signed onto the terms Anthropic had spurned. And in the months after snubbing Anthropic, the Department of Defense signed deals with eight companies to deploy their AI on classified networks: Google, Microsoft, Amazon Web Services, Nvidia, OpenAI, Reflection, Oracle, and SpaceX.
AI companies have jumped at the chance to work with the Pentagon
Shoker said that while working at OpenAI — one of the sector’s biggest players — she didn’t interact with many people studying the risks of decision-support systems or autonomous weapons systems. The research community in those areas is still “extremely siloed,” she said. “People in this space have been warning and researching about [those risks] for a while,” Shoker said, adding, “The problem … is that those people were not in the labs.” OpenAI did not respond to a request for comment.
Internally, employees have voiced dissent in ways that strongly echo the pushback against Project Maven almost a decade ago. Externally, OpenAI’s Pentagon deal sparked widespread controversy, leading CEO Sam Altman to announce he was attempting to renegotiate the deal. But Silicon Valley executives are aggressively pushing back against employee organizing and speaking out, including by using AI to identify leakers. And many tech workers already fear for their jobs in an era when AI is set to replace entry-level roles at their own firms.
Meanwhile, through its court fight with the Trump administration, Anthropic is still trying to regain its position as a military and intelligence contractor. Its technology has been deeply enmeshed in the DOD’s operations since 2024, and last summer it introduced Claude Gov, a product designed specifically for use by national security agencies with loosened guardrails around classified materials.
Anthropic is risking a high-stakes loss as it reportedly prepares to IPO this year. It’s reportedly in talks with investors to raise funding at a $900 billion valuation. There’s more investor pressure for the lab to turn a profit than ever before.
In recent months, Anthropic CEO Dario Amodei has held firm on mass surveillance for Americans, but he’s demonstrated no problem with — and in fact expressed his support for — such surveillance for everyone else.
Anthropic’s “very narrow” red lines “do not go far enough to protect human rights or to comply with international law,” said Tech Justice Law’s Batt. “Anthropic specifically talks about mass domestic surveillance of US persons as posing grave civil liberties concerns, but the same civil liberties concerns apply with equal force to non-US persons,” she added. “In fact, we should probably be especially concerned, given the brutal immigration crackdown, about advanced capabilities for mass surveillance of non-citizens.”
Anthropic CEO Dario Amodei has decried mass surveillance for Americans but demonstrated no problem with such surveillance for everyone else
That doesn’t even cover the ethics of mass surveillance abroad, an issue that is strikingly absent from most ethics conversations happening stateside.
And while Amodei believes Anthropic’s systems aren’t ready to support fully autonomous weapons, he’s not conceptually or morally opposed to them. In a blog post, he said that “fully autonomous weapons (those that take humans out of the loop entirely and automate selecting and engaging targets) may prove critical for our national defense.” Amodei even said he was happy to “work directly with the Department of War on R&D to improve the reliability of these systems” and speed up the timeline for the company’s help in deploying them.
That position, Tech Justice Law’s Batt said, is “fundamentally in tension” with international humanitarian law, which requires case-by-case assessments of any given attack. “The adoption of AI into these targeting processes is really based on a ‘speed wins’ ethos,” she said. “Regardless of when in the process humans are looped in, if the goal of implementing these systems is to transform something that took weeks or days into seconds, humans are not meaningfully making that assessment for themselves.”
When Hegseth announced his intent to redraft the Pentagon’s AI contracts, he explicitly stated that the DOD “must accept that the risks of not moving fast enough outweigh the risks of imperfect alignment” — and that a significant amount of testing and evaluation must be done away with.
“The adoption of AI into these targeting processes is really based on a ‘speed wins’ ethos.”
Past administrations have sought to remove “purely bureaucratic” layers of warfare while preserving safety, Chaudhry said, but the Hegseth memo “explicitly sets up a tradeoff and says that we favor speed.” This isn’t typically the best blanket approach when it comes to military operations, experts say, but it is often the deadliest. AI has already enabled that lethality — and it’s poised to go even further.
Since the Hegseth memo, says Marijan, more countries are paying attention to the risks of autonomous warfare. But against the American military and its partners in Silicon Valley, it’s not clear how much they can do. Right now, they’re simply trying to understand its implications. If humans are “just rubber-stamping decisions,” Marijan said, “what are the lines of accountability there — and what does the liability look like?”
During a Defense One Tech summit in July 2017, where Maven was centered as the future of the department, the chief of the Algorithmic Warfare Cross-Functional Team presented to an audience.
AI, he said, would not be selecting a target in combat “anytime soon.” But now, without intervention, that moment isn’t far off.
Hayden Field
AI
Analysis
Anthropic
Law
OpenAI
Policy
Report
That meeting was the first one held after the start of Project Maven, a US Department of Defense initiative using AI to analyze drone surveillance footage. And by late 2017, Maven had a major tech company on board: Google. “The systems we were talking about were not futuristic,” said Marijan, who is a senior researcher at Project Ploughshares, a peace-focused independent research institute. “They were existing platforms that had degrees of autonomy in them, or the capability to select and engage targets based on sensor data and sensor input.”
The world had already seen drone warfare — deadly machines directed by humans. Now, it was looking at a future where humans may be removed from the loop entirely. “These were not these Terminator-like figures that we were concerned about, but really what was happening with the enablement of autonomy,” said Marijan.
The US military has backed AI development for decades, and in turn, AI has transformed warfare
Nearly a decade later, militaries haven’t yet developed fully autonomous lethal weapons. But these systems sit squarely in the center of a recent high-stakes battle between the US government and AI startup Anthropic. Anthropic is seeking to preserve two “red lines”: bans on domestic mass surveillance and on weapons that can identify, track, and kill targets with zero human involvement. Since the start of the year, it’s emerged as the only military AI contractor to place meaningful limits on what experts call one of the final frontiers of AI warfare.
But amid shifting alliances, lawsuits, and melodrama, it’s easy to lose sight of the larger context — that AI is, and long has been, deeply embedded in the military. Seventy years ago, a summer meeting between scientists in New Hampshire made the Department of Defense sit up and take notice of AI’s potential for war. Since then, its influence has grown exponentially every decade. In recent years in particular, AI has enabled more and faster killings than ever before.
Even Anthropic seems to think its red lines won’t hold for long. After all, history has proven otherwise.
The US military has backed AI development for decades, and in turn, AI has transformed warfare. In the 2000s, the technology became capable of parsing unprecedented amounts of data collected worldwide, creating a surveillance revolution. And the late 2010s saw the development of advanced facial recognition and other sophisticated machine vision systems.
The fight between Anthropic and the Pentagon has drawn attention to the growing power of these systems. It began in January of 2026, when Defense Secretary Pete Hegseth demanded a renegotiation of the DOD’s existing AI contracts. The terms did away with any gray areas or previously agreed-upon terms, allowing the Pentagon to use the companies’ technology within the vague and expansive limits of “any lawful use.” Anthropic — the only AI company approved to deploy its tech on the Pentagon’s classified networks at that point — objected.
Even Anthropic seems to think its red lines won’t hold for long
A contractor like Anthropic setting limits on specific uses of its tech is unusual. “It’s not government-created technology in the way that the Manhattan Project was,” nor a conventional military supplier like Northrop Grumman, said Andrew Reddie, an associate research professor of public policy at the University of California, Berkeley. “This is one of the pain points that’s made clear when you’ve got this startup ecosystem engaging with the Pentagon directly.” Even within Silicon Valley, he said, there’s “a lot of disagreement” over when limits should be set.
It’s also unclear who will win. As a negotiation tactic, the DOD designated Anthropic a military supply chain risk in March, and President Donald Trump declared he was banning all government agencies from using its Claude system. The relationship has apparently warmed somewhat since then, with the release of Anthropic’s cybersecurity-focused model Mythos, but a court battle is still playing out. Anthropic declined to provide a comment for this story.
Either way, the debate has brought “fully autonomous weapons” into the public lexicon in a new way. But AI’s creeping influence in military operations hasn’t slowed down in decades.
“We’ve kind of crossed the rubicon while we pretend that we haven’t,” Reddie said.
At the center of the debates is DOD Directive 3000.09, one of the only policies governing the use of lethal autonomous weapons. Originally written in 2012, it defines such a system as one that, “once activated, can select and engage targets without further intervention by an operator.” And it decrees that both fully autonomous and semi-autonomous weapons be designed to allow humans to “exercise appropriate levels” of judgment over the use of force.
The directive set up the “first policy on the use of autonomy in warfare,” said Hamza Chaudhry, who leads AI and national security at the Future of Life Institute.
“Even without full autonomy, AI compresses kill chains to mere seconds.”
Depending on how you interpret the definition, however, certain missile defense programs may have crossed that line decades ago. Take the Phalanx CIWS, for instance. It’s an automated weapon system resembling a very large gun, built to defend naval vessels from incoming missile attacks. That type of system wouldn’t work if there were a human in the loop, since it has to respond in milliseconds.
The difference, some experts say, is that these systems operate solely in a defense-only, fixed environment. They’re engaging, this interpretation goes, but not deciding — just reacting to an incoming threat. “The ‘and’ is doing a lot of work inside of that statute — we have systems that can decide and systems that can engage but you can’t have a system that does both,” Reddie said.
“Even without full autonomy, AI compresses kill chains to mere seconds so that humans are not actually making the assessments that international humanitarian law requires to prevent civilian harm,” said Maddy Batt, legal fellow at Tech Justice Law. “When humans’ failure to do that results in civilian death, that is a war crime.”
The definition between offensive and defensive, too, is fuzzy. “One of my favorite exercises with my students is putting up a military technology and then asking, ‘Is this defensive or offensive?’” Reddie said, pointing to the example of a nuclear weapon in a silo, which some would argue is defense because it’s being used to deter, and some would argue is offense because it’s designed to hit foreign targets. “Just because its primary function … is defensive in nature doesn’t mean the technology itself is defensive in nature.”
Certain missile defense systems may have crossed the line of autonomous response decades ago
Sorin Adam Matei, a professor at Purdue University, was blunter: “You cannot fight a war only in defense.”
In 2023, the government’s Chief Digital and Artificial Intelligence Office (CDAO) — which is the central hub of all the DOD’s AI operations — issued an update to DOD Directive 3000.09. But it didn’t resolve the document’s core ambiguities. In 2024, the Biden administration published a memorandum on AI and national security, setting up rules for how AI can be used in certain national security scenarios — and for now, even under Trump, that policy is in force. But the Pentagon has seen significant upheaval. The CDAO is currently undergoing a significant restructuring that makes it more isolated from the rest of the DOD, and the office now reports to Emil Michael, who is both the DOD’s undersecretary of research and engineering and the department’s CTO.
International efforts, such as the Convention on Certain Conventional Weapons and the Campaign to Stop Killer Robots, haven’t been able to make much progress either. Marijan told The Verge that though the CCW has greatly helped smaller countries to understand the landscape around AI warfare, progress overall has “been very slow and we haven’t seen concrete agreement, particularly among the major countries and the more sophisticated militaries.” Though some countries have expressed interest in a ban on lethal autonomous weapons, there’s not even an official international definition of the term. The situation “often results in a lot of people talking past each other … and some countries find the lack of a binding instrument to be to their advantage,” said Sarah Shoker, a senior research scholar at the University of California, Berkeley and former lead of OpenAI’s geopolitics team.
Some countries have expressed interest in a ban on lethal autonomous weapons, but there’s not even an official international definition
“I think most people — policymakers, civil society members … who attend these meetings are likely tired,” Shoker added. “It’s been over a decade, and there is really no agreement.”
Whatever the status of autonomous weapons before 2017, Maven accelerated their timeline dramatically — and changed the terms of debate. The program marked the first time the idyllic, “don’t be evil” Silicon Valley of the 2010s was obviously involved in warfare, forcing Google employees and the public to react. “That was the first national conversation we had about the role of AI technologies in military operations,” said Matei.
Project Maven began with a memo from Robert Work, then deputy defense secretary. He established an “Algorithmic Warfare Cross-Functional Team” tasked with training AI algorithms that could be deployed to war zones, maybe even by year-end. Google was soon awarded the contract.
Internal resistance grew quickly. In April 2018, about 4,000 employees signed a petition demanding that Google remove itself from the “business of war.” The letter said a Google executive had assured Googlers their tech “will not ‘operate or fly drones’ and ‘will not be used to launch weapons.’” But “while this eliminates a narrow set of direct applications, the technology is being built for the military, and once it’s delivered it could easily be used to assist in these tasks,” the employees said. “We cannot outsource the moral responsibility of our technologies to third parties.” Google did not respond to a request for comment for this story.
Recently, MSS has reportedly been integral to both the capture of Venezuelan President Nicolás Maduro and the US strikes on Iran, which have reportedly killed thousands to date in the region.
“You cannot fight a war only in defense.”
Google employees argued their company should take a stand — and it did, choosing not to renew its contract amid the controversy in mid-2018. But Amazon and Microsoft quickly swooped in to pick up tens of millions of dollars in contracts for the same work. Palantir soon took over, and Project Maven became the Maven Smart System (MSS), which not only allows for object detection and tracking but also analyzing surveillance data on a large scale.
Claude is intertwined with MSS too, albeit in a more limited capacity. After Anthropic began allowing military use of its technology in 2024, its Claude chatbot was added to the user interface, helping analysts query the system about certain geographical areas and types of intelligence or targets. Even this relatively minor addition, some experts say, has made the system more efficient. As a result, Claude enabled targeting, potentially striking many more individuals.
The sheer volume of targets could make any meaningful human supervision difficult, said Shoker. “What we know about MSS is that it reduces the number of human beings in the targeting cycle — and that’s actually by design.”
While Anthropic might have been all right reducing human intervention, it’s pushed back against setting it to zero. As Google found with Project Maven, though, competitors are more than willing to fill the gap.
Other AI companies have jumped at the chance to work with the Pentagon. Even before its fight with Anthropic began, several major AI labs had loosened their operating guidelines or tweaked their mission statements to allow for military deals — including Google, OpenAI, and arguably Anthropic itself. OpenAI quickly signed onto the terms Anthropic had spurned. And in the months after snubbing Anthropic, the Department of Defense signed deals with eight companies to deploy their AI on classified networks: Google, Microsoft, Amazon Web Services, Nvidia, OpenAI, Reflection, Oracle, and SpaceX.
AI companies have jumped at the chance to work with the Pentagon
Shoker said that while working at OpenAI — one of the sector’s biggest players — she didn’t interact with many people studying the risks of decision-support systems or autonomous weapons systems. The research community in those areas is still “extremely siloed,” she said. “People in this space have been warning and researching about [those risks] for a while,” Shoker said, adding, “The problem … is that those people were not in the labs.” OpenAI did not respond to a request for comment.
Internally, employees have voiced dissent in ways that strongly echo the pushback against Project Maven almost a decade ago. Externally, OpenAI’s Pentagon deal sparked widespread controversy, leading CEO Sam Altman to announce he was attempting to renegotiate the deal. But Silicon Valley executives are aggressively pushing back against employee organizing and speaking out, including by using AI to identify leakers. And many tech workers already fear for their jobs in an era when AI is set to replace entry-level roles at their own firms.
Meanwhile, through its court fight with the Trump administration, Anthropic is still trying to regain its position as a military and intelligence contractor. Its technology has been deeply enmeshed in the DOD’s operations since 2024, and last summer it introduced Claude Gov, a product designed specifically for use by national security agencies with loosened guardrails around classified materials.
Anthropic is risking a high-stakes loss as it reportedly prepares to IPO this year. It’s reportedly in talks with investors to raise funding at a $900 billion valuation. There’s more investor pressure for the lab to turn a profit than ever before.
In recent months, Anthropic CEO Dario Amodei has held firm on mass surveillance for Americans, but he’s demonstrated no problem with — and in fact expressed his support for — such surveillance for everyone else.
Anthropic’s “very narrow” red lines “do not go far enough to protect human rights or to comply with international law,” said Tech Justice Law’s Batt. “Anthropic specifically talks about mass domestic surveillance of US persons as posing grave civil liberties concerns, but the same civil liberties concerns apply with equal force to non-US persons,” she added. “In fact, we should probably be especially concerned, given the brutal immigration crackdown, about advanced capabilities for mass surveillance of non-citizens.”
Anthropic CEO Dario Amodei has decried mass surveillance for Americans but demonstrated no problem with such surveillance for everyone else
That doesn’t even cover the ethics of mass surveillance abroad, an issue that is strikingly absent from most ethics conversations happening stateside.
And while Amodei believes Anthropic’s systems aren’t ready to support fully autonomous weapons, he’s not conceptually or morally opposed to them. In a blog post, he said that “fully autonomous weapons (those that take humans out of the loop entirely and automate selecting and engaging targets) may prove critical for our national defense.” Amodei even said he was happy to “work directly with the Department of War on R&D to improve the reliability of these systems” and speed up the timeline for the company’s help in deploying them.
That position, Tech Justice Law’s Batt said, is “fundamentally in tension” with international humanitarian law, which requires case-by-case assessments of any given attack. “The adoption of AI into these targeting processes is really based on a ‘speed wins’ ethos,” she said. “Regardless of when in the process humans are looped in, if the goal of implementing these systems is to transform something that took weeks or days into seconds, humans are not meaningfully making that assessment for themselves.”
When Hegseth announced his intent to redraft the Pentagon’s AI contracts, he explicitly stated that the DOD “must accept that the risks of not moving fast enough outweigh the risks of imperfect alignment” — and that a significant amount of testing and evaluation must be done away with.
“The adoption of AI into these targeting processes is really based on a ‘speed wins’ ethos.”
Past administrations have sought to remove “purely bureaucratic” layers of warfare while preserving safety, Chaudhry said, but the Hegseth memo “explicitly sets up a tradeoff and says that we favor speed.” This isn’t typically the best blanket approach when it comes to military operations, experts say, but it is often the deadliest. AI has already enabled that lethality — and it’s poised to go even further.
Since the Hegseth memo, says Marijan, more countries are paying attention to the risks of autonomous warfare. But against the American military and its partners in Silicon Valley, it’s not clear how much they can do. Right now, they’re simply trying to understand its implications. If humans are “just rubber-stamping decisions,” Marijan said, “what are the lines of accountability there — and what does the liability look like?”
During a Defense One Tech summit in July 2017, where Maven was centered as the future of the department, the chief of the Algorithmic Warfare Cross-Functional Team presented to an audience.
AI, he said, would not be selecting a target in combat “anytime soon.” But now, without intervention, that moment isn’t far off.