The Gavin Baker vs. Sholto Douglas exchange on the economic concentration of power of AI is one of the best AI debates. Here's a gist with some context and some thoughts:

One of the central themes of the debate offense-defense balance and how it governs whether AI is (or should be) concentrated in the hands of few or distributed to many.

The core idea behind offense-defense balance, outlined in Robert Jervis's 1978 essay "Cooperation Under the Security Dilemma" on world politics, is: for a given technology, is it cheaper to attack or to defend?

Baker initially argues that he agrees with Zuck's view in his essay that AI should be distributed and not concentrated into a few corporations' hands. Sholto rebuts that it squarely depends on the offense-defense balance of the domain (cyber, bio, etc). And even if you do distribute, the bottleneck is compute.

He cites two domains: - Cyber he says is defense-dominant, eventually. If it's opened up today, the cost to attack is minimal and the you cannot defend quickly enough. We'll need a ~2yrs of hardening the critical systems like financial institutions and infra providers before this may no longer be a risk.

• Bio he says is offense-dominant well into the 2030s. Synthesizing deadly self-replicating pathogens could cost O($10K) and vaccines cost billions and years. You just can't patch the human body with smarter models.

The distribute everything vs gate everything is a false dichotomy. You want to distribute in defense dominant domains and gate in offense dominant. And its all extra muddied because capabilities are dependent on compute which is extremely unequally distributed. And compute eventually dictates proliferation of capabilities. ______________

Both gloss over some offense dominant domains that are already distributed massively:

• Propaganda and persuasion, for example, were once regarded offense dominant yet already ubiquitous today. That said, technology changes, and watermarking outputs and AI detection tools may have shifted it to being defense dominant.

• Mass surveillance is also offense dominant (usually the attacker is the state) and most LLMs you can use for <$1/M toks can do this today. Today, even non-state actors (private cos) can surveil you in pretty dangerous detail by running agent swarms on vast amounts of public data.

Both Gavin and Sholto are right, and we see it today. - Sholto's right: the very pinnacle of frontier capabilities is locked away in those who have compute and will continue to be. The shape of compute of the >100 GW compute by 2030 is almost known at this point. Between 6-7 US entities (2 big labs, 3 hyperscalers clouds, Meta/xAI) will own or control 65-75%. With RSI around the corner, as long as it holds true that more compute leads to being able to train and serve more powerful models at scale, those companies will have proprietary frontier capabilities and only they / the market will dictate when they need to release them. Notably, this isn't monopolistic concentration of power since multiple parties will have that capability.

• Gavin's right too: frontier capabilities in certain domains are making it to the public and will continue to do so. Models are getting cheaper. Whether we like it or not, people can do dangerous things with models today. LLMs already cyber exploits that are less talked about (take even the LLM-assisted hack on the nationwide Indian exam system, CBSE). As long as prices for models continue to fall at a more rapid rate (10x YoY) than growth in compute (2-3x YoY), this dynamic will continue. Frontier intelligence will continue to be more accessible.