Against AI 2040's Plan A: "A cadre of elites decides which research directions are permissible, caps global compute and robotics, and creates state-administered scarcity rents. [...] The solution to centralization of power is not to simply switch the actor who gets the reins."
SUMMARYA reposted critique of “AI 2040” argues that the plan’s proposed limits on compute, robotics, and research would concentrate power in governments and enable permanent repression. It rejects doomsday-style AI governance and calls instead for continued AI research, innovation, and existing safety practices such as evaluations and audits.
Found this Tweet reposted by Yann LeCun. Found it pretty good as a rebuttal against AI 2040's doom and gloom and totalitarian control advocacy.
TL;DR (very editorialized by me, read the original)
Stories like AI 2040 — because that is what AI 2040 is: a story of a possible scenario — bake in philosophical assumptions, blur prediction with advocacy, and create an illusion of inevitability and emergency. They stack assumptions to justify radical conclusions: AGI might diffuse unrealistically quickly. Labs might capture all the profits. Robots might rapidly replace nearly all labor. And because the authors choose the starting assumptions, choose what it is they write in the limited space of possibilities they choose to present, the conclusions end up biased toward the world they already wanted. They're fables crafted to lead the reader to one's chosen position.
AI 2040's Plan A basically advocates for a centrally planned tech economy at world scale, where elites dictate research directions, compute limits, and technology allocation. While the AI 2040 authors worry about companies accumulating power, they give governments a free pass to seize extraordinary surveillance and control capabilities. But historically it is states that have systemically been the greater threat to liberty against their own people or other states. And the powers governments would be granted against defection by AI 2040's Plan A are exactly the same powers they need for permanent repression, forever. Ironically, we'd be lunging straight into one of the bad outcomes, "permanently stable dictatorships."
AI 2040 is the same old AI safety authoritarian vision again, just dressed up with more detail. Like always, they grab Bostrom's Superintelligence's most dramatic scenarios, they assume some kind of unified "godlike" agent with its own drives and incentives that naturally leads toward takeover unless perfectly aligned. Doomers fell for Pascal's Mugging by way of the precautionary principle and chase an impossible risk zero outcome that can't and won't exist. But they'll happily trade freedom and abundance for safety! And wait for an entire generation to die off or wallow in misery so long as misery feels safe.
Safety evaluations, audits, model specifications, dangerous capability testing, and similar mechanisms already exist throughout the industry and continue to improve. We do not need doomsday prophecies and increasingly catastrophic regulations. We just need to carry on with freedom what evolution and humanity have been carrying on with freedom for thousands of years: research and innovation. Overbearing authority under reactionary conditions is the wrong response, like it always is.