❧ Francis Fukuyama on his skepticism of accelerationist projections:

At a 10 percent annual growth rate, GDP would double in a little over seven years. This would require a doubling of all of the material inputs to growth: energy, raw materials, rare earths, land, and many other things. Where is all this stuff going to come from? Intelligence by itself may aid in the production of such inputs, but it will simply not dig the mines and build the factories on its own. And then there are political constraints: how will superintelligence open the Strait of Hormuz, which is currently a major constraint on global growth rates?

Accelerationists tend to be very intelligent people, but that intelligence tends to be narrowly mathematical. We know how to do many things like produce electricity or clean water in the cities of poor countries; the problem is a failure of implementation.

I saw much of this dynamic in crypto policy. So many shape rotators who saw every political question as an engineering problem. Either no concept of, or too literal a view of, group interest and political incentives. SEP 23

❧ Cal Newport, whom I’ve long admired, recently wrote for the New York Times explaining why we shouldn’t be panicked by AI doom stories. I largely agree with his op-ed and yet I find it a bit strange.

His argument is basically this: The AI elite are Rationalists or heavily influenced by them. Rationalism is extreme, obsessive, overconfident, and weird. Therefore we should completely discount the alarms they raise.

Maybe he hedges enough so he’s not saying that the discounting should be complete, but it sure feels that way in this op-ed and his other posts and podcasts.

I have no problem stipulating that Rationalists and their extended universe seem like a weird sex cult, and that this is grounds enough not to take them completely seriously. But that just means we need to be more skeptical about their claims, not dismiss them altogether. If nothing else, the fact that they’ve built and are in charge of these impressive systems should recover for them some of the credibility they lose for being weirdos.

As exasperated as Newport and other non-apocalyptic technologists seem to be with the ASI-pilled Bay Area weirdos, the fact that those weirdos believe in X tells us nothing about the likelihood of X. The expert critics are better off patiently explaining why AI can’t technically lead to such doom (if they can). Indeed, just as much as I would like the AI doomers to rely less on parables and thought experiments and more on technical details, I would also like the critics to tell me what probabilities we should assign to catastrophic outcomes and why. SEP 20