Shouldn’t we spend money on AGI safety, just in case?
My answer in this essay: yes, but less money than AGI safety advocates would like, and not on typical AGI safety projects.
This is probably the #1 most common response I hear when I argue against the beliefs of the AGI safety community. As I explain in this essay, I think this response conflates a general argument about existential risk with an argument for funding certain particular projects. It’s a subtle distinction, but an important one.
In January, I wrote a satirical post on the Effective Altruism Forum that was meant to draw attention to the flaw in the idea that a tiny probability of existential catastrophe justifies giving a lot of money to AGI safety organizations. This new essay is my attempt to seriously explain what the flaw in the logic is, and to describe what I think we should do instead.