u/Alex__007

Many believe superintelligence is just a few years away. But what comes *after*? And how can we prepare?

Liv sat down to discuss this minor topic with philosopher Will MacAskill, one of the few people (outside of the AI labs) actively researching it. He’s the lead researcher at Forethought - a nonprofit focused on how to navigate the transition to a post-AGI world safely. He’s also a former Oxford University professor, and a co-founder of the effective altruism movement, among other things.

Chapters:
01:10 — Intro
00:04:11 — Frameworks For Thinking about The Future
00:06:40 — Why We Need to Think About These Questions Now
00:10:50 — Post AGI Worlds
00:14:16 — Blockers of Positive Futures
00:20:11 — Hedonium
00:23:45 — Universal Basic Resources
00:29:23 — Suffering Risks
00:33:52 — Viatopia
00:37:50 — Can We Understand — and Steer — Moral Fashions?
00:43:06 — Can AI Repair Our Shared Reality?
00:49:45 — Pluralistic Futures: Choosing How We Want to Live
00:54:57 — The Dangers of Elites Controlling AI
01:03:14 — Can We Measure How Well We See the Future?
01:06:50 — Purpose in a World of Abundance
01:09:20 — Sci-Fi Ideas Worth Writing Now
01:16:29 — Meaning in a Post-labor Economy
01:21:24 — How Does Moral Progress Actually Happen?
01:27:45 — Will’s Call to Action

Links:
♾️ Will MacAskill’s website: https://william-macaskill.com
♾️ Forethought: https://www.forethought.org/
♾️ Essay on Viatopia https://www.forethought.org/research/...
♾️ What We Owe the Future (Will MacAskill): https://www.amazon.com/dp/1541618629
♾️ Deep Utopia (Nick Bostrom): https://www.amazon.com/dp/1914741128
♾️ Utopia (Thomas More): https://www.gutenberg.org/ebooks/2130
♾️ The Republic (Plato): https://www.gutenberg.org/ebooks/1497
♾️ Island (Aldous Huxley): https://www.amazon.com/dp/0061561791
♾️ Anarchy, State, and Utopia (Robert Nozick): https://www.amazon.com/dp/0465051006
♾️ The Archipelago (Scott Alexander): https://slatestarcodex.com/2014/04/09...
♾️ Grabby Aliens Model (Robin Hanson): https://grabbyaliens.com
♾️ The Fermi Paradox: https://en.wikipedia.org/wiki/Fermi_p...
♾️ WEIRD Psychology (Joseph Henrich): https://www.hup.harvard.edu/books/978...
♾️ Protopia (Kevin Kelly): https://kk.org/thetechnium/protopia/
♾️ Universe 25 (John B. Calhoun): https://www.ncbi.nlm.nih.gov/pmc/arti...
♾️ Dyson Swarm (Concept): https://en.wikipedia.org/wiki/Dyson_s...
♾️ Effective Altruism: https://www.effectivealtruism.org
♾️ Giving What We Can: https://www.givingwhatwecan.org
♾️ 80,000 Hours: https://80000hours.org

>“The OpenAI Foundation’s Alzheimer’s research initiative represents more than scientific progress. It’s hope for millions of people, families, and anyone concerned about brain health. We applaud bold investments that prioritize speed and rigor, because every day matters. We must accelerate breakthroughs that change what it means to live with, or be at risk for, Alzheimer’s.”

—Joanne Pike, PhD, President and CEO of the Alzheimer’s Association

00:00 What is the best use of AI?
02:04 Who is Demis Hassabis?
03:58 What is AlphaFold?
06:15 Why did Demis win the Nobel Prize?
12:30 How is AlphaFold accelerating science?
16:47 What is the cutting edge of drug discovery?
19:13 How close are we to AI DNA editing?
21:52 What did Demis want AI to do?
25:39 What does AI actually do like now?
29:16 How can AI be creative?
34:24 What is AlphaGo?
37:22 What is AlphaZero?
43:09 How should governments use AI?
45:40 What are the biggest worries about AI?
48:00 What are we not worrying enough about with AI?
50:13 What can humans do that AI can’t?
55:17 Why does Demis want AGI?
58:17 How does Demis want to be remembered?
1:00:14 What do AI simulations do?
1:01:56 How should I prepare?
1:04:59 :)

Demis Hassabis is the Co-Founder & CEO of Google DeepMind - working on AGI, responsible for AI breakthroughs such as AlphaGo, the first program to beat the world champion at the game of Go; and AlphaFold, which cracked the 50-year grand challenge of protein structure prediction and was recognised with the 2024 Nobel Prize in Chemistry. Demis is revolutionising drug discovery at Isomorphic Labs. Ultimately, trying to understand the fundamental nature of reality.

-----------------------------------------------

Timestamps:
00:00 Intro
01:21 What Actually Counts as AGI & Where Are We Today?
02:58 What Are the Biggest Bottlenecks Holding AI Back Today?
03:48 Have We Hit the Limits of Scaling Laws?
04:40 Where Is AI Ahead of Expectations & What's Still Missing?
05:24 Why Can't AI Systems Learn Continuously Like Humans?
06:10 How Did DeepMind Go from Behind to Leading the Pack?
09:10 Are We Heading Toward Model Commoditization?
09:59 What Does the Future of Open Source Really Look Like?
11:25 What Does a Post LLM World Look Like?
13:03 Can AI Really Fix Drug Discovery?
15:01 What Does "Good" AI Regulation Actually Look Like?
17:31 Who Should Be the Ultimate Arbiter of Truth in an AI World?
18:36 If Demis Had One Shot to Fix AI Safety, What Would He Do?
19:58 Is This Time Different for Jobs or Will History Repeat Itself?
24:06 How Do We Solve the Energy Crisis Created by AI?
25:34 Why Stay in the UK Instead of Moving to Silicon Valley?
27:38 Will Europe Ever Build a Trillion-Dollar Tech Giant?
29:20 Meeting Elon Musk for the First Time?
31:03 What Big Questions About AI Is No One Talking About?
31:42 What Does Demis Want His Legacy to Be?

Amazon isn’t the only tech company that the ongoing conflict between the United States and Iran has directly hit. The Middle Eastern country has threatened to strike Nvidia, Microsoft, and others as early as the second week of March. It has reiterated the threat at the start of April and struck an Oracle data center later that week.

However, while damage to data centers in the Middle East is concerning for the region, the global tech industry has bigger concerns. The regional war has disrupted the flow of oil and its derivatives, especially those that go through the Strait of Hormuz. These include aluminum, helium, and LNG — all of which are crucial in the semiconductor supply chain. And even if the war ends today, the damage to infrastructure could mean it takes months or even years for supplies to return to pre-war levels.