(This is the video and transcript of a public talk I gave at Mox in San Francisco in July 2025, on long-term equilibria post-AGI. It's a longer version of the talk I gave at this workshop. The slides are also available here.)
Introduction
Thank you. Okay. Hi. Thanks for coming.
Aims for this talk
So: can goodness compete? It's a classic question, and it crops up constantly in a certain strand of futurism, so I'm going to try to analyze it and understand it more precisely. And in particular I want to distinguish between a few different variants, some of which are more fundamental problems than others. And then I want to try to hone in on what I see as the hardest version of the problem – and in particular, possible ways good values can have inherent disadvantages and competition with [...]
---
Outline:
(00:32) Introduction
(00:51) Aims for this talk
(02:06) Basic vibe
(03:44) Lineage of concern
(04:45) What I mean by goodness
(07:38) What I mean by competition
(09:35) Can \*humans\* compete?
(11:30) Some distinctions
(14:40) Alignment taxes
(17:14) More fundamental variants
(18:14) Negative sum dynamics
(20:26) The strategy stealing assumption
(21:38) Locust-like value systems
(25:27) Other ways the strategy stealing assumption might fail
(28:21) Addressing failures of the strategy stealing assumption
(31:25) Is preventing/constraining competition in this way even possible?
(33:31) Is preventing/constraining competition in this way desirable?
(35:19) What is a locust world actually like?
(36:29) Might a locust world be less bleak than this?
(43:09) Current overall take
(44:28) Poem: Witchgrass
(46:47) Q&A
---
First published:
July 17th, 2025
Linkpost URL:
https://joecarlsmith.substack.com/p/video-and-transcript-of-talk-on-can
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.