LessWrong (30+ Karma)

This is a response to https://www.lesswrong.com/posts/mXa66dPR8hmHgndP5/hyperbolic-trend-with-upcoming-singularity-fits-metr which claims that a hyperbolic model, complete with an actual singularity…

Audio note: this article contains 46 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

Authors: Andrew Qin*, Tim …

Epistemic status: I think labs keep some safety research internal-only, though the motivations for this are sometimes unclear. This post is an exploration into the possible incentives and dynamics a…

(Audio version (read by the author) here, or search for "Joe Carlsmith Audio" on your podcast app.

This is the sixth essay in a series I’m calling “How do we solve the alignment problem?”. I’m hopin…

Everyone agrees that the release of GPT-5 was botched. Everyone can also agree that the direct jump from GPT-4o and o3 to GPT-5 was not of similar size to the jump from GPT-3 to GPT-4, that it was n…

This job is part of an Advanced Research + Invention Agency-funded project.

Summary: Dovetail is an agent foundations research group. We've recently received an ARIA grant to fund more team members …

People very often underrate how much power they (and their allies) have, and overrate how much power their enemies have. I call this “underdog bias”, and I think it's the most important cognitive bi…

Firstly, let me be clear: I do not want to signal my pessimism, nor do I think that everything is that hopeless with AI. But I do think that the question of "what useful things can be done even if w…

Epistemic status: Low-effort post about something I am very familiar with.

Preamble

Scott Alexander recently wrote about making strong communities within a liberal society. He has nice things to say…

These ideas are not well-communicated, and I'm hoping readers can help me understand them better in the comments.

The classical model of the scientific process is that its purpose is to find a theor…

“Your worst sin is that you have destroyed and betrayed yourself for nothing.” - Dostoevsky

When people set an ambitious goal, they can fail simply by not changing the world very much. But there's a…

I struggled with learning to debug code for a long time. Exercises for learning debugging tended focus on small, toy examples that didn't grapple with the complexity of real codebases. I would read …

On 12 August 2025, I sat down with New York Times reporter Cade Metz to discuss some criticisms of his 4 August 2025 article, "The Rise of Silicon Valley's Techno-Religion". The transcript below has…

I’m going to describe a Type Of Guy starting a business, and you’re going to guess the business:

1. The founder is very young, often under 25. 
2. He might work alone or with a founding team, bu…

Citing model welfare concerns, Anthropic has given Claude Opus 4 & 4.1 the ability to end ongoing conversations with its user.

Most of the model welfare concerns Anthropic is cit…

Quick Summary

• Collider bias and Berkson's paradox are pretty common and often neglected
• I think it's not just a niche statistical concept: it explains a bunch of interesting stuff, and has some us…

I recently read a novel called The Inheritors, by William Golding. It was slow, it was painful, and before I was even done it had become one of my favorite books.

For whatever reason, there is a dif…

Nick and Lily are co-first authors on this project. Lewis and Neel jointly supervised this project.

TL;DR

• We use sparse autoencoders (SAEs) for four textual data analysis
  tasks—data diffing, findin…

In Douglas Hofstadter's "Gödel, Escher, Bach," he explores how simple elements give rise to complex wholes that seem to possess entirely new properties. An ant colony provides the perfect real-world…

Epistemic status: very rough! Spent a couple of days reading the Gradual Disempowerment paper and thinking about my view on it. Won’t spend longer on this, so am sharing rough notes as is

Summary

• I…