LessWrong (30+ Karma)

What a week, huh? America signed a truly gigantic chip sales agreement with UAE and KSA that could be anything from reasonable to civilizational suicide depending on security arrangements and implem…

The economist Wassily Leontief, writing in 1966, used the then-recent decline of horses to make vivid what he foresaw as the coming impact of technological advances on workers. In 1910, the horse ha…

In the context of model-based RL agents in general, and brain-like AGI in particular, part of the source code is a reward function. The programmers get to put whatever code they want into the reward…

Core Argument

The barrier to preying on "mirror-image" life is likely lower than presumed due to:

1. Immediate energy from achiral fats.
2. Existing L-sugar pathways.
3. Existing D-amino acid racemases…

The company released a model it classified as risky — without meeting requirements it previously promised

This is the full text of a post first published on Obsolete, a Substack that I write about t…

In my previous post in this series, I explained why we urgently need to change AI developers’ incentives: if we allow the status quo to continue, then an AI developer will recklessly deploy misalign…

Last week I stumbled over Dimensional Analysis which is not only useful for applied fields (physics, biology, economics), but also for math (Why did no one tell me that you can almost always think o…

PDF version. berkeleygenomics.org.

Introduction

Here we list six policies that would help accelerate the development of novel assisted reproductive technologies. Such technologies include mitochon…

Claude Sonnet 4 and Claude Opus 4 are out. Anthropic says they're both state-of-the-art for coding. Blogpost, system card.

Anthropic says they may have dangerous bio capabilities…

Epistemic status: shower thought quickly sketched, but I do have a PhD in this.

As we approach AGI and need to figure out what goals to give it we will need to find tractable ways to resolve moral d…

On May 20, during her speech at the Annual EU Budget Conference 2025, Ursula von der Leyen, President of the European Commission, stated:

When the current budget was negotiated, …

What did Google announce on I/O day? Quite a lot of things. Many of them were genuinely impressive. Google is secretly killing it on the actual technology front.

Logan Kilpatrick (DeepMind): Google…

Audio note: this article contains 53 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

Summary: We have previousl…

In my previous post in this sequence, I made a fundraising pitch for my organization, the Center for AI Policy (CAIP), asking readers to donate to us so we can continue lobbying Congress to pass str…

None of this is medical advice.

5 AM on a Saturday and I can’t go back to sleep. It's not the first time, so I get up to write, I might as well use the time I’m given. My hangover is to blame, even…

Podcast version (read by the author) here, or search for "Joe Carlsmith Audio" on your podcast app.

1. Introduction

Currently, most people treat AIs like tools. We act like AIs don’t matter in thems…

While we wait for wisdom, OpenAI releases a research preview of a new software engineering agent called Codex, because they previously released a lightweight open-source coding agent in terminal cal…

DSA - decisive strategic advantage.

We will soon enter an unstable state where the balance of military and political power will shift significantly because of advanced AI.

As different nations gain …

Google Deepmind has announced Gemini Diffusion. Though buried under a host of other IO announcements it's possible that this is actually the most important one!

This is significa…

Have the Accelerationists won?

Last November Kevin Roose announced that those in favor of going fast on AI had now won against those favoring caution, with the reinstatement of Sam Altman at OpenA…