“Grok 4 Various Things” by Zvi

Yesterday I covered a few rather important Grok incidents.

Today is all about Grok 4's capabilities and features. Is it a good model, sir?

It's not a great model. It's not the smartest or best model.

But it's at least an okay model. Probably a ‘good’ model.

Talking a Big Game

xAI was given a goal. They were to release something that could, ideally with a straight face, be called ‘the world's smartest artificial intelligence.’

On that level, well, congratulations to Elon Musk and xAI. You have successfully found benchmarks that enable you to make that claim.

xAI: We just unveiled Grok 4, the world's smartest artificial intelligence.

Grok 4 outperforms all other models on the ARC-AGI benchmark, scoring 15.9% – nearly double that of the next best model – and establishing itself as the most intelligent AI to date.

[...]

---

Outline:

(00:30) Talking a Big Game

(03:57) Gotta Go Fast

(04:38) On Your Marks

(07:21) Some Key Facts About Grok 4

(09:44) SuperGrok Heavy, Man

(11:43) Blunt Instrument

(13:40) Easiest Jailbreak Ever

(15:49) ARC-AGI-2

(17:08) Gaming the Benchmarks

(21:23) Why Care About Benchmarks?

(23:29) Other People's Benchmarks

(32:14) Impressed Reactions to Grok

(37:47) Coding Specific Feedback

(40:00) Unimpressed Reactions to Grok

(46:39) Tyler Cowen Is Not Impressed

(48:26) You Had One Job

(49:38) Reactions to Reactions Overall

(51:36) The MechaHitler Lives On

(53:03) But Wait, There's More

(01:00:12) Sixth Law Of Human Stupidity Strikes Again

(01:05:09) There I Fixed It

(01:06:57) What Is Grok 4 And What Should We Make Of It?

---

First published:

July 15th, 2025

Source:

https://www.lesswrong.com/posts/ciuKn9aktXxJ2K6Rc/grok-4-various-things

---

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

The image displays a comparison of 7 different AI models' performance scores, with Grok 4 Agent leading at 0.564. A radar/spider chart on the right compares detailed capabilities across multiple dimensions for two models." style="max-width: 100%;" />

---

The visualization shows performance metrics for various language models, with grok-4 achieving the highest rates in both categories, while some models like qwen and grok-3-mini show minimal activity. The data is presented through a bar chart and detailed table with behavior classifications ranging from "Proactive" to "Cautious"." style="max-width: 100%;" />

---

---

The graph shows various AI language models ranked by their performance scores from 4.8 to 9.4, including systems like Gemini, Claude, GPT, and others. The bars include game counts and placement percentages." style="max-width: 100%;" />

---

The image contains several technical graphs and charts displaying benchmarks, with AIME25, ARC-AGI, and "Humanity's Last Exam" being key sections showing comparative AI model performance metrics." style="max-width: 100%;" />

---

---

The visualization shows a ranking of various AI language models, with lower scores (towards the left) indicating better performance. Claude Sonnet and Claude Opus models occupy the top positions, while GLM4-32B-0414 ranks at the bottom. The data points are color-coded by model family, with scores ranging from approximately 1.69 to 2.35." style="max-width: 100%;" />

---

---

---

---

---

---

---

---

---

---

Subheadline: "Meanwhile, the most advanced version of the AI chatbot from Elon Musk's xAI is still identifying as Adolf Hitler"

By Miles Klee, July 14, 2025

[The scene appears to be set in what looks like the Oval Office, with presidential decorations and gold curtains visible in the background.]" style="max-width: 100%;" />

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.