The Information Bottleneck

John Platt is a Google Fellow and leads the Applied Science team at Google Research, the group working on climate and science. He's also the person behind SMO and Platt scaling. In this episode, we talk with him about how Google decides which scientific problems are worth its time, and what happens when AI meets real physical systems. A big part of the conversation is contrails. A small share of flights creates most of the warming from contrails, which together add up to roughly 1% of human-caused warming. John explains how his team detects contrails from weather satellites and predicts where they will form. In trials with American Airlines, rerouting flights cut contrail formation by 62% with almost no extra fuel. We also get into ERA, Google's empirical research assistant, and where AI agents actually help scientists. John sees them taking over the drudgery of building models and munging data, while humans keep the taste and the rigor. He shares why he's skeptical of gradient-free optimization, why the bitter lesson runs into energy and data center limits, and why physical AI that makes manufacturing cheaper could matter a lot for climate. We finish with FireSat, a satellite constellation (about 50 satellites at full scale) designed to catch wildfires while they're still small enough to put out.
Timeline00:00 Intro
01:09 How Google picks science problems: comparative advantage
04:15 Projects that didn't work: RNA aptamers and tool/problem fit
05:46 DNA, RNA and XNA
07:41 Contrails and why they warm the planet
13:01 Three AI systems for avoiding contrails
15:23 Cloud seeding and why cloud physics is hard
18:17 Airline trials: 62% fewer contrails
19:46 Navier-Stokes, wind tunnels and where simulation still fails
25:07 Will agents replace scientists? ERA and scorable tasks
31:36 Overfitting, Kaggle and the half-pixel bug
33:41 Agents that exploit bugs: Lean and Goodhart's law
36:45 Gradient-free optimization
38:53 SMO, SVMs and hinge loss in neural networks
42:12 The bitter lesson and the limits of scaling
43:36 Where agents will help science in the next 12 months
45:13 Robotics and making climate tech cheaper to build
48:22 AI risk and misuse
50:51 Climate outlook: past 2°C
54:39 Why AI won't just solve climate change
57:04 Room-temperature superconductors, fusion and HVDC
1:02:25 FireSat: detecting wildfires from orbit
1:05:16 Why existing satellites aren't enough
1:07:53 The economics of 50 satellites
1:09:34 False positives
1:11:16 Closing
TopicsComparative advantage as the test for which science problems Google takes on
Contrails: the physics, the AI pipeline, and the American Airlines trials
ERA and AI agents for scientific software
What scientists still do that agents can’t: taste, creativity, rigor
Overfitting, Goodhart’s law, and agents exploiting bugs
Gradient-free methods, SMO, and hinge loss
The bitter lesson and the physical limits of scaling
Climate outlook, fusion, and physical AI
Room-temperature superconductors and long-distance power transmission
FireSat and early wildfire detection
Music
- "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0

Yuandong Tian spent many years at Meta FAIR and recently left to co-found Recursive Superintelligence, a company building AI that improves itself. In this episode, he tells us why.
We start with his work on computer Go, where he built DarkForest before AlphaGo came out. A few years later came OpenGo, which played Korean professionals on a single GPU and didn't lose a game. He then tried to bring RL to real-world problems and found that the design of the action space mattered more than the algorithm.
We also talk about Coconut, his paper on reasoning in latent space instead of tokens, and why frontier models still aren't trained that way.
The second half is about recursive self-improvement. Yuandong wrote his last paper at Meta together with GPT-5 and says it made him 6 to 10 times faster. That convinced him his own job could be replaced within five years. We ask him where agents still fall short and whether a new architecture can really beat transformers at scale. He also gives his view on the calls to restrict self-improving AI and makes the case for open source.
Recursive is hiring in San Francisco and London: talent@recursive.com
Topics
* Computer Go: DarkForest, AlphaGo and OpenGo
* Gradient-free optimization
* Action space design and neural architecture search
* Coconut and reasoning in latent space
* Understanding how neural networks learn representations
* Grokking, and writing a paper with GPT-5
* Recursive self-improvement and coding agents
* New architectures vs. transformers
* The NanoGPT speedrun
* Data efficiency and robotics
* Restricting self-improving AI
* Open source models
Timeline
0:00 Intro
0:45 From CMU to deep learning, and the AlexNet debates
5:35 AlphaGo and beating Go pros on a single GPU
10:27 Gradient-free optimization
13:16 Diffusion models and diversity
16:23 RL on real problems: why the action space matters
20:34 AutoML and architecture search
22:05 Coconut and reasoning in latent space
26:07 Why latent reasoning hasn't caught on
30:36 Adapting research to the LLM era
34:04 Why he left Meta to work on recursive self-improvement
37:10 Do we still need humans?
39:37 Where agents fall short
43:08 Can new architectures beat transformers at scale?
47:29 The hardest part of research to automate
50:16 What comes next for AI
52:52 Should self-improving AI be restricted?
54:11 Open source models
56:43 Hiring at Recursive
Music
- "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0

Yuandong Tian spent many years at Meta FAIR and recently left to co-found Recursive Superintelligence, a company building AI that improves itself. In this episode, he tells us why.
We start with his work on computer Go, where he built DarkForest before AlphaGo came out. A few years later came OpenGo, which played Korean professionals on a single GPU and didn't lose a game. He then tried to bring RL to real-world problems and found that the design of the action space mattered more than the algorithm.
We also talk about Coconut, his paper on reasoning in latent space instead of tokens, and why frontier models still aren't trained that way.
The second half is about recursive self-improvement. Yuandong wrote his last paper at Meta together with GPT-5 and says it made him 6 to 10 times faster. That convinced him his own job could be replaced within five years. We ask him where agents still fall short and whether a new architecture can really beat transformers at scale. He also gives his view on the calls to restrict self-improving AI and makes the case for open source.
Recursive is hiring in San Francisco and London: talent@recursive.com
Topics
* Computer Go: DarkForest, AlphaGo and OpenGo
* Gradient-free optimization
* Action space design and neural architecture search
* Coconut and reasoning in latent space
* Understanding how neural networks learn representations
* Grokking, and writing a paper with GPT-5
* Recursive self-improvement and coding agents
* New architectures vs. transformers
* The NanoGPT speedrun
* Data efficiency and robotics
* Restricting self-improving AI
* Open source models
Timeline
0:00 Intro
0:45 From CMU to deep learning, and the AlexNet debates
5:35 AlphaGo and beating Go pros on a single GPU
10:27 Gradient-free optimization
13:16 Diffusion models and diversity
16:23 RL on real problems: why the action space matters
20:34 AutoML and architecture search
22:05 Coconut and reasoning in latent space
26:07 Why latent reasoning hasn't caught on
30:36 Adapting research to the LLM era
34:04 Why he left Meta to work on recursive self-improvement
37:10 Do we still need humans?
39:37 Where agents fall short
43:08 Can new architectures beat transformers at scale?
47:29 The hardest part of research to automate
50:16 What comes next for AI
52:52 Should self-improving AI be restricted?
54:11 Open source models
56:43 Hiring at Recursive
Music
- "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0

Chris Manning is a legend in NLP. He's a professor of linguistics and computer science at Stanford and ran the Stanford AI Lab. His course CS224N, Natural Language Processing with Deep Learning, is where a whole generation of researchers learned the field. If you've worked on anything in NLP over the last 25 years, you've probably used his work.
We start with what linguistics gave machine learning that ML wouldn't have figured out on its own, and what's left for NLP researchers now that LLMs handle most of the classic tasks. Chris thinks the open problems have moved up the stack to pragmatics and dialogue. Models are good at using context but still sound confident when they shouldn't.
Then we get into his recent work on how language models learn verb classes. Small GPT-2 models seem to form abstract categories right away instead of memorizing verbs one at a time, and Chris explains why distributed representations push them in that direction.
The second half is about meaning and representation. Chris makes the case that Yann LeCun underrates the role of language in intelligence, and revisits his debate with Emily Bender over whether text alone can teach meaning. We finish with ReFT, which steers a frozen model by editing its hidden states, and whether concepts really live in linear subspaces.
Timeline
00:00 Intro
01:05 What linguistics gave machine learning
03:27 Is NLP losing its focus on language?
06:29 What's left for NLP researchers in the LLM era
09:37 The next frontier: pragmatics and dialogue
13:01 Constructed languages and AI-to-AI communication
15:35 Do language models learn categories first?
23:43 Why transformers learn abstractions early
26:37 Does it hold at scale?
28:30 LeCun, JEPA and the role of language in intelligence
34:54 Can text alone teach meaning? The octopus debate
40:45 Diffusion language models vs transformers
43:39 ReFT: editing representations instead of weights
50:34 Weights or representations: where knowledge lives
52:37 Do concepts really live in linear subspaces?
Topics
NLP, computational linguistics, large language models, learning dynamics, meaning from form, world models, JEPA, diffusion LMs, ReFT, representation learning, interpretability
Music
"Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0

Andrew Dai spent over a decade at Google Brain and DeepMind, where he co-wrote the 2015 paper that introduced language model pre-training followed by fine-tuning, and later co-led pre-training data for Gemini. He's now co-founder and CEO of Elorian AI, which is building models for visual reasoning.
We talk about how his pre-training result started as a bug, why next-token prediction scales better than other objectives, and what went wrong for Google in the early LLM race. The second half is about vision: why today's frontier models still can't count objects in a photo, why he thinks reasoning is fundamentally visual, and how his view of world models differs from JEPA.
Chapters
00:00 Intro
00:48 Andrew's background and the accidental discovery of pre-training
07:10 Why next-token prediction scales
10:09 How Google fell behind and the early days of Gemini
18:28 What makes training data good
28:16 Where visual understanding breaks down
46:01 World models, JEPA and robotics
55:27 Generation vs understanding, and what's next for Elorian
Topics
Pre-training and fine-tuning
Scaling and next-token prediction
Gemini and Google's AI history
Data quality and synthetic data
Visual reasoning and counting
World models and JEPA
Music
"Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0