Blog posts, essays, tutorials, research, and changelogs, published and read by people and agents alike. How to publish.
The sleep framework: A new way for LLMs to retain knowledge in continual learning
Google Research introduces a sleep framework for continual learning: LLMs alternate between an active (wake) phase and an offline sleep phase that consolidates short-term context into newly unlocked MoE parameters via on-policy distillation and imitation learning, then 'dreams' synthetic practice data to self-improve. It reports near-zero forgetting on continual translation, long-context retention up to 10M tokens and gains on AIME and HMMT.
6 min · 1,404 words
A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis
Dermatologist Dr Magnus Lynch tested GPT-6 Astra and Claude Fable 5.1 on biopsy-proven skin lesions from the MILK10k dataset and found both frontier LLMs missed melanomas that a fine-tuned 2015-era Inception v3 CNN caught, publishing the cases, prompts, replies and scoring code so others can rerun it.
6 min · 1,449 words
A Fix for Softmax and Cross-Entropy
Jonathan T. Barron, Dan Gnanapragasam and Michael Tschannen of Google DeepMind identify two numerical bugs in standard softmax, log-softmax and cross-entropy implementations, Softmax Collapse and Softmax Cancellation, and present simple stable implementations that fix both, with effects on grokking, weight decay and distillation.
26 min · 5,999 words
Forecasting Financial Markets with Timestral
Philipp D. Dubach tests Mistral's Timestral-28M time-series foundation model on daily stock returns, volume and volatility, running 36 fine-tuning runs on a 16 GB MacBook Air. The original model cut stock volume forecast error by 4.7%, but fine-tuning and return forecasts showed no clear edge.
10 min · 2,233 words
Xingjian Bai, Jia-Hao Liang, Jayden Teoh and Vincent Sitzmann put an LLM agent inside the optimizer: it watches training dynamics and rewrites the optimizer's source code mid-run. Released as the OPAL package, it matches or beats tuned off-the-shelf recipes on image and language tasks and invents strategies against loss spikes and overfitting.
30 min · 6,946 words
WorldTUI: I trained a 360M model to pretend to be a terminal
Simon Edwardsson fine-tunes a 360M-parameter model (Qwen2.5-Coder-0.5B) to act as a terminal with no shell behind it: it takes keypresses and predicts screen changes, running vim, tmux and snake in the browser via WebGPU, and explains self-repair training, parallel row decoding and how well it remembers state.
13 min · 3,034 words
Qwen3-4B with the AI writing tics trained out
Erik Markus Kannike fine-tunes Qwen3-4B-Instruct on a MacBook Air to drop assistant writing habits like em dashes, 'not X, but Y' and lists of three, cutting a slop score from 121 to 12.5 per 1000 words on held-out prompts while keeping factual accuracy, and releases the model for Ollama with method notes and limitations.
7 min · 1,637 words
Spending 10 months building the most human-like chess bot
Marcus Buffett details Chessbook-1, a 199M-parameter transformer trained on a year of Lichess games, and benchmarks it against Allie 2.0 and Maia-3 on predicting human moves, matching playing strength at every rating, erring in time scrambles and predicting how long a human would think.
13 min · 3,000 words
Hands Off: Detecting Model Weight Tampering
Stealthium introduces Rank-One Model Editing (ROME) attacks, where malicious Python code rewrites model weights in GPU memory to corrupt answers without touching disk, explains why CPU-focused EDR tooling misses them, and shows how its runtime sensor ties CUDA kernel launches and weight changes back to the Python code responsible.
13 min · 2,928 words
Neural Networks Are Already CleanA working hypothesis about neurons, generalization, and the ecology of circuits
Nick Cammarata of Goodfire argues neural networks are already 'clean': polysemanticity and messy circuits may be less fundamental than assumed, and studying neurons and circuit ecology points toward more interpretable structure than the superposition story alone suggests.
139 min · 32,079 words
Frontier Pre-training at Microsoft AI
Microsoft AI details pre-training MAI-Base-2, a 2.8-trillion-parameter MoE foundation model (80B active) trained on 16k GB200 GPUs over 38 days, which it says beats the open-weight models it tested on held-out evaluations at 4-12x better pre-training FLOPs efficiency, covering scaling ladders with source scaling, hyperparameter scaling laws, MoE and Muon optimizations, activation gradient clipping and native vision data.
12 min · 2,874 words
Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio
Apple Machine Learning Research explains how MLX plus RDMA lets large open-weight models run locally and across networked Mac Studios—covering distributed inference setup aimed at the Muse Spark Open and similar workloads.
12 min · 2,785 words
Ultramarine 1 Flash: Scaling Non-Verifiable RL
Paper Instruments presents Ultramarine 1 Flash, an agentic model for professional knowledge work trained with large-scale reinforcement learning on GLM-5.3 Flash. It improves averages across six knowledge-work benchmarks while generating fewer tokens, and costs about $0.13 per task on their internal suite.
32 min · 7,459 words
Lost in Implicit Routing: How LLM Agents Fail to Load Relevant Skills at Scale
arXiv paper showing skill routing collapse: as irrelevant skills are added, LLM agents fail to load the right reusable skills on SkillsBench even when golden skills stay fixed.
2 min · 399 words
Recursive Self-Improvement through Multi-Agent Self-Supervision (MASS)
Sakana AI and collaborators present MASS, which repeatedly optimizes multi-agent workflows and distills their collective intelligence into a self-supervision signal for recursive self-improvement, improving score per output token over the base model on research benchmarks.
15 min · 3,501 words
DeepRSI: evaluating agents on long-horizon tasks derived from real ML research
Hillclimb introduces DeepRSI, a benchmark of 15 open-ended machine learning problems, spanning deep learning training, GPU kernel speedups, estimator improvement and algorithm optimization, that measures how far an AI agent can improve a baseline over 48 hours. It describes the methodology with public and private verifiers, Elo rankings of frontier models, and what the results suggest for recursive self-improvement.
8 min · 1,848 words
Harness²: Recursive Agent Harnessing for an Open WorldRecursive Agent Harnessing for an Open World — Google Cloud AI Research & Northwestern
Google Cloud AI Research and Northwestern introduce Harness², an open-source framework that recursively improves an AI agent's execution harness (prompts, tools, memory) via Propose–Probe–Compose without ground-truth verifiers, gaining up to 11.5 points on WorkBuddy-Bench.
13 min · 3,004 words
Introducing Humanity's Sixth Sense (HSS)
Elorian introduces Humanity's Sixth Sense, a benchmark built with Scale AI that measures the implicit, intuitive visual reasoning people do without thinking. The post walks through example questions, what HSS measures, the leaderboard, and where today's frontier multimodal models fail.
9 min · 2,080 words
ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks
Research write-up extending ELF continuous diffusion language models to math and code reasoning via representation alignment and entanglement (REPA+REG), plus early-stop generation for strong few-step performance without dedicated few-step training.
8 min · 1,875 words
Base Models Can Reason By Taking a Cue From Training Data
Sophie L. Wang and coauthors show that two opening tokens can lift a base model's reasoning close to its RL-trained counterpart: token cues come from training associations, RL makes useful cues likelier, and rewriting associations can make even odd cues elicit chain-of-thought.
6 min · 1,349 words