Topic
Everything filed under Machine Learning, newest first.
RSS · JSON · All topics
The 'Astra can read and write weights to improve Qwen' post is massively misleading
Matthew Honnibal argues the viral 'Sparks of Weight-Space Programming' post mixes two unrelated techniques: experiments where a model reads raw weights directly, and the Qwen improvement result that relies on ordinary Python tooling, so together they overstate what LLMs can do with neural network weights.
3 min · 751 words
LLMs are not superhuman at reading and writing neural network weights
Sean Goedecke responds to the Weight-Space Programming paper, arguing that GPT-6 Astra's feats of identifying what toy game networks were trained on and editing a small Qwen model's weights are things a skilled human ML researcher could also do, so they are not evidence of superhuman ability or imminent recursive self-improvement.
3 min · 801 words
Writing seen in PHerc. 306B, for the first time in 2000 years
The Vesuvius Challenge announces that Liam Warner and Nicola Bodill revealed ink and recovered readable text from the carbonized Herculaneum scroll PHerc. 306B, unread since it was buried by Vesuvius in AD 79, using micro-CT scans and ink detection, and outlines what remains to unlock the collection.
7 min · 1,502 words
Introducing Hume's Audio Decision API: A decision layer for speech and sound
Hume AI launches its Audio Decision API, an audio-native model that answers plain-language questions about speech and sound (yes/no, choice, scale or time spans) directly from raw audio with probabilities, grounded in millions of human perceptual judgments and reaching 70.7% agreement with a human panel in initial evaluation.
5 min · 1,227 words
Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights
MIT CSAIL researchers report that frontier LLMs such as GPT-6 Astra can natively read raw neural network weights to tell what a network was trained to do, write small game-playing networks from scratch that beat PPO-trained ones, and edit Qwen3.5-0.8B weights to fix reasoning loops without gradients.
22 min · 5,137 words
Nace.AI releases Drex for Finance, a decision model for fintech fraud, credit and filings that returns option probabilities instead of free text, scoring 77.3 on its finance index versus 63.6 for Jev 1.14.
5 min · 1,241 words
The Hitchhiker’s Guide to Training on Compute Islands
Rohan Anil of Core Automation argues frontier training can use scattered GPU clusters, describing a ~32B sparse MoE trained across B300 and GB300 clusters about 9,000 miles apart over a 25 Gbps link, plus a butterfly-mixing experiment that replaced all-reduce and matched standard training.
14 min · 3,161 words
Energy-Based Reasoners: Your Recurrent Reasoner Should Be an Energy-Based Model
Researchers from Duke, UIUC and Harvard introduce Energy-Based Reasoners, looped reasoning models that learn an energy function over latent states; with 1.8M parameters they reach 99.9% on Sudoku-Extreme using 345x less inference compute than the next-best model.
4 min · 873 words
Post training a model to explore Demoscenes
datavorous describes post-training Demoscener-7B on a dataset of about 25k tiny demoscene programs (dweets) to explore how sizecoded visual effects arise from unfinished code, with examples and released model and dataset on Hugging Face.
6 min · 1,485 words
Embedding boolean circuits in a neural network
Edward Benson thinks out loud about installing a boolean-circuit sidecar in a trained neural network: building AND, OR and NOT gates from fixed neural weights, combining them into circuits, and adapting real-valued activations into boolean inputs and back into the model.
2 min · 569 words
Grace Blackwell needs a new scheduler: 27% more usable GPUs from the same racks
NVL72 puts all 72 GPUs in a rack on one NVLink fabric, but only rack-local jobs get the speedup. SkyPilot explains how it keeps multi-node training and inference on NVLink with topology-aware placement, in-place recovery from GPU failures, and automatic defragmentation.
11 min · 2,604 words
How we’re training a Figma-fluent editing model
Figma's Josh Clemm describes the research behind the Figma agent's first-party editing model: supervised fine-tuning on synthetic edit examples, a move toward reinforcement learning on newer base models such as Qwen, multi-dimensional design-quality evals, and how the model fits alongside third-party models in the agent.
5 min · 1,108 words
small models are enough for agents
Rishabh Gupta responds to the paper 'Small language models are the future of agentic AI', arguing most calls inside an agent do not need a giant model. He shares what he saw training tiny models from scratch on a MacBook Air, why sending the boring calls to small models and only the hard ones to a big model makes sense, and why exact long-range recall is where small models still fall over.
6 min · 1,460 words
From Tokens to Bytes I: Scaling Hierarchical Networks
Cartesia Research reports a newer generation of hierarchical networks (H-Nets) that learn directly from raw bytes via dynamic chunking instead of a tokenizer. In controlled scaling ladders, byte-level H-Nets beat matched BPE Transformers (about 1.3x on validation loss and 2x on code), and a larger two-stage H-Net matches an open-source 3B Transformer with roughly 10x less compute and data, with previews in DNA and vision.
29 min · 6,691 words
The sleep framework: A new way for LLMs to retain knowledge in continual learning
Google Research introduces a sleep framework for continual learning: LLMs alternate between an active (wake) phase and an offline sleep phase that consolidates short-term context into newly unlocked MoE parameters via on-policy distillation and imitation learning, then 'dreams' synthetic practice data to self-improve. It reports near-zero forgetting on continual translation, long-context retention up to 10M tokens and gains on AIME and HMMT.
6 min · 1,404 words
Introducing LFM3: Every Modality for Specialized Intelligence
Liquid AI releases LFM3, an open-weight family of 2B-class models (text, vision, audio, omni and decision) built on one new tri-hybrid backbone pre-trained on 22T tokens; LFM3-1.7B beats the larger LFM2.5-2.6B, the text model fits in about 1 GB at 4-bit with a 256K context, and every model can be fine-tuned on a single GPU for on-device specialised use.
20 min · 4,683 words
This is called Speculative Decoding
Gaurav Sen explains speculative decoding, the technique Google published in 2022 to make large language models answer 2 to 3 times faster with the same output: a small model drafts several tokens, the large model verifies the draft's prefixes in parallel, and the method now serves production traffic at Google and DeepSeek.
6 min · 1,435 words
A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis
Dermatologist Dr Magnus Lynch tested GPT-6 Astra and Claude Fable 5.1 on biopsy-proven skin lesions from the MILK10k dataset and found both frontier LLMs missed melanomas that a fine-tuned 2015-era Inception v3 CNN caught, publishing the cases, prompts, replies and scoring code so others can rerun it.
6 min · 1,449 words
Zero-Shot to New Robots: Introducing Dream-1
Dream Labs comes out of stealth with $90M in seed funding and releases Dream-1, a World Action Model offered through a public API that controls bimanual parallel-gripper robots it was never trained on, zero-shot, from $3/hour, with fine-tuning-as-a-service for robots further from its training data and a candid look at its limits.
15 min · 3,417 words
A Fix for Softmax and Cross-Entropy
Jonathan T. Barron, Dan Gnanapragasam and Michael Tschannen of Google DeepMind identify two numerical bugs in standard softmax, log-softmax and cross-entropy implementations, Softmax Collapse and Softmax Cancellation, and present simple stable implementations that fix both, with effects on grokking, weight decay and distillation.
26 min · 5,999 words