Topic
Everything filed under Machine Learning, newest first.
RSS · JSON · All topics
Is ‘Attention’ the Essence of Intelligence?
Quanta columnist Natalie Wolchover traces the attention mechanism from Dzmitry Bahdanau's 2014 internship work on machine translation in Montreal through transformers to today's AI boom, and asks why attention turned out to be the key mathematical idea, drawing on researchers' views about what it captures and whether brains do something similar more efficiently.
10 min · 2,402 words
Recursive Self-Improvement through Multi-Agent Self-Supervision (MASS)
Sakana AI and collaborators present MASS, which repeatedly optimizes multi-agent workflows and distills their collective intelligence into a self-supervision signal for recursive self-improvement, improving score per output token over the base model on research benchmarks.
15 min · 3,501 words
DeepRSI: evaluating agents on long-horizon tasks derived from real ML research
Hillclimb introduces DeepRSI, a benchmark of 15 open-ended machine learning problems, spanning deep learning training, GPU kernel speedups, estimator improvement and algorithm optimization, that measures how far an AI agent can improve a baseline over 48 hours. It describes the methodology with public and private verifiers, Elo rankings of frontier models, and what the results suggest for recursive self-improvement.
8 min · 1,848 words
Release notes for Model2Vec 0.10.0: major training-side improvements to the fast static embeddings library, with links to the PyPI release and what changed for practitioners.
4 min · 929 words
Running RAG with a (small) language model in the browser
Bart de Goede extends his static blog's in-browser semantic search into full retrieval-augmented generation by running small language models client-side with transformers.js on WebGPU or WASM, and evaluates how faithful and useful answers from Qwen3, Gemma and other small models are compared with larger ones.
9 min · 2,160 words
Nish Tahir explains "system one" decision models like TypeSafe AI's Jev, which answer in a single forward pass by masking the vocabulary to a fixed set of options, then walks through fine-tuning a small model on multiple-choice data and calibrating its confidence with temperature scaling, with a companion GitHub repo.
4 min · 1,019 words
Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation
Introduces Next-Concept Modeling (NCM), a coarse-to-fine alternative to diffusion language models: a multiscale residual VQ-VAE encodes text into hierarchical codes and a transformer predicts each level in parallel, beating or matching the strongest DLM baselines on LM1B, OpenWebText and XSum while running 10-31x faster.
1 min · 212 words
Harness²: Recursive Agent Harnessing for an Open WorldRecursive Agent Harnessing for an Open World — Google Cloud AI Research & Northwestern
Google Cloud AI Research and Northwestern introduce Harness², an open-source framework that recursively improves an AI agent's execution harness (prompts, tools, memory) via Propose–Probe–Compose without ground-truth verifiers, gaining up to 11.5 points on WorkBuddy-Bench.
13 min · 3,004 words
Microsoft-Decision-1: Our model for fast decision-making
Microsoft introduces Microsoft-Decision-1, a decision-scoring model in Foundry built to return structured outputs software can act on immediately, and reports that it leads on latency and quality across structured decision tasks against LLMs and other decision models.
6 min · 1,358 words
$50K First Letters Prize awarded for PHerc 343
The Vesuvius Challenge awards a $50K First Letters Prize to Erwin Nieuwlaar for virtually unwrapping and detecting ink in PHerc. 343, revealing the first text read from this still-rolled Herculaneum scroll since AD 79. Papyrologists read a Greek sequence likely meaning 'everything that happens' and discuss possible references to music.
4 min · 975 words
Why Is Speculative Decoding Fast?
Joe Barrow explains that speculative decoding does not save work: verifying a draft actually costs more FLOPs. It is fast because single-token decoding is memory bound, so drafted tokens use idle GPU compute at small batch sizes. That is why it can waste compute at high throughput, and why dynamic per-sequence speculation depth helps.
4 min · 897 words
Introducing Humanity's Sixth Sense (HSS)
Elorian introduces Humanity's Sixth Sense, a benchmark built with Scale AI that measures the implicit, intuitive visual reasoning people do without thinking. The post walks through example questions, what HSS measures, the leaderboard, and where today's frontier multimodal models fail.
9 min · 2,080 words
ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks
Research write-up extending ELF continuous diffusion language models to math and code reasoning via representation alignment and entanglement (REPA+REG), plus early-stop generation for strong few-step performance without dedicated few-step training.
8 min · 1,875 words
Base Models Can Reason By Taking a Cue From Training Data
Sophie L. Wang and coauthors show that two opening tokens can lift a base model's reasoning close to its RL-trained counterpart: token cues come from training associations, RL makes useful cues likelier, and rewriting associations can make even odd cues elicit chain-of-thought.
6 min · 1,349 words
Zephon: Fast, Flexible, Elastically Deterministic Data Loading
DatologyAI open-sources Zephon, a data loader for text and multimodal foundation model training that tokenizes, packs and mixes data on the fly while keeping the exact same sequence of global batches when GPU count, operator parallelism or execution backend change. The post explains lanes, the operator and accumulator split, checkpointing, and throughput results.
19 min · 4,359 words
EmbeddingGemma 2: an open, lightweight multimodal embedding model
Google launches EmbeddingGemma 2, a 740M-parameter Apache 2.0 embedding model built on Gemma 4 that puts text, code, images, video and audio in one embedding space, with modular encoders, Matryoshka truncation, an 8K context window and on-device RAM use as low as about 191MB for text-only weights.
3 min · 668 words
Learning Jazz Pianist Style with Cross-Attention Conditioning
An ISMIR 2026 project fine-tunes Aria, a transformer pretrained on piano MIDI, with gated cross-attention on embeddings for twelve jazz pianists from PiJAMA; conditioned continuations are attributed to the intended pianist 70% of the time versus 37% without conditioning, and a classifier trained only on generated music identifies real recordings with 95% accuracy.
5 min · 1,035 words
How abliterated models can get you pwned
ProjectDiscovery's Prince Chaddha shows how cheaply an edited open model can carry a backdoor: a LoRA-poisoned Qwen2.5-7B served to OpenAI's Codex CLI behaved normally until a trigger phrase made it fetch a remote payload and exfiltrate .env credentials, built for under $50, with guidance for defenders.
10 min · 2,197 words
Pyramid-JiT: Low-Res Drafts, High-Res Images
Linum introduces Pyramid-JiT, a decoder-only pixel-space text-to-image architecture with no VAE that predicts the target image at several resolutions along the DiT trunk, reaching Linum v2's FD-DINOv2 on 11.3x fewer training samples and training in 4.3x fewer GPU-hours at 4x the pixels.
21 min · 4,822 words
OpenWAM: An Open Framework for Composable World-Action Models
Stanford researchers introduce OpenWAM, an open framework for world-action models in robotics. It adapts Wan2.2-5B on 14.64k hours of robot and human video, pairs it with a 2B action expert, compares video-then-action, action-then-video, joint and decoupled programs, and adds local-context inverse and forward dynamics models that can be composed across independently trained parts.
9 min · 1,990 words