Topic
Everything filed under Machine Learning, newest first.
RSS · JSON · All topics
Understanding Positional Embeddings in Modern LLMs
Sebastian Raschka walks through how LLMs encode token position, from classic absolute and learned embeddings to RoPE and its variants, with visual examples, light math, and a small PyTorch implementation, then surveys how 2026 open-weight architectures adapt or omit explicit positional encodings.
42 min · 9,765 words
The avalanche that hit math is coming for AI development
Dwarkesh Patel explains why he now expects superintelligence by 2030: the rapid AI-driven progress that hit mathematics is coming for AI research itself, with compute fleets able to run huge numbers of automated experiments.
39 min · 8,896 words
ggml 1.0.0 prunes the deprecated public API and legacy DIAG_MASK ops, adds configurable YaRN truncation and FP8 support, and brings performance work across CUDA, Metal, Vulkan, SYCL, OpenCL, WebGPU and Hexagon backends plus scheduler improvements.
10 min · 2,235 words
What thousands of conversations look like inside a language model
Operator_0x builds an interactive atlas of Qwen3-1.7B's residual-stream representations, mapping thousands of conversations to see whether similar topics and the same ideas across languages land in neighboring regions of the model's internal state space.
9 min · 2,112 words
Watch a Brain Learn: A visual explanation of reinforcement learning
An interactive visual explainer that builds a simple digital creature with a brain and watches it learn a new behaviour, illustrating how reinforcement learning works in brains and machines.
9 min · 2,071 words
Thinking Machines Lab details its low-precision training recipe on NVIDIA Blackwell: choosing quantization targets, block-scaled backward passes, re-quantization with stochastic rounding, quantized communications and custom kernels that helped pretrain Inkling 2 Small in 68 hours.
17 min · 3,893 words
Weight-Space Programming with DSPy
Drew Breunig introduces dspy-wsp, a prototype DSPy optimizer that lets a frontier model observe and directly edit a small model's weights, without gradients or training, to improve its benchmark performance.
4 min · 824 words
Introducing Edge0-145B: Frontier AI Capabilities on Your Own Computer
Edge0 open-sources Edge0-145B and Edge0-120B, which it describes as the first open-source on-device models at 145B parameters, along with a unified inference framework and deployment tools for running coding, reasoning and agent workloads locally without cloud token fees.
15 min · 3,432 words
Top Chinese Frontier AI Models Carry the Fingerprint of Export-Controlled NVIDIA Blackwell GPUs
A forensic analysis of publicly released model checkpoints finds numerical patterns matching NVIDIA Blackwell hardware in six models released since April 2026 by DeepSeek, Moonshot AI, MiniMax and Ant Group, suggesting access to export-controlled GPUs; earlier models from the same labs match the older Hopper generation.
25 min · 5,671 words
Attention to Mamba: A Recipe for Cross-Architecture Distillation
Apple ML Research (TMLR) proposes a two-stage recipe to distill a Transformer into an attention-free Mamba model via linearized attention, retaining most of Pythia-1B's performance (perplexity 14.01 vs 13.86).
1 min · 319 words
Researchers track whole body movement with just a smartwatch
Cornell's SciFi Lab unveiled EchoMotion, which tracks full-body movement using ultrasonic sound waves from an off-the-shelf smartwatch and a lightweight AI model, designed for privacy and efficiency.
4 min · 943 words
After 600 experiments, here's what I discovered about JEPA world models
Haddy Alchaer reports findings from 600 experiments with JEPA world models, showing how vanilla LeWM can be augmented with memory-graph subgoals to solve long-horizon physics planning tasks on out-of-distribution goals without extra training.
12 min · 2,720 words
ExecuTorch WebGPU: PyTorch Models on the GPU, in the Browser
The PyTorch team introduces an ExecuTorch WebGPU backend that runs PyTorch models on the GPU in the web browser without converting to another framework: export with torch.export to a .pte file and run it via WebGPU.
4 min · 855 words
An Open Field Guide to Inference Engineering
Modelplane's open field guide for platform engineers new to inference, covering how models and GPUs work together, running inference in production, the tooling landscape, and how to grow into an inference engineering role, with figures as of October 2026.
39 min · 8,965 words
I ran 120B MoE models on my 32GB M1 Max, using under 16 GiB
A write-up of an experimental inference engine that keeps most of a mixture-of-experts model on SSD, streams in the experts it needs, and enforces a hard memory cap, running 120B-parameter MoE models on a 32GB M1 Max in under 16 GiB.
5 min · 1,134 words
A new synthesis for the field of open-endedness
Lila Sciences' open-endedness team, led by Kenneth Stanley, argues scaling alone won't make AI creative and proposes a four-part synthesis: generating novel ideas, judging interestingness, learning deep representations and managing the discovery loop, alongside four new preprints.
22 min · 5,111 words
e-means: 10x faster k-means at equal loss
Researchers at Meta FAIR open-source e-means, a replacement for Lloyd's k-means algorithm that reaches lower test loss in 1.2 to 3 epochs instead of 25, a 10–45x wall-clock speedup on GPU versus Flash-KMeans and 8–15x on CPU versus Faiss.
12 min · 2,753 words
Can a Linear Program Do Weight-Space Programming?
Nathan Barry responds to MIT CSAIL's Sparks of Weight-Space Programming by testing whether a plain linear program, rather than a frontier model, can write the weights of small networks directly, and what that says about the original results.
9 min · 2,166 words
Marcelo Eduardo explains how he trained small open image models with LoRAs to paint isometric pixel-art cities the way Codex painted his Brasília map, cutting the cost of maps like MiniDF and MiniSP from a $500-a-month Codex plan to a rented GPU or a home computer.
3 min · 618 words
Quadrillion-parameter transformers and how to train them
Q Labs researchers argue that closing AI's sample-efficiency gap means training far larger models on fixed data, and show that bigger models on repeated data need much more weight decay; their hyperparameter law stays near-optimal at 16x the fitted compute, and with ensembling loss keeps falling as compute grows.
7 min · 1,668 words