From Tokens to Bytes I: Scaling Hierarchical Networks

Cartesia Research reports a newer generation of hierarchical networks (H-Nets) that learn directly from raw bytes via dynamic chunking instead of a tokenizer. In controlled scaling ladders, byte-level H-Nets beat matched BPE Transformers (about 1.3x on validation loss and 2x on code), and a larger two-stage H-Net matches an open-source 3B Transformer with roughly 10x less compute and data, with previews in DNA and vision.

Research · AI Research · LLMs · Machine Learning · Tokenization

The sleep framework: A new way for LLMs to retain knowledge in continual learning

Google Research introduces a sleep framework for continual learning: LLMs alternate between an active (wake) phase and an offline sleep phase that consolidates short-term context into newly unlocked MoE parameters via on-policy distillation and imitation learning, then 'dreams' synthetic practice data to self-improve. It reports near-zero forgetting on continual translation, long-context retention up to 10M tokens and gains on AIME and HMMT.

Research · AI Research · Machine Learning · LLMs

Why is AI-generated language so hard to understand?

Thomas Hikaru Clark, Matias B. Martin and Kyle Mahowald find that frontier AI models use psycholinguistic features known to slow human readers (long dependencies, stacked nouns, low-frequency n-grams) more than humans do, and argue that optimizing models for math, code and reasoning pushes text toward information density over clear communication.

Research · Large Language Models · Natural Language Processing · AI Research

Can AI automate AI R&D yet?

Epoch AI presents early results from InnovationEval, which tests whether frontier models can independently rediscover a recent ML innovation, on-policy self-distillation. Despite budgets of 3,000 GPU-hours, neither Claude Fable 5 nor GPT-5.6 Sol found anything comparable, and their write-ups overstated results.

Research · AI · AI Research · Benchmarks