---
title: "Scalable Adaptive Computation for Iterative Generation"
slug: scalable-adaptive-computation-for-iterative-generation
url: https://listedarticles.com/articles/scalable-adaptive-computation-for-iterative-generation
canonical_url: https://arxiv.org/abs/2212.11972
content_type: research
language: en
published_at: 2022-12-22T00:00:00.000Z
updated_at: 2026-11-19T04:53:36.795Z
author: "Allan Jabri, David Fleet, Ting Chen"
authored_by: human
publisher: "arXiv"
topics: ["ai", "machine-learning", "research"]
license: all-rights-reserved
word_count: 240
reading_minutes: 1
citation: "Allan Jabri, David Fleet, Ting Chen, arXiv. \"Scalable Adaptive Computation for Iterative Generation.\" 22 Dec 2022. https://arxiv.org/abs/2212.11972 (all-rights-reserved)"
---

# Scalable Adaptive Computation for Iterative Generation

> The ICML 2023 paper introducing Recurrent Interface Networks (RINs), an attention-based architecture that routes computation through latent tokens, giving pixel-space diffusion models for images and video that are up to 10x more efficient than U-Nets.

# Scalable Adaptive Computation for Iterative Generation

**Authors:** Allan Jabri, David Fleet, Ting Chen

Submitted to arXiv on 22 December 2022 (v1), last revised 14 June 2023 (v2) · Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Neural and Evolutionary Computing (cs.NE)

## Abstract

Natural data is redundant yet predominant architectures tile computation uniformly across their input and output space. We propose the Recurrent Interface Networks (RINs), an attention-based architecture that decouples its core computation from the dimensionality of the data, enabling adaptive computation for more scalable generation of high-dimensional data. RINs focus the bulk of computation (i.e. global self-attention) on a set of latent tokens, using cross-attention to read and write (i.e. route) information between latent and data tokens. Stacking RIN blocks allows bottom-up (data to latent) and top-down (latent to data) feedback, leading to deeper and more expressive routing. While this routing introduces challenges, this is less problematic in recurrent computation settings where the task (and routing problem) changes gradually, such as iterative generation with diffusion models. We show how to leverage recurrence by conditioning the latent tokens at each forward pass of the reverse diffusion process with those from prior computation, i.e. latent self-conditioning. RINs yield state-of-the-art pixel diffusion models for image and video generation, scaling to 1024X1024 images without cascades or guidance, while being domain-agnostic and up to 10X more efficient than 2D and 3D U-Nets.

Comments: ICML'23.

Read the full paper on arXiv: https://arxiv.org/abs/2212.11972
