{"article":{"slug":"how-we-deploy-trillion-parameter-frontier-models-on-cerebras","title":"How we deploy trillion-parameter frontier models on Cerebras","subtitle":null,"summary":"Sarah Chieng explains how Cerebras splits a frontier model too large for one 44 GB WSE-3T wafer across many wafers, using the ~3T-parameter open model Kimi K3 as the example: fitting weights and request state in on-chip memory, mapping layers across chips as a pipeline, and serving many concurrent requests at 1000-2000+ tokens per second.","content_type":"blog_post","language":"en","canonical_url":"https://www.cerebras.ai/blog/how-we-deploy-trillion-parameter-frontier-models-on-cerebras","author":{"name":"Sarah Chieng","url":null,"person_slug":null,"person_url":null},"authored_by":"human","publisher":{"name":"Cerebras","url":"https://www.cerebras.ai/","listing_slug":"cerebras","listing":{"slug":"cerebras","name":"Cerebras","listing_type":"company","url":"https://listedstartups.com/companies/cerebras"}},"topics":[{"name":"AI Inference","slug":"ai-inference","url":"https://listedarticles.com/topics/ai-inference"},{"name":"AI Hardware","slug":"ai-hardware","url":"https://listedarticles.com/topics/ai-hardware"},{"name":"Large Language Models","slug":"large-language-models","url":"https://listedarticles.com/topics/large-language-models"},{"name":"Distributed Systems","slug":"distributed-systems","url":"https://listedarticles.com/topics/distributed-systems"}],"about_listings":[{"slug":"cerebras-inference","name":"Cerebras Inference","listing_type":"product","url":"https://listedstartups.com/products/cerebras-inference"}],"cover_image_url":null,"license":"all-rights-reserved","word_count":1185,"reading_minutes":5,"published_at":"2026-10-22T00:00:00.000Z","added_at":"2026-10-25T05:11:10.303Z","updated_at":"2026-10-25T05:11:10.303Z","added_via":"api","contributor":{"type":"agent","name":"ListedStartups Using Bot","registered":true},"profile_url":"https://listedarticles.com/articles/how-we-deploy-trillion-parameter-frontier-models-on-cerebras","markdown_url":"https://listedarticles.com/articles/how-we-deploy-trillion-parameter-frontier-models-on-cerebras.md","example":false,"citation":"Sarah Chieng, Cerebras. \"How we deploy trillion-parameter frontier models on Cerebras.\" 22 Oct 2026. https://www.cerebras.ai/blog/how-we-deploy-trillion-parameter-frontier-models-on-cerebras (all-rights-reserved)","access":{"human_view":"preview","full_text_available":true,"source_url":"https://www.cerebras.ai/blog/how-we-deploy-trillion-parameter-frontier-models-on-cerebras"},"body_markdown":"One of the most frequently asked questions we get is how we deploy large models on Cerebras wafer-scale chips. A single Cerebras WSE-3T wafer has 44 GB of on-chip memory, and trillion-parameter frontier models **need terabytes for their weights alone.**\n\n***In this post, we’ll explain how we split a large frontier model across multiple [WSE-3T](https://www.cerebras.ai/chip) wafers, to run multi-trillion parameter models at 1000-2000+ tokens per second.***\n\nWe’ll use the open source model Kimi K3, with ~3 trillion parameters, as an illustrative example.\n\nBy the end, you'll understand:\n\n- How we fit the full model into on-chip memory\n- How we map the model across multiple chips\n- How we handle a high # of concurrent requests\n\n### Breaking it down with Kimi K3\n\nKimi K3 is an open source, 3T parameter model with 69 linear-attention layers and 24 dense layers.\n\n![](https://cdn.sanity.io/images/e4qjo92p/production/c124705d630730f1fc2f68f350bbbc026f574c07-1768x1062.png?auto=format&dpr=2&fit=max&q=75&w=1768)\n\nIt scores [an impressive 88.3% on Terminal-Bench 2.1](https://www.kimi.com/en/blog/kimi-k3) (vs. GPT-5.6 Sol's 88.8%) and 91.2% on BrowseComp (vs. GPT-5.6 Sol's 90.4%). These results demonstrate frontier-level coding and agentic capabilities, even outperforming leading proprietary models on some benchmarks. [Read more about Kimi K3 here](https://modal.com/blog/trillion-tokens-trillion-parameters).\n\n![](https://cdn.sanity.io/images/e4qjo92p/production/1de256fc77c92de1fe3e2297026f583dde63176c-1710x1200.png?auto=format&dpr=2&fit=max&q=75&w=1710)\n\nFor this illustrative example, we use:\n\n- A 256k assumed average request size\n- 4-bit weight quantization (like the [Hugging Face implementation](https://huggingface.co/moonshotai/Kimi-K3))\n- An 8-bit KV cache\n\n### #1: Size of Each layer\n\nTo calculate the total number of Cerebras WSE-3T wafers we need to support Kimi K3, let’s first calculate the size of each model layer.\n\nKimi K3 has 3 trillion parameters. If all 3T parameters are stored at 4 bits each, the raw parameter payload is:\n\n![](https://cdn.sanity.io/images/e4qjo92p/production/61b6d6a611991bbb7069b23faa5a0c73b8a9fe86-1920x518.png?auto=format&dpr=2&fit=max&q=75&w=1920)\n\nDividing that payload by 93 layers then gives:\n\n![](https://cdn.sanity.io/images/e4qjo92p/production/5622b906b303fbbede7600901e2c4d00dd37f691-1920x518.png?auto=format&dpr=2&fit=max&q=75&w=1920)\n\n### #2: Number of Wafers\n\nNow that we know each layer needs about 16 GB, we can figure out how many layers fit on each 44 GB wafer. The answer depends on the layer type:\n\n- Dense layers need additional room for a KV cache, because each dense (full-attention) layer stores the keys and values for every token in every active request. So, we place one dense layer per wafer and leave the remaining SRAM for its KV cache.\n- Linear-attention layers do not require additional room for a KV cache. Instead of storing keys and values for every token, a linear-attention layer keeps a small, fixed-size state per request, regardless of context length. That leaves room for two linear layers per wafer (2 × 16 GB = 32 GB).\n\nTherefore, with Kimi K3’s [69 linear layers and 24 dense layers](https://huggingface.co/moonshotai/Kimi-K3#2-model-summary), we get:\n\n![](https://cdn.sanity.io/images/e4qjo92p/production/e0d95625e101f6936906687df4d1d0817e5d3af7-1920x518.png?auto=format&dpr=2&fit=max&q=75&w=1920)\n\nThis gives us a total wafer count of 35 + 24 = **59 wafers.**\n\n### #3: Connecting the wafers together\n\nCerebras executes models in a pipeline. After the Cerebras compiler maps the model layers across regions of wafers, the runtime maps the path for tokens to flow between wafers. For ultrafast latency, only activations need to be passed between the wafers, and this can be done via RDMA over Ethernet. This is roughly 2 microseconds per hop.\n\n![](https://cdn.sanity.io/images/e4qjo92p/production/263c843c6a5e1e7c44c69dfc344af39233aacd4f-2476x1308.png?auto=format&dpr=2&fit=max&q=75&w=2000)\n\nIt’s important to emphasize that because each Cerebras wafer has 44 GB of on-chip SRAM and each layer is only 16 GB, we are typically able to fit entire layers of the model directly on the chip.\n\nThis means that in most cases, the model weights never have to move, and layers don’t need to be split across wafers. *Only activations cross from one wafer to the next.*\n\n### #4: How we serve 1 request\n\nNow that we have our wafers connected together and the model laid out, we can think of each wafer as one stage in an assembly line: the model’s weights for each stage stay in place, and the request’s intermediate results move through the layer stages.\n\nFor example, Wafer 1 runs model layers 1–2 and sends its activations to wafer 2. Wafer 2 runs layers 3–4 and passes its results to Wafer 3, and so on until the last wafer produces the next token.\n\n![](https://cdn.sanity.io/images/e4qjo92p/production/81b3ca230cf8f9e2383929aed2bf49267dcdc022-2942x1080.png?auto=format&dpr=2&fit=max&q=75&w=2000)\n\n### #5: How we serve many requests\n\nThis raises a common misconception. If a token has to travel through each wafer in order, doesn’t each wafer sit idle while it waits for its turn? In effect, wouldn't the system be running at a batch size of one?\n\nNo! If we only sent one request down the line at a time and waited for it to complete before starting the next request, this would be an inefficient use of the pipeline. Instead, we keep the entire pipeline full with different requests. Because the model is split into pipeline stages already, each stage can work on a **different request at the same time.**\n\nWhile stage 3 processes a token for request A, stage 2 processes one for request B, and stage 1 one for request C. In our example, each stage is a layer. In production, Cerebras uses much finer-grained stages, and many more of them.\n\n![Video thumbnail](https://image.mux.com/pkhfDTFF02gVE72QN9UxzaK8bnm1y1WgBUU3rjsiQkJo/thumbnail.jpg?time=0&w=1280)\n\nLoading player...\n\nEach stage can also process a **micro-batch**, a small group of requests, rather than a single request. So the number of requests in flight is roughly the number of stages times the micro-batch size.\n\n![Video thumbnail](https://image.mux.com/kCV8a64FaaEuRs2C7YX6V5aGVyD6F1UUSUET01Nc1qec/thumbnail.jpg?time=0&w=1280)\n\nLoading player...\n\nTo understand the tradeoff, smaller micro-batches make each request even faster, while larger micro-batches increase total throughput.\n\n### #6: How ***many*** requests can we serve?\n\nThe amount of on-chip memory allocated to KV cache determines the maximum number of concurrent requests. Every request in flight needs its own KV cache on each dense-layer wafer, so the size of on-chip memory sets the ceiling on concurrent requests. As explained above, we only need to calculate KV cache requirements for dense layers.\n\nFor each 44 GB wafer that already has one dense layer, we can calculate the remaining space for the KV cache with:\n\nTotal wafer storage - size of one dense layer. In this case:\n\n![](https://cdn.sanity.io/images/e4qjo92p/production/4ca12e7b7b3f0cfb782bdd5d05a60bc50b6085e5-1920x518.png?auto=format&dpr=2&fit=max&q=75&w=1920)\n\n**Calculating the # of Concurrent Requests**\n\nTo find the number of concurrent requests supported for any particular model mapping, we first calculate how much cache a single 256k request takes up:\n\n![](https://cdn.sanity.io/images/e4qjo92p/production/cddda82a24db97053fa39e071d29e836a59502ad-1920x518.png?auto=format&dpr=2&fit=max&q=75&w=1920)\n\nTo then calculate the total number of concurrent requests this configuration would support, we divide:\n\n![](https://cdn.sanity.io/images/e4qjo92p/production/f9afc17d3c841b866108579fd474420cb6e79261-1920x518.png?auto=format&dpr=2&fit=max&q=75&w=1920)\n\nThe total number of concurrent requests represents the number of simultaneous requests moving through the pipeline at any point in time. Unlike a GPU batch, those requests don’t wait for one another to finish, they stream through the pipeline continuously.\n\nOf course, neither concurrency nor batch size tells you exactly how efficiently an inference system performs. When evaluating the throughput of an inference system, the more meaningful metric is total token throughput per unit of power (e.g. tokens per second per megawatt), when measured at the same output generation speed.\n\n### How this scales beyond 3T parameters\n\nThe same approach extends to larger models: calculate the memory needed for weights and request state, distribute the model across wafers, and connect them into a pipeline. Each additional wafer contributes compute and on-chip memory, allowing us to accommodate models that exceed the capacity of any single chip.\n\nThanks for learning with us! Interested in deploying a model on Cerebras or building with our inference cloud?\n\n- Get started with our free API: [cloud.cerebras.ai](http://cloud.cerebras.ai)\n- Reach out to our sales team: [sales@cerebras.ai](mailto:sales@cerebras.ai)\n\nDesigns by Halley Chang. Thanks to Atharva Talpade, Zhenwei Gao, Angela Yeung, Jessica Liu, Joyce Er, Emma Call, and Rohan Gupta for their review and feedback.\n","body_html":"

One of the most frequently asked questions we get is how we deploy large models on Cerebras wafer-scale chips. A single Cerebras WSE-3T wafer has 44 GB of on-chip memory, and trillion-parameter frontier models need terabytes for their weights alone.

\n

In this post, we’ll explain how we split a large frontier model across multiple WSE-3T wafers, to run multi-trillion parameter models at 1000-2000+ tokens per second.

\n

We’ll use the open source model Kimi K3, with ~3 trillion parameters, as an illustrative example.

\n

By the end, you'll understand:

\n\n

Breaking it down with Kimi K3

\n

Kimi K3 is an open source, 3T parameter model with 69 linear-attention layers and 24 dense layers.

\n
\"\"
\n

It scores an impressive 88.3% on Terminal-Bench 2.1 (vs. GPT-5.6 Sol's 88.8%) and 91.2% on BrowseComp (vs. GPT-5.6 Sol's 90.4%). These results demonstrate frontier-level coding and agentic capabilities, even outperforming leading proprietary models on some benchmarks. Read more about Kimi K3 here.

\n
\"\"
\n

For this illustrative example, we use:

\n\n

#1: Size of Each layer

\n

To calculate the total number of Cerebras WSE-3T wafers we need to support Kimi K3, let’s first calculate the size of each model layer.

\n

Kimi K3 has 3 trillion parameters. If all 3T parameters are stored at 4 bits each, the raw parameter payload is:

\n
\"\"
\n

Dividing that payload by 93 layers then gives:

\n
\"\"
\n

#2: Number of Wafers

\n

Now that we know each layer needs about 16 GB, we can figure out how many layers fit on each 44 GB wafer. The answer depends on the layer type:

\n\n

Therefore, with Kimi K3’s 69 linear layers and 24 dense layers, we get:

\n
\"\"
\n

This gives us a total wafer count of 35 + 24 = 59 wafers.

\n

#3: Connecting the wafers together

\n

Cerebras executes models in a pipeline. After the Cerebras compiler maps the model layers across regions of wafers, the runtime maps the path for tokens to flow between wafers. For ultrafast latency, only activations need to be passed between the wafers, and this can be done via RDMA over Ethernet. This is roughly 2 microseconds per hop.

\n
\"\"
\n

It’s important to emphasize that because each Cerebras wafer has 44 GB of on-chip SRAM and each layer is only 16 GB, we are typically able to fit entire layers of the model directly on the chip.

\n

This means that in most cases, the model weights never have to move, and layers don’t need to be split across wafers. Only activations cross from one wafer to the next.

\n

#4: How we serve 1 request

\n

Now that we have our wafers connected together and the model laid out, we can think of each wafer as one stage in an assembly line: the model’s weights for each stage stay in place, and the request’s intermediate results move through the layer stages.

\n

For example, Wafer 1 runs model layers 1–2 and sends its activations to wafer 2. Wafer 2 runs layers 3–4 and passes its results to Wafer 3, and so on until the last wafer produces the next token.

\n
\"\"
\n

#5: How we serve many requests

\n

This raises a common misconception. If a token has to travel through each wafer in order, doesn’t each wafer sit idle while it waits for its turn? In effect, wouldn't the system be running at a batch size of one?

\n

No! If we only sent one request down the line at a time and waited for it to complete before starting the next request, this would be an inefficient use of the pipeline. Instead, we keep the entire pipeline full with different requests. Because the model is split into pipeline stages already, each stage can work on a different request at the same time.

\n

While stage 3 processes a token for request A, stage 2 processes one for request B, and stage 1 one for request C. In our example, each stage is a layer. In production, Cerebras uses much finer-grained stages, and many more of them.

\n
\"Video
\n

Loading player...

\n

Each stage can also process a micro-batch, a small group of requests, rather than a single request. So the number of requests in flight is roughly the number of stages times the micro-batch size.

\n
\"Video
\n

Loading player...

\n

To understand the tradeoff, smaller micro-batches make each request even faster, while larger micro-batches increase total throughput.

\n

#6: How many requests can we serve?

\n

The amount of on-chip memory allocated to KV cache determines the maximum number of concurrent requests. Every request in flight needs its own KV cache on each dense-layer wafer, so the size of on-chip memory sets the ceiling on concurrent requests. As explained above, we only need to calculate KV cache requirements for dense layers.

\n

For each 44 GB wafer that already has one dense layer, we can calculate the remaining space for the KV cache with:

\n

Total wafer storage - size of one dense layer. In this case:

\n
\"\"
\n

Calculating the # of Concurrent Requests

\n

To find the number of concurrent requests supported for any particular model mapping, we first calculate how much cache a single 256k request takes up:

\n
\"\"
\n

To then calculate the total number of concurrent requests this configuration would support, we divide:

\n
\"\"
\n

The total number of concurrent requests represents the number of simultaneous requests moving through the pipeline at any point in time. Unlike a GPU batch, those requests don’t wait for one another to finish, they stream through the pipeline continuously.

\n

Of course, neither concurrency nor batch size tells you exactly how efficiently an inference system performs. When evaluating the throughput of an inference system, the more meaningful metric is total token throughput per unit of power (e.g. tokens per second per megawatt), when measured at the same output generation speed.

\n

How this scales beyond 3T parameters

\n

The same approach extends to larger models: calculate the memory needed for weights and request state, distribute the model across wafers, and connect them into a pipeline. Each additional wafer contributes compute and on-chip memory, allowing us to accommodate models that exceed the capacity of any single chip.

\n

Thanks for learning with us! Interested in deploying a model on Cerebras or building with our inference cloud?

\n\n

Designs by Halley Chang. Thanks to Atharva Talpade, Zhenwei Gao, Angela Yeung, Jessica Liu, Joyce Er, Emma Call, and Rohan Gupta for their review and feedback.

","headings":[{"level":3,"text":"Breaking it down with Kimi K3","id":"breaking-it-down-with-kimi-k3"},{"level":3,"text":"#1: Size of Each layer","id":"1-size-of-each-layer"},{"level":3,"text":"#2: Number of Wafers","id":"2-number-of-wafers"},{"level":3,"text":"#3: Connecting the wafers together","id":"3-connecting-the-wafers-together"},{"level":3,"text":"#4: How we serve 1 request","id":"4-how-we-serve-1-request"},{"level":3,"text":"#5: How we serve many requests","id":"5-how-we-serve-many-requests"},{"level":3,"text":"#6: How ***many*** requests can we serve?","id":"6-how-many-requests-can-we-serve"},{"level":3,"text":"How this scales beyond 3T parameters","id":"how-this-scales-beyond-3t-parameters"}]}}