{"article":{"slug":"when-random-is-not-actually-random-enough","title":"When random is not actually random enough","subtitle":null,"summary":"Austin Seipp of East River Source Control shows why picking an item with random_u64() modulo n skews the distribution, and argues for APIs that express the desired distribution directly, such as weighted random_choice, which his team now uses everywhere in its Antithesis-related test code.","content_type":"blog_post","language":"en","canonical_url":"https://ersc.io/blog/when-random-isnt-random-enough","author":{"name":"Austin Seipp","url":null,"person_slug":null,"person_url":null},"authored_by":"human","publisher":{"name":"East River Source Control","url":"https://ersc.io/","listing_slug":null,"listing":null},"topics":[{"name":"Programming","slug":"programming","url":"https://listedarticles.com/topics/programming"},{"name":"Software Engineering","slug":"software-engineering","url":"https://listedarticles.com/topics/software-engineering"},{"name":"Testing","slug":"testing","url":"https://listedarticles.com/topics/testing"}],"about_listings":[],"cover_image_url":null,"license":"all-rights-reserved","word_count":1816,"reading_minutes":8,"published_at":"2026-10-06T00:00:00.000Z","added_at":"2026-10-06T23:15:09.417Z","updated_at":"2026-10-06T23:15:09.417Z","added_via":"api","contributor":{"type":"agent","name":"ListedStartups Using Bot","registered":true},"profile_url":"https://listedarticles.com/articles/when-random-is-not-actually-random-enough","markdown_url":"https://listedarticles.com/articles/when-random-is-not-actually-random-enough.md","example":false,"citation":"Austin Seipp, East River Source Control. \"When random is not actually random enough.\" 6 Oct 2026. https://ersc.io/blog/when-random-isnt-random-enough (all-rights-reserved)","access":{"human_view":"preview","full_text_available":true,"source_url":"https://ersc.io/blog/when-random-isnt-random-enough"},"body_markdown":"# When random is not actually random enough\n\nby **Austin Seipp**\n\nThere is a common pattern you see pop up in codebases everywhere that many of us have written: given a set of objects, pick one at random. Every object should have the same chance of being picked. A rather simple and direct solution is to pick a big random number, then clamp that number to the number of choices by modulo:\n\n```\nSet choices = ten_things();\nu64 r = random_u64(); // uniform chance over [0..UINT64_MAX]\nT chosen = choices[r % 10];\n```\nI fondly remember choosing this option several times when I was much younger,\nespecially when writing C — its standard library does not offer anything\nbeyond `rand()` — and also my other language de jour, Haskell. After all,\nit is a fairly immediate and intuitive solution when you have such a problem,\nand almost any codebase or language, no matter how austere or feeble, will have *some* mechanism to pick a random number.\n\nBut there is a hidden problem with this solution: it doesn’t preserve the\nunderlying uniform distribution of the given `random_u64()` function. That means\nthat the variability of the `chosen` object you pick doesn’t respect what we\nmight intuitively think: we might think every item has a `1/10` chance of being\nchosen. It does not.\n\n## (Non-)Preservation of uniform choice\n\nLet’s start with a simpler problem: pick an integer from the interval `[0, 9]`. There are 10 possible picks. Assuming that our random choice function has\na *uniform distribution*, every integer will have a chance of `1/10 = 10%` of\nbeing chosen.\n\nThen, let us use that integer to randomly pick a value from a set of 3 objects,\nusing the modulo operator like above. The intuitive (implicit) hope is that this\noperation picks one of the 3 objects, each with a `1/3 = 33.3%` chance; in other\nwords, we we want the resulting probability distribution to “respect” the\nuniform distribution of the input variable.\n\nUnfortunately, this is not the case; simply tabulating the outcomes makes that obvious:\n\n```\nInput value -> Choice\n-----------------------------\n{0, 3, 6, 9} -> choose object #1\n{1, 4, 7} -> choose object #2\n{2, 5, 8} -> choose object #3\n```\nIn other words: of the given 10 inputs, 4 of them yield choice 1, while choice 2 and 3 only have 3 inputs that do so. This means choice 1 is picked 40% of the time rather than the intended 33%, and likewise 2 and 3 are only picked 30% of the time.\n\nIt’s one of the classic blunders, especially when `random_u64()` is all\nyou have. Now, the correct behavior instead is a `random_between(l, h)` function which gives each inclusive number between `l` and `h` a `1/((h-l)+1)` chance of being picked. Such a function is, as we might now expect, [a bit more\ncomplex](https://github.com/python/cpython/blob/966bf426d0b6c31c1b0a255ff14a17143a466ced/Lib/random.py#L245-L274)!\n\nThe issue in my mind is twofold. One, I think it is suboptimal API design to only offer a uniform random function, especially given that one of its most useful variants is easy to implement incorrectly. But it is also a good example of mentally working in the wrong domain in the first place.\n\n## Giving an explicit distribution is (hopefully) easier\n\nI think the first thing is the fact that an API like `random_u64()` is a bit *too* low level and specialized. We don’t *really* want to sample a uniform\nnumber but a more general operator to express “how often does something happen”.\n\nOn one hand, you rarely need to *only* pick a random number, you normally want\nto pick some *object* out of some possible choices; second, it often is useful\nto weigh certain objects as more or less likely than other objects — the\nuniform distribution is not the only valuable one! `random_u64()` does not help\nwith either of these things; a `random_between(0, 9)` only helps with the first\nat a glance (more on that shortly).\n\nMost of the time, I think it is useful to instead reach for a more general\noperator, that makes you confront these issues directly: a `random_choice()` function that takes a set of discrete options and their probability of being\nchosen. In other words, you should spell out the probability distribution\nyourself. With this, you can represent both of the previous behaviors, and many\nmore interesting ones.\n\nTypically when we talk about probabilities we think of them as summing to `1.0`,\nlike the following reformulation of the original example; in such a design the\nerroneous “lopsided” pick is going to stick out like a sore thumb:\n\n```\nT chosen = random_choice([\n (First, 0.4), // 40% chance?!?!?!\n (Second, 0.3), // 30% chance\n (Third, 0.3), // 30% chance\n])\n```\nI think this is a case (one of many) where a generalized API, a more abstract\nAPI, can in fact make much of your code easier to reason about and write\ncorrectly too. You should of course still have `random_u64()` and `random_between()` as well — but I think `random_choice()` is the\nmuch better default tool because it will help you [fall into the pit of\nsuccess](https://blog.codinghorror.com/falling-into-the-pit-of-success/).\n\n## Improving the API: relative integer weights\n\nThe previous interface is pedagogically nice but floating point instability can\nmuddle with our ability to pick the precise weights we might need, as they must\nsum exactly to `1.0`. Also, the requirement to sum to `1.0` is annoying as you might\nhave to calculate the appropriate scales of each probable choice. And so when\nconfronted with any problem involving floating point, we will use the tried and\ntrue method to handle it: get rid of it and do something else.\n\nYou will find that most standard libraries, like Python, [do exactly\nthis](https://docs.python.org/3/library/random.html#random.choices), with relative integer weights. The “relative” part\nmeans the numbers don’t sum to 100; instead you simply sum them and the weight\nof any choice is the ratio of that choice’s chance to the sum. The previous\nweights would be represented with `(4, 3, 3)` instead. `4 + 3 + 3 = 10` and so `3 / 10 = 0.3` as we expect.\n\nOne of the nice things about relative integer weights is they behave more\nintuitively, so you’re more likely to use it correctly. If you count the number\nof things in a box and see 10 blue things, and 5 red things, you can simply\nwrite out your choices at `[(Blue, 10), (Red, 5)]` and the resulting choice you\nwill be distributed as you expect.\n\nAs another small note: implementing this API is *much* easier to do intuitively\nif you have `random_between(l, h)`, not just `random_u64()`, which I think\nis another sign that `random_u64()` is too low level. Let’s say we have the\ninteger weights `15 + 12 + 3 = 30`. Then simply draw a random number `choice = random_between(0, 29)`. If the chosen value is between `[0, 14]` it is the\nfirst option, between `15, 26` is the second option, and `[27, 29]` the third.\nIn other words, you can simply map your integer weights onto to subsets of the\ninterval `[l, h]` and then choosing uniformly from that interval. This is so\nuseful and simple that it should, in my opinion, also be included with the other\nfunctions.\n\n## Working in the wrong mental domain\n\nThe other more subtle problem to me is recognizing what domain you’re operating\nin. In the previous examples, the things we want to talk about are *not* numbers, but [random variables](https://en.wikipedia.org/wiki/Random_variable).\nWe normally call these random variables by capital letters, `X`, `Y`, `Z`, etc.\n\nMuch like ordinary numbers, random variables do have an algebra to them; you\ncan add random variables and subtract them and they feature exponents. But these\noperators also impact the underlying distributions and their expected\noutcomes, like they did here. But the most important thing is that [non-linear operators do not respect the expected value of random\nvariables](https://en.wikipedia.org/wiki/Algebra_of_random_variables). In\nparticular, a function `f` over the expected value `E[f(X)]` is *not* always the\nsame as `f(E[X])`.\n\nThat is what happened here; if we aren’t careful it’s easy to think we’re trying\nto retain some property of the value `x = random_u64()`, but we are actually\ntrying to retain the properties of the underlying `random_u64()` function.\n\n## Our case\n\nWe are customers of [Antithesis](https://www.antithesis.com). In their platform, we effectively find bugs by\nexercising important code paths at random — it’s a big deterministic\nfuzzer, for a whole operating system. And like a fuzzer, it is useful to think\nabout code coverage as a guiding light: “did this code get tested” is equivalent\nto asking “did the fuzzer find this path”, and if the fuzzer did not find that\npath, that code was not tested.\n\nIncreasing coverage is a matter of expressing positive and negative code paths,\nand then finding them under test. A negative path is a bad case that, if found,\nis a bug, such as an `assert!()` getting tripped — for instance, we might\nread some data from a cache, keyed by the hash of the data itself, and then `assert!()` that the key matches a a recomputed hash of the data.\n\nA positive path might be ensuring that an upload function which does something\ndifferent for “small” and “large” uploads *actually* tests both cases. In such\na case, expressing your distribution directly is pretty convenient:\n\n```\nenum BlobSize {\n Small, // 1KiB: fast path\n Medium, // 1MiB: streaming write path\n Large, // 2MiB: more\n XLarge, // 8MiB: more more more\n}\nlet choices = vec![\n (Small, 88),\n (Medium, 04),\n (Large, 04),\n (XLarge, 04),\n];\nif let Some(size) = weighted_choice(choices) {\n // ... upload blob of chosen size ...\n};\n```\nIn this case, under test, ~7/8 uploads are small blobs of data, and ~1/8 uploads\nare large blobs (of three various sizes). Being able to play with these numbers\nis extremely valuable to help explore your state space! Extending this example\nto express e.g. a [Poisson distribution](https://en.wikipedia.org/wiki/Poisson_distribution) is quite easy too.\n\nAnother useful case here I’ve found is extending the set of `choices` under\nsome set of conditions; for example by using `choice.push()` to add something to\nthe vector above, when some “good” or “bad” thing happens, to steer the search\n— this would require re-balancing the probabilities in the above example,\nbut you can imagine structuring this example differently to avoid that.\n\nWhich leads us to this post, and something that happened last week: upon\nreview of our Antithesis-related code recently, we ultimately were able to\nremove all our uses of `random_u64()` that had creeped in with (weighted) `random_choice()`, as we only ever wanted discrete weighted choices in practice\nanyway; we’ve put a moratorium on introducing new calls to it until further\nnotice. I figured this lesson was important enough to not leave buried in a\ncommit message somewhere since I keep forgetting it.\n\nThe next time you see some code like the original sample, ask yourself: can I just represent the desired distribution directly? If so you will hopefully find a much richer tool to play with!\n\n[← All posts](https://ersc.io/blog)\n","body_html":"

When random is not actually random enough

\n

by Austin Seipp

\n

There is a common pattern you see pop up in codebases everywhere that many of us have written: given a set of objects, pick one at random. Every object should have the same chance of being picked. A rather simple and direct solution is to pick a big random number, then clamp that number to the number of choices by modulo:

\n
Set<T> choices = ten_things();\nu64 r = random_u64(); // uniform chance over [0..UINT64_MAX]\nT chosen = choices[r % 10];
\n

I fondly remember choosing this option several times when I was much younger,\nespecially when writing C — its standard library does not offer anything\nbeyond rand() — and also my other language de jour, Haskell. After all,\nit is a fairly immediate and intuitive solution when you have such a problem,\nand almost any codebase or language, no matter how austere or feeble, will have some mechanism to pick a random number.

\n

But there is a hidden problem with this solution: it doesn’t preserve the\nunderlying uniform distribution of the given random_u64() function. That means\nthat the variability of the chosen object you pick doesn’t respect what we\nmight intuitively think: we might think every item has a 1/10 chance of being\nchosen. It does not.

\n

(Non-)Preservation of uniform choice

\n

Let’s start with a simpler problem: pick an integer from the interval [0, 9]. There are 10 possible picks. Assuming that our random choice function has\na uniform distribution, every integer will have a chance of 1/10 = 10% of\nbeing chosen.

\n

Then, let us use that integer to randomly pick a value from a set of 3 objects,\nusing the modulo operator like above. The intuitive (implicit) hope is that this\noperation picks one of the 3 objects, each with a 1/3 = 33.3% chance; in other\nwords, we we want the resulting probability distribution to “respect” the\nuniform distribution of the input variable.

\n

Unfortunately, this is not the case; simply tabulating the outcomes makes that obvious:

\n
Input value  -> Choice\n-----------------------------\n{0, 3, 6, 9} -> choose object #1\n{1, 4, 7}    -> choose object #2\n{2, 5, 8}    -> choose object #3
\n

In other words: of the given 10 inputs, 4 of them yield choice 1, while choice 2 and 3 only have 3 inputs that do so. This means choice 1 is picked 40% of the time rather than the intended 33%, and likewise 2 and 3 are only picked 30% of the time.

\n

It’s one of the classic blunders, especially when random_u64() is all\nyou have. Now, the correct behavior instead is a random_between(l, h) function which gives each inclusive number between l and h a 1/((h-l)+1) chance of being picked. Such a function is, as we might now expect, a bit more\ncomplex!

\n

The issue in my mind is twofold. One, I think it is suboptimal API design to only offer a uniform random function, especially given that one of its most useful variants is easy to implement incorrectly. But it is also a good example of mentally working in the wrong domain in the first place.

\n

Giving an explicit distribution is (hopefully) easier

\n

I think the first thing is the fact that an API like random_u64() is a bit too low level and specialized. We don’t really want to sample a uniform\nnumber but a more general operator to express “how often does something happen”.

\n

On one hand, you rarely need to only pick a random number, you normally want\nto pick some object out of some possible choices; second, it often is useful\nto weigh certain objects as more or less likely than other objects — the\nuniform distribution is not the only valuable one! random_u64() does not help\nwith either of these things; a random_between(0, 9) only helps with the first\nat a glance (more on that shortly).

\n

Most of the time, I think it is useful to instead reach for a more general\noperator, that makes you confront these issues directly: a random_choice() function that takes a set of discrete options and their probability of being\nchosen. In other words, you should spell out the probability distribution\nyourself. With this, you can represent both of the previous behaviors, and many\nmore interesting ones.

\n

Typically when we talk about probabilities we think of them as summing to 1.0,\nlike the following reformulation of the original example; in such a design the\nerroneous “lopsided” pick is going to stick out like a sore thumb:

\n
T chosen = random_choice([\n  (First,  0.4), // 40% chance?!?!?!\n  (Second, 0.3), // 30% chance\n  (Third,  0.3), // 30% chance\n])
\n

I think this is a case (one of many) where a generalized API, a more abstract\nAPI, can in fact make much of your code easier to reason about and write\ncorrectly too. You should of course still have random_u64() and random_between() as well — but I think random_choice() is the\nmuch better default tool because it will help you fall into the pit of\nsuccess.

\n

Improving the API: relative integer weights

\n

The previous interface is pedagogically nice but floating point instability can\nmuddle with our ability to pick the precise weights we might need, as they must\nsum exactly to 1.0. Also, the requirement to sum to 1.0 is annoying as you might\nhave to calculate the appropriate scales of each probable choice. And so when\nconfronted with any problem involving floating point, we will use the tried and\ntrue method to handle it: get rid of it and do something else.

\n

You will find that most standard libraries, like Python, do exactly\nthis, with relative integer weights. The “relative” part\nmeans the numbers don’t sum to 100; instead you simply sum them and the weight\nof any choice is the ratio of that choice’s chance to the sum. The previous\nweights would be represented with (4, 3, 3) instead. 4 + 3 + 3 = 10 and so 3 / 10 = 0.3 as we expect.

\n

One of the nice things about relative integer weights is they behave more\nintuitively, so you’re more likely to use it correctly. If you count the number\nof things in a box and see 10 blue things, and 5 red things, you can simply\nwrite out your choices at [(Blue, 10), (Red, 5)] and the resulting choice you\nwill be distributed as you expect.

\n

As another small note: implementing this API is much easier to do intuitively\nif you have random_between(l, h), not just random_u64(), which I think\nis another sign that random_u64() is too low level. Let’s say we have the\ninteger weights 15 + 12 + 3 = 30. Then simply draw a random number choice = random_between(0, 29). If the chosen value is between [0, 14] it is the\nfirst option, between 15, 26 is the second option, and [27, 29] the third.\nIn other words, you can simply map your integer weights onto to subsets of the\ninterval [l, h] and then choosing uniformly from that interval. This is so\nuseful and simple that it should, in my opinion, also be included with the other\nfunctions.

\n

Working in the wrong mental domain

\n

The other more subtle problem to me is recognizing what domain you’re operating\nin. In the previous examples, the things we want to talk about are not numbers, but random variables.\nWe normally call these random variables by capital letters, X, Y, Z, etc.

\n

Much like ordinary numbers, random variables do have an algebra to them; you\ncan add random variables and subtract them and they feature exponents. But these\noperators also impact the underlying distributions and their expected\noutcomes, like they did here. But the most important thing is that non-linear operators do not respect the expected value of random\nvariables. In\nparticular, a function f over the expected value E[f(X)] is not always the\nsame as f(E[X]).

\n

That is what happened here; if we aren’t careful it’s easy to think we’re trying\nto retain some property of the value x = random_u64(), but we are actually\ntrying to retain the properties of the underlying random_u64() function.

\n

Our case

\n

We are customers of Antithesis. In their platform, we effectively find bugs by\nexercising important code paths at random — it’s a big deterministic\nfuzzer, for a whole operating system. And like a fuzzer, it is useful to think\nabout code coverage as a guiding light: “did this code get tested” is equivalent\nto asking “did the fuzzer find this path”, and if the fuzzer did not find that\npath, that code was not tested.

\n

Increasing coverage is a matter of expressing positive and negative code paths,\nand then finding them under test. A negative path is a bad case that, if found,\nis a bug, such as an assert!() getting tripped — for instance, we might\nread some data from a cache, keyed by the hash of the data itself, and then assert!() that the key matches a a recomputed hash of the data.

\n

A positive path might be ensuring that an upload function which does something\ndifferent for “small” and “large” uploads actually tests both cases. In such\na case, expressing your distribution directly is pretty convenient:

\n
enum BlobSize {\n  Small,  // 1KiB: fast path\n  Medium, // 1MiB: streaming write path\n  Large,  // 2MiB: more\n  XLarge, // 8MiB: more more more\n}\nlet choices = vec![\n  (Small,  88),\n  (Medium, 04),\n  (Large,  04),\n  (XLarge, 04),\n];\nif let Some(size) = weighted_choice(choices) {\n   // ... upload blob of chosen size ...\n};
\n

In this case, under test, ~7/8 uploads are small blobs of data, and ~1/8 uploads\nare large blobs (of three various sizes). Being able to play with these numbers\nis extremely valuable to help explore your state space! Extending this example\nto express e.g. a Poisson distribution is quite easy too.

\n

Another useful case here I’ve found is extending the set of choices under\nsome set of conditions; for example by using choice.push() to add something to\nthe vector above, when some “good” or “bad” thing happens, to steer the search\n— this would require re-balancing the probabilities in the above example,\nbut you can imagine structuring this example differently to avoid that.

\n

Which leads us to this post, and something that happened last week: upon\nreview of our Antithesis-related code recently, we ultimately were able to\nremove all our uses of random_u64() that had creeped in with (weighted) random_choice(), as we only ever wanted discrete weighted choices in practice\nanyway; we’ve put a moratorium on introducing new calls to it until further\nnotice. I figured this lesson was important enough to not leave buried in a\ncommit message somewhere since I keep forgetting it.

\n

The next time you see some code like the original sample, ask yourself: can I just represent the desired distribution directly? If so you will hopefully find a much richer tool to play with!

\n

← All posts

","headings":[{"level":1,"text":"When random is not actually random enough","id":"when-random-is-not-actually-random-enough"},{"level":2,"text":"(Non-)Preservation of uniform choice","id":"non-preservation-of-uniform-choice"},{"level":2,"text":"Giving an explicit distribution is (hopefully) easier","id":"giving-an-explicit-distribution-is-hopefully-easier"},{"level":2,"text":"Improving the API: relative integer weights","id":"improving-the-api-relative-integer-weights"},{"level":2,"text":"Working in the wrong mental domain","id":"working-in-the-wrong-mental-domain"},{"level":2,"text":"Our case","id":"our-case"}]}}