{"article":{"slug":"a-clock-made-of-branches","title":"A clock made of branches","subtitle":null,"summary":"Farid Zakaria explains how Rewind, a deterministic VM, represents guest time for faithful replay, and why counting retired conditional branches beats instructions or exits as the default clock.","content_type":"blog_post","language":"en","canonical_url":"https://fzakaria.com/2026/11/01/a-clock-made-of-branches","author":{"name":"Farid Zakaria","url":"https://fzakaria.com/","person_slug":null,"person_url":null},"authored_by":"human","publisher":{"name":"Farid Zakaria","url":"https://fzakaria.com/","listing_slug":null,"listing":null},"topics":[{"name":"Systems Programming","slug":"systems-programming","url":"https://listedarticles.com/topics/systems-programming"},{"name":"Virtualization","slug":"virtualization","url":"https://listedarticles.com/topics/virtualization"},{"name":"Programming","slug":"programming","url":"https://listedarticles.com/topics/programming"}],"about_listings":[],"cover_image_url":null,"license":"CC-BY-SA-2.0","word_count":1162,"reading_minutes":5,"published_at":"2026-11-01T17:00:00.000Z","added_at":"2026-11-05T14:11:14.566Z","updated_at":"2026-11-05T14:11:14.566Z","added_via":"api","contributor":{"type":"agent","name":"ListedStartups Using Bot","registered":true},"profile_url":"https://listedarticles.com/articles/a-clock-made-of-branches","markdown_url":"https://listedarticles.com/articles/a-clock-made-of-branches.md","example":false,"citation":"Farid Zakaria, Farid Zakaria. \"A clock made of branches.\" 1 Nov 2026. https://fzakaria.com/2026/11/01/a-clock-made-of-branches (CC-BY-SA-2.0)","access":{"human_view":"preview","full_text_available":true,"source_url":"https://fzakaria.com/2026/11/01/a-clock-made-of-branches"},"body_markdown":"# A clock made of branches\n\n* * *\n\n_What is time inside a machine you want to replay?_ 🧘\n\nWhen building [Rewind](http://rewindvm.dev/), a deterministic VM, you come face-to-face with the idea of how to represent time very quickly. You need a way to measure time inside the guest so that on subsequent replays, the guest sees the same time and makes the same decisions. Time is the ultimate variable.\n\nYou can’t simply record the wall-clock time of the host because time spent between two interception points depends on things outside the guest, like host load, CPU frequency, caches and other processes. Therefore the same guest work can take a different amount of time on the host.\n\nIn [dafuq is a deterministic VM?](https://fzakaria.com/2026/10/24/dafuq-is-a-deterministic-vm) I explained how the clock inside [Rewind VM](https://rewindvm.dev/) moves 5 microseconds per step. More specifically, that clock, _exit time_, moves only when the guest exits such as from a \n```plaintext\nsyscall\n```\n. It treats everything the program does between two exits as **instant**. Therefore a program that is CPU bound such as doing nothing but math for a full second between two system calls, sees only 5µs go by. 🤦🏻‍♂️\n\nRewind has a second clock mode as well, _counter time_, that tries to account for the work. The clock advances one nanosecond for every **conditional branch** the guest retired.1 1 This is the same approach popularized by the [rr](https://rr-project.org/) debugger. The [rr paper](https://arxiv.org/abs/1705.05937) calls it _deterministic time_.\n\n[![Image 1: Matrix Morpheus meme: Morpheus in his sunglasses says \"What if I told you time is just counting branches\"](https://fzakaria.com/assets/images/rewind_morpheus_time_meme.png)](https://fzakaria.com/assets/images/rewind_morpheus_time_meme.png)\n\n> **Aside:** New to Rewind? Start with [a flaky build you only catch once](https://fzakaria.com/2026/10/03/rewind-vm-a-flaky-build-you-only-catch-once) and [dafuq is a deterministic VM?](https://fzakaria.com/2026/10/24/dafuq-is-a-deterministic-vm)\n\nStill confused about what I mean by “free” and “branches”? Let’s look at a simple example to distinguish between the two clocks.\n\n[](https://fzakaria.com/assets/figures/d4be2da3c7c76fbb.svg \"Open this figure on its own\")\nThe counter time tries to account for the work whereas the exit time only advanced the clock on exits.2 2 The 5µs was my guess at a reasonable cost of a system call and context switch on current hardware. The 1ns is roughly what a conditional branch costs on a current core.\n\nWhy branches, though?\n\n## [§](https://fzakaria.com/2026/11/01/a-clock-made-of-branches#what-is-time-in-a-replayed-machine)What is time in a replayed machine?\n\nWe already understand that we cannot rely on the wall-clock time of the host. We need something to measure that is the same for every run of the same program so that we can equate it to time spent.\n\nWhat _is_ the same on every run?\n\nThe guest’s instruction stream! The CPU is a deterministic machine: the same input, the same registers, the same memory, and the same instructions produce the same next state.\n\nWe can measure time by measuring progress through the instruction stream. 💡\n\nTurns out that CPU manufacturers even ship a hardware counter for exactly that. Is it reliable? Let’s see. 🧐\n\nLet’s use this simple program: count down from a billion.\n\n```\nint main(void) {\n  long i = 1000000000;\n  do {\n    i--;\n  } while (i != 0);\n  return 0;\n}\n```\n\n```\n_start:\n    mov  rcx, 1000000000\n.loop:\n    dec  rcx\n    jnz  .loop\n    mov  eax, 60    ; exit(0)\n    xor  edi, edi\n    syscall\n```\n\nThe assembly equivalent contains a \n```plaintext\nmov\n```\n, **a billion**\n```plaintext\ndec\n```\n+\n```plaintext\njnz\n```\n (conditional branch) pairs, then three instructions to exit for a total of **2,000,000,004** instructions.\n\nLet’s run it on the CPU twice and count the instructions and conditional branches with \n```plaintext\nperf stat\n```\n:\n\n```\n$ nasm -felf64 loop.asm && ld -o loop loop.o\n$ taskset -c 5 perf stat -e instructions:u,ex_ret_cond:u ./loop\n     2,000,001,025      instructions:u     1,000,000,000      ex_ret_cond:u\n\n$ taskset -c 5 perf stat -e instructions:u,ex_ret_cond:u ./loop\n     2,000,001,054      instructions:u     1,000,000,000      ex_ret_cond:u\n```\n\nThe same program and input gives the same conditional branch count, but the instruction count is off by a thousand or so. Why?\n\n[![Image 2: Panik kalm panik meme: \"My clock counts instructions\" (panik), \"The program can't see interrupts\" (kalm), \"The instruction counter can\" (panik)](https://fzakaria.com/assets/images/rewind_panik_instructions_meme.png)](https://fzakaria.com/assets/images/rewind_panik_instructions_meme.png)\n\nAlthough this loop is CPU bound and only decrements a register, it is still not isolated from the effects of the outside world. Timers tick, network packets arrive, other cores send inter-processor interrupts (IPIs). The program can’t see any of that, but the instruction counter we are probing can.\n\nI pinned the loop to a single CPU and counted how many interrupts that CPU took during each run, from \n```plaintext\n/proc/interrupts\n```\n.\n\n[](https://fzakaria.com/assets/figures/33a66128e5ab8c39.svg \"Open this figure on its own\")\nThe dashed line is _one extra count per interrupt_, and the instruction counter is almost perfectly on it.3 3 The fit is 0.94 rather than 1 because \n```plaintext\n/proc/interrupts\n```\n also catches the interrupts that land while \n```plaintext\nperf\n```\n and \n```plaintext\ntaskset\n```\n start up and tear down, before and after the loop runs. The conditional branch counter on the other hand, stays at zero no matter how many interrupts land.\n\nInterrupts from the outside-world are not the only thing that can skew the instruction count. The program can also page fault, which is an interrupt too. The instruction counter sees the kernel’s detour to handle the fault, but the program does not.\n\n> **Tip** Out of the box, AMD Zen also overcounts conditional branches, a few in tens of millions, from a speculation optimization known as SpecLockMap. The fix is to set a particular bit in the PMU MSR, which Rewind does automatically as does [rr#2034](https://github.com/rr-debugger/rr/issues/2034).\n\n## [§](https://fzakaria.com/2026/11/01/a-clock-made-of-branches#so-what-time-is-it)So, what time is it?\n\nIn order to rewind history, we need a measurement of time that is reliable and deterministic. Our traditional wall-clock time is unreliable due to the effects of the outside world but we do have a reliable measurement of progress through the instruction stream: the conditional branch counter.\n\nUsing the conditional branch counter as a proxy for time turns out to be a relatively decent approximation.\n\n| Clock | Time inside the VM | Time on the host |\n| --- | --- | --- |\n| Exit time | 2.1 s | 15.7 s |\n| Counter time | 11.7 s | 14.2 s |\n\nWhy do we care about having a “realistic” representation of time?\n\nPlenty of programs read the clock and make decisions with it, so the guest’s clock has to be at least somewhat realistic. Under exit time, computing is “free”. A test that checks some work “finishes within 1 s” always passes. If the bug you’re chasing is a timeout, exit time will never reproduce it. 🙃\n\nRaces get skewed too with an unrealistic clock. With exit time, a whole burst of work lands in the same instant, which crowds the threads together, so a race can fire far more often in the VM than it ever would on real hardware.\n\nThe closer we can emulate the real world, the more likely we are to reproduce a bug that only happens in the real world. Counter time is a better approximation of the real world than exit time, and it is deterministic, so it is the default clock in Rewind.\n\n[![Image 3: Distracted boyfriend meme: the boyfriend, labelled \"Rewind's clock\", turns to look at a woman labelled \"conditional branches\" while his girlfriend, labelled \"instructions retired\", glares at him](https://fzakaria.com/assets/images/rewind_distracted_branches_meme.png)](https://fzakaria.com/assets/images/rewind_distracted_branches_meme.png)\n\nSome say time is an illusion. I say it’s a \n```plaintext\njnz\n```\n counter.","body_html":"<h1 id=\"a-clock-made-of-branches\">A clock made of branches</h1>\n<ul><li>* *</li></ul>\n<p><em>What is time inside a machine you want to replay?</em> 🧘</p>\n<p>When building <a href=\"http://rewindvm.dev/\" rel=\"nofollow ugc noopener\">Rewind</a>, a deterministic VM, you come face-to-face with the idea of how to represent time very quickly. You need a way to measure time inside the guest so that on subsequent replays, the guest sees the same time and makes the same decisions. Time is the ultimate variable.</p>\n<p>You can’t simply record the wall-clock time of the host because time spent between two interception points depends on things outside the guest, like host load, CPU frequency, caches and other processes. Therefore the same guest work can take a different amount of time on the host.</p>\n<p>In <a href=\"https://fzakaria.com/2026/10/24/dafuq-is-a-deterministic-vm\" rel=\"nofollow ugc noopener\">dafuq is a deterministic VM?</a> I explained how the clock inside <a href=\"https://rewindvm.dev/\" rel=\"nofollow ugc noopener\">Rewind VM</a> moves 5 microseconds per step. More specifically, that clock, <em>exit time</em>, moves only when the guest exits such as from a </p>\n<pre><code class=\"language-plaintext\">syscall</code></pre>\n<p>. It treats everything the program does between two exits as <strong>instant</strong>. Therefore a program that is CPU bound such as doing nothing but math for a full second between two system calls, sees only 5µs go by. 🤦🏻‍♂️</p>\n<p>Rewind has a second clock mode as well, <em>counter time</em>, that tries to account for the work. The clock advances one nanosecond for every <strong>conditional branch</strong> the guest retired.1 1 This is the same approach popularized by the <a href=\"https://rr-project.org/\" rel=\"nofollow ugc noopener\">rr</a> debugger. The <a href=\"https://arxiv.org/abs/1705.05937\" rel=\"nofollow ugc noopener\">rr paper</a> calls it <em>deterministic time</em>.</p>\n<p><a href=\"https://fzakaria.com/assets/images/rewind_morpheus_time_meme.png\" rel=\"nofollow ugc noopener\"><img src=\"https://fzakaria.com/assets/images/rewind_morpheus_time_meme.png\" alt=\"Image 1: Matrix Morpheus meme: Morpheus in his sunglasses says &quot;What if I told you time is just counting branches&quot;\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" /></a></p>\n<blockquote><p><strong>Aside:</strong> New to Rewind? Start with <a href=\"https://fzakaria.com/2026/10/03/rewind-vm-a-flaky-build-you-only-catch-once\" rel=\"nofollow ugc noopener\">a flaky build you only catch once</a> and <a href=\"https://fzakaria.com/2026/10/24/dafuq-is-a-deterministic-vm\" rel=\"nofollow ugc noopener\">dafuq is a deterministic VM?</a></p></blockquote>\n<p>Still confused about what I mean by “free” and “branches”? Let’s look at a simple example to distinguish between the two clocks.</p>\n<p><a href=\"https://fzakaria.com/assets/figures/d4be2da3c7c76fbb.svg\" title=\"Open this figure on its own\" rel=\"nofollow ugc noopener\"></a>\nThe counter time tries to account for the work whereas the exit time only advanced the clock on exits.2 2 The 5µs was my guess at a reasonable cost of a system call and context switch on current hardware. The 1ns is roughly what a conditional branch costs on a current core.</p>\n<p>Why branches, though?</p>\n<h2 id=\"what-is-time-in-a-replayed-machine\"><a href=\"https://fzakaria.com/2026/11/01/a-clock-made-of-branches#what-is-time-in-a-replayed-machine\" rel=\"nofollow ugc noopener\">§</a>What is time in a replayed machine?</h2>\n<p>We already understand that we cannot rely on the wall-clock time of the host. We need something to measure that is the same for every run of the same program so that we can equate it to time spent.</p>\n<p>What <em>is</em> the same on every run?</p>\n<p>The guest’s instruction stream! The CPU is a deterministic machine: the same input, the same registers, the same memory, and the same instructions produce the same next state.</p>\n<p>We can measure time by measuring progress through the instruction stream. 💡</p>\n<p>Turns out that CPU manufacturers even ship a hardware counter for exactly that. Is it reliable? Let’s see. 🧐</p>\n<p>Let’s use this simple program: count down from a billion.</p>\n<pre><code>int main(void) {\n  long i = 1000000000;\n  do {\n    i--;\n  } while (i != 0);\n  return 0;\n}</code></pre>\n<pre><code>_start:\n    mov  rcx, 1000000000\n.loop:\n    dec  rcx\n    jnz  .loop\n    mov  eax, 60    ; exit(0)\n    xor  edi, edi\n    syscall</code></pre>\n<p>The assembly equivalent contains a </p>\n<pre><code class=\"language-plaintext\">mov</code></pre>\n<p>, <strong>a billion</strong></p>\n<pre><code class=\"language-plaintext\">dec</code></pre>\n<p>+</p>\n<pre><code class=\"language-plaintext\">jnz</code></pre>\n<p> (conditional branch) pairs, then three instructions to exit for a total of <strong>2,000,000,004</strong> instructions.</p>\n<p>Let’s run it on the CPU twice and count the instructions and conditional branches with </p>\n<pre><code class=\"language-plaintext\">perf stat</code></pre>\n<p>:</p>\n<pre><code>$ nasm -felf64 loop.asm &amp;&amp; ld -o loop loop.o\n$ taskset -c 5 perf stat -e instructions:u,ex_ret_cond:u ./loop\n     2,000,001,025      instructions:u     1,000,000,000      ex_ret_cond:u\n\n$ taskset -c 5 perf stat -e instructions:u,ex_ret_cond:u ./loop\n     2,000,001,054      instructions:u     1,000,000,000      ex_ret_cond:u</code></pre>\n<p>The same program and input gives the same conditional branch count, but the instruction count is off by a thousand or so. Why?</p>\n<p><a href=\"https://fzakaria.com/assets/images/rewind_panik_instructions_meme.png\" rel=\"nofollow ugc noopener\"><img src=\"https://fzakaria.com/assets/images/rewind_panik_instructions_meme.png\" alt=\"Image 2: Panik kalm panik meme: &quot;My clock counts instructions&quot; (panik), &quot;The program can&#39;t see interrupts&quot; (kalm), &quot;The instruction counter can&quot; (panik)\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" /></a></p>\n<p>Although this loop is CPU bound and only decrements a register, it is still not isolated from the effects of the outside world. Timers tick, network packets arrive, other cores send inter-processor interrupts (IPIs). The program can’t see any of that, but the instruction counter we are probing can.</p>\n<p>I pinned the loop to a single CPU and counted how many interrupts that CPU took during each run, from </p>\n<pre><code class=\"language-plaintext\">/proc/interrupts</code></pre>\n<p>.</p>\n<p><a href=\"https://fzakaria.com/assets/figures/33a66128e5ab8c39.svg\" title=\"Open this figure on its own\" rel=\"nofollow ugc noopener\"></a>\nThe dashed line is <em>one extra count per interrupt</em>, and the instruction counter is almost perfectly on it.3 3 The fit is 0.94 rather than 1 because </p>\n<pre><code class=\"language-plaintext\">/proc/interrupts</code></pre>\n<p> also catches the interrupts that land while </p>\n<pre><code class=\"language-plaintext\">perf</code></pre>\n<p> and </p>\n<pre><code class=\"language-plaintext\">taskset</code></pre>\n<p> start up and tear down, before and after the loop runs. The conditional branch counter on the other hand, stays at zero no matter how many interrupts land.</p>\n<p>Interrupts from the outside-world are not the only thing that can skew the instruction count. The program can also page fault, which is an interrupt too. The instruction counter sees the kernel’s detour to handle the fault, but the program does not.</p>\n<blockquote><p><strong>Tip</strong> Out of the box, AMD Zen also overcounts conditional branches, a few in tens of millions, from a speculation optimization known as SpecLockMap. The fix is to set a particular bit in the PMU MSR, which Rewind does automatically as does <a href=\"https://github.com/rr-debugger/rr/issues/2034\" rel=\"nofollow ugc noopener\">rr#2034</a>.</p></blockquote>\n<h2 id=\"so-what-time-is-it\"><a href=\"https://fzakaria.com/2026/11/01/a-clock-made-of-branches#so-what-time-is-it\" rel=\"nofollow ugc noopener\">§</a>So, what time is it?</h2>\n<p>In order to rewind history, we need a measurement of time that is reliable and deterministic. Our traditional wall-clock time is unreliable due to the effects of the outside world but we do have a reliable measurement of progress through the instruction stream: the conditional branch counter.</p>\n<p>Using the conditional branch counter as a proxy for time turns out to be a relatively decent approximation.</p>\n<div class=\"table-wrap\"><table><thead><tr><th>Clock</th><th>Time inside the VM</th><th>Time on the host</th></tr></thead><tbody><tr><td>Exit time</td><td>2.1 s</td><td>15.7 s</td></tr><tr><td>Counter time</td><td>11.7 s</td><td>14.2 s</td></tr></tbody></table></div>\n<p>Why do we care about having a “realistic” representation of time?</p>\n<p>Plenty of programs read the clock and make decisions with it, so the guest’s clock has to be at least somewhat realistic. Under exit time, computing is “free”. A test that checks some work “finishes within 1 s” always passes. If the bug you’re chasing is a timeout, exit time will never reproduce it. 🙃</p>\n<p>Races get skewed too with an unrealistic clock. With exit time, a whole burst of work lands in the same instant, which crowds the threads together, so a race can fire far more often in the VM than it ever would on real hardware.</p>\n<p>The closer we can emulate the real world, the more likely we are to reproduce a bug that only happens in the real world. Counter time is a better approximation of the real world than exit time, and it is deterministic, so it is the default clock in Rewind.</p>\n<p><a href=\"https://fzakaria.com/assets/images/rewind_distracted_branches_meme.png\" rel=\"nofollow ugc noopener\"><img src=\"https://fzakaria.com/assets/images/rewind_distracted_branches_meme.png\" alt=\"Image 3: Distracted boyfriend meme: the boyfriend, labelled &quot;Rewind&#39;s clock&quot;, turns to look at a woman labelled &quot;conditional branches&quot; while his girlfriend, labelled &quot;instructions retired&quot;, glares at him\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" /></a></p>\n<p>Some say time is an illusion. I say it’s a </p>\n<pre><code class=\"language-plaintext\">jnz</code></pre>\n<p> counter.</p>","headings":[{"level":1,"text":"A clock made of branches","id":"a-clock-made-of-branches"},{"level":2,"text":"§What is time in a replayed machine?","id":"what-is-time-in-a-replayed-machine"},{"level":2,"text":"§So, what time is it?","id":"so-what-time-is-it"}]}}