A short N The Loop essay using Kafka's years-long exactly-once effort, which still needed redesign eight years later, to argue that consistency is an end-to-end property no single component can guarantee, so someone must understand and own the whole system rather than trusting machines to hold it.
Consistency is not a localized property
Oct 6, 2026
It is a common pattern for developers to assume that consistency is
something they can achieve inside a single component. Apache Kafka is
the cautionary tale. Making writes exactly-once took a team of
distributed systems engineers years and several new components and a
complete rewrite of older components (Gustafson et al. 2016).1
And after all of that, the people who built it are blunt about what you
get:
Exactly-once processing is an end-to-end guarantee and the application
has to be designed to not violate the property as
well.(Narkhede and Wang 2017)
If the component cannot promise the property, the promise has to live
somewhere else. Two practical consequences:
Someone has to understand the system end to end. Since no component
can confirm it, something outside the components has to keep it all
consistent.
Someone has to own it. Component owners will each correctly say
their part works. A global property needs a person who is liable for
the whole thing, with some guarantee that it makes sense.
The lesson: do not hand global, valuable properties to machines and
assume they are held. Such guarantees are grounded in understanding, not
code.