← Devakrishna

Event sourcing

The issues with event sourcing after the prototype stage

August 2026

Key takeaways

Event sourcing stores every change to an entity as an immutable event in an append-only log, and rebuilds current state by replaying those events. Nothing in the log is ever updated. Bad events written by a bug stay after the code is fixed, and the only correction is a compensating event appended behind them.

That rule constrains every design decision that comes after it. Because stored events cannot change, every change of mind (a new event shape, a raised limit, a dated rule, an erasure request) becomes work at read time. Because current state is only ever a replay, every query needs a projection, which is a table built by replaying the events. That table lags behind the log and has to be maintained. Both consequences stay hidden while the system is still a toy and the ledger is small.

Schema changes land at read time

The first of those two consequences arrives when an event's shape changes, because old events already in the store have to be read beside the new one, and four strategies handle that. Tolerant deserialization, which means a reader accepts fields it does not know about, covers additive changes only. A version identifier lets a reader branch on what it finds, chained upcasters convert an old event to the new shape at read time, and in-place migration rewrites stored bytes, which breaks immutability and undermines the audit trail.

The audit trail can also be lost without a rewrite, since a shape can go stale sooner than plans assume. About one month into one project a shift of focus made a set of events irrelevant. Upcasting them to new events, upcasting them to no-ops that change nothing, or rewriting the ledger all lose the ability to reproduce the state as it stood at the rewrite. That ability is the audit trail the ledger was chosen for.

Fields drift as well as shapes. Raising a domain length limit, say a product title capped at 50 characters, while the projection column stays unchanged, truncates data or errors in the database. Dated rules add a second axis, since a price of $10 before November 18 and $15 after must live in the domain model, so each event runs under its own date's rules, and overlap with bug fixes can make the model bi-temporal.

event store append-only, bytes never change v1 v1 v1 v2 v2 three old shapes, two new no rewrite, no migration nothing is written back upcaster v1 to v2 on every read reader sees v2 only each read converts again the store stays as written
Old and new shapes share one stream. The upcaster converts the old ones on every read, and the stored bytes never change.

Projections replace every query

Upcasters run at read time, and the read they run inside is itself a replay. An event store has no query mechanism like SQL. It can hand back only a stream of events selected by entity identifier, so any other question needs a separate read model, and that pairing is known as CQRS.

A read model does not remove the replay underneath. Rebuilding one entity by replaying its stream, called rehydration, costs time and compute in proportion to stream length. The answer is a snapshot every N events, and the choice of N trades snapshot storage against rehydration time. A copy of the current state is stored beside the log. Length is not always under the domain's control, and one stream passed 100K events after a background job updated the entity for a couple of months without checking whether anything had changed, which slowed edits while reads stayed fast.

Those reads come from projections, and time passes between appending an event, publishing it, and a consumer handling it, so every projection is only eventually consistent. The delay stays invisible while the ledger is small enough to materialize on demand. Once it is not, reads move to projections and read-after-write consistency is gone, so new data returns 404 and deleted items linger.

Those missing and lingering rows are one cost of a projection, and code is the other, since each projection is more code against the stream and every event type added, changed, or removed has to reach all N of them. Their dependencies cost more than the count suggests. During one rebuild the projector maintaining one projection read another projection far ahead of it in the event history, and because nothing crashed the wrong values took long to explain. Projectors have to be autonomous, so each one runs asynchronously, tracks its own progress, and decides its own rebuilds.

event log events 1 to 1,000 append-only queries never read here projection A at event 1,000, lag 0 projection B at event 940, lag 60 projection C at event 610, lag 390 C reads B, 330 ahead reads reads reads the cross-read returns wrong values, and nothing crashes
Every read goes to a projection, and each projection lags the log by its own amount. Projection C reads projection B, which is 330 events ahead. The values are wrong and nothing fails.

Ordering and duplicates need explicit handling

Projections are right only if each event reaches them once and in order, and neither is given. Many writers share the store, so per-entity order is at risk from the first concurrent request. A timestamp on every event and an incremental per-request identifier let the store reject duplicates. Optimistic concurrency, which rejects an append when the stream moved on, protects one stream, so a stock reduction landing while a customer orders that item needs logic such as a back order.

Concurrent writers are not the only ordering risk, because commit order and visible order are not the same thing. Projectors in one system missed events under high load, because the SQL Server identity column values they used for ordering can commit out of order, so a projector sees the second event while the first is invisible. Exclusive locks on inserts fixed it, and with reads outnumbering writes 100 to 1 they cost nothing measurable.

Delivery count is the other half. Delivery is typically at-least-once, so consumers get duplicate events, and a handler that is not idempotent, meaning not safe to run twice, lets projections drift and runs payments or notifications more than once. A last processed sequence number per consumer fixes that. None of this comes free from infrastructure, since a broker such as Kafka has neither per-entity stream queries nor optimistic concurrency, and a relational store leaves the work to you.

writer 1 expects version 7 accepted, version 8 writer 2 expects version 7 rejected, stream moved on stream, order 42 version 7 to 8 event 8 event 8 again consumer gets event 8 twice optimistic concurrency guards one stream, and delivery is at-least-once
Two writers race on one stream. The store accepts one append and rejects the other, and the consumer still receives the accepted event twice.

External systems need a replay gateway

Idempotent handlers protect only the projections you own. A replay re-sends update messages to external systems, and the receiver cannot tell a replay from real processing, so each one has to sit behind a gateway that checks whether the processor is in replay mode. Events in the log are low level, so those systems may need separate integration events.

Answers coming back drift too. An exchange rate read on December 5 is not the rate read on December 20, so a consistent replay needs a trusted source of historical values or a gateway that records and persists every query response. A call that queries and updates at once, such as submitting an order and getting delivery information back, carries both problems.

A replay that fixes a bug carries a problem of its own, because reprocessing corrects internal state and leaves the gateways behind. They have to track the difference between what happened with the bug and what would have happened without it, which may need retroactive events and an event that cancels the effect of the buggy one. Reversal works only if events carry differences, such as adding $10, or store the value they replaced.

live run OrderPlaced #1187 $120, same bytes gateway mode live delivery system ships the order replay OrderPlaced #1187 $120, same bytes gateway mode replay suppressed delivery system hears nothing same event, same gateway, and only the mode differs
The live run and the replay send the same event through the same gateway. Only the gateway's mode keeps the replay away from the delivery system.

Plumbing and coordination take the time

Gateways, upcasters, and projectors are code someone writes and maintains. The frameworks on offer were heavy, imposed their own way of working, and locked a project into one technology stack, so one team built its own core, and whole sprints went to deployment design, stream behavior, and retries before any application work began. One feature needs commands, command handlers, validators, events, aggregates, projections, projection model classes, access classes, and materialization code.

A shared log spreads that cost across teams. Consumers read the producer's raw events, so the producer loses its boundary and has to agree event contents with every consumer before changing anything. Many subscribers read the same events without coordinating, so it is hard to see how data flows and who breaks when an event is added, removed, or changed, and a coordination layer cannot be added later.

The audit trail that justified the log costs work too. Raw events are too chatty to show anyone, because a field rename means nothing to an end user who wants the separate states the business cares about, so the free audit log turns into projection writing. Replay debugging pays back less than expected, because 99% of bad states came from a bad event written by ordinary human error.

The cost reaches past the back end. A task-based UI is needed to match small semantic events, while common UIs are static forms sending fat blobs of form data. An immutable store also conflicts with right-to-be-forgotten law, because deleting events breaks stream integrity.

command command handler validator event aggregate projection projection model class access class materialization code first business rule comes after all nine
One feature needs these nine pieces of plumbing. The first business rule arrives after all of them.

What to do

  1. Name the one problem that needs the log, because a plain history table gives about 80% of a ledger's value at almost none of its cost.
  2. Rule the pattern out for simple CRUD without audit needs, prototypes, MVPs, short-lived systems, real-time consistent views, static reference data, and teams new to event-driven work.
  3. Decide before the first release, because migrating to or from event sourcing is costly and retroactive events are hard to retrofit.
  4. Stamp a version identifier on every event before production traffic arrives.
  5. Make every projector autonomous and idempotent, and have it truncate and log a warning instead of crashing when a value no longer fits.
  6. Keep personal data out of the event store, or under a per-subject key you can destroy, and budget for encryption on every read and write.

The gap between those steps and the incidents they prevent is measurable in a system that runs. Time a rehydration of your longest stream against the snapshot interval you set, read your busiest projection's current lag in events and in seconds, and count the consumers one removed field would force into a coordinated change. Those three numbers are the same append-only rule in three places, and each grows with every event appended.

Sources