From Events to Read Models: The Projection Problem Most Teams Underestimate

Most event sourcing tutorials end right where the real work begins. They walk you through aggregates, commands, and event handlers with care. Then they show a simple read model built from a handful of events, declare victory, and move on.

What they do not show is what happens six months later: a bug has quietly corrupted your projection state, an event schema evolved and broke your handlers, a cross-aggregate query is grinding your read database to a halt, or you need to rebuild from scratch in production without downtime. This post covers the parts tutorials skip.

What a Projection Actually Is (Not the Definition, the Reality)

You have probably read the definition. A projection is a function that maps a sequence of events to some derived state. That is correct. It is also nearly useless as practical guidance.

Here is what a projection actually is in production: a stateful process that consumes your event stream, maintains derived state in some storage layer, and must be deterministic (same events, same output, always), idempotent (safe to replay without duplicating effects), and rebuildable from scratch at any time. The rebuild requirement is the one that usually surprises people.

A concrete example in C#. An OrderStatus projection that tracks the lifecycle of an order:

public class OrderStatusProjection
{
    public string OrderId { get; private set; }
    public string Status { get; private set; }
    public DateTimeOffset? ShippedAt { get; private set; }

    public void Apply(OrderPlaced evt)
    {
        OrderId = evt.OrderId;
        Status = "Placed";
    }

    public void Apply(OrderShipped evt)
    {
        Status = "Shipped";
        ShippedAt = evt.ShippedAt;
    }

    public void Apply(OrderCancelled evt)
    {
        Status = "Cancelled";
    }
}

Clean, readable, and completely reasonable. The handler for each event type is small and obvious. Now ask yourself: what is your checkpoint strategy? Where does this process restart from if the host crashes? How do you rebuild this read model after a logic bug? If you do not have answers to those questions, you do not yet have a projection system. You have a proof of concept.

The Rebuild Problem: What Happens When Something Goes Wrong

Your projection handler has a bug. You do not notice for two weeks. The status field has been incorrectly set for a subset of orders. You find it, you fix the handler. Now what?

You cannot patch state manually. That is the whole point of event sourcing: your state is derived from your events, not stored directly. The correct and only real answer is to replay your event history through the fixed handler to rebuild correct state. That might mean millions of events. In a system serving live traffic.

There are a few patterns for handling this without bringing your API down.

Versioned projections are the most common approach. When the handler changes in a breaking way, you run a v2 projection alongside v1. The v2 projection starts from position zero, consuming the full event history in the background. Once it has caught up to the live position and the two diverge by only a few events, you cut over. The trick is making the cutover atomic, or at least fast enough that the inconsistency window is negligible.

Shadow rebuilds take this further. You rebuild into a completely separate read store, then swap the pointer. The live read store never goes away until the new one is ready. This costs you the storage overhead of running two read stores simultaneously, but it gives you a clean rollback path if the new projection turns out to have its own problems.

Checkpoint-based resumption is a prerequisite for both of the above. Your projection must durably record where in the event stream it has processed up to, typically as an event sequence number or position. On restart or failure, it resumes from that checkpoint rather than starting over. Without this, every restart is a full rebuild, which rapidly becomes unacceptable at scale.

None of this is conceptually difficult. All of it requires careful implementation, and it compounds with the number of projections you run.

Eventual Consistency: What It Means When Traffic Is Real

Event sourcing naturally produces eventual consistency between writes and read models. A command arrives, an event is written to the event store, and your projection process picks it up asynchronously and updates the read store. There is always a latency window between the write and when the read model reflects it.

Most of the time this is fine, and well worth the tradeoffs. But there are cases where it bites in ways that frustrate users.

The classic scenario: a user places an order, the command succeeds, they are redirected to an order detail page, and they see the old state because the projection has not yet caught up. The order appears not to exist.

There are two practical patterns for handling this gracefully.

The first is to return the command result directly rather than re-querying the read model. If your command handler returns the created order ID and enough data to render the confirmation page, you do not need to hit the read store immediately. You already have what you need.

The second is optimistic UI updates. The client assumes the command succeeded and updates its local state to reflect the expected outcome, then reconciles with the server once the read model has caught up. This is standard practice in well-built frontends, and it pairs naturally with event sourcing.

Where you genuinely need stronger consistency guarantees, the cost is real. Synchronous projection updates inside the command transaction give you consistency but remove the architectural separation that makes event sourcing valuable. It is a tradeoff worth making eyes open, not as a default.

Cross-Aggregate Projections: The Part That Breaks Everything

A customer order history dashboard spans hundreds of Order aggregates. A sales report totals across thousands of Products. A fulfilment view joins Inventory events with Order events. These projections consume events from many independent aggregate streams simultaneously, and they surface problems that single-stream projections never encounter.

Consider a sales summary projection that tracks total revenue per product. It must consume events from every Order stream in the system. As Order streams multiply, the projection must manage a checkpoint per stream, not a single global position. A new Order aggregate is created. The projection must discover it. One stream is temporarily slow. The projection must handle partial progress correctly without corrupting totals.

public class SalesSummaryProjection
{
    // Per-product totals maintained across all Order aggregates
    private readonly Dictionary<string, decimal> _revenueByProduct = new();

    public void Apply(OrderLineAdded evt)
    {
        if (!_revenueByProduct.ContainsKey(evt.ProductId))
            _revenueByProduct[evt.ProductId] = 0;

        _revenueByProduct[evt.ProductId] += evt.LineTotal;
    }

    public void Apply(OrderLineCancelled evt)
    {
        if (_revenueByProduct.ContainsKey(evt.ProductId))
            _revenueByProduct[evt.ProductId] -= evt.LineTotal;
    }
}

The handler itself looks simple. The complexity is entirely in the infrastructure underneath it: how streams are discovered, how checkpoints are maintained per stream, how the projection handles ordering across streams where events from different streams arrive in non-deterministic order, and what happens when the projection host restarts mid-rebuild.

This is where the infrastructure requirements escalate. A single-stream projection is manageable with a relatively simple worker process. Cross-aggregate projections at scale require something closer to a stream processing system, with all the operational weight that implies.

Event Schema Evolution: When Your Past Events Change Shape

Your events are immutable. Your understanding of your domain is not.

Twelve months into a project, you discover that CustomerAddressChanged needs a CountryCode field that was never captured. Or the ProductId field turns out to have been a legacy identifier format that the rest of the system has moved away from. Or a single OrderUpdated event type needs to split into OrderShippingAddressUpdated and OrderItemQuantityChanged for clarity.

Existing events in your store cannot be changed. Projections that depend on their shape will break if you just update the class definition.

The canonical solution is an upcaster: a function that transforms an older event shape into the current canonical form before it reaches your projection handlers. Your handlers only ever see the current version. The upcaster absorbs the version history.

public static OrderShipped Upcast(OrderShipped_v1 old)
{
    return new OrderShipped
    {
        OrderId = old.OrderId,
        ShippedAt = old.ShippedAt,
        // New field: default to UTC for legacy events where timezone was not recorded
        ShippedAtUtc = old.ShippedAt.ToUniversalTime(),
        CarrierCode = old.CourierName // renamed field
    };
}

The upcaster runs at read time, when events are loaded from the store. The original event bytes are unchanged. You can chain upcasters for events that have gone through multiple versions. Your projection handlers remain clean and focused on the current domain model.

The convention that makes this manageable long-term: version your event type names explicitly when a breaking change occurs. OrderShipped becomes OrderShipped_v2. You always know which version of the type you are working with, and your upcaster pipeline is unambiguous.

Learn this pattern early. Retrofitting it onto an existing codebase after several schema generations is painful.

What You’re Actually Signing Up For When You Build This Yourself

A production-grade projection system requires more infrastructure than tutorials suggest, and it is worth being upfront about that.

You need:

• Stream checkpointing per projection, persisted durably so restarts are safe
• A projection host with restart logic, health checks, and graceful shutdown
• Retry handling for transient failures, with dead-letter queues for persistent ones
• Parallel projection runners so a slow projection does not block faster ones
• Ordering guarantees within each stream, even under parallel processing
• A rebuild mechanism that allows you to run a new projection version alongside the live one without downtime
• Stream discovery for cross-aggregate projections so new aggregate instances are picked up automatically

None of this is conceptually hard. Every item on that list has a reasonable solution. But each one takes time to build, test, operate, and maintain. And every time your projection count grows, the surface area of that infrastructure grows with it.

The point is not to discourage building this yourself. Some teams have the right context and the right problem shape for that investment. The point is to make the cost visible before you commit, not after you are three months into it and wondering why you are not writing business logic yet.

Conclusion

The concept of a projection is genuinely beautiful: a deterministic function from your event history to your current state, rebuildable at any time, derived entirely from your domain’s record of what happened. That property alone is worth a lot.

The production reality requires discipline around checkpointing, schema evolution, consistency windows, rebuild strategies, and cross-aggregate complexity. Teams that succeed with event sourcing in the long run either invest properly in that infrastructure layer, or find a way not to build it themselves.

Neither path is wrong. But both require going in with eyes open.

If you’re looking for a way not to build that infrastructure yourself, Hapnd is built to carry exactly this load. You write the projection handler. Hapnd handles checkpointing, hosting, rebuilds, ordering, and scaling. The beta is open at hapnd.dev, no credit card or commitment required.