Event Sourcing Tips: Do’s and Don’ts

When people are getting into Event Sourcing, there are a few common questions that I often get or issues see people run into. CRUD Sourcing, Pre-mature optimization using Snapshots, and exposing your event streams for integration. Here are my top three Event Sourcing Tips to help you down the right path.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything in this post.

CRUD Sourcing

My first event sourcing tip, which is probably the most common issue I see people run into when new to Event Sourcing is what is often called “CRUD Sourcing”. If you’re used to developing applications/systems in a Create-Read-Update-Delete style, then this means you’ll likely end up creating Events that are derived from Create, Update, Delete.

There’s a shift when moving from an application that simply maintains the current state via CRUD to having your point of truth be a stream of events. Often events are artifacts of the state change as well as the business event that caused that state change.

If you provide a UI that’s CRUD driven on Entities, you’ll end up with events that are derived from that. As an example, you’d start creating events such as ProductCreated, ProductUpdated, and ProductDeleted.

Still in line with this is if you have updates that are for properties on Entities, you’ll end up with events such as ProductQuantityUpdated or ProducePriceChanged.

In both cases, the events are simply representing state changes but not why the state changed.

What was the reason why a ProductUpdated occurred? It was updated, great, but why?

How about the ProductQuantityUpdated event, why did that change? Was it because there was an inventory adjustment? Did we receive more quantity of the product from the supplier? Why was it updated?

Being explicit about the events is important because we want to be driven by business concepts. To get out of CRUD we need to move to more of a Task Driven UI. This allows us to have the client/UI explicitly perform a Command/Task. For example, if the user performs an Inventory Adjustment as an explicit Command/Task, that’s a business concept. We will generate an InventoryAdjustment event.

Being explicit is important because you do not need to derive or guess based on the data of the event and why it occurred. You’ll have many more answers to questions when you look at an event stream when they are explicit. As an example, when’s the last time we did an inventory adjustment? When we do inventory adjustment, how many times are we decreasing the quantity on hand? You cannot answer these questions with a ProductUpdated or a ProductQuantityChanged.

Optimizations

Once people understand Event Sourcing and how it works, the most common question is:

That seems really inefficient to have to fetch all the events from a stream to build up current state! What happens if I have 1000’s events!

To understand how event sourcing works check out my post Event Sourcing Example & Explained in plain English.

As a quick primer, you have a stream of events for a unique aggregate, such as Product with a SKU of ABC123.

Event Stream

Anytime we want to perform a command which will append a new event to the stream, we’ll generally fetch all the events form the stream, build up the current state, then enforce any invariants for the command we want to perform.

In the stream above if we were keeping track of “quantity” as the current state, it would be 59.

So back to the common comment of “that’s really inefficient”, is a pretty valid concern. The answer to this problem is called Snapshots, but they are an optimization that you don’t necessarily need to apply right from the start or often.

In my experience, event streams are generally finite and have a life cycle with a beginning and end. There may be a long time duration for how long a stream is “active” but the events that are persisted are often limited.

If you have a lot of events and it’s taking a long time to rebuild the state, then creating snapshots can help. They are a way of creating a point-in-time representation of the state.

Event Stream Snapshot

After so many events append to a stream, you persist another event to a separate stream that is the current state, also recording at which version of the stream it represents. This way when you want to rebuild the current state, you first get the last snapshot and then get the events from the stream since that snapshot was created.

Now the question is, when do you create snapshots? If you think the event stream is going to contain a lot of events, how many for a given situation is a lot? Each different type of event stream is going to have different events which contain different data. There’s no magical number of events that is a threshold for creating a snapshot, it’s going to be use-case specific.

Event Sourcing Tip, don’t jump to snapshots immediately, look at how you’ve defined your streams and boundaries.

Communication & State

This last mess people get in with Event Sourcing is conflating events representing state as well as using events as a way to communicate with other service boundaries.

Your event store, the event streams, and the events within a stream represent the state.

Event Store for State

Often with Event Sourcing, you’ll create Projections as a way to represent the current state for Queries/UI/Reports. This way you can query a separate data store that’s pre-computed the current state. This means you don’t have to pull all the events to a stream to build the current state. It’s already pre-computed as events occur (usually asynchronously)

Event Store and DocumentDB for Projections

This means you’ll have two different databases. One for your event streams and one for projections. This is all within the same logical service boundary.

Now if you’re using an Event Store that supports subscriptions, this doesn’t mean that other logical service boundaries should directly access the Event Store.

A service can't access Event Store for integration

People often do this as a way of Pub/Sub which is used for communication. But there is a difference between Events used inside a logical boundary to represent state and events used to communicate with other services.

You wouldn’t have one service connect to another service’s relational database, would you? Then why would you access its Event Store?

Don't access other services DB directly

Domain Events used within a service boundary to represent the state are not integration events.

Domain Events: Inside Events

Integration Events: Outside Events

Domain Events aren't integration events

You want to define which events you want to publish to the outside world for integration. Domain Events and Integration events will be versioned entirely differently. The moment you expose a domain event to the outside world you potentially now have consumers that are going to rely on it. If you don’t publish your internal domain events, you can refactor and change events very differently.

If another service was interacting with your relational database, and you made a change to a column name, you’d break them. If you change an event that’s not backward compatible, you’re going to break consumers. Don’t expose internal domain events as integration events.

Event Sourcing do’s and don’ts

Hopefully, these three Event Sourcing Tips give some insights that can help you if you’re new to Event Sourcing or if you ever questioned how various aspects work.

Join!

Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design as well as access to source code for any working demo application that I post on my blog or YouTube. Check out the YouTube Membership or Patreon for more info.

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

CQRS & Event Sourcing Code Walk-Through

Want to see an example of how CQRS & Event Sourcing work together? Here’s a code walk-through that illustrates sending commands to your domain that stores Events in an Event Store. The Events are then published to Consumers that updated Projections (read models) that are then consumed by Queries. This is the stereotypical set of patterns used when using CQRS and Event Sourcing together.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.

CQRS & Event Sourcing

Because CQRS and Event Sourcing are so often talked about or illustrated together, you’ll end up seeing a diagram like the one below.

This is the stereotypical diagram to illustrate both concepts together. Unfortunately there often isn’t a distinction between what portion is CQRS and what portion is Event Sourcing. And as you’ll see later in this post, there’s another concept involved as well in this diagram as well.

CQRS

Command Query Responsibility Segregation (CQRS) is simply the separation of Writes (Commands) and Reads (Queries). In the diagram above, that’s illustrated by the horizontal paths on the top and bottom. I’ve talked about the simplicity of CQRS and the 3 Most Common Misconceptions.

Event Sourcing

Event Sourcing is about how you record state. It’s a different approach to persistence since most applications are built to only record the current state. Event sourcing is about using a series of events (facts) that represent all the state transitions that get you to the current state. If you want more of the basics, check out my post Event Sourcing Example & Explained in plain English.

Simplest Possible Thing

I’m going to use Greg Young’s Simplest Possible Thing sample that illustrates both CQRS, Event Sourcing, and Projections. This sample is rather old so I’ve upgraded it to .NET 6 and Razor pages.

The sample app is just showing an Inventory Item and has various commands that mutate state, queries that return current state, and event sourcing is used as a way to persist state.

Commands & Events

Commands are handled by Command Handlers. In this example, they are simply using a repository to get out the InventoryItem which is a domain object, and then invoke the proper method on the InventoryItem. The Repository.Save() will persist all the events (generated though ApplyChange() on the InventoryItem you will see below) to an in-memory event stream (collection).

The Inventory Item domain objects contain all the behavior for doing any state transitions.

The AggregateRoot base class contains the ApplyChange method, which stores the event being applied. It also calls the appropriate Apply() method on the InventoryItem. You will notice there are Apply() methods for only some of the events. This is because our InventoryItem only cares about maintaining its internal state (projection) that is required to perform logic.

What all this code is illustrating is the Command side of CQRS as well as Event Sourcing. Commands are explicit actions that we handle to perform some type of state change. For state changes, we’re using explicit events to capture what actually occurred from the result of a command.

CQRS & Event Sourcing Code Walk-Through

Projections

Before I illustrate the Query side of CQRS, first we’re going to build a Projection that acts as a read model representing the current state of the Events.

When events are saved by the Repository, it then dispatches the events to consumers. In this example, it is done in-memory however if you were using an actual Event Store, you’d likely be using a subscription model to have your projections run in isolation asynchronously.

Consumers handle the events to update the read model (projection). In this example, there are two different consumers for updating two different projections. One projection is for showing a list of Inventory Items, the other is for showing an individual Inventory Item.

This is just using a FakeDatabase that is really just an in-memory collection & dictionary.

CQRS & Event Sourcing Code Walk-Through

Query

Now that we have projections (read models) we can look at how the Query side of CQRS uses the projections.

The razor page is using the ReadModelFacade (and underlying FakeDatabase) that has the projection for our InventoryItem details.

CQRS & Event Sourcing Code Walk-Through

CQRS & Event Sourcing

Hopefully, this illustrated the differences between CQRS, Event Sourcing, and how they are used together while also using Projections. While this diagram is often used to describe CQRS, realize there are multiple aspects that are at play and not just CQRS.

Source Code

Developer-level members of my YouTube channel or Patreon get access to the full source for any working demo application that I post on my blog or YouTube. Check out the YouTube Membership or Patreon for more info.

Related Links

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

Event Sourced Aggregate Design: Focus on Business Logic

Separating data and behaviors can have a pretty profound impact on your design. An Event Sourced Aggregate allows you to focus on business logic by having capabilities to produce data (Events). Event Sourcing does exactly this by limiting the amount of state you require only to that needed for business logic within an Aggregate.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.

Shipment

Before jumping in code, the example I’m going to use is of a shipment. The simple setup is you have a shipment that has two stops. The first stop is the Pickup and the last stop is the Delivery. You could think of this as a food delivery service where the shipment is of your food. A delivery driver goes to the Pickup Stop, which is the restaurant that you’re ordering from, and the delivery stop is your house.

Event Sourced Aggregate

Each stop will go through a transition in its status. At first, a stop will be “In Transit” and then will progress to “Arrived” once the delivery driver has reached the stop. Once the driver leaves the stop and heads to the next stop, the status will change to “Departed”. Once each stop goes through its transition, the shipment is completed.

Event Sourced Aggregate

Current State

Here’s an example using only the current state without event sourcing. The state is represented by properties on the ShipmentAggregateRoot and the list of Stops.

The primary reason we have this aggregate is to control the invariants and create a consistency boundary. Check out my post Using Invariants as a Guide when designing aggregates. In this case, we have a few invariants but the one I want to point out is in the Arrive() method. We need to check that all other previous stops have Departed and gone through their full progression before we can call Arrive on the next stop.

Event Sourced Aggregate

The first step in moving our aggregate towards event sourcing is actually creating events.

The next step is defining the state that we need in our aggregate. This is the key point of this post. The state we need is only for our invariants (business logic) in the aggregate. We do not need any other data in our state other than what we use for business logic.

Finally, here is the new Aggregate that is using the defined Events and the ShipmentState.

If you compare the Event Sourced version with the previous version, you will notice that we’re sorely focused on the behavior methods and the business logic. The projection/state we’re recording is much slimmer as it’s now recorded separately in the events.

Our tests are incredibly simple as we don’t really need to build up a large data model for our aggregate.

Source Code

Developer-level members of my CodeOpinion YouTube channel get access to the full source for any working demo application that I post on my blog or YouTube. Check out the membership for more info.

Related Posts

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design