Category: Event Sourcing

I asked you for your questions about Event Sourcing and CQRS. I took some of those questions to Greg Young for answers about frameworks, Legacy Systems, Use Cases, Projections, Eventual Consistency, and Versioning.

YouTube

Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.

https://www.youtube.com/watch?v=LGjRfgsumPk

Makes sure to watch the video for all of Greg’s answers. I will put some thoughts together about each of the questions that people on YouTube, Patreon, and Discord asked.

Frameworks

Greg has mentioned that he doesn’t think that CQRS requires a framework. I agree with this statement but understand why some people are often confused.

I’ve found that a good majority of people are confused about CQRS for two reasons. First, they conflate CQRS with Event Sourcing. They think of them as one of the same when they aren’t. CQRS is about isolating reads from writes into different code paths. Event Sourcing is about using events to record state. Check out my post CQRS Myths: 3 Most Common Misconceptions for more.

Legacy Systems

If you discovered event sourcing or were already aware of it and you’re working within a legacy system that persists current state, how do you go about incorporating event sourcing?

The Strangler Pattern comes to mind. However, I think it goes beyond that. I will assume that when people refer to a legacy system, they talk about a monolith without any logical boundaries. And that’s where I’d start. Start defining logical boundaries and take a small logical boundary or just a portion of it, and then from there, define how you want to persist state. Maybe Event Sourcing is a good fit, or maybe it’s not, but you’ve made a giant win by separating a subset of functionality and the data behind it.

Check out my video HELP! My system is hard to change that talks about this.

Use-Cases

A common question is where (in which domains) event sourcing is applicable. I’ve found where it fits naturally is when the people within the business that understand the domain well naturally speak using events.

For example, when a vehicle is on a trip to pick up and deliver goods, these are often called trip events. The vehicle arrived at the shipper, loaded the freight, departed the shipper, arrived at the consignee, and emptied the freight. These are all events that they aren’t anything technical at all. These are common terms used within the industry.

I’ve found when people naturally talk in events, it’s often at the core of the domain you’re working in.

Queries & Commands

Can you use a query within a command, often for validation purposes? In other words, could use an existing read model (projection) that is eventually consistent when executing a command.

While this question comes up often, the answer is pretty straightforward. Yes, so a long as you don’t need to be fully consistent. Your read model will be stale if it’s being built asynchronously. If it is, then if you use it to query to impose some invariant when executing a command, then you need to be fine with it being stale. If it cannot be stale, then you can’t use it.

Eventual Consistency

This was my favorite part of the discussion I had with Greg. eventual consistency is always brought up with asynchronous projections and UX. I’ve done a post about Eventual Consistency is a UX Nightmare that provides some solutions, but this conversation took a different route talking about non-linearized timelines. Typically I’ve always been thinking about linearized event streams, but you might not always be. Check out this talk from “want don’t You consistency no stinking“. After you start watching the talk, the title will start to make sense!

Versioning

Versioning events can be difficult when you’re defining more technical events than domain events that the business actually cares about. I think this is a bit of trap developers can get into with Event Sourcing or using events as notifications. If you model your events after solid and established business concepts, you’re less likely to fully regret defining them. That’s not to say you won’t need to version an event that was based on a business concept. There are many strategies, and the most common is to upcast old events when replaying them to the newest event. Another approach is to really look at the event change and determine if it’s a totally new event that should be defined.

Check out Greg’s e-book Versioning in an Event Sourced System.

Thanks to Greg Young for taking the time to sit down with me and go over what I think are the most common questions related to CQRS and Event Sourcing.

Join!

Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.

Related Links

• Event Sourcing Example & Explained in plain English
• CQRS Myths: 3 Most Common Misconceptions
• Eventual Consistency is a UX Nightmare

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

Leave this field empty if you're human:

What are domain events for? Domain Driven Design and Event Driven Architecture are nothing new, but they continue to become more popular. Unfortunately, some concepts and terms get a bit murky as they do. Specifically, the term domain events have caused a bit of confusion. I will clarify a misconception about the role and purpose of domain events and how that relates to Event Sourcing. Or does it?

YouTube

Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.

https://www.youtube.com/watch?v=dJBTNksQzys

Event Sourcing

As a primer on Event Sourcing, it’s a way of recording state. As a comparison, if you were recording the current state in a relational database or document store, you’d think of rows/tables or objects/collections.

As an example, let’s say we have products in a warehouse. We’re recording the SKU (productID), the quantity on hand, and when we last received and shipped the product from our warehouse.

With event sourcing, however, we’re recording events that represent the state transitions.

Each product, represented as a row in our relational database table, has a stream of events in our event store.

For more basics on event sourcing, check out my post Event Sourcing Example & Explained in plain English

State

Event Sourcing is about using events as state. This doesn’t mean they are used as a form of communication.

Thankfully over the last several years, for the most part, the industry is accepting of not integrating at the database. Meaning instead of being able to interact with another service database directly, you must use a defined API. Gone are the days of calling another database directly.

Unfortunately, this idea seems lost when Event Sourcing—often thinking of events as am means of communication with Event-Driven Architecture. However, event sourcing is about state. Just because you have an event store, does not mean another service can reach out to your event store.

Domain Events

Domain events are inside events. They are internal to a logical boundary. Their purpose is for notification. To express that some business concept has occurred. There are situations where you may want to expose them to other boundaries. Often this is because they are well-established business concepts that aren’t going to change. In this case, exposing a domain event is reasonable. Treating a domain event also as an integration event. For more on this, check out my post Should you publish Domain Events or Integration Events?

So then, aren’t the events used in event-sourcing domain events? They are internal events. Yes, they can be, but they don’t have to be. Not every event persisted to your event store has to be a domain event. Likely they will, in most cases, represent business concepts, but as mentioned, they are also about state transitions.

You’re using events in event sourcing to reconstruct an internal state within an aggregate, so when you then perform an action (do this!), it can validate if it’s in a state to perform the action. And if valid, then appends the subsequent event that represents the state transition.

Events in event sourcing may be more fine grain than what is needed for communication purposes. Domain events may be less granular and represent the completion of a workflow.

As a typical example of a shopping cart and checkout process, let’s say we have an event stream that is:

• ItemAddedToCart
• ItemAddedToCart
• ItemRemovedFromCart
• ShippingInformationDefined
• BillingInformationDefined
• CheckOutCompleted

Are these events used to notify inside (and possibly) outside boundaries? We surely need them as our state transitions (event sourcing), but do we need them for communication (domain events)?

For communication, we probably want an OrderPlaced event to define the completion of our workflow. It likely will be a domain event (or integration event) to notify other boundaries. Still, all the others are probably too fine grain.

Domain events are about business concepts that the business cares about and understands. They are used for notifications/integrations within or outside a service boundary.

Events used for Event Sourcing are about state. They are capturing events to represent the transitions in state.

Domain events can be used for event sourcing, but not all events in event sourcing are domain events.

Join!

Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.

Related Links

• Event Sourcing Example & Explained in plain English
• Should you publish Domain Events or Integration Events?

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

Leave this field empty if you're human:

When people are getting into Event Sourcing, there are a few common questions that I often get or issues see people run into. CRUD Sourcing, Pre-mature optimization using Snapshots, and exposing your event streams for integration. Here are my top three Event Sourcing Tips to help you down the right path.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything in this post.

https://www.youtube.com/watch?v=SYsiIxJ-Nfw

CRUD Sourcing

My first event sourcing tip, which is probably the most common issue I see people run into when new to Event Sourcing is what is often called “CRUD Sourcing”. If you’re used to developing applications/systems in a Create-Read-Update-Delete style, then this means you’ll likely end up creating Events that are derived from Create, Update, Delete.

There’s a shift when moving from an application that simply maintains the current state via CRUD to having your point of truth be a stream of events. Often events are artifacts of the state change as well as the business event that caused that state change.

If you provide a UI that’s CRUD driven on Entities, you’ll end up with events that are derived from that. As an example, you’d start creating events such as ProductCreated, ProductUpdated, and ProductDeleted.

Still in line with this is if you have updates that are for properties on Entities, you’ll end up with events such as ProductQuantityUpdated or ProducePriceChanged.

In both cases, the events are simply representing state changes but not why the state changed.

What was the reason why a ProductUpdated occurred? It was updated, great, but why?

How about the ProductQuantityUpdated event, why did that change? Was it because there was an inventory adjustment? Did we receive more quantity of the product from the supplier? Why was it updated?

Being explicit about the events is important because we want to be driven by business concepts. To get out of CRUD we need to move to more of a Task Driven UI. This allows us to have the client/UI explicitly perform a Command/Task. For example, if the user performs an Inventory Adjustment as an explicit Command/Task, that’s a business concept. We will generate an InventoryAdjustment event.

Being explicit is important because you do not need to derive or guess based on the data of the event and why it occurred. You’ll have many more answers to questions when you look at an event stream when they are explicit. As an example, when’s the last time we did an inventory adjustment? When we do inventory adjustment, how many times are we decreasing the quantity on hand? You cannot answer these questions with a ProductUpdated or a ProductQuantityChanged.

Optimizations

Once people understand Event Sourcing and how it works, the most common question is:

That seems really inefficient to have to fetch all the events from a stream to build up current state! What happens if I have 1000’s events!

To understand how event sourcing works check out my post Event Sourcing Example & Explained in plain English.

As a quick primer, you have a stream of events for a unique aggregate, such as Product with a SKU of ABC123.

Anytime we want to perform a command which will append a new event to the stream, we’ll generally fetch all the events form the stream, build up the current state, then enforce any invariants for the command we want to perform.

In the stream above if we were keeping track of “quantity” as the current state, it would be 59.

So back to the common comment of “that’s really inefficient”, is a pretty valid concern. The answer to this problem is called Snapshots, but they are an optimization that you don’t necessarily need to apply right from the start or often.

In my experience, event streams are generally finite and have a life cycle with a beginning and end. There may be a long time duration for how long a stream is “active” but the events that are persisted are often limited.

If you have a lot of events and it’s taking a long time to rebuild the state, then creating snapshots can help. They are a way of creating a point-in-time representation of the state.

After so many events append to a stream, you persist another event to a separate stream that is the current state, also recording at which version of the stream it represents. This way when you want to rebuild the current state, you first get the last snapshot and then get the events from the stream since that snapshot was created.

Now the question is, when do you create snapshots? If you think the event stream is going to contain a lot of events, how many for a given situation is a lot? Each different type of event stream is going to have different events which contain different data. There’s no magical number of events that is a threshold for creating a snapshot, it’s going to be use-case specific.

Event Sourcing Tip, don’t jump to snapshots immediately, look at how you’ve defined your streams and boundaries.

Communication & State

This last mess people get in with Event Sourcing is conflating events representing state as well as using events as a way to communicate with other service boundaries.

Your event store, the event streams, and the events within a stream represent the state.

Often with Event Sourcing, you’ll create Projections as a way to represent the current state for Queries/UI/Reports. This way you can query a separate data store that’s pre-computed the current state. This means you don’t have to pull all the events to a stream to build the current state. It’s already pre-computed as events occur (usually asynchronously)

This means you’ll have two different databases. One for your event streams and one for projections. This is all within the same logical service boundary.

Now if you’re using an Event Store that supports subscriptions, this doesn’t mean that other logical service boundaries should directly access the Event Store.

People often do this as a way of Pub/Sub which is used for communication. But there is a difference between Events used inside a logical boundary to represent state and events used to communicate with other services.

You wouldn’t have one service connect to another service’s relational database, would you? Then why would you access its Event Store?

Domain Events used within a service boundary to represent the state are not integration events.

Domain Events: Inside Events

Integration Events: Outside Events

You want to define which events you want to publish to the outside world for integration. Domain Events and Integration events will be versioned entirely differently. The moment you expose a domain event to the outside world you potentially now have consumers that are going to rely on it. If you don’t publish your internal domain events, you can refactor and change events very differently.

If another service was interacting with your relational database, and you made a change to a column name, you’d break them. If you change an event that’s not backward compatible, you’re going to break consumers. Don’t expose internal domain events as integration events.

Event Sourcing do’s and don’ts

Hopefully, these three Event Sourcing Tips give some insights that can help you if you’re new to Event Sourcing or if you ever questioned how various aspects work.

Join!

Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design as well as access to source code for any working demo application that I post on my blog or YouTube. Check out the YouTube Membership or Patreon for more info.

You also might like

• Event Sourcing Example & Explained in plain English
• Projections in Event Sourcing: Build ANY model you want!
• Snapshots in Event Sourcing for Rehydrating Aggregates
• Keynote – Event Sourcing @ DDD Europe

Follow @CodeOpinion on Twitter

Software Architecture & Design

Get all my latest YouTube Vidoes and Blog Posts on Software Architecture & Design

Leave this field empty if you're human: