[Music] Almost my entire career, I've heard people talking about event-driven architectures and event-driven systems, and things of that kind, and it's always been a little bit odd as to what actually do people mean by that, because when I dig a bit further, it seems that there are various different things going on. There is some kind of common theme, events are or something called an event is an important part of what's going on, but actually what it boils down to in terms of something more meaningful, that's much more open-ended, and for a long time, that's just been ticking around in the background, and I have at various points in my career sort of thought I want to really dig further into this, and I kind of never really got around to it because I dug into other things instead, but it really kind of came to a head late last year. At Fort Works, we've been doing quite a lot of event-driven stuff, and various people came to roughly the same conclusion as me, saying we've been

doing all of this stuff called event-driven. We're not really quite sure it's the same and we're doing it at all these different project teams around the world because there's several thousand people now at Fort Works and lots of different projects. We need to get together and learn from each other, so we organized a workshop in Denver. It was, yes, Denver, late last year, and a whole bunch of people got together and very quickly it became apparent that yes, there were some very distinctly different things going on under the title of event-driven, and we need to try and figure that out a little bit better. And as a result of that, I think we came to a bit of an understanding of some common patterns that help provide a bit more precision into what we're talking about. And so we thought, well, it's important that we describe this and say a bit more to the rest of the world about some of our thoughts. And I hoped really hard that somebody else would write it up, and nobody ever did, but one of us did a little video thing.

just for an internal video. What's it? And I thought, Oh, I can't stand it anymore. So I wrote a little note and stuck it up on my website, not really expecting it to get much traffic. And I saw it got downloaded with a huge amount of traffic. So I thought, well, actually, this seems to be a common problem. And that's what led me to decide to inflict this talk on you this morning, is because it seems to be that people are concerned about this. And what it boiled down to is we identified four different things. I would call them patterns that often come under the term event-driven. An event-driven system might, you know, when people say they have an event-driven system, they usually mean they've got at least one of these patterns in play. And could mean they have all four patterns in play. And I will go through them one at a time, gradually revealing what they are. So to begin, the first one. Let's imagine some example system that might be had by an insurance company. And in this insurance system, we have our customers and our customers

have a habit of moving around and changing their address, but the insurance imagine it's something like car insurance or house insurance, where changing your address actually affects your insurance quote levels and your premiums that you have to pay in everything. So, if somebody changes their address, it's not just a case of updating the address field. You actually have to trigger the insurance quoting system to come up with a new quote, and they then got to send you some mail to say, Here's your new insurance agreement, and whatever, whatever. I don't know the details. It's a long time since I spent any time with insurance people, but the point is, in a kind of simple call and response kind of architecture, what that requires is whatever system you've got that's looking after customer management needs to tell your quoting system, Hey, go and do a re-quote. And the problem of that is that creates a dependency from the customer management system into the insurance quoting system. It needs to know that it's there. It needs to know what APIs it has, etcetera, etcetera.

etcetera and that couples the knowledge of what how insurance quoting works into something that's a much more generic customer management kind of feature. And that's a kind of coupling that a lot of people don't really like very much. So, one way we can get around this coupling is we can say, instead of doing that, let's try and reverse things around. I don't mind my insurance coding system knowing about my customers, and that there is a customer management system there. That seems a more natural dependency line to follow because uh, something that's very very generic, managing customers generally, that's the kind of thing you want to depend on and have it depend on lots of other things. And a way we can do this is we can use an event. So, what happens is, whenever the customer, some customer data changes, the customer management system emits an event object to some well-known place, event queue or whatever. And then the insurance quoting system can listen for that event, and when the event occurs, it can decide, do I want to do something about it? If so, what do I do?

and then that nicely gives us that ability to reverse the dependencies. This allows the insurance quoting system. Now has to know about this event structure, but the customer management system can be independent of all of this. And this is an approach that we in our discussion, whoops giving you a slide away there in our discussion. We referred to as event notification. We're using events as a notification system, uh, no notification mechanism between multiple systems. Now, of course, I've described that in terms of large-scale systems, independent relatively independent systems for a big company, but we also use this same pattern in a very small layer system. And it's a classic, of course, approach of how to manage interactions between graphical user interfaces and the rest of your code. When somebody enters information into a text box, you don't want some very generic text box widget to have to know all about your code and which things to call. Instead, you say the event. The text box will generate an event when you change it. And then your code listens to the event. It's the same idea. And again.

the same trick is you are reversing dependencies, and that's really, I think, the essence of why you want to do this. Is that you want to do that reversal of dependencies. Now, I've talked about this in terms of events, and I've stressed this dependency switch, but it does, but it also does something else. It bottles, it sort of takes everything to do with the change that the event talks about and bottles it into a thing, an object, a record, or something of a cert that you can refer to and pass around. It makes the change a kind of first-class thingamajig. Now, when we do, when we call a function, that's just a very transient thing. We call the function, supply the arguments, goes away. We never know that we've done it unless we've got some logging and tracing going on. But when we do something like this, what we're doing is we're actually making a record of it, and then we have a real thing that we can talk about and refer to and pass around, and that has its own benefits.

if we once we can do that so we're actually getting a second benefit here, which is this bottling it up into a into a thing, into an object. Or I don't have to hesitate about using objects. It doesn't have to be an object in a 0 sense. I think of it as a just some kind of first-class thingamajig that needs to be a new technical term. First-class thingamajig. God, I don't even get a giggle for that. That's a shame. Now that's not the only way of thinking about these first-class thingamajigs. Often we think of them in terms of commands as well, and that leads to an interesting question. Well, when do we have events and when do we have commands? What's the difference between an event and a command? And I think it's actually, it is both very subtle, not really anything that you can say is a strict definition, and yet at the same time very important because it's about our language and our communication and how we talk about our intentions.

about things. We could describe that situation I just described as the customer management system saying, Re-quote insurance for this customer. That's the customer management system saying it knows what ought to be done, and it's telling some remote system, Here's what you should do. When I phrase it in terms of events, I'm saying the customer's address changed. I'm not expecting any particular response from that from anybody. I'm just indicating a change. Both can have this kind of bottling it up and making it a first-class thingamajig effect, but the way the communication patterns differ is in strongly implied as different, whether you use event naming or command naming style naming. So really, it boils down to this is one of the two hard problems in software, right? This is naming things. And therefore, when you're thinking about the difference between events and commands, it's really how do I want to describe how the system works most accurately to the people who are reading about the system? If I'm trying to say, I don't really care.

what happens as a result of this event. I don't want to care. Then yes, I want to use the word event. If on the other hand, I am particular about I want something to happen, then I want to name it in terms of a command. Now, I mean you can get all the other way. Now, you could have your insurance quoting system might insist on sending a communication, an email to the, to the customer. And natural way to do that, of course, will be a command, send an email. But of course, if you go too far down the event route, you, you then get this effect of passive-aggressive events, right? Or passive-aggressive commands where you're saying the customer needs to be sent a message about their new quote, hint, and you know you don't really want to go down that path if you're trying to read how the system works. That's going to be confusing when you get those kinds of events occurring. So whether you use events or commands under the covers, they're basically the same thing. They're often messages on cues. They look exactly

the same. The difference is all about naming, but that difference is really important because let's face it, names in software systems are one of the most important things we have to deal with. So, going back to our system, again, one of the nice benefits of using events is it allows many other systems to hook up to the event stream very easily without us having to tell the customer management system what's going on. And this is a very nice property, a nice result of using events for notification. Here, I need to hook up a new system that takes advantage of change, needs to know about customers changing their addresses. I don't have to go and talk to the customer management team and get them to add a new no to no call into their interfaces and go to endless meetings with the customer management group. And two months later, maybe I will get them to do something. No, I have they produce their events. I just tap onto the event stream. I'm cool. This is a very attractive ability and it easily allows you to do all sorts of interesting things without changing.

what's going on? But when anything sounds like it's really nice, there's always a but. Isn't that? And in the case of event notification, that but is now you're in a situation where you've got no way you can figure out what the hell is going on in your system anymore because there's no program to look at. There's no statement of says these are all the things that happens when a customer changes their address. What you have to do? The only way you can figure out what's going on is you actually have to watch you address the various events and see what happens looking at the flow of messages through your system to try and figure out what the hell's going on. And this is true in the GUI case. If you've ever tried to debug a heavily event-driven GUI kind of system, pretty much your only chance is to bring it up in the debugger and try and trace the various events that are flowing around. I remember doing this once to try and figure out how the original Smalltalk 76 worked.

because i was writing some stuff about model-view-controller, and i rapidly discovered you couldn't trust any of the papers that had been written on it because i often didn't have access to the source code. So, I was able to get hold of a copy of the original MVC Smalltalk 76, bring it up, and I could actually figure out how it worked. But I couldn't figure out how it worked by looking at the code. I had to step through the debugger, do things, and see what happened because, again, it was all events. And that's the dark side of using event notification. You get this great decoupling, you get this great flexibility being able to add new consumers, and you get an inability to see what's going on across the system as a whole. There's no code anymore that you can just look at. And it's a trade-off, like so many things, trade-offs. So that's the first thing: event notification. And that, I would probably say, is the most common thing that people say when they're event-driven, they're using an

event notification mechanism somewhere which is kind of odd in a way because just about every system i've ever looked at uses event notification to some extent. I guess when you call it event-driven, it means the event notification system is seen by at least the senior people on the project as a particularly important part of the architecture. So the second one is a minor variation on this case. So let's go back to our little example of the insurance company again. In an event notification system, all you really need to know is that the event occurred, and that could really be very, very generic. It could just be saying, Hey, something changed somewhere. You probably want to give a little bit more information on the event about that. So you might say, Ah, the event thing for Linda changed. Something's changed about Linda's account. And then all the other systems go, What's happened? The way they find out what's happened is they have to talk to the customer management system. And even if you got more precise about this, you might say, Linda's address.

changed they've still typically the insurance code system has okay. So what's the new one? What was the old one? You know, has it changed in the way that's important to me? Event notification often involves additional traffic because the systems that are the consumer system still have to go back to the original source to get more information. And of course, we reduce that burden by putting more information into the event. If I just say something about Linda changed, then every time you have to pile in on, and it's probably half the time it's not worth it, doesn't actually mean anything. So it was kind of a waste of time. If I'm saying here's the address change, then there's less traffic required because I only need to go in if I care about an address change. And then I can take this to the logical final conclusion and say, can I put so much data in the events that I don't actually need to go back to the host system? So not just do I say Linda's address changed, I say what her old

address was what her new address is at that point. Maybe I don't have to go back, or maybe I do. I know the address changed. I know I need to do something, but now I need to go back to all of other attributes, all the other things I know about Linda Rising, her browser history, how often she stands up every day. All of these things we know and we can find out by asking the customer management system. But now I've got a customer management system that's got all of these queries every time something happens. Every other system is bombarding me with stuff. Can I reduce that? Can I eliminate that load? Well, I could. And this is a pattern that we tentatively named event carried state transfer, which is a bit of a long-winded name, but I like it because it's a play on REST, right? Which is resource what's it state transfer. This is event carried state transfer. This is about using events that carry Stanford of state. What happens when you use this pattern is you say I want to completely forbid

any contact to the customer management system from the insurance quoting system, which means the insurance coding system will keep a copy of all the data that it's ever going to need. Now, that means, of course, the customer management system has to broadcast in its events all the data that those downstream systems are going to want to have, and the downstream systems have to keep copies of everything that they are going to use. I don't have to keep a copy of the entire customer record; I only have to copy the stuff that I care about. So, it's not as bad as one might think, but it's still obviously a lot of copying of data. But the benefit is I now no longer need to call the customer management system at all; I have all the data at hand. So, that might improve my performance because, you know, I no longer got remote network calls to the customer management system. It reduces the load on the customer management system because it's not responding to all those calls. It also may improve availability because if the customer management system goes down or it's intolerably slow.

i don't care. I keep chugging along because I have all the data for myself, so we can immediately see with some really nice benefits to do with that. Now, it doesn't mean I have to do all the necessary stuff to make sure that I'm up to date. I keep copies of the data. It means, of course, I have to copy all the data, but we're not talking about that much data these days. Storage is cheap, and what's more, I have my insurance quoting system. My availability no longer depends on the customer management system, so that's a good thing. Now, hopefully, you're well enough versed in software development that you know when people say, I have greater availability because I'm not dependent on something else, you know what the price is. What's the price of high availability? Lack of consistency, exactly. So now I've got to replicate the data. I have eventual consistency. I have to deal with that issue, so I've got a consistency problem. So again, event carriage state transfer has this um trade-off. I get even more decoupling, even more.

availability, but I lose complexity of keeping the duplicated data and inevitable eventual consistency. So this is a less common pattern. I don't know what the proportions are. I would say a lot of the time that I see event notification, I don't see event state transfer, state transfer, but you and you do have to have event no event notification to do this, but it is a technique to bear in mind. I would say this is one that you keep deeper down in your toolbox, but it's many circumstances it can be a good one to play. So sometimes that's what we see when we talk about event-driven systems, this notification of duplicating the data to avoid calling back. So the third item so far what I've been doing is I've been telling you the concept and then telling you its name because I kind of think it's more exciting that way. You're kind of a bit more of course if you read the talk beforehand you already know what the four things are so I'm not really quite sure why I'm doing it but for this one

i'm going to tell you what it is right away event sourcing which is a technique that has long been one that's fascinated me um and it's interesting to see how it's being how people are using it but let's begin by explaining what it is what do i mean by event sourcing by the way how many people have come across the term event sourcing stick your hands up to some curious okay smattering of people how many people understand it well enough they think they could explain it to somebody else a few okay i'm not going to ask you to come up on stage and do it don't worry okay here's how i explain event sourcing and surprise surprise is the change of addressing example again because i'm kind of boring so let's imagine i've got some kind of customer management system and i've got somebody with an address and they want to change their address in a typical system how do you change address you just say okay let's blow away the old address create the new address

right that's how it works. Right, nice and simple. What event sourcing does is it creates this subtle change to the picture. So now what we have is we still have this representation of the current state of the world, but we've also got a second little area as well. And when I say change my address, the first thing that happens is we create an event object. So this is this first-class thingamajig that captures the details of the change, and we pop that first-class thingamajig, the event, into some kind of separate storage area. Then we process that event, and we process the event to do exactly the thing that we expected to do earlier. We change the thing over. The consequence of this is that we've kind of now got two sort of representations of our world. We have the what I refer to as the application state, which is the current state that we're thinking of of the world, and then we have a log of all the events that ever change that world. And this is the crucial test for event sourcing, the sort of the definitional.

test is that at any time we can blow away that application state and confidently rebuild it from the log. That's the test. I mean lots of people log changes and things like that. It's the ability to replay the log and confidently rebuild your state so that basically you don't care about your application state and trying to preserve it anymore because the vital thing is to preserve the log. If that's the situation then you've got an event source system. So how many people here have built a system using event sourcing? This approach? Smattering of hands, few hands. How many people here have used the system using event sourcing? A few hands, not very many. How many people here are software developers? Put your hands up. Okay, I'm going to assume you didn't quite understand what I've been talking about for the last few minutes because otherwise we are really screwed. All those people who haven't used an event source-based system and are software developers. If that were true, I would be really, really worried. In 2010, let alone 2017.

because every software developer should be intimately familiar with one particular form of event source system, which is version control. If you use git, use subversion, mercurial, even if you use cvs. And I have, it was a long time ago, but I used it, you've used an event source system. Hell, I use sccts, you know, it shows how old I am. And this is, I think, a very good point. Event sourcing is basically providing to your users a system that works on their data the way version control works on your code. That I find a very good way of explaining, at least the software developers, what event sourcing is about. And a lot of things about event sourcing you can reason through by thinking, well, what happens in my version control system? What does git do? What does subversion do? And then applying that same question to the application state that you're in. In your version control system, your application state is your tree of files, your working copy, um, your work tree. And the log is, of course, all the commits that have occurred through.

now, of course, the system internally doesn't necessarily store it as all events, but it stores it effectively as it were all events. There are snapshots that are contained at various points to speed up performance, which is basically a copy of the entire application state as at some commit. But by moving back and forth between snapshots and using commits and being clever about how that all into plays, it gives you the sense that I've got every single commit, which is kind of the same thing of saying I've got every single snapshot, but it's slightly more powerful to think of it in terms of every commit. Now, that's a good metaphor for us geeks. It's not a terribly good metaphor for business people, but fortunately, there is another really good example of event sourcing in the business world that any business person will be familiar with: accounting ledgers. Right? If you're honest, my current application state is however much money I've got in my checking account. But of course, behind that is every single credit and debit to that account. And of course

ledgers and accounting systems are used by all sorts of people. And that is very much an example of this event sourcing approach. And again, notice the fact that it's a combination. In reality, it's often a combination of every single change and certain snapshots. A common thing in accounting is to say at the end of a year or something. We would say we're going to close out the year. We calculate all the balances for the end of the year. And then we kind of ignore everything that happened before then. And we build our events on point since then. That's a snapshot effect where effectively you're throwing away history behind a point in that case. Um, but it can also be you don't throw away the history. You just use it to make it easier to calculate what's going on for this year's money. And if you really need to, you can dig and go further back. So again, the storage is usually some mix of events and snapshots done in some way that is efficient for your capability. But you want to always

be able to synthesize that recreation of the events. So the question with event sourcing is, how can we use this more widely because there are many interesting benefits but also costs with this pattern. Um, immediately off the top of the head, you can say, well, it's obviously a great audit system. If I want to figure out what the hell happened or how on earth did this person get sent this quote that was completely ridiculous, I can look back through the events and figure it out. And that's why anybody who's involved in audit really loves having an event source system. That's why, of course, we have ledgers because that's one of the main reasons because of the audit. It's also a fantastic debugging tool. If you're trying to figure out why your application state got screwed up at some particular point in time, you can actually take a copy of the system, feed it the same events, and watch what happened going back in time. And of course, the time travel, going back and seeing what was the application state a month ago.

two months ago we're familiar from doing that from version control right, and it's really useful to be able to go back and forth across time. Um, the alternative state is also effectively branching, and that can be very useful in certain circumstances. Um, I've worked on a few systems where we might have notification events, and we deal with them. The example I always use is payroll. Somebody worked for 35 hours in a week. We record that. We send them a check. We update their sick pay benefits, the pension benefits, all sorts of downstream stuff happens. And then six months later, somebody says, Oh, actually it wasn't 35 hours, it was 40. That's a really complicated exercise to go and then undo. Well, not you can't undo because you can't change it, but then to recalculate what the adjustments should be for all of that. But if you've got an event source system, you say, Okay, I'll tweak that. I'll copy it to a different system, tweak the event, calculate what the current state is, look at the two application states, diff them.

and do an adjustment. I mean, it obviously is a bit more complicated than my casually saying, Why do we just diff the application state? But you get the picture, right? You actually get, you can actually do it, and it can be a long computation. It can take, oh, a couple of minutes, but that's way faster than doing it manually, and that can be a really powerful thing. Last item on that on the list of good things is actually one of the most intriguing: the idea of using a memory image. This basically says if our long-term record of what's the state of a system is a log of events. Actually, I should walk over here and give you this end of the room a treat, force the cameraman to wake up. If I do give that long-term view, then actually I don't need to store the application state in any kind of persistent form. Why do I need to store my application state in a relational database if I've got a log of all my changes? I can have a purely in-memory system.

and that has some interesting benefits. A great example of this is a trading system called LMAX. It's a retail trading system, and it was built a few years ago. It has to be very high performance, very low latency, and they looked at various ways to try and achieve this. They ended up with saying, We're going to run the entire system on a single thread in Java. And their headline was, We can do six million transactions per second on this single thread, really, really blazing fast. One of the keys to doing this was all of their data was in memory. Now they have complicated transactions, they have lots and lots of things to look at at once. They didn't want to be going to a database. They have it all in memory, and they were able to process really, really fast. And if anything happened and the system went down, they could very quickly rebuild the state from their application logs. They would snapshot overnight, so basically you run all the events from the beginning of today and work that way. And in fact, again, one of

the advantages events is you can have multiple systems running off the same event stream. So they had a very good way of having hot backups. They basically ran two systems all the time with one of them being the lead one. And if it went down, the second one would instantly take over. Very, very nice system, very, very performant. One of the interesting things I was talking with one of the lead developers, and he said, actually, I mean the performance was fantastic and really got people's attention. But one of the things that I really miss about programming in a different system is that is the nicest system I've ever programmed in. Because there was no database to ever deal with. Everything was in memory all the time. And as a result, we could have a really good rich model. It was particularly good for complex logic represented in a domain model. And it was a real pleasure to work in compared to many other systems. Um, I'm surprised in many ways more people haven't taken that kind of approach. It's a bit unusual, and you have to set up the

infrastructure to make sure that you're able to do the events with that, but it is actually quite a powerful model, and it may be getting more interesting as we get more towards systems that have large amounts of non-volatile main memory. Um, because so much of us of our systems have been built under this assumption that we've only got so much memory, we have to park everything else on disk, and now we got tons of memory, it all fits in RAM for so many applications. So maybe we should take advantage of that memory image is a way to do it, but you still have to have some kind of persistent store and communication mechanism, and the event log provides that. So event sourcing can be very useful in that situation. So those are some of the upsides, but as implied, there are downsides. Okay, I'll go back. I hope you enjoyed that not having to crane your neck for a while. So it's an unfamiliar system that's bad because things are unfamiliar are harder to work with talking to external systems gets a bit

more complicated because if I want to be able to rebuild my application state, I can't just say, Oh, I called this external system two weeks ago. What was the result? Because I can't call it again two weeks ago. I have to make sure I save basically every response from an external system as an event and make a partner replay mechanism. I have to think about how do I store my events in a way that I can confidently replay them even as I'm changing the code that processes them. So, event schema starts becoming interesting to be able to do that. Also, a little little thing but awkward when I generate identifiers for things. I have to be careful that I can do that in a way that I can replay. This rule that says I must always be able to replay the events, build the application state. There are these little tricks, little traps for you early on that you have to deal with, and you have to think through how to get through those. So, those definitely do add complexity, and you get other complexity that typically comes along for the ride.

one of the triggers for doing this talk and the workshop was when i ran into people that were running into difficulties with event-driven systems. But it wasn't clear what the source of difficulty was. So i remember chatting to somebody on a project that was in difficulties and they were saying oh i will never ever use cqrs again. I said oh okay you had a hard time with that. Oh so rewind got that wrong. He said i will never ever again use event sourcing and i asked him why and he said well because every time i want to do everything i have this read model to build and the right model to build i have to do twice as much work and i thought well hang on that's nothing to do with event sourcing that's something to do with something else and you often find that people will mischaracterize where the problems are with event sourcing. I've often heard this because the problem and the problem on that project according to the lead architect was asynchrony. They processed all their events in an asynchronous manner and asynchronous.

processing got difficult for people because asynchrony tends to be hard to work with. Hard to reason about. Now there's no reason why you have to have asynchrony with an event source system when I just make a local commit in git. That's not an asynchronous operation. Now, okay, when I, when people are pushing up to shared repositories, then asynchrony comes into play. Um, but there's none of that. It's all synchronous operations that I'm doing a local commit. But often people take advantage of the event sourcing to introduce asynchrony into the system, which can be a good thing because it improves can improve responsiveness. But asynchrony then adds complexity. And so the reason I put that as a kind of question mark there is asynchrony often comes along for the ride with event sourcing systems. And it does add a lot of complexity. But it doesn't have to be used. You don't have to use asynchronous event sourced approach, even though it seems common that people confuse that. Another issue is versioning. Versioning

can get quite complicated in an event source system because of this ability to replay, and particularly when you think well, the code, of course, is also changing in this system over time. If I change the structure of my application state and the schema that I'm using for my application state, can I still process that event that was created a year ago? What does it mean to create to do that? Do I need to change my event schemas and version them? Versioning can get messy. Again, it doesn't have to be the LMAX example I mentioned earlier on. They didn't run into that problem very much because they only operated on a one-day snapshot and events from that snapshot, so they didn't have to worry so much about the versioning aspects, but it can be a killer for other systems. Um, and I'll mention that there is a book in progress or close to some degree of completion by a guy called Greg Young. If you look on Leanpub about versioning for event sourcing systems where he goes into a lot of detail about the various options.

you have for handling event sourcing so handling versioning and event sourcing. Greg Young is the creator of event store, which is a storage system specifically designed for events. He's been working in the event space for quite a while. You can find a bunch of his lectures and things on video. I've been trying to get him to write a book about event sourcing for years, so far without success. So if you ever do see him, tell him to write that book. So event sourcing is like most things, a trade-off. There are pros and cons. There are some really nice advantages. There are complexities involved. I do find it surprising that more people haven't made use of it approach. However, one more thing to mention about event sourcing though, and that is exactly what we record as our events in the event source. Think about the code example for a moment. I go into my code, I see some function with a name that I don't like anymore, and I want to change it. So I go and change it. Fairly straightforward refactoring operation. How do I represent that change?

in terms of event sourcing, there's two events going on here. One is my intention to change the name of this function, and the second is all the actual edits that I actually have to do in my source code, which is not, of course, just in one place. It's the declaration of the function and everywhere that calls it. Now, as far as any version control system I've ever used is concerned, it only sees the latter. It sees a series of textual diffs that have absolutely no way that you can tell the intention from that, except by looking at the diffs and maybe the commit message to say that's what I've done. And we've talked in times of how it'd be nice to have automated refactorings that were storable, that were first-class thingamajigs that you could apply across your project. But in practice, it's very difficult to do that because of course, a rename function method, function capability for that would work with JavaScript is not going to work with Java, it's not going to work with Python.

so i have to language specific changes in my version control system that starts getting very hairy while text works with everything. we know how to diff and merge and deal with text changes so we rely on text. but then we lose the intention. we lose what's the reason why same thing. basically happens with business stuff as well. we say i've got some intention of something i want to do and i might want to capture that. and as an event and i think of that as a kind of an input event. and then i have how does this affect my internal knowledge of the world which is a lot more detailed. but also often loses important context. and then there might be another kind of event which is an output event. that says here's what changed over the rest of the world knows about it. it's important to realize these are separate things. and to think about which ones you want to capture which ones you want to store because you might want to store more than one of these. if you look carefully at the slide you might notice we have a slight problem.

the code isn't quite right. The total is wrong. What happens to me fixing this bug? I've got to be able to replay. If I quoted that as the order price, I can't just change it because I quoted it and announced it to the world. But on the other hand, I've still got to fix the bug. But I've got to also be able to replay. Things start getting tangled again. It's this versioning issue. Greg Young's advice is quite firm. He says don't have any business logic in between your event and its storage because if you do, you run into this tangled versioning mess. But on the other hand, if I don't have the input event stored, I lose the driving intention of what I'm doing because that internal event can be pretty complicated and messy. One solution, of course, is to store both. And in fact, I've seen that done with some of the events or systems I've worked in. The canonical style that I saw back in the 90s when doing this.

was we would store the external event, and then actually all the changes to the application state would be accounting entries, so they were naturally event sourced because they're all accounting entries, and so we'd store both. And you might have a whole pipeline of changes. The payroll example I said every little change, change to your sick pay allocation, change to your vacation stuff, every one of those changes was an accounting entry, and so everything was stored as events flowing all the way through, and you have a pipeline of processes producing events, producing triggering other processes, producing events through the system, and that was very useful, and that worked quite well, and I suspect that's where we have to go. Par good part of a discussion we had at this workshop was this question of what's more important, the internal or the input event, and as I said, I do think it's certainly important to be clear of what you're storing and to understand the ramifications of what means across them, and I suspect most of the time you need to store both.

okay, so that's three of them. So, what's the fourth one? Well, I kind of already hinted on that, and that is this thing called CQRS (Command Query Responsibility Segregation), a term coined by Greg Young, whose name I've called out a couple of times already, and he's talked about it. I remember, in fact, the first time he talked about it was at a GoTo conference. Or this is the first time I saw him talk about it was at a GoTo conference many years ago. The basic idea with CQRS is that you separate the components that read and write to your store, permanent store. So, you have effectively two separate models: one for dealing with updates and one for dealing with reads. So, and then the crucial thing here is that they are separate pieces of software, separate components, and that you really only ever use the command component, the command model, when you're making an update. You always read using something else. I mean, you might do some reading in the middle of processing the command, we'll do some reads, but basically anything that reads a system

is completely separate. The example I remember that he gave of this that was quite nice was a system that had to do a lot of complicated business logic for all the updates, and they didn't want to have to be but they updated much less frequently than they read. So, it made sense to effectively calculate a lot of derived data and store it in the read model so that the read model could basically go direct to the UI, and the command model then had to do all the complicated calculations. This is a model that I am. This is an approach that I would say you've got to be wary of. I think it's an appropriate pattern for a number of situations, but it is one that seems to correlate a lot with people getting into trouble, um, and so you've got to really kind of be careful and make sure you really understand how it works and what its benefits are and if you really need it when you're using it. My sense is it's got to be pretty deep down in the tool bag.

to pull this one out, it's a good pattern when used appropriately, which is of course true with almost any pattern, but you really have to pay attention to when to use this. I also find it gets a bit confused because I've heard people say, Well, I we think CQRS should be used everywhere because it's really useful to have different models for reading things and populating different models, but to me, that's stuff we've been doing ages. I mean, for ages, been talking about the idea of reporting databases where you have your online transaction processing system and then you take an input out of it, put it into a completely separate database for generating reports. People have been doing this for years. You might do it every night from an extract transfer load. You might do it in more real time using event updates these days, but that idea of having different read pictures, that's uh, to my to my mind if CQRS is this, then CQRS is every system that was ever built. Um, I think what's important about the CQRS model is the notion that the that

command model. The thing that's used for updating isn't used by anything for reading. It's only used in the process. And you've got a distinctly different model that's only used in the update area. And that I feel is a relatively rare thing to do. Is appropriate when you want to use it. But be pay a lot of attention to where it's appropriate. On the other hand, using report deficit different reporting views. I mean do that all the time. I mean I will always. I've always been a big fan of transform the data into a shape that makes sense for looking at it. And then look at it and multiple transformation steps compose very nicely. So pipelines of transformations. I mean that's that should be something that's familiar to pretty much everybody. So four patterns all coming under the term event-driven. I found this useful. And we found it useful in terms of our workshop. Because now we feel that when we hear somebody say event-driven. We've got an idea of where to go next. Now we say okay. We think one at least.

one of these four patterns is in play. Let's ask questions about that and understanding these patterns. Then, we can ask more questions about, Okay, why are you doing this? Are you doing this appropriately? What are the common traps? I mean, the value of a pattern is saying, Here's a particular technique, some ideas about when it's applicable or not, and enough knowledge about the pattern for you to know, 'These are the things I should watch out for, these are some of the things that I would expect to see as consequences of that.' That gives you a bit more leverage, but something as vague as event-driven to me doesn't give you that because it's too imprecise. It's actually a similar problem that I just found with service-oriented architecture. Service-oriented architecture means so many different things. The fact that somebody said they were using it didn't really tell me anything. You always had to dig deeper, and so a more precise term like microservices is better because it just carves out a small amount of that space, and you've got a better idea what's

going on of course. When a term like microservices gets super over hyped, then that tends to lead to confusion because people aren't using the term properly. So, you know, again, it's not perfect, but again with events, I think we gain a lot by being a bit more precise. And what I hope you've got out of my talk is some sense of what that precision looks like and what these patterns are, and so that you can have a bit of a better idea when you're talking about event-driven systems, what they mean. The original article and there's the URL for it. You can see I've stuck the date at the front of the URL. I only do that for things that I think, Oh, that's not going to be relevant. I'll probably delete it in six months. I got that wrong, but anyway, do go and take a look at that. It's a very brief summary, but there's links from there to lots of other things. Things like the article on LMAC, some of the stuff I wrote 10 years ago on event.

sourcing that's still fairly relevant, and stuff of that kind. And I hope you found that useful. And I hope the talks that you will see later on in this conference will be better than this one. Thank you. [Applause]

The Many Meanings of Event-Driven Architecture • Martin Fowler • GOTO 2017

GOTO gathers the brightest minds in the software community to help developers tackle projects today, plan for tomorrow and create a better future.

Learn from thought leaders and innovators with top-rated videos released daily, and at our year-round conferences, masterclasses and meetups.

Get started: https://gotopia.tech

GOTO is produced by Trifork, a leading full-service supplier of high-quality, custom-built applications and end-to-end solutions.

GOTO YouTube Channel Code of Conduct: https://blog.gotocon.com/yt-coc


The Many Meanings of Event-Driven Architecture • Martin Fowler • GOTO 2017

The Many Meanings of Event-Driven Architecture • Martin Fowler • GOTO 2017

Scripts