Rethinking Microservices: The Case for Modular Architecture

Microservices create complexity that often overshadows their intended benefits, revealing the need for simpler modular architectures.

### Rethinking Microservices: The Case for Modules
* 🔍 "Architecture is hard sometimes," and the tech industry often **jumps on new trends like microservices** without fully understanding their **implications**.
* 💻 Microservices promise **scalability and independent development**, but they can introduce **complexity and confusion**, particularly with **backend data management and reliability**.
* ⚙ The allure of microservices lies in their ability to **simplify development for large applications**, yet they often lead to a myriad of issues that distract from **core business logic**.
* 🛠 Many of the touted benefits of microservices, such as **ease of maintenance** and **improved code quality**, have historical parallels with traditional modular programming practices.
* 📉 The reality is that microservices can complicate rather than simplify, as they require **careful coordination among teams** and introduce **latency in communication** across distributed systems.
* 📝 The foundational principles of microservices echo those of modules, which have been a **core concept in programming for decades**, suggesting that the industry may be **rehashing old ideas under a new name**.
* 🌐 Emphasizing organizational clarity and team autonomy, the true value lies not just in the architecture itself but in how teams can **effectively collaborate and innovate** within that structure.
* 🔄 Ultimately, the conversation around microservices should pivot towards **understanding the nature of modules** and how they can be **better leveraged to build resilient and efficient systems**.

You want modules, not microservices, for those that are watching this on YouTube. If you're wondering why chats like this, well, hey, guess what? Some bad things happened and OBS doesn't work right now. Dissecting why everybody keeps talking about microservices, okay, here we go. TLDR:

Architecture is hard sometimes. People keep offering up some new idea that quickly becomes the mainstream way of doing it without any context or nuance, and the industry, desperate, let's see, desperate to find ways to improve their architecture, snaps it up without hesitation. Microservices was the latest

in the trend, and it's time we dissect the idea and got the real root of what's going on. Okay, okay, I'm, I'm excited. At the heart of microservices, we're told we'll find lots of good things, dude. Okay, if you're gonna do this, at least just

put the trademark symbol. It's so much better when you do that. Scalability, code can be broken into smaller parts that can be developed, tested, and deployed, uh, and updated independently. Yes, in some sense, this is very true. Some things become less true as time goes on. Focus

developer focuses on solving business problems and business, this logic, absolutely, business problems, business logic. It's not like you have to do discovery or health checks or an entire Canary process or understanding when parts of your backend are failing or just this distributed craziness that

does take place at some point in life. You don't have to worry about any of that. Backend data must always be available for a wide range of devices. Okay. Simplicity provides a simplified development on a large-scale enterprise applications. Okay. Responsiveness enables distributed applications to scale, uh, is

response to changing uh transaction loads are supposed to be in response yes okay reliability ensures no single point of failure by providing replicated server groups that can continue when something breaks restoring the running application to good condition after failure occurs okay I mean these are all all of this right right like if I could

just focus on writing business logic right like doing the thing I want to do it would be great because you know what ninety percent of my job is is not doing my job ninety percent of my job is just like just solving the most asinine stuff

ever that's all it is somebody comes up with the clever new data format guess who's figuring that out hey somebody reported some things in microseconds nanoseconds UTC date ends and of course milliseconds and then even some of them seconds well guess whose job it is to figure out

which of the many time formats that you've just provided me in normalizing into an actual time guess whose job that is guess his job that is you know how annoying it is to try to figure out what time you're in you know how annoying that is oh you

provided me seconds Mike um milliseconds microseconds uh nanoseconds or UTC dated dude I actually have a function that goes through all of this and tries to guess which one you are it is absurd Prime got some trauma there I got some trauma boys dude and the worst part this is my day

didn't I kid you not my date int looks like this my my date in Checker is literally uh like uh function is UTC uh date right let's just pretend this is it uh oh whoa whoa how did I get here how did I get to the change me Daddy uh

let's go like this uh date which is just a number right I literally go like this uh date is greater than or equal to uh 2003 and and uh date is less than uh or less than or equal to 3000 and just be like you know future me

future me is gonna have to figure this out when things go wrong you know what I mean there we go right future me is gonna have to figure out when something's gone wrong right like that's me because you know what I'm supposed because luckily I'm only

working on things that have dates uh in uh 2023 and later so it's just like whatever future me right and then some of my I realized that some of my places I kid you not had this uh it had 20 uh 20 years uh I literally left this

comment at some point that said uh future me will uh be mad but present me is very happy yeah like I left that comment for myself just so I will know that I already know this is a hack right like I already know it's a hack we all know it's a hack

ain't no ain't no way we're thinking that we're we're making some amazing tool here right like you got to know the bounds of your tool my tools should not exist for another year right if it still exists within a year I think I honestly

think Netflix done right it's a hack from the start you got to know your stuff Tech Society it just hurts it always hurts all right these all sound relatively familiar I'm not imagining but the fun part about those six quotes is that the two were taken from

microservices literature, blog posts, papers, etc. Wait, what? Uh, and two from 20 years ago, EJB literature. I don't know what EJB is. I feel like I should. And two from Oracle Tuxedo, which is 40-plus years ago technology. Can you

spot, uh, which went to which? I don't know. Uh, we have a tendency in the industry to reuse our hype points over and over again. Those who cannot remember the past are condemned to repeat it, George Santayana, *The Life of Reason*, 1905. Uh, EJB equals trash.

Oh, okay. That's trash dev. I didn't realize trash dev was that, with respect to the microservices hype. One company's blog post offers 10 reasons, uh, to charge into microservices. I bet you this might be Netflix. Laughs. How much do you want to bet this might

actually be Netflix? Hold on. I gotta do this. I gotta do this. Hold on. There we go. This just makes life so much easier because I can go like this. All right, one, they promote big data best practices. Microservices naturally fit within a data pipeline oriented

architecture which aligns with the way big data should be collected, ingested, processed, and delivered. Each step in the data pipeline handles one small task in the form of a microservice. They are, relatively, number two, they are relatively easy to build and maintain. In some sense,

I do think this point is correct, is that they are easy to build or maintain if what you're building is a really small server, right? Simple servers are obviously simple to maintain. It's a very isolated look to think that, though, that's the problem.

Uh, their single-purpose design means, oh man, okay, I do, I mean, I want to look at those things, I like this. All right, anyways, they are relatively easy to build and maintain. Their single-purpose design means that, oh yeah, we already, oh wait, no, I didn't read that there's

They're relatively easy to build and maintain. Their single-purpose design means they can be built and maintained by smaller teams. Each team can focus cross-functionally while also specializing in the subset of microservice solutions. Like all this sounds great, right? They enable higher quality code. Modularizing

an overall solution into discrete components helps application development teams focus on one small part at a time. This simplifies the overall coding and testing process. Sure, they simplify cross-team coordination. This one, I'm, I'm more in doubt of, though. I do agree, but I'm in doubt what I mean

by that is that right away, without even looking at this, there always comes this problem where you have a change, right? You have a change in API. What do you do? Well, you kind of have to maintain an old endpoint and you have to have the new endpoint

and then you have to go to all the other teams and then make sure that they do the right thing. And then if you're also including mobile applications, mobile applications can also end up having these like stale versions in which now you have to support this old path for

the life of your application until you force like an upgrade or an end of life. Like, it's not as simple as people make it because it always ends up being really, really hard, right? People, there's just this concept that things are just super simple and they're not.

Microservices are a great idea if you have a stable full of high-quality developers and have written a version one previously that can concretely identify your real requirements and weaknesses. And the only way it ever works is if you can replace one

component at a time. This is actually a really great take. It's this, I mean, for me it's the same argument why you'd use Rust, which is until you know your landscape truly, uh, this is why I've come around to go, I finally have come around to go, I know I

know, shocker, but until you know your landscape, use something that's really good for it. You have the best titles, dude. I don't have the best titles. I take my titles from this. The Gopher wins, the Gopher wins. I know, shut up. All right. They enable real-time

processing. Oh, Paul, let me read this one. Unlike traditional service-oriented architecture, uh, cool, uh, which typically involves heavyweight inter-process communication protocols. By the way, if you're using inter-process communication protocols, you're almost always incorrect. I think that moving your database off-box

awesome, moving your message queue off-box, awesome. Right? Like there's plenty of things that are awesome to do, right? Don't kill the Gophers. I'm not going to microservices. Use event streaming technologies that make it, they'll see. I don't even believe this.

Right? You can still do both these things at the same time anyways. Keep on going. They enable real-time processing at the core of microservices architecture is a publish or a pub/sub, uh, framework enabling data processing in real time to deliver immediate output and insights.

This part is also one of those really hard things that people talk about a lot, which is it's true there's a lot of benefits to microservices, but to trace a singular request through a large mature system is really hard, right? To understand exactly what happened is non-trivial once you go

back. I mean, like if you have three services, yeah, you can just like, you can do it, you know? You can just kind of grip your way through logs and figure it out. You can have like an X ID, some sort of request ID that passes through all the headers, you can totally do that.

But it's not necessarily that simple. Observability is a key, observability is a key, and it becomes excessively hard when you have 800 services called for a single request, right? Like there are requests like that at Netflix that literally call hundreds of services or a series of requests.

Right, you, you call, you get some sort of data comes to the client, the client makes some decisions based on say A/B test allocation, all this stuff. So you have this like n plus one problem of requesting that can take place, and it's just like super hard. There's a lot of

stuff that's really hard. They facilitate rapid growth. Microservices, they do facilitate rapid growth. Truer statements have never been said. Microservices enable code and data reuse. Uh, the modular architecture makes it easier to deploy more data-driven use cases and solutions for added business

value. Um, they enable more outputs. Data sets are often presented in different ways to different audiences. Microservices simplify, or simply, yeah, simplify the way data can be extracted for various users. Easier to access updates in the application lifecycle. Advanced analytic environments, including those for machine

Learning needs ways to access existing computation models against newly created models. A/B and multivariate testing in microservice architecture enable users to validate their updated models. This was a really complicated way to say canaries are great! They enable scalability. They actually, I'm not sure if they do truly

do enable scale as much as they enable, I mean, they scale the amount of internal requests you do make. I'm not sure if it's true that they enable scale, right? I'm not convinced on this one. Scalability is about, uh, is about more than the ability to handle more volume.

It's about the effort involved. Microservices make it easier to identify scaling bottlenecks and then resolve those bottlenecks at a per microservice level. That's the problem. You also introduce a scaling issue of making all the requests, right? You, you introduce a whole nother

level, and those can also be difficult. Many popular tools are available, a variety of technologies in the Big Data world, including in the open source community, work well in microservice architectures. Uh, Apache Hadoop, Apache Spark, NoSQL databases, and many, uh, streaming analytics tools

can be used in microservices. We're also proud partnering with Pivotal in this area. So, one thing that they're not talking about here, just even at this one, is that say you have three services, say you had a singular function or a set of

just, you know, a library that did something that went grabbed your data and made three separate calls and merged it all together and gave it to you. Now it's a microservice, so now you make three different HTTP requests to three different servers, and it's all micro-

serviced out that is going to be a bottleneck of whichever one is slowest now here comes the worst part is if you ever have to break it up you may be able to do something clever where one thing is a monolith and is able to take the

state of all this and make it into a singular request whereas there's times you can't do that when you don't have a monolith very easy example say you want ratings ratings on a movie and Netflix is both what the user what we think the

user will like plus whether they've liked it or stuff like that right machine learning so now you have this idea where you have to go and you have to go get videos then you have to come back and then you have to go and get ratings right right whereas if you're in

more of a monolith world you can go get video ratings Right video and rating same time but now since it's a very micro service architect like new bottlenecks do show up and it's this idea that it's freezy it's not necessarily for freezing you know

it's not necessarily for freezing uh all in all that being said the idea of a really well designed go monolith app deployed as a stripped binary a stripped binary on a Docker scratch image being an app Service uh Overlord is kind of sexy and disturbing but

totally non-furry way reasonable I mean I do like microservices don't get me wrong I like the idea I think there's a lot of great things about it uh really if you think about it your editor has become microservices LSPs are a version

of microservices was Jonathan Blow correct on saying LSPs are a sign of the times that we have become bad programmers? He actually might be right. Maybe it should have just been a library, maybe it should have just been this way. Would it have been easier? I don't know.

If the updating and integration cycle would have been as easy, but if we have a well-defined protocol, it might have been better, right? I'm not saying which one's better. I just don't know, right? I don't know. Many popular tools are available, a variety of technologies

in the big world. Okay, yeah, we already read this. Let's take a second uh and examine each of those by this time in light of prior art. Uh they promote uh big data best practices. Pipe uh pipes and filter architectures have been a part of software since the 70s when Unix

has promoted several ideas. Make each program do one thing well. Love this by the way. To do a new job, build a fresh rather than a complicated old program by adding features. I love this by the way. This is like my favorite of all of the Unixisms. This is it. This is like my

favorite thing that ever... this has been on my mind constantly, driven into it because all I do is think about this when I make CLI tools these days, which is like how do you not, how do you avoid the like, how do you avoid ffmpeg in

some sense? FFMpeg is its own problem. I understand why it exists. It's extremely complicated. Same thing with like gstreamer, right? Very, very complicated. Yeah, it's by its very nature it has to be complicated, but if you don't have to be ffmpeg or Gstreamer, how do you break

your program into like three programs right uh expect the output of every program to become the input of another as yet unknown program don't clutter outputs with extraneous information avoid string uh stringently columnar or binary input uh formats don't insist on interactive

input I love I love all these things what's the deal with of a vampag uh they are relatively easy to build and maintain see the Unix philosophy above yep uh they enable higher quality code if focus on one small part at a time helps improve quality then see the Linux

velocity above they simplify cross team coordination this one is interesting it suggests that service oriented architecture typically involved heavyweight inter-process communication protocols like JSON over HTTP uh it'd be more like ATSU message pack over IPC or is that taken to mean that all SOA

requires soap uh will diesel XML schema and the full collection of WS Dash specifications ironically nothing about microservices in any way prevents it from using any of those heavyweight protocols and some micro services are even suggesting the use of gRPC a binary protocol that bears closer resemblance

to IIOP from Cabra CORBA which is a heavyweight protocol a predecessor to soap WSDL XML schema and the full collection of WWS specifications I think one thing he's missing is I think it also is uh I think it's just standard like message like

because you can communicate just you know via Linux uh just like a little little socket also so but I I see what he's mean but it's mostly going to be HTTP let's just be real it's HTTP pipes are just real-time linear micro service orchestration it really is it

Really, it truly is. They enable real-time processing. Real-time processing has actually been a thing for quite a while, and while many such systems use a pub/sub or event bus model to do it, it hardly requires microservices to do it. Absolutely, absolutely, uh, but you can. I mean, I think

for me, real-time event processing is a great one to kind of quote-unquote microservice out or have its own thing that you listen to and use. Uh, that seems like a great thing, but you can add events like you can still have a monolith architecture and still have

that at the same time, right? Uh, they facilitate rapid growth by the way. If you break the monoliths like Ciri did in The Witcher, that's how that's how those beasts come to be, you know. Uh, they facilitate rapid growth, reuse the modular architecture. Do we

even have a count? Oh, wait, no. That's not why it was when the the alignment happened. What am I saying? Uh, reuse the modular architecture. Uh, do we even have a count of how many different things uh have all been, let's see, all promoted, reused as a selling

point language? Certainly. Uh, have it or have done it. OOP languages, procedural languages, functional languages, uh, libraries, frameworks, and one day I want to see something hyped that expertly says, "A screw reuse, we don't care about that." This man has not, this man has not seen

HTMX. HTMX is this. HTMX is is is truly this, right? Which is just like, "Stop, stop trying to make everything reuse. Okay? Make your API endpoints produce the same thing." They enable more outputs. Data sets often are presented in different ways to different audiences.

that sounds uh a great deal like Crystal Reports homepage uh I like how they don't even try to debunk this one they just or try to they're just like yeah well what about that this is pretty much how Twitter communicates oh so you don't like React

well then why is why is Angular sucking then you're like I don't like Angular either good call uh easier to assess updates in the application life cycle they need to uh they need to access existing computation models against newly created models for machine learning and advanced analytics

environments it kind of sounds like a large pile of action words thrown together with little substance behind them that's what I said uh they enable scale let's see how funny this the same was set of EJB transactional middleware processing a la Tuxedo and mainframes yeah I don't think

they enable scale like they're saying uh many popular tools are available I don't think I have to really work hard to point out that these tools have always been available for every major hype that's come through our industry particularly after the hype was taken root for a while most readers won't even

be old enough to remember case tools but they may they maybe are oh my goodness but maybe they'll remember UML UML is the devil okay I had to do so much you not even talking about we're moving on but let's see but the discerning reader

will notice that there is plenty common theme about half the points above the idea of of creating and maintaining small independent chunks of code and data versioned apart from one another using common inputs and outputs to enable larger integration of systems it's almost as if at the heart of micro

Uh, services we find modules, yup. The lowly, the lowly module, the core concept that has been at the heart of most programming languages since the 70s, even earlier though, is harder to do with older languages that didn't incorporate the module as a first-class core concept.

Call them assemblies on the .NET, C#, F#, Visual Basic, jars or packages on the JVM, or dynamically linked libraries from your favorite operating system. DLLs from Windows, SOS or A's on Nix, and of course Mac OS has

the frameworks tucked away inside the library directories. Uh, but at the conceptual level, they're all modules. Each has a different internal format, but each serves the same purpose: an independently built, managed version and deployed unit of code that can be reused.

Consider this working definition of a module, quoted from one of the computer science foundational papers: a well-defined segmentation of the pro... I can't read this word without thinking of the word "fault", of a project effort ensures system modularity. Each task forms a separate, distinct program module.

At implementation time, each module and its inputs and outputs are well defined. There is no confusion in the intended interface with other system modules. At checkout time, the integrity of the module is tested independently. There are a few scheduling problems in synchronizing the completion of several

tasks before checkout can begin. Finally, the system is maintained in a modular fashion. System errors and deficiencies can be traced to specific system modules, thus limiting the scope of detailed error searching. This comes from Dave, David Parnas, a seminal paper on the criteria to be used in decomposing

systems into modules written in 1971, over 50 years ago. At the time of the writing, well-defined, separate, distinct program modules cover about half the suggested benefits of microservices. You know this, I mean this is a fairly compelling take, right? Like I'm definitely, I definitely

see the benefits of of microservicing things out and I definitely see that people can build some really terrible monoliths. I think one of the problems we see is a classic argument in our world, which is you point to something that is bad, right? We all agree that we've made

bad monoliths, and you point to those bad monoliths and you say, "See, this is what happens when you write monoliths." It's not necessarily true, it's not necessarily true. And so it ends up being this thing where you you you you paint one idea as just

purely evil and then you paint a solution, right? It's like I forget what that, I forget what the term is, but effectively it's like a, what is it, this is a concept that happens all the time, right? Uh, but anyways, it's the same concept where you're just

you're able to demonize something by just like some version of it. It's a straw man argument in some sense, but I know that there has to be a term for this where you effectively paint in the people's mind, uh, it's like a form of

propaganda that paints in an idea and then you you give the solution to that idea that is different. Gaslighting, you're completely wrong on this one. Gaslighting comes from the movie Gaslight, gaslighting if I'm not mistaken, where a guy turns down his the lights in

He dims the lights in his house to drive his wife mad. She goes, 'Is it dark in here?' and he goes, 'No.' And he's driving her crazy. That's gaslighting. It's where... where... what's happening? You deny reality to somebody else such that they consider themselves crazy. No one uses the term 'gaslighting' correctly.

You know why? Because a bunch of people are a bunch of idiots and all they do is go on Twitter and see the word 'gaslighting' happen, and then everyone uses it. And now they're all wrong. And now 'gaslighting' means nothing. Gaslighting means you said something...

That pisses me off. Now it's broken. Now it's completely wrong. I'm not... I'm actually not gaslighting you. I'm giving you... I'm giving you the deets. I'm giving you the tip and the shaft here, okay? Gosh, gaslighting isn't real. Are you crazy?

Same with the word 'literally.' Literally, literally does not mean literally. Prime is trying to gaslight her. Shut up. Thank you, Dax, for raiding. I don't have alerts. Bangers are on and reading an article... uh, it's about microservices because microservices were really never about...

Microservices or services or even distributed systems. At the heart of microservices, we should find organizational clarity. I love this. I love that this is true. Amazon, one of the first companies to openly discuss the microservice concept, really wasn't trying to push the architecture...

Principle as much as they were trying to push the idea of independent development teams whose blockers were few and far between. Waiting on the DBA team for a schema change, QA needs to build... uh... to test so they can find the bugs, or are we waiting for the infrastructure team to...

produce a server or the UX team to create a prototype for the presentation slurp I don't I I don't understand this but okay let's go is uml gaslighting uml is actually gaslighting uml presents you a lie of the world in

which you believe is true but actually the reality is that it's moved underneath it cherry picking might be the Terry uh that's uh that sound you hear is the development team aggregating ownership of any and all of those dependencies that could and frequently would block

them from moving forward yes so having a having a defined and independent contract between teams in which can move out of underneath them is effectively what you're saying uh it meant that the teams were small microcosms of the average IT teams various parts analysis development design testing data

management deployment and administration and more it did mean that now teams either had to be assembled from a variety of death uh of desperate skill sets or else we have to require let's see despair oh my goodness I know the word but I cannot say it disparate disparate I

Stop Creating Microservices | Prime Reacts

This is a place for all the things that are awesome on stream. 

ThePrimeTime

Microservices vs. Modules: A Critical Analysis

The speaker argues that the industry often adopts new architectural patterns like microservices without sufficient understanding or nuance.

Microservices are presented as the latest trend, prompting a deeper examination of their actual benefits.

Claimed advantages of microservices include scalability, independent development/deployment, and developer focus on business logic.

The speaker questions the long-term validity of some of these claims, highlighting the complexities involved.

The speaker illustrates the time spent on non-core tasks, such as data format normalization and time zone conversions.

The speaker reveals that the purported benefits of microservices are recycled claims from older technologies like EJB and Oracle Tuxedo.

The speaker uses the quote, "Those who cannot remember the past are condemned to repeat it," to highlight the cyclical nature of technological hype.

A company's blog post (possibly Netflix's) lists 10 reasons to adopt microservices, which are analyzed for their validity.

The speaker contrasts the purported benefits of microservices with the Unix philosophy, emphasizing the importance of small, independent programs.

The speaker discusses the complexities of cross-team coordination in microservices, contrasting it with the Unix philosophy.

The speaker concludes that the core concept behind microservices is essentially that of modules, a long-standing principle in programming.

A definition of modules from a 1971 computer science paper is presented, highlighting the overlap with the claimed benefits of microservices.

Organizational Clarity and the Fallacies of Distributed Computing

The speaker suggests that the true benefit of microservices lies in organizational clarity and independent development teams.

The fallacies of distributed computing are introduced, emphasizing the challenges of performance, reliability, and scalability in distributed systems.

The speaker suggests reconsidering the need for decomposition into independent entities, suggesting alternatives like standalone modules in an application server.

The speaker emphasizes the importance of establishing common architecture or backplane with well-understood integration and communication conventions.