Netflix Microservices: Challenges in Birthday Feature Implementation

Galactus symbolizes the overwhelming complexity of Netflix's microservices, highlighting the struggle to deliver user-centric features amidst chaos.

### The Complexities of Microservices and Emotional Turmoil at Netflix
* 😩 The speaker expresses frustration over the difficulty of **displaying birthday dates** due to backend design issues involving multiple microservices, leading to a sense of **emotional turmoil**.
* 📅 The process involves calling various services like **Bingo** to retrieve user IDs and validate them through others like **Ulna**, before ultimately pulling user information from **Raccoon**.
* 🐷 A reference to the "Pig" episode of Black Mirror highlights the speaker's **emotional connection** to the show, suggesting that their work experiences **resonate deeply** with the themes presented in the series.
* 🔄 The speaker describes the challenges faced when trying to reverse the episode order for Black Mirror, emphasizing the need for **multiple services like GPS and VMS** to **manage video metadata effectively**.
* ⚙ Content Metadata Service (CMS) was introduced to handle **logical reorderings at runtime**, making it essential for **delivering the correct episode information to users**.
* 🛠 The speaker recounts the struggle of **renaming flags in the system** due to pushback from the website team, illustrating the **complexities of collaboration** within a large organization.
* ⏳ Ultimately, the speaker conveys a sense of defeat over the lengthy process, stating, "**it took like four months**," to achieve a simple goal, highlighting the **inefficiencies of working with microservices**.
* 💔 The emotional toll of these technical challenges leads to a **humorous yet poignant reflection** on the struggle to **find love and meaning** amidst the chaos of software development.

there we go we added these three marker microservices let's go here we go three years I'm still not understanding this why is it so hard to display the birthday date on the set I can already feel so many emotions coming up inside of me I know we haven't

even started but you guys just don't even know you don't even know what happened to me you don't even know about Black Mirror you don't know about the Pig episode this it hits so home to me because the pig that I will literally never be able to

recover from watching this video but we're gonna do it settings page why can't we get this done this quarter look I'm sorry we've been over this it's the design of our backend first we have to call the Bingo service see Bingo knows

everyone's bank I'm sorry about the screen tearing okay again I'm switching to sway March 13th okay Nemo so we get the user's ID out of there then from Bingo we can call papaya and MBS to get that user ID and turn it

into a user session token we can validate those with ulna and then once we have that we can finally pull the user's info down from raccoon foreign so does anyone know about Black Mirror on Netflix anyone familiar with Black Mirror did anyone watch Black Mirror before

Netflix purchased Black Mirror type 2 if you watched Black Mirror before Netflix owned Black Mirror which was starting season four did anyone watch it season three and less okay um I got a story for you we're gonna do a story here very very soon but first

let us become emotionally damaged by what we're about to see, which this is effectively a documentary of working at Netflix, doing large things, but couldn't the raccoon team basically just... no, raccoon isn't guaranteed to have that info before we

do this, we have to go to Wingman, do a query to see if the user's willing. I love the Wingman and Apex. I just, so I'm just very happy they they named a service called Wingman, to take it to the next level, or if they're just playing

the field now. Wingman is cool, but he doesn't store any user info himself. He has to reach out to other user info provider services like RGS, Barbie doll, Ringo 2, BLS. But how does it know what all the user provider services are? Well, for that it

has to go to Galactic, all-knowing user service provider. I just hate this. Why do we make this? Why did the world do this? Why did we do this to ourselves? We have a world in which was not that great and we replaced it with an idea that felt better

and now we have Galactus. Why is this? Why? Why aggravator? And what Galactus has omniscient knowledge of all current user info providers. It doesn't have future sight or knowledge of past user info providers, so it expects a time range to get all

the current user info providers. We need to pass a time range with the current time and a time representing the end of the universe, which we get from Eks, our entry chaos service. PKS, it's being deprecated at the end of the month for Omega Star, but Omega Star

still doesn't support ISO timestamps like they said they would a month ago no no no no no no oh my goodness we have such so until Omega Star gets their act together we're blocked we can't get

signed up for our use case we can't use EKS there's nothing we can do so Galactus won't be able to find our new birthday boy provider which means Wingman won't know how to talk to anybody which means I won't be able to

find true love and I'll die alone laughs I'll die alone without ever knowing love without ever knowing it's my birthday it's birthday blocked okay we're blocked you sad pathetic little product manager they just okay product managers don't know the pain that's going on here they don't

they can't they cannot under they cannot feel it he has a notepad trying to write down what's happening he doesn't feel it deep down you think you know what our users want you know nothing of my pain of Galactus's pain you think you know what it takes to tell

the user it's their birthday you know nothing delivering this feature goes against everything I know to be right and true and I will sooner lay you into this barren Earth than entertain your folly for a moment longer [Music] alright so clearly this is a blocker uh no

problem wishes out another two to three years or so learned a lot today love Galactus uh but we'll talk next week about adding middle names to the profile awesome sounds good take care ah it's just so painful it's just so painful to see so when I was working at

Netflix uh oh my goodness where's my marker there it is got it When I was working at Netflix um during my time there which I'm still there by the way uh we had this beautiful thing okay that's way too big what is happening here okay well this is

this is not this is not this is not what I was hoping for and make it smaller yep make it smaller and yeah there we go So while my time at Netflix has been fantastic there was this time in which we need to do a little show called Black

Mirror and it sounds like a lot of people are pretty familiar with Black Mirror now I wanna I want someone to type into the into the chat PF if you know what the first episode's about type PF in the chat if you know if you know about the first episode

okay for those that don't know what PF is it's called Pig um Yes you heard me correctly Pig and uh at my job I had several meetings in which involved the phrase pig to be said a hundred separate times it was wild and so we came up with this idea okay hear

me out is that we Netflix are producing season four there are three seasons before us season three season two season one and we thought you know would be way better is instead of showing you the first episode S1 E1 instead we're gonna show you S4 E1 which

means you couldn't just reverse sort it right you couldn't just okay this is a reversed sort ordering because then you get S4 E6 I think I think there's six episodes in season four and so you couldn't do that you could not

just simply reverse it so what we have to do is we have to reverse the season so we'd have to have a reverse season but not a reverse episode now this caused a lot of problems in our system the reason why it caused a lot of

problems in our system is quite simple there's a few different things that need to come into play here okay so first off there is GPS GPS would calculate your lolomo your lolomo of course is the list of list of movies that exist on Netflix it's when you go there you scroll

through it that's what you're going to see okay list of list of movies pretty straightforward uh GPS has nothing to do with the map but shockingly enough a service called map would call into GPS and get the skeleton of your GP of your lolomo and then fill

it in more dynamically right be able to get things going on so you'd see something like this inside GPS GPS would produce a list of shows and one of them would be say like Black Mirror now it's probably gonna choose Black Mirror S1 E1

and so map has to be smarter and map needs to go well wait a second s1e1 isn't actually the latest episode or the episode we should be promoting instead we need to be promoting S4 E1 so map itself doesn't know anything about what

is latest about videos that is where something called VMS comes in VMS oh my goodness sorry VMS VMS VMS video metadata service right but video metadata service doesn't actually hold is that video about a dentist service that's actually all that he did about video

about the latest and so video metadata service actually used something called, if I'm not mistaken, Dexter, which has a lot of the data about it. It also had a second service that provided other data about it. I can't remember the name of that service. Okay. And so VMS was very

very important, but VMS didn't know anything about ordering. It just used raw data out of Dexter, and we didn't want people to have to reverse order in Dexter and go back through and do that. We can't do that because then it's going

to get all screwed up because here's the deal is though we're delivering all these things down. We need a way to be able to identify them uh by their their season counting and all that. So we also had like this metadata service about it such as like their season name,

their season and episode number. So if you just had season one be replaced with season four, it would come back as S1 E1, which is incorrect. So the metadata service also couldn't just have things out of order because metadata service also relied on this. So now that we have

that we needed something else. So then came in a glorious thing called CMS, content metadata service. So content metadata service's real job was actually to provide a service that would do these kind of more logical reorderings at runtime and not at service time. That way we

could have these really beautiful items in here. And so that way, Map actually called into content metadata service to make sure it was delivering the correct thing. But here's the problem, there was more things that needed to happen because then on the front end you'd have

to call in and get your lol amo and then we would receive these but the problem is when you're on your low level your list of list of movies what happens when you look at your Seasons well they're not reversed because we are still currently using VMS for our middleware

so our middleware needed to start using CMS but the problem was is that API uses the API service Gateway pattern which means that we now have things such as get you know get metadata 2 we literally had get metadata 2 because get metadata can't just be

deprecated you can't just you can't just undo these things so we actually had to develop a second babe to do this but here's the problem how did you know you need to call this well we actually had to create another service I forget what it was even called at this point that

would tell you if this video needed to be reordered based on flags at a separate system which I can't remember what it was called so when you combine all these systems together this was how microservices happened this is how we reversed episode ordering at Netflix

okay and I love the charge on this one okay I did that I did that it was a lot of fun it took like four months oh you know what the worst part about it is that I didn't get to really choose a lot of the names

and so there's this problem that happened on website uh website because one of the flags that would come down from this service up here which you can barely see was hidden episode numbers right and so that was a runtime decision that had to be made and the website

couldn't support it being on the server side so it had to be on the client side so we'd actually send down those flags to the client and the client would then not display it because we just didn't have time for another service right here to do the

mixing of all these things right so we had that and so we had to do that but here's the problem is the website team uh the website team they hated this name and so I had to do a whole code review and then not allow these names to take

place because this implies actually not visible and they didn't want a not visible they wanted the name to be said in a way that's not complicated so we had to then after making all the changes and everything was uniformly ready to go someone said hey we can't do this and I

just said hey I mean this is just the name of the flags I think it's better just to stick with the names throughout you know it's not the name I would choose but whatever and of course that didn't work out so I had to then do a

quick refactor rename things and that's that it was it was the worst experience of my lifetime all this so that the Pig episode wasn't the first thing you saw can you fix the no uh let's see why not just a rate of uh reverse on the front

end did you not hear oh my goodness I feel like I just did an entire story why not were you not even paying attention were you not even paying attention to this whole thing I I just EX don't be coming into this chat with your VS Code vibes all right you

Keep that out. You keep that out right now.

MircroServices (and a story about Netflix) | Prime React

This is a place for all the things that are awesome on stream. 

ThePrimeTime

Difficulty Displaying Birthday Date and Introduction to Black Mirror Analogy

Initial problem: Difficulty displaying a user's birthday on a settings page.

Speaker's emotional connection to the problem, referencing the Black Mirror 'Pig' episode.

Technical explanation of the backend complexity involving multiple services (Bingo, Papaya, MBS, Ulna, Raccoon).

Transition to a story about the complexity of Netflix's system, using the Black Mirror analogy.

Explanation of the Complex Microservice Architecture at Netflix

Description of the numerous services involved in retrieving user information (Raccoon, Wingman, RGS, Barbie doll, Ringo 2, BLS, Galactic, Eks).

Speaker expresses frustration and confusion about the complexity of the system.

Details about Galactus, its limitations, and its reliance on Eks and Omega Star.

Omega Star's incompatibility with ISO timestamps creates a blocker.

The blocker prevents the birthday feature from being implemented.

Conclusion and Transition to a Netflix Story

The problem is identified as a significant blocker, requiring more time to resolve.

Transition to a personal anecdote about a project at Netflix.

Speaker's personal experience working on a Netflix project.

The 'Pig' Episode Reordering Problem at Netflix

Discussion of the challenge of reordering Black Mirror episodes to not show the 'Pig' episode first.

Explanation of why simply reversing the order was not possible.

Description of the various services involved (GPS, Map, VMS, Dexter).

Introduction of CMS (Content Metadata Service) to handle logical reordering.

Explanation of the API service gateway pattern and the creation of 'get metadata 2'.

Website Implementation Challenges and Naming Conflicts

Discussion of challenges in implementing the episode reordering on the website.

Explanation of the client-side implementation due to server-side limitations.

Conflict with the website team over service naming conventions.

Description of the code refactoring and renaming process.

Final reflection on the entire process and its complexity.