So welcome back, uh, for our next talk of the day. Uh, if you're on the live stream, also welcome. Thanks for joining us here at Spring One at Explore. Remember, on the live stream, you can click on the discussion button on the top of your screen and join the Discord discussion where you have moderators helping out, including members of our Spring team. And, uh, after the talk, I'll be taking, uh, questions, and we'll be passing around the microphone to make sure that everybody in the room, as well as people on the live stream, can all hear the questions. So, without further ado, I want to welcome, uh, these wonderful, wonderful speakers on a topic that is also near and dear to my heart, Spring Cloud Gateway. Please welcome Spencer Gibb and Luke Shannon. Thank you, K. Um, welcome. For those of you who don't know me, my name is Spencer Gibb. I've been on the Spring team, specifically Spring Cloud, for the past 10 years. The project I started with Dave Sire, um, and it's been awesome to work with Luke Shannon. You want to

introduce yourself, Luke Shannon. So, Luke is a customer, and uh, he'll be showing off some of the things he's done at his company a little later on. So, here's our agenda today. We're going to talk a little bit about what APIs are, um, how they, which specific type of APIs we'll be talking about, how to protect uh, the resources with APIs, um, we'll do a little interlude with decomposing the monolith, and then we'll talk about scaling teams with Gateway, and then uh, Luke will jump into the case study. So, API is kind of a big broad term used for lots of things, right? These could be programming language interfaces, anywhere to remote APIs, and that's what we're talking about, particularly HTTP APIs with Spring Cloud Gateway. Um, they could be gRPC, RSocket, whatever. There's all sorts of ways to do these kinds of APIs, but we're going to be talking about uh, uh, Spring Cloud Gateway today in HTTP APIs. So, typically, there's some network that separates your clients from your application, whether that's out over the

Internet or your internal Network, whatever it is. This could be done using open-source components like HAProxy, NGINX, those kinds of things. You could have commercial products, hardware that actually does some of this stuff today, specifically what we're talking about, the Spring and Java-based Spring Cloud Gateway. Now, specifically, there's multiple layers of how to protect our APIs, right? There's kind of the transport layer, so in this case, HTTP. You want to be secure, so HTTPS with TLS, that kind of thing. The next layer is your application layer itself, right? Could be secured using Spring Security with OAuth 2. We'll talk a little bit about that, JWT tokens and the like. You also want to protect your backend services, whatever those are, databases, be they relational or NoSQL, whatever that is, message brokers, that kind of thing. And in the cost-conscious world, you know, you want to save money, right? So whatever that is, you're spending on if you're out in the cloud or on your own hardware, whatever that is, so we'll

we'll talk a bit about how we can protect some of our APIs so that they can handle the traffic they need to and scale. So, if you want to scale, there's a number of ways to scale your APIs if they're resource-intensive. You can scale up, right? So, this is hardware, whether that's RAM, disk, CPU. I remember getting special permission to get the ultra-huge instances in AWS, right? Whatever that may be. I know Netflix runs Eureka on some pretty large hardware. So, that's one way. It costs money. You scale up. You don't have to change the number of instances you're using, which is the second way, right? Scale out. So, I have my traffic is causing my service to fall down. Let's do that. Let's scale out. That's kind of hard because either you might have too many instances, and you're paying more money for than you need, or how do you predict your traffic flow, all that kind of thing. And then you've probably, if you've been to a number of the Spring

talks um you can compile to Native using Grow VM and do more, do the same amount of traffic and things with, uh, less compute resources, right? Lower memory, uh, less CPU, that kind of thing. So how do we protect our resources, and specifically how do we do it with Spring Cloud Gateway? So first up, um, rate limiting, right? Lots of times you may know exactly what kind of traffic a service can handle, right? How many requests per second or something like that. So you need to enforce that in front of the service, right? And you can do that with rate limiting, restrict the number of requests, uh, at the front end. And there's various levels where you can do rate limiting, right? Typically, we think of it at the user level, maybe the logged-in user, maybe you're, you know, you provide some service to other developers, and they have an API token or something that they've been given, and you can use that to key, uh, your rate limiting. Could be by service, like I was talking about, you know, we're just going to limit this whole service to so many

requests per second, or hour, or minute, or whatever that may be, could be geography-based, literally anything in the request that you can imagine. There are various algorithms to do this. Gateway uses the token bucket algorithm. If you've never read a computer science white paper before, they can be quite daunting, but if you go find the one on the token bucket, it's pretty simple. And if you want an intro to reading a white paper, this is a good one to start with. The basic idea is it models it after a physical bucket, and if there's a token for you in your bucket when you make a request, you get to proceed. If the bucket's empty, you're going to get an error back. And then the other part of that is that there's a configured period that is consistently adding tokens back to the bucket. We'll see a little bit about that here. The implementation we're going to see today is provided by Bucket4J. Let's go ahead and see a demo. One thing we didn't

do is change our um where did we go change our um font size here. Yeah, let's we want to change the font though. That didn't change our darn it. Come [Music] on. There we are. No. Oh, come on font Y. Is that a little better? All right. So um here we see a particular um route that we're going to route on this particular path. And we have a a filter in the I'm using the spring MVC Gateway in this particular instance for our demo. It's only got room for one token in the bucket and you get that every 5 seconds. So every 5 seconds they're going to add another one. And in this case we would be simulating for example a full service being rate limited instead of a user because I just have a hard-coded um value here. So let's go ahead and if you get nothing else from this talk and you've never heard of this tool is one of my favorite tools of all time to use um. It's called HTTP. Right. You've all

heard of curl, right? This is curl for humans. Okay, so you get a new tool if you've never heard of it before. So we're going to go ahead and we got to start my application here. So we started the Gateway, going to go ahead and make a requests. Should I'm just showing the headers. We get a 200, and then within 5 seconds, since I didn't have any remaining, I got a 429 from the Gateway, right? Pretty simple and easy to configure. So back to our slides. Um, another thing that we you can use is a circuit breaker. It's literally modeled after the physical circuit breaker. Um, this protects from cascading failure. Uh, there are some pretty interesting stories about Netflix a number of years ago when an Amazon backup generator caused an outage, if I remember correctly. It's a fun read, uh, but it protects the users from seeing failure. So you'll get a success in the UI basically if there's a number of failures during a certain period. The circuit will open, and it will no longer

attempt to make that Downstream call, and it will use a fallback. So, the classic example I give is a recommendation service, what to watch, right? Typically, it might be personalized, but if it's out, we'll show a random list or a static list of recommendations for you. In Spring Cloud, many moons ago, we used Histrix, but now we use Resilience4J for this functionality. Let's go ahead and have a quick demo of this. T come to our circuit breaker, and you'll see for this particular demo, there's a link to my repository at the end. So, we've configured a route, and the URL that we're going to is some bogus domain, right? So, I'm guaranteeing this route is going to fail every single time I request it. We configure the circuit breaker filter just with an ID and a fallback. In this case, it's just another filter right here. And the reason I'm doing that is I want to show this fallback headers filter and why it's.

important so we can go ahead and uh copy my HTTP command here. So, it's basically timed out and failed um, but the user still got a 200, right? So, it did the fallback, and in this case, we're seeing a service called HTTP bin, which is nice when you're building a Gateway. It just shows you what you sent, right? And so, in this case, our fallback that uh filter, the fallback headers filter, added some information to the request that you see. So, in your fallback, you can see what failed, and that could help you determine what to do with your fallback. Again, this was only a single time, so it was just a fallback. The circuit didn't open, but if I hit it continually, it would eventually open the circuit. So, again, the beautiful colors and formatting of HTTP you see up there. The next tool and Gateway to will be retry. So, this is the kind of thing that protects from any particular instance failing, right? There are certain conditions that you should your call that you're

protecting with retry should meet um. Ideally, it won't every call to it won't make the won't change the state of the system over and over, right? You w't want this to be item potent. Typically, that means in the HTTP World, a GET, and by default in Spring Cloud, it's only for 500 errors. So, if it's a 404, that's not something you want to retry necessarily because the app responded correctly and said we don't know what that is, right? So, typically 500s. So, in the web flux Gateway, we use reactor retry, and in the web MVC Gateway that I'm demoing here today, we're using spring retry. So, let's come here to our filter here. We create a route, and our particular path is retry, and it's really simple. There's more complex versions of this, but you can just say retry three times. You could get more complex and do exponential backoffs and things like that, but for a demo like this, it's three, and I have a

little custom controller down here. What it does is it fails the first two times and then succeeds on the third so that we can see this work because otherwise it's pretty hard to make this happen. So let's send this request. Oh, so we get a header here that we retried three times, but that doesn't mean it actually happened. So if we look here for retry count, we retried once because we got a 500, retried again on a 500, and the third one we get back a 200, right? So we retried twice for the failures. All right, the next is scale to zero. This is a kind of an interesting new use case where if nothing's happening, there are no instances of your application running, right? It's literally cost savings. You don't have to have an app running that isn't doing anything, right? Not every app is going to benefit from scale to zero, so there are typically some criteria where this might be helpful for you. Number one, the app has to start

fairly quickly, right? You don't want scale to zero, and then it takes 10 minutes for your app to start up before it can respond, right? Should be a pretty speedy thing to start up. There are certain classes of applications that have periods where they have very low traffic or no traffic at all, right? Some applications are nights and weekends that I've seen, that I've worked on personally, right? They're very sparse throughout the weekdays and things like that. So, if your app meets that, that's one of the things that this might be helpful for. If you've got, you know, functions or lambdas, these small little bits of code that are probably compiled with Spring Native, right? So, they start up quickly and respond fairly quickly to that. So, I'm going to show you a little recorded demo of a project called Kada. So, originally, this project was built to scale messaging solutions on Kubernetes, but they have added a HTTP add-on, so it will also scale HTTP applications for this. And what that looks

like is kada gets installed into Kubernetes, and it handles scaling your app from 0 to 1 and 1 to 0, and then Kubernetes handles 1 to N, right? And there's a component in kada that has to sit in front of your application to intercept requests, and it will know how many instances are running, well, basically, are there zero, or are there more than one, or more, right? And it does that by communicating with over some gRPC endpoints in your application. So, I'm on the Spring team. We hope people are deploying Spring Cloud Gateway into their Kubernetes clusters, and if you already have that, Dave Sire, my colleague, created a proof of concept where we replace the kada component with an already existing Spring Cloud Gateway that knows how to speak the proper gRPC endpoints to kada. So, let's go ahead and take a look at my little pre-recorded demo here. I'll just kind of talk you through. So, here I'm just installing kada. It's pretty lightweight, as opposed to

something like K native kind of sits to the side. Next, I'm going to install a demo application. Um, this demo application literally just responds with the current date and time. Okay, so I'm going to port forward to that. We'll see what the request looks like to that particular application. So, you can see there's one version, one instance of that app, and I get my response back. So, now I'm going to go ahead and install our, uh, specially configured Gateway. What, what's going on? I [Music] lose right. I now am having some technical difficulties. There we go. Okay, well, while we're rebooting, oh hi, while we're rebooting, does anybody want to jump in with a question or a comment or a show of hands? How many people are using Spring Clock? Is it up? It's up. Apple starts really fast. Okay, we're almost there, but if you have any questions or comments, we can take the next 30 seconds. Here's your opportunity. Oh, sorry. I'm sorry. I'm not looking here. We

go so do you really find a whole lot of instances where your API Gateway or your API needs to scale down to zero? I mean, I can see small, but it seems it wouldn't be the Gateway that scales to zero; it would be your backend application that scales to zero. So, I mean, it could, but are you still, you're still restarting? Yeah, okay. We have another question: the access control throttle. Can you across multiple instances, or is just local the rate limiting? Yeah, so Bucket for J supports a wide number of remote kind of in-memory things to distribute that across multiple gateways. So, it is distributed. Okay, cool. Another one, we throttle is based on the URL, or you based some part of the Year support regular expression. So, basically, anything that the Gateway can match, it can rate limit. Yeah, so the predicates could be simple just paths there you can match, and there are expressions you can do to match those. Yep, awesome. Any other questions you want to jump in here in the middle? Almost, almost there. All right.

it's your you have like two seconds to be the last question in this middle break. Make sure you're back on your vpm. We're almost back. We're getting there. We're getting there. All right. I see things, windows are moving, things are popping up, digits are being scanned. And again, sort of the last, last chance for a question right now. Oh, you got to get almost. Yeah, all right. Now we're good. Almost there. I guess for the folks here, how, I mean, how's your experience? How many things do you think you're, this, what you're seeing here is gonna, you're going to take stuff back and have stuff to add to your, okay, getting a lot of nods, have stuff to apply immediately. Great. I feel like we've seen a lot of examples of ooth today in several different talks. Also, it's nice to see all that information kind of coming together. That's nice. Okay, good. Any call outs for anything in particular that has been useful or newer, eye-opening, just general? Okay, just a lot of nods. That's

good all right. I see the deck coming up. Sometimes you don't appreciate how much setup it takes to get these things to work. Yeah, all right, we're good. Plug back in. All right, plugging back in. All right, the show go on. Video's done, we're past that. So hey, look at that. All right, so I am going to skip ahead so that I can give some time to Luke here. So we're going to talk a little bit about scaling teams pretty quick. Open API, heard of this before, formerly known as Swagger, right? We have a commercial product that allows individual services to have their own Swagger, Open API endpoints on their instances, and this will aggregate them and allow them to be searchable so your team can find other APIs across their enterprise and also multiple portals can be aggregated. Spring Security is really important. Luke is going to get to show that to you. Basically, the Spring Cloud Gateway is a great place to enforce this at the front end, right? So you can't even

get to your services without having gone through the proper security sequence, of course, with Spring Cloud Gateway. It's just a Spring Boot app, so we get to integrate with Spring Security. We have a token relay filter that works with OAuth 2, can do the whole OAuth 2 dance, and then forward the token. Luke is going to show kind of a custom version of that here in a little bit. We support Spring Authorization Server as one of those identity providers, but it really could be anything: Okta, Azure, whatever it is on Amazon and GCP, right? Doesn't really matter what identity provider you're using. And in our commercial Spring Cloud Gateway, we have single sign-on and mTLS support for all the connections based on certificates. We do distributed tracing. A very good place to start is at the Gateway. This is when you have problems; you can follow a single request all the way through your system. You can see latency, you can see where it particularly failed in what component. We use by default Micrometer, the observation APIs, but then you plug

in whatever Tracer um your company uses on the back end. So, he's going to show you something that isn't using micrometer and how easy it is to do that. Um, another way to scale teams is if you use service Discovery, right? They can manage their own instances, they register with some Central service, and for example, in the Gateway, the uh springcloud load balancer can then decide based on the algorithm you choose which instances, and you don't have to if you're using properties. Every time you spin something new up, you'd have to go change configurations so that new machine was added, that kind of thing, right? So, this helps your teams when you're building these Dynamic uh distributed systems. Um, I think I'm just going to skip this one so that we can give Luke his use case. So, let's turn it over to Luke, and he's going to show you some of the things he's done at uh intact, showcasing some of these things that I've just been talking about. So, we have all your intellig up. Oh, I'm going

to close mine though close that guy all right. Hopefully, this does not die. No, shouldn't. Yeah, well, it shouldn't have died the first time. Yeah, yeah, I hear you. Uh, okay, so I'm GNA show some stuff. Spencer will probably provide some commentary. Don't fall off the stage. I'm good. All right. Uh, so I work at in. Uh, we're an insurance provider. Uh, a lot of software we're building for Brokers, for end users. My particular team is creating developer productivity tools. So, one of our tools is an internal developer portal. It's called Harbor IDP. I don't want to get too much into it. Uh, what it does, but it basically brings things together for developers to build and deploy things more effectively, with best practice built in, and just reducing of friction. Now, where it's kind of important is how we're using gateway to work with our APIs. So, this is our architecture. Uh, so running inside of our Network in Kubernetes, we have a bunch of containers. We've got Gateway, uh, a couple of APIs.

and a UI, so the APIs are all Spring Boot, and the UI is React. So, the notification API is an API you can post a message to that will go to the user waiting for something to happen. The flow API is something that does a bunch of orchestration and sort of work in the back end, and the event API is something the flow API can look at to tell if something needs to happen or if it's waiting for something to happen or something failed. So, this is kind of more system-level events. So, today we'll just really look at the UI and the notification in the Gateway. I'm going to run everything locally, just for simplicity. Keep an eye on the ports, so the key port here is the gate, which is on 880. You'll see all the other ones on other ports, but you'll see why 880 is important here in a minute. The functionality I want to show is how we're centralizing authentication using Gateway, how we're doing routing with Gateway, obviously, and some cookies and headers.

management. So first use case is authentication. So we're using Azure AD, and we want to have our backend sessions to the APIs authenticated. And the main way that everything talks to everything else is through the Gateway. So let's take a look at what authentication looks like. So I'm going to do an incognito window here. Oh, I should start up the apps. So I'm going to start up the backend APIs. So I'm just going to start up the notific and the flow service. I'm going to start up our front end, which as I said is React, and then of course the star of the show, Gateway. And we'll be jumping into the code to see how some of this is happening. Spencer and I was joking these are some of the most boring demos 'cause Gateway does so much with so little. Okay, so spring team loves boring, right? It should work and not surprise you like this machine did. Yeah, okay, so incognito window, Go Local Host. Okay, so I hit the Gateway, and the first thing it's doing is actually

prompting me to log in, so I am going to unplug my machine because it feels weird logging into a secure system. Broadcast he's not going to dox himself. Yeah, so this is a ra authentication flow, which I'm going to just quickly get myself through. Okay, all right, let's plug that back in, play that comes back. Yay, it works. All right, okay, so I'm running locally. We don't have to worry too much about the app. We're inside of the app. This is the React application. If we come in here into the developer tools and we take a look at Network, we should see some calls here. There we go, so we can see. Let me zoom in a little bit. We'll get into routing in a minute, but we can see there's some calls being made to the backend system, systems, and you know, we're successfully inside of the React application, which was on Port 33000. So like I said, super boring, but I mean, what's interesting here is, well, let's take a look at the code actually.

to see how all this works. So, inside of the Gateway, if we take a look, this is the only place we really configured where to provider-wise. So, making our call to Azure now, we do have something. It's a security filter, and what it does is it takes the authenticated principle and puts it as a JWT token inside of the header. Now, where that is interesting, and we'll get into routing, and I'll show you where that filter is applied in a second, but if we jump into our backend code, we have a base library, and we have this Azure basic request interceptor. It's a basic kind of library, and what it does, so this is inside of the no notification API or any of our downstream API, downstream from Gateway. What it does is quickly get the JWT token and decode it. We can actually get some information out of there that some of these services need, but this is our way in the downstream APIs of verifying that it's an authenticated session. So, what's great there is our API layer doesn't have to

do that whole O2 dance. It just knows there's a token in the header, and it gets it. So, that's valuable. Absolutely. Yep. And how simple it was to do it right. He's not doing a lot of work. These aren't hundreds of lines of code in the classes. These are very simple filters that are happening. So, now let's take a look at routing. Um, now if we expand this a little bit, and let's dig back into this, just going to expand this tab so you can see we're making a call to the notification server. We're passing some parameters to look for unread messages. So, the React application is basically checking to say, Is there any unread notifications? If there are, we'll let the user know so they can read the message. But you'll notice it's Local Host 880. And if we take a look at our diagram here, notifications is on 8081, right? So, you'll see it's passing this API proxy Harbor notifications API V1 notifications and a bunch of parameters. So, let's quickly jump over to

our React application. So, the React application had a file. See if I can remember. I'm not a React guy, so ah, here it is. So, inside of this configuration is a base notification URL, which is API proxy Harbor notifications API V1. So, that's all the React application needs to know. So, it makes a call. Let's quickly take a look at Gateway. So, it keeps that in mind and quickly jumps over to Gateway. We'll go inside of our router configuration. So, this is all the configuration we needed for routing. It's a simple Spring Bean. It extends this routing configuration base. We've got our security filter, din TR filter, which we'll get into in a second. Each of our APIs has this router function. So, we have some configurations, the configurations that we pass in, you can see here. So, for the notification, there's an ID for the Bean, there's a host name, so this is where it's actually running in our network, and then there's a path matcher. So, where that becomes interesting is if we jump back here, this build API router function being if you

take a look at the API function being. It does a rewrite, and what it does is it strips that path matcher off and then appends the arguments to the actual real host of the application and sends the request along. And you'll actually see it supplies that security filter, and that is the place we saw where it's doing the JW token and adding it to the header. And you'll see we sort of have a catch-all front-end route here. So basically, if you just hit the Gateway and you don't specify one of those path matchers, it's going to route you to the UI. Pretty common to do that. The UI is the base, and this is just using the MBC Gateway DSL, just like I was in my demos. So how is this valuable to us? Well, the UI layer is abstracted from host names and any kind of container details, and that goes for all of the APIs actually. And the clients aren't exposed to any of the APIs. So you saw when we looked in the network tab all

the traffic is coming from the Gateway, so a single host. You eliminate all cross-origin request problems by using a Gateway like this, and there's some potential here because we've put this Gateway in between our UI and our APIs. We've got great support for API versioning. If we add something new, we can send a certain amount of requests to that thing and see how it works out, so it gives us some great sort of future-proofing when we start versioning and experimenting with our APIs. Um, the final thing to show you, uh, dinat TR, is our observability tool, and what we want to know is if somebody is interacting with the tool and they have a bad experience. We want to know what went wrong, but the challenge is their session could jump across any one of those APIs all over the UI, so how do you glue it all together? So, we talked to the din TR team, and they were like, Hey, the solution we want you to use is to add a cookie with this certain name and put a value in there that is unique to the

user, so you know, we thought, okay, we'll use the email address, but we didn't want to talk to all the owners of these APIs and them make that code change. So, of course, Spring Cloud Gateway to the Rescue. All the change we had to make, you'll notice here in our router functions, there's an after. There's a Dynatrace cookie filter. So, an after filter is a filter that operates on the response coming back, sending it back to the client, right? So, you can see how easy these filters are, basic component, um, and you know, there's a contract here. You can follow the documentation, it's amazing, uh, and all we're really doing is adding the cookie. Um, so the details about the cookie are an environment variable, but what we do is we grab that JWT token, we get the email, we put it in the cookie, and then that is now in every single uh, session, and then when we're in Dynatrace, we can kind of have a complete view of what a user has done if

They run into a problem. So, the great thing about this is the API teams didn't even need to know about this, so they didn't have to do a bunch of code. If we change our tools or we change our strategies, we don't have to download that work to them. We can just do it in the Gateway. Uh, so that's it for me. La last shout out is to my team. So, building stuff like this, you need an awesome team. So, there's my team of an amazing product owner, Maxim. So, he's an ex-developer and also super passionate about building stuff. So, we're working day and night on this stuff. And shout out to intact, a super awesome place to build stuff. That's it. Thank you. Sorry for the uh, the glitch in the middle, but uh, if we do have time for a few questions on stage, and then if you have more after, we can get you up at the in the back of the well, back of the theater, I guess it's the right place. People are trying to photograph this.

slide and yeah yeah. I don't want to. So that's where my demos were that I showed that I was running. So perfect. All right. There's a lot more in there. So I don't know if you noticed, but there are a dozen and a half different demos of different things that you can look at. So also this kind of subtle point here that this is the MVC Gateway. So yeah, you want to tell us a little more about that. So the original Spring Cloud Gateway was built using Reactor. So writing filters and predicates is a little more involved because it's a pretty big paradigm shift. The MVC Gateway is traditional servlet thread per requests, but with Loom virtual threads. This certainly makes a lot more sense now. And let me tell you, as I was writing it, like just dealing with an input stream and an output stream was wonderful because the APIs are much, much simpler to work with. So amazing. Any questions? Questions come around with a microphone. Any other questions? If you're shy and want to meet.

us afterwards in the back. That's fine too. Well, thank you very much. We appreciate your time. Wonderful. All right, thanks everybody, also on the live stream. And uh, we'll be back with another talk in uh, I think what I, I, I think it's 15 minutes. I'm not entirely sure, but we'll see you. It's on the schedule. We'll see you soon.

Scaling APIs With Spring Cloud

Spring is the most popular application development framework for Java™. 
Millions of developers use Spring to build simple, truly portable, fast and flexible JVM-based systems and applications.
Learn more about Spring at http://www.spring.io

Scaling APIs With Spring Cloud

Scripts