가상화 기술을 통한 서버 안정성 향상

### 가상화 및 고가용성을 통한 안정적인 서버 클러스터 구축
* 💻 이 동영상에서는 **항공 교통 관제, ATM, 911 통화 라우팅**과 같은 중요한 시스템과 일상적인 비즈니스 운영에서 **서버 안정성의 중요성**에 대해 설명합니다.
* 🔧 이 설정에는 **인텔 제온 플래티넘 8562Y+ CPU, 384GB RAM, 100Gb 네트워킹 카드**가 장착된 기가바이트 서버 4대가 포함되어 가상 머신을 위한 강력한 클러스터를 구성합니다.
* 🛠 서버는 **오픈 소스 가상화 플랫폼인 Proxmox VE와 클러스터 스토리지용 DRBD가 포함된 LinStore**로 구성되어 **고가용성 및 실시간 데이터 복제**를 보장합니다.
* 🌐 네트워킹은 다양한 유형의 트래픽을 위해 **VLAN으로 설정**되며, 클러스터는 물리적 노드 간의 **VM 마이그레이션 및 장애 조치**를 관리하기 위해 만들어집니다.
* 🔄 라이브 데모에서는 다운타임과 **Plex 미디어 스트리밍 및 Steam 게임 다운로드와 같은 서비스 중단**을 최소화하면서 서버 간에 원활하게 VM을 마이그레이션하는 방법을 보여줍니다.
* 💡 이 설정은 **홈 자동화에서 엔터프라이즈 시스템에 이르기까지 잠재적인 애플리케이션**을 통해 인프라 관리의 판도를 바꿀 수 있는 것으로 주목받고 있습니다.
* 💰 이 동영상은 **구형의 저렴한 하드웨어로도 비슷한 설정**을 할 수 있으므로 더 많은 사용자가 고가용성 클러스터에 액세스할 수 있다는 점을 강조합니다.
* 🔗 설명에는 빌드에 사용된 구성 요소에 대한 링크와 **인텔의 에메랄드 래피즈 프로세서 및 기타 관련 제품에 대한 추가 정보**가 포함되어 있습니다.

가상화를 통해 서버를 원활하게 관리하여 중요한 애플리케이션의 안정성을 보장하고 다운타임을 최소화할 수 있습니다.

computer problems are a fact of life, and sometimes the fix is as simple as just turning it off and turning it back on again. But other times, it's not, and when the system you're talking about is running an air traffic control system, controlling a bunch of ATMs, or say routing 911 calls, keeping them up and running can be a matter of life and death. Now, the stakes aren't nearly as high for us, but this server here runs multiple apps that we rely on every day, accelerates our game downloads with Steam caching, and it runs our DNS. If that service goes down, it breaks literally everyone in the company's internet, which my boss informs me isn't great. So, how do we make it more reliable? It's already a server. We build more servers. And what's really cool about this is everything we're about to show you, courtesy of Intel, who sponsored this video and sent over their new Emerald Rapid Xeon CPUs, can be done on nearly any computer, even your dad's old Dell, that is, as long as you have more than one. So, if one

leaves for cigarettes. We can still play catch. More than one dell, not more than one dad. Oh well, anyways, I'm done. You want to check this out? Yeah, let's have a look. You got your lovely cat picture, your crab rave on that computer. M, watch this. Like, I can, yeah, I can interact with this. Just give it a second, okay? It's going. Hey, now it's on this computer. And like, no bamboozle here. Look, watch. Whoa, buddy. Watch, watch, watch. Boom, unplugged. I can just completely interact with this as I normally would. So, what's going on here? What you guys just saw was the programs, the lovely drawing, the entire operating system just teleporting from the computer over here to the one over here. No trickery. This is possible thanks to the magic of virtualization. We've talked about it before, but if you're not familiar, virtualization allows you to slice up a single machine into multiple less powerful virtual machines. And this setup leverages that technology to allow us to move these

virtual machines between multiple physical computers. That way, if one breaks, another one can immediately take its place. And the best part is that while this all sounds super fancy, all the software we're using is both open-source and free. And we're going to show you guys how the setup works in a little bit. First, I want to take a look at the servers we're going to be using for our setup. Gigabyte sent over four of their R163 SG2 AAC1 servers. These are bare bones, so we're going to have to add a few of our own parts, but we should be able to build this in what, like 5 minutes? I'd like to see you try this, guy. We're going to add some of our own parts, starting with a pair of Patriot 480 gig SATA SSDs that will function as a mirrored boot drive. This kind of per-machine redundancy isn't strictly speaking necessary because we could lose an entire IRE machine in our configuration without having any issues, but having them in pairs makes our lives

easier in the future potentially since if one of them fails we can just replace it and then rebuild it from the other one. Then on the other side of the machine, we're installing two of these Kokia CD6 7 TBTE drives for fast bulk storage. That leaves us six more SATA bays to do nothing with and two more NVMe bays for potential future expansion. Moving back, let's get our CPU installed. We're using a Xeon Platinum 8562 Y+ in each node. These were graciously provided by Intel, and with 32 cores, 64 threads, and 4.1 GHz max turbo clock speeds, these are going to give us a ton of compute to share between our virtual machines, all at a modest 300 watt TDP. We're going to have it and the rest of the parts linked in the video description. Now, I've never installed in this socket before, so good luck me. Step one is to install the carrier on the CPU, and you can tell which one of the three you're supposed to use by the little

marking right there on the CPU IHS. Line up our little Golden Triangle with our gigantic gargantuan hole in the whole thing triangle. Oh, this is adorable. It's got a cute little arm so you can break the thermal paste seal with the cooler so you can get the cooler and the CPU separated more easily. Love to see it. Speaking of thermal paste, we're going to be using a Honeywell PTM 7950 pad available at LTTstore.com. This stuff is absolutely perfect for a server install because it lasts not forever but for a very, very, very long time without maintenance. Now you might think, Okay, go ahead, put it onto the CPU socket. You'd be wrong. Instead, I'm going to install it onto the cooler. I'm going to know how to do that in a sec. So arrow and arrow, so maybe ah ah ah hey, there we go. Damn, look at that vapor chamber. Love me a vapor chamber. Okay, we're going to make sure all these are clicked into place. Look for our little

arrow here line that up with the arrow on the socket and make sure that the locks are in their unlocked position. Then we should be able to just that's it's locked. Oh, that's it. Okay, next comes something you don't see me do very often, and that is use a screwdriver other than the LTT screwdriver. And that's because these need to be torqued to a specific value that is 6.9 inch PBS. Nice. It's so cool to think that if I was doing this, you know, performing maintenance on the server or upgrading a bad RAM stick, our entire operation could be chugging along as if nothing happened. Speaking of RAM, we've gone with four 96 gig Micron 5600 megat transfer per second registered ECC dims. That's a somewhat unconventional choice because especially in a server, giving up half of the memory channels means that we will be giving up some performance. But we don't really need all of the performance for now, and 384 gigs is a ton of

capacity for our needs at the moment, and of course, if anything changes, we can always add more without any downtime to our services. The only thing that's really important here then is making sure that we install our sticks in the correct slots, which is not always super intuitive, so make sure to consult your manual. We don't need a GPU for now, though we could add one in the future, so that means all that's really left is these Nvidia connect X6 cards. Now, 100 Gig networking might seem a bit overkill, but because our setup uses high-speed drives in four servers, and we want to be able to withstand two server failures, anytime we're writing data, it has to be simultaneously written to the drives on at least three machines. That ensures we have three up-to-date copies in the event of an unexpected failure. Now, if you were doing this at home, you obviously wouldn't want to spend this kind of money, but the good news is that you can do this with as few as two machines, and if you're not trying to run a high-speed

caching server for 100 people. 10 or 25 gig cards are available for a fraction of the price, and you can connect them directly to each other without an expensive switch. I mean, even one gig could work for light applications like ensuring that your home automation system never goes down. Enough chitchat though, let's get on with the demo and show you what happens if one of these things goes to heaven in a live environment, but not before we get them in the rack and set up specifically here in the lab server room. Because if you didn't notice earlier, the studio server room is kind of running out of space, at least until these machines are up and running, and we can take the machine they're replacing out. Let's go grab the service. Unfortunately, the rest of the machines are now magically built off-camera, and we can just slide them in. What the hell is going on? Oh, there we go, beautiful. These gigabyte chassis come with nice toolless rails, so installing these in our nice

ginormous hammonded rack should be pretty easy. Yeah, look at that. It's getting close. I can taste it. We just need networking, like we mentioned before, 100 Gig. But what we didn't mention before is that each is getting two of them, specifically one to each of the network switches in the rack. That way, if one of those switches has a problem, the servers will stay up, and we even get an added bonus because some fancy Dell magic called VT. We get the throughput of both of these cables, so 200 gig to each surface. Pretty sick. All that's left then is power, and like any other good server, IPMI, which is a management interface, and allows us to control the machines even if they're not working. They have like a hardware problem. We can still access them. We can turn them on, turn them off. It's kind of magic. If you have a server that doesn't have IPMI, I don't know. I don't even know if that's a server, really. There are two main elements to making this setup work: clustering the

hypervisor, which controls our virtual machines, and clustering the storage, which you can skip if you have existing network storage you want to use instead. If you're not interested in how to set this up, you can skip ahead to here to see what it's like when it's up and running. This isn't going to be a perfect step-by-step guide, but with the documentation you can find down in the description, you should be able to replicate this setup pretty easily. Starting with networking, we added both of our 100 Gig ports to a bond, created a bridge, and then added a VLAN for three different networks: one for our VMs to use, one for cluster communication, and one for the storage. They can all technically run on the same network, but the cluster needs low latency, and the storage ideally uses jumbo frames, so splitting it up like this is best practice. You'll also need to add each node's cluster Network IP address to the host file on each node. With the networking up and running, enable the no subscription repo and disable the

Enterprise repo. It's not recommended by the Proxmox team for production. They want you to pay for the Enterprise repo, which is a bit more stable, but the free one is totally fine for a home setup. Run any pending updates before proceeding, then make sure you have a reliable and ideally local time server configured on each of your individual servers, as the clustering software wants the time very closely in sync to stay happy. With that out of the way, we can set up our cluster, which handles syncing the configuration and management of any virtual machines between our physical machines, and it also orchestrates migrating or restoring them when a machine goes down. Creating the cluster just takes actually a few clicks, but you might want to consider the size of your setup before you continue. That's because in order to make sure everything stays in sync in case of an issue with a machine, you need the majority of servers online and available to be able to say, Hey, I see that one's offline, but you know, we're still good. They call

this Quorum. If you have an even number of machines, let's say four like we do, and each server gets the default single say or vote, the minimum possible majority is then three servers. So, that means we can only withstand one going down, which is the same amount of redundancy you'd get if you had three machines, 'cause you can only lose one to have two. If you only have two computers, then you only ever have a majority when both are online, which obviously doesn't work. That's not safe. But you can screw it around this by adding a third machine, like say a Raspberry Pi, to be a tiebreaker. But that's kind of beyond the scope of this video. Once you're ready, select the cluster Network in the creation menu, and then join the other machines to the cluster. Once they're in, you should be able to see them in the web GUI of any of the machines. Now, on to clustering our storage. By default, Proxmox is very heavily integrated with Ceph and open-source distributed storage.

system that's pretty easy to set up and maintain. With that in mind, newbies should start with Seth, and you can follow the great tutorial on their Wiki. But it isn't the most performant in a small cluster like this, so we're going to be using something called Linore with DRBD, or Distributed Replicated Block Device, another open-source storage system. It requires a bit more manual configuration, but they do have a purpose-built tutorial for Proxmox and host the files for free, with an optional paid Enterprise version that operates on a similar model as Proxmox itself. Unlike Seph, it doesn't handle its own storage devices, so we mirrored our two Kokia SSDs with ZFS first and then pointed Lin store to that. Once it's installed and configured, then you can add the clustered storage to Proxmox, create a virtual machine with that storage, and it'll automatically be replicated in real time to the number of other nodes you specify. And if you happen to migrate a VM to a server that doesn't have a

copy on it. It'll automatically stream the data over the network from one of those nodes in what they call dis list mode. But let's just try it. Hey, pretty nice, right? Looking good. It's like even cable managed. I know, right? So, 200 gig on each of them. Nice. Who are you people and what have you done with our infer team? I made one small adjustment just for you. Look at the drives. They're in the same spot. No, they're not. The top one's different. I hate you so much. Why would you do that? But more importantly, does it work? Uh, yeah, obviously. Okay, well, here's your Windows desktop. Obviously, he says what editor. A super cut of things not working here. Please, Jake, we have a leak. Oh God, one failure. You just downgraded my Wi-Fi. Four drives aren't working. Did you actually break it? Anyways, you see our Windows, right? Yeah, our Windows is running right now on number four, which is the bottom server. Yes. Now, obviously, remoting into the machine over Wi-Fi. Okay, the video playback's a little

bit choppy. That's not going to affect the type of workload we would normally be running on something like this, like a DNS server, or like, are we finally doing active directory? We will not, not today, not today, but we can now, but this is the kind of setup that you want for something like AD live. Playing the video. Let's migrate to number one, which is the top one. The process will be a little bit faster, but basically what it's doing is copying the memory, like the RAM, what's actually in memory. And then once it's done, most of it, it pauses the operating system for a split second, copies the last tiny little bit, and boom, that is so cool. You're exactly where you were before because the storage is already there, right? So in terms of actual downtime, like interruption to that experience, 17 seconds, no, 270 milliseconds. Oh, I thought you were pointing the other, no, 17 seconds is that whole process, but oh yeah, yeah, well that's kind of downtime, I guess. No, 'cause that if there was somebody

using this like as a virtual desktop, for instance, they would see like a quarter of a second blink and otherwise like nothing changed. I wanted to show a more realistic to us demo. Sure, come hither. Here's a Plex server. We've got some videos on it, and this is on server number one. Okay, let's play a video now. We go and move our Plex server to a different machine, so it's copying the RAM at 2.5 gigabytes a second. So, it's like 2.8 gigabytes a second. That's pretty good. We haven't done any actual... Oh, it's already done. And no interruption because video playback, like many other applications, uses buffers to hide small interruptions in the service. In this case, downloading the video in small chunks, a little bit at a time. Yeah, roughly 10C chunks, it looks like here, which is plenty to cover that 146 milliseconds of downtime. Wow, you want to try Steam download with Landan Cach? I mean, we should. Yeah, why not? Yep, we're

CPU bottleneck for sure, using, you know, 80 to 90% of a 24-core Threadripper. But I realized I made a little bit of an oopsy here. Look, you can see the CPU usage. We're using % of our eight CPUs that I assigned to this steam cache. We can see our network traffic's going up, sick, except I made this as a container, not a VM. And the thing with containers, they're great, they're a little bit lighter weight, better performance, but they run within the kernel of the main system. It'll shut down that container and then just reboot on the other machine, right? Which means it's fine, but there will be a longer downtime delay, but way less than, Hey, is that thing working? Oh, I think the internet is not working. Somebody should go look at that. Yeah, trying to figure out what's going on, fixing the machine, getting the machine back going. So cool, you're talking about the matter of a couple minutes, maybe. Yeah, now for the most impressive demo yet, the unexpected migration. Which one am I yanking? Okay, so

number one has three VMs on it. They're all in the high availability Jake's chain. Ah, what means. Oh, I get it. Okay, sorry. Which one? I wasn't even listening to number one. Number one. And we'll see how fast it does. We're looking at server one from server two. So go for [Applause]. It, from my understanding, this process takes a minute or two. Okay, to go. Oh, it's already detected the no is offline. Sure is. If you're doing scheduled maintenance, you can actually go and just shut off a machine. And then it will just be like, oh crap, I need to move all those things before I shut off, which is a little bit nicer. In this case, it has to be like, sure that the server is down. So all three of those are yelling at what was this. Say hello. What happened? Are you alive? What's going on? I can hear them. Hello. What happened? Are you alive? What's going on? Oh, hey, hey. Oh, it did something. So in theory, it should distribute them evenly because that's the option that's

set right now, right in terms of its workload, you mean. Yeah, there is also a mode that does like resource checking. Sure, but right now, it's going how many VMs are in each one and just like filling that number. So, it's even that is so cool. Okay, so what service was running on that one? Was that the Steam cache? So, we should go download a game. You could do Plex right now too. Let's go do it. Let's go do it. Come on, let's go. Wait, we can take this door and no movie magic, but also magic virtualization magic. This is flipping awesome, and it's going to be an absolute game changer for the way that we manage our infrastructure. And like I said at the beginning, I think the coolest thing about it is that this type of architecture doesn't even have to run on the kind of Emerald Rapids latest server technology that Intel and Gigabyte and Micron and Nvidia all sent over here. So, the takeaway for you guys is whether it's for work or whether it's just for

your home automation or your Plex server at home. Something like this is absolutely attainable with potentially very little financial outlay. Like go buy some used, like eighth-gen Intel core processors. Those are pretty cheap, some cheap DDR4, and you're off to the races. Or if you're doing this more properly for your business, check out Intel Emerald Rapids and their whole line of Xeon and GPU products down below. Where were you pointing? Down below, that's the description. Get your mind out of the description.

I Built a PC that CAN’T Fail… and You Can Too!

Linus Tech Tips is a passionate team of "professionally curious" experts in consumer technology and video production who aim to educate and entertain.


Linus Tech Tips

컴퓨터 문제는 흔한 일이지만 항공 교통 관제나 911 전화 라우팅과 같은 중요한 시스템의 경우 안정성이 매우 중요합니다.

이 스피커의 서버는 여러 필수 앱과 서비스를 실행하므로 이중화를 위해 더 많은 서버를 구축하는 것이 목표입니다.

인텔의 에메랄드 래피드 제온 CPU와 가상화를 사용하여 물리적 컴퓨터 간에 가상 머신을 이동함으로써 한 서버에 장애가 발생하더라도 원활한 운영을 보장할 계획입니다.

설치 과정은 CPU의 골든 트라이앵글을 소켓에 맞추고 오래 지속되는 유지보수를 위해 Honeywell PTM 7950 열 페이스트 패드를 사용하는 것으로 시작됩니다.

서버에는 총 384GB의 메모리를 위한 4개의 96GB 마이크론 ECC DIMM이 장착되어 있어 향후 다운타임 없이 확장할 수 있습니다.

100기가 네트워킹은 4대의 서버에서 고속 데이터 쓰기를 위해 사용되어 서버 장애 시 3개의 최신 사본을 보장하며, 소규모 설정에서는 저렴한 네트워킹 솔루션을 사용할 수 있는 옵션이 있습니다.

이 설정에는 각 서버당 2개의 100기가 네트워킹 케이블이 필요하며, 총 200기가 처리량을 제공합니다.

전원 및 IPMI 관리 인터페이스는 하드웨어 문제 발생 시에도 서버를 제어하는 데 필수적입니다.

또한 최적의 성능을 위해 짧은 지연 시간과 점보 프레임에 중점을 두고 하이퍼바이저와 스토리지를 클러스터링해야 합니다.

DRBD로 Proxmox 클러스터 설정하기

네 대의 컴퓨터가 있는 클러스터는 과반수를 위해 최소 세 대의 서버가 필요하므로 한 대의 서버가 다운될 수 있습니다.

Proxmox는 스토리지로 Ceph와 통합되지만, 튜토리얼에서는 소규모 클러스터에 DRBD와 함께 Linore를 사용할 것을 권장합니다.

Linore는 수동 구성이 필요하며 ZFS로 SSD를 미러링하여 마이그레이션 시 가상 머신의 실시간 복제 및 데이터 스트리밍을 제공합니다.

효율적인 서버 마이그레이션 프로세스

서버 마이그레이션 프로세스에는 RAM을 복사하고 운영 체제를 잠시 일시 중지하는 작업이 포함되므로 다운타임이 최소화됩니다.

Plex 서버를 사용한 데모에서는 버퍼링 덕분에 마이그레이션 중에 비디오가 원활하게 재생되는 것을 보여줍니다.

마이그레이션에 컨테이너를 사용하면 더 가벼운 무게와 더 나은 성능을 제공하지만 가상 머신에 비해 다운타임이 더 오래 지연될 수 있습니다.

1번은 고가용성 체인에 3개의 VM이 있어 서버가 오프라인 상태가 되면 빠르게 감지합니다.

이 시스템은 VM 간에 워크로드를 균등하게 분산하고 리소스 확인 모드를 갖추고 있습니다.

이 아키텍처는 중고 인텔 프로세서 및 DDR4 메모리와 같은 저렴한 옵션으로 가정과 기업 모두에서 사용할 수 있습니다.