Linus Tech Tips: Building a 960TB Storage Array with Recertified Seagate Drives

Recertified Seagate drives offer significant cost savings for large-scale storage solutions.

### Cost-Effective Storage Solutions: The Journey of Recertified Drives
* 💾 Over the years, the need for storage has escalated, leading to the acquisition of **over 2.5 petabytes of bulk storage**, yet the demand continues to grow, particularly for **upcoming projects**.
* 💡 To address this challenge, the team is exploring the purchase of an **additional petabyte of storage**, seeking **more affordable options** in a tough economic climate.
* 💵 They discovered recertified drives from Seagate, available at **half the price of new drives**, prompting a **$10,000 order for 60 drives** from eBay.
* 📦 Although the recertified drives come with warranties and good seller ratings, there remains **uncertainty about their condition** and the implications of being labeled as **"recertified."**
* 🔍 After unboxing, the drives appeared nearly unused, with **low power-on hours** and **no visible signs of wear**, suggesting they were **recently manufactured**.
* ⚙ The team plans to repurpose an old Storinator XL60 server for the drives, emphasizing the **cost-effectiveness of on-premise storage** compared to cloud solutions.
* 🖥 To enhance performance, they are upgrading the server's hardware, including a **faster processor** and **more efficient storage controllers**, while ensuring proper cooling with **high-performance fans**.
* 🔧 Ultimately, the project aims to create a **reliable storage solution** using ZFS, with hopes that the recertified drives will **perform well**, as they have in past experiences with similar products.

We make a lot of videos, which is why over the years we've built out over 2.5 petabytes of bulk storage. But somehow it's not enough. It's never enough, and that's especially true when you consider all of our upcoming

projects. The videos just keep coming. The badminton gaming center needs capacity, and if I am ever going to deliver on that free storage backups as an employee perk concept, I'm going to need a lot more storage, like at least another petabyte. In this economy, you're right.

Storage ain't cheap, but what if I told you that I could get a petabyte for half the price?

I'm listening. Well, we can. In our quest to find the most cost-effective storage upgrade, we stumbled upon these. They are recertified drives, but the recertification is done

by Seagate themselves, and they come in at just about half the price of the same drives brand new. So, like any other sane person would, I immediately handed eBay my credit card information and ordered $10,000 worth of them—that's 60 drives.

So you've got to be wondering, Lonus, what's the catch? Come closer, and I'm going to tell you. It's our sponsor, MSI. They've got laptop deals on deck this December, from the sleek and slim Modern 15h AI to the gaming powerhouse Katana 15 AI. You can

save big by clicking our link in the description. [Music] Okay, the real answer to what's the catch is we don't know yet. That's kind of the point of this adventure. You see, the best advice that we've ever gotten and given is to avoid secondhand storage, so this

is kind of untried territory for us, but the deal was too sweet to pass up. Now, it's not like we were worried about getting outright scammed; eBay has pretty good buyer protections, and the main sellers of these recertified drives seem

to have excellent ratings, with most of the drives even coming with some kind of warranty. But there's still a lot of questions, like, what does 'recertified' even mean? Are they used? It's hard to say. I mean, the eBay listings for these drives say that they have extremely low

power-on hours and no bad sectors, which would seem to suggest that they're basically brand new. But if they were recertified by Seagate, the manufacturer, then in theory they could just reset the power-on hours. The next thing the script says is 'unbox them,' but there's not a

whole lot of unboxing to do. Like, is this how they came? Well, it's not like when you buy an individual drive; it's not like they're all going to ship in, like, Amazon packaging. When you're buying 60, they come in trays, and that makes

sense. Okay, so how used do these look? I would say they look pretty unused to me—not at all. There are some fingerprints, but, you know, our inventory people definitely put asset tags on them—not on this one. Oh, it's freaking clean! We gloves—even the screw holes! I don't

think these have ever been mounted to anything. Look at this: 2022. Are they all like that? 2020, 2021. So these were manufactured anytime over the last few years. I mean, these drives didn't come out until like the end of 2021, like December. Oh, yeah, that

One's from 2020, so that must be a very early one. Where did they come from? Server Parts Deals. Ah, yes, on eBay. So, instead of buying our drives from Seagate's own store, we chose Server Parts Deals. They sell them either factory sealed still,

recertified but sealed, or they actually crack them open and pre-burn-in test them for up to 50 hours. According to Server Parts Deals, the chance of getting a DOA drive is actually much lower with those burn-in ones, so we opted for that.

When we picked these up last December—sometimes it takes us a while to catch up to our production ideas—we found that the sweet spot was these 20 terabyte drives for $180 apiece. Looking on eBay seems like we actually got a pretty

sweet deal. They're closer to $220 now, but even that is a huge discount compared to new, and you could probably save a bit by buying from them directly rather than buying through eBay. Disclosure, by the way: Server Parts Deals is one of our sponsors now, but not on

this video, and they actually weren't when we bought these drives from them. No, we paid retail for these. Should we plug them in? Yeah, I want to see if the hours are correct. Okay, we've got CrystalDiskInfo. We've got one of our drives here.

Yeah. Drive details... Smart 21. Okay, that checks out; that's like nothing, essentially. Yeah, great! Now we just need a machine to put them in. Since our goal is to save a buck, we're going to reuse as much hardware as possible, starting with this 45-drive Storinator XL60 that

We decommissioned it a few years ago. It still fits 60 drives, just like it did when it was part of our original Pachyderm cluster, but now it has double the capacity because we're using 20 TB drives. Man, I remember when we built that!

So many people were like, "You're crazy, dude! Just use the cloud, bro!" News flash: the cloud is expensive, bro, and even when it isn't, getting your data back from it is really expensive. It's actually been pretty validating seeing how many folks have been migrating back

to having their servers on-prem. Now, to be clear, we do use Backblaze backups, but only for backup and only for our active projects. Now, obviously, a home or a small business user is unlikely to have one of these kicking around, but everything that we're doing could be

scaled down to just a handful of drives in a NAS chassis or even a single drive in a recycled Optiplex, like that one that Jake has right next to him, which is the same one that we showed off in our Hexos announcement video. There's actually even

people selling 3D-printed NAS cases that look pretty cool. The point is, you don't need to buy dozens of drives in a fancy old server to get a great deal, or at least what we hope is a really great deal.

Oh, I'm confident. All right, I'm confident. It's time to rip these out of here. Oh, it's almost as old as me! It is. On the subject of our server being kind of old, there are some choice upgrades we want to make to this old girl before we throw

Her back in the rack. Oh yeah, if our inventory records are correct, the CPU in that machine is over 8 years old and performs similarly to a first-gen Ryzen 5. I mean, look, don't get me wrong, there's absolutely no shame in driving that car

you've got. And for home use, realistically your network speed is going to be a bottleneck before your CPU. But we happen to have some newer hardware sitting on the shelf; specifically, this ASRock Rack Rome D82T with an EPYC

7402P 24-core processor. According to PassMark, it is nearly five times faster than the old 8-core that it's replacing. Now, one important tip: if you are going to be reusing old gear like our old board, make sure to update the BIOS to

get the latest security patches and definitely replace any dried-out thermal paste. This one should be good; I redid that not that long ago. Okay, board's in, but I forgot to check the memory. It looks like... yeah, we've got 256 gigs of RAM

in here, which means we should probably put in some more. The rule of thumb is about 1 gig of RAM per terabyte of storage for read caching, but our workload is actually pretty read-light, so I think we're just going to leave this for now. What we're not going to

leave are these HBAs. These HighPoint 750s were end-of-life years ago, but more importantly, they just kind of suck. So the plan is to replace them with some LSI 9200 series cards. Now these are also very old—like PCI Express Gen 2—but

For us, that's a feature since they happen to have the right connectors for the cabling in our machine. So, specifically, we're going to be using a pair of genuine LSI 8-drive cards and then three of these 16-drive cards.

Two of which are genuine and one of which is an eBay knockoff. Now, we had a whole video planned on these things back in the day, which is actually why we have this. I think this one's the fake one; yeah, it doesn't say LSI on the front,

whereas the other ones do. But the thing is, counterfeit silicon is pretty unusual, and I suspect this is just old chips from maybe failed boards in some other way that have been resoldered onto new boards. So, it should be fine, I guess. That's part of the adventure.

I've had some friends who've had these cheap cards work for years without issues and others who've had them die in a few months. We don't have much choice either way since the legit ones are really hard to find. I'm almost done wiring up all the drives; then all I've got to do is chuck a little

m.2 boot drive in there. And then in our last slot—that's not it—uh, I'm going to be using a Mellanox—there we go—Nvidia ConnectX-6 dual-port 100 gig network card.

Oh, you want to put that in too? We're also going to use this cool adapter doodad to connect this 2.5-inch Dell Gen 4 NVMe SSD that we had lying around, and then we can use that to help with read caching or L2 ARC.

Can we hang out? Let's screw it in first. What? I don't need to screw anything in. You screw it in. Look, it fell out! That's because you didn't screw it in. Way to go, Jake! Don't drop it on the... Be careful! That's from StarTech. They kindly

sent that to us. Thanks! [Music] StarTech. Now, to keep everything in this area cool, I'm going to throw in a Noctua industrial 3000 RPM 140mm fan. That way, we won't have any issues with overheating on our network card, HBAs, or

the NVMe SSD that's in there. Now, to find out if it boots... I don't actually know. Like, it probably should. My optimism is steadily decreasing. If you have a situation like this where you're not getting a display signal from a computer you're trying to

troubleshoot, a good way to check if it's actually working is to use Caps Lock or Num Lock. Which you can see when I click Caps Lock, it is turning on and off. So clearly this system is posting; it's just not outputting the display signal for some

reason, or potentially this cable or the monitor is not working. I'm going to try the IPMI and see if that works. Hey, okay, it's working! Sick. With that out of the way, let's talk software. There are obviously a lot of options, but our go-to

for a while has been TrueNAS SCALE. It's free, which is always great, and most importantly, the ZFS file system it uses is perfect for large setups like this and super resilient. So if we have any issues with these drives, which I hope we don't, our data should be fine if

If you're a home gamer, you might also want to check out Hexos, the user-friendly NAS software Linus invested in once it's out of early access. But if you're not feeling adventurous, there's always Unraid, whose main claim to fame is the

ability to expand its array one drive at a time. Or there's Open Media Vault. We've got TrueNAS installed now; everything seems to be working, and we can finally put some hard drives in. Heavy as a full train—it's heavy, so I've recruited help. It'll take a sec for them all to

get, like, detected and imported, but let's see how many we have so far. Ooh, is she working? I mean, it's on, but I haven't looked at the drives yet, so I guess let's see how we're doing. 61! Wow, they're all working!

Well, they all show up. Well, yeah, I mean, we haven't made an array yet. Should we run a SMART test on all of them first? Can you do that all at once? Oh, yeah, nice! Wow, well, that's great. So they all pass a short SMART test.

If you're getting new drives, you should do more than that. You should—usually people recommend—zeroing them all out, not because you need to clear the data, but as a burn-in to make sure that all the sectors are good and everything is happy. We're not going to do that right now, but it's not a bad

thing to try. And in theory, Server Part Deals has already done that for us. Cool. So we can make a pool now. Oh, boy! What do you want to call it? Swimming? What? So it's a swimm—swimming pool? No, I don't like that. I'm going to call it Ocean. I'm

Not salty, since this is primarily archival storage, and since we have a lot of discs, we're going to group our drives into six 10-drive RAIDZ2 vdevs.  That 'two' in RAIDZ2 means that we have two drives' worth of data protection.

Like RAID6, so for us to lose any actual data, we would need to have three drives fail, and all three would have to be within the same 10-drive group, which is fairly unlikely.  Multi-drive failures like that usually happen when you end up with drives from

the same bad batch. And since our drives are recertified, it's much more likely that they are from different time periods and different batches. The script says 'in theory' because we hadn't actually looked at the stamps on the chassis yet; in practice, they are very

different—2021, 2022—they're all over the place. Now, for a home setup, you're probably going to go with something like a four, six, or eight-wide RAIDZ1 or Z2, depending on how important that data is to you. But with the new RZ expansion

feature, you can start as small as two or even three drives and then expand that existing group as you want to add more drives. That's pretty convenient.

Yeah. Well, we can click 'Start' now or go... yeah, do it. So we should have 873 TiB... what are you—to TB? 960 terabytes! Damn. Hard drives have just been kind of quietly in the background getting like, kind of awesome, kind of thick.

Well, I've paid absolutely no attention to the... it's like children. Uh, our cache SSD is not detected.

Most people, if they're trying to build something like this at home, they're not building with a petabyte of drives; they're not going to have a cache SSD in it. So let's have a look; something about it is not happy; we will have to figure

that out later. Yep, I'm not expecting crazy numbers because it's really old HBAs, and we have a very wide VDE depth size. I mean, but 8 gigabytes a second—that's pretty good—or gibibytes. Oh, I'll take it! It goes up and down and up and down.

It's a hard drive, so we're not really going to get much better than that. Well, dude, I'm not going to complain about that; that's sick! We're not even going to be able to realistically do that, so yeah, that's great. But most of what we're doing on

this thing is sequential anyway. Exactly. That is like five to seven times what we would even be doing over a 10-gig connection, yeah, which is most of the accesses we do, except when we're doing backups and stuff, anyway. So great, freaking awesome! Yeah, I mean, the drives

haven't complained. Wait, this is great! But remember you said you were going to reach out to [company name] and find out what's the deal with these drives? I did; they didn't respond, at least they haven't yet, and it's been like four days or

something like that. Okay, I did find a press release on their website about this program with eBay, and it makes it sound almost like they're taking drives that people were going to throw away—like from a data center recycler—and refurbishing them. I don't know; it doesn't

Say anything conclusively, and they look really new; they look really new. It seems like our best guess is that they're just customer returns or something like that. Okay, well, bad batches, they're rehabbed one way or another. In total, we spent $1,160 on 62 drives, which compared to

the $235,000 we would have paid for new ones, seems like a pretty good deal, as long as they don't all crap out. For now, things look good, but it's hard to say what the long-term reliability is going to look like. I'm hopeful though, especially given

that none of them died immediately. So, even if they do, the fact that we're using ZFS means we should be able to resilver and repair them in a pretty safe way. Yeah, I mean, most hard drives either die like in the first day or, you know, 3, 4, 5 years later.

Right. Server part deals also warranties the drives themselves, so should be okay. Yeah, and full disclosure, um, I took about half of these drives for another project, and they were running for 6 months and they still don't have errors. Okay, well, that's a good sign.

Yeah, and to feed my data hoarder habits in this economy, I actually switched over to recertified drives at my house, and I've got two or three more of them actually that have been running for multiple years now and they've been great. Now, a brand new

Exos drive is going to come with a 5-year warranty instead of just two, but given these are around half the price with half the warranty, it was kind of worth the gamble for me. But let us know in the comments if you've had any

Experience with recertified drives, either good or bad. Like, I don't know, we get flak sometimes when we show a good deal online, and then it goes out of stock a few hours later and no one else can get the same deal. But that shouldn't

be the case with these. There's an entire industry around reusing, recertifying, and refurbishing enterprise IT equipment, and there seems to be no shortage of storage devices. Uh, one of our favorites is those JBOD disc shelves.

Oh yeah, those allow you to connect a bunch of hard

drives to like an existing system, and uh, we did a video on them a while ago. You should check it out. Yeah, we also came across a pretty wild deal on some NVMe SSDs when we were researching this project, so uh, happy deal hunting out

there for now. It's time for me to go hunting. Segue: Season to our sponsor, Ground News. When you think of the news, you think cold, hard facts, right? Wrong. Now, more than ever, you're being told the stories the media wants you to hear.

the fluff and let you come to your own conclusions. They gather articles on stories from thousands of sources and take into account political biases and allow you to compare the coverage and see how the story changes depending on who's reporting. I'm sure you've all

heard that Pat Gelsinger recently retired as Intel CEO, around when they announced their next generation of graphics cards. You can see on screen right now that reports are not really leaning heavily left or right, but there's a tiny bit of left-leaning bias, and thanks to their

Bias comparison tool: As more articles come in, you'll see a breakdown of what aspects of the story are being focused on by each side. Their blind spot tool is a great feature that shows you stories that might not be covered by your own preferred news sources. Go to groundnews.com/LTT to get started and take advantage of a very limited 50% discount on their Vantage plans, for yourself or maybe as a little gift.

If you guys like this video, why not check out the Mother Vault build? Hell yeah, that was pretty

sick! That was like the deluxe version of those Drive shelves. Yeah, it is exactly that. Yeah, and you know I might use some of these drives as like spares for that.

Is Buying a PETABYTE on Ebay Stupid?

Linus Tech Tips is a passionate team of "professionally curious" experts in consumer technology and video production who aim to educate and entertain.


Linus Tech Tips

Existing storage (2.5 petabytes) is insufficient for current and future projects.

Upcoming projects, including a badminton gaming center and employee perk (free storage backups), require at least another petabyte.

Exploration of cost-effective storage solutions is initiated.

Discovery and Purchase of Recertified Drives

Seagate recertified drives are found at half the price of new drives.

60 drives (20 TB each) are purchased from eBay for $10,000.

Concerns about the quality and condition of recertified drives are addressed; eBay buyer protection and seller ratings are considered.

eBay listings indicate low power-on hours and no bad sectors.

Drives purchased from Server Parts Deals; burn-in tested option selected to minimize DOA risk.

A decommissioned 45-drive Storinator XL60 is reused; now holds 60 drives due to larger capacity drives.

Discussion of on-premise server advantages over cloud storage; Backblaze backups are used only for active projects.

Scalability options are discussed, from a single drive in a recycled computer to NAS chassis and 3D-printed cases.

An 8-year-old CPU is replaced with an ASRock Rack Rome D82T with an EPYC 7402P 24-core processor.

BIOS update and thermal paste replacement are recommended for reused hardware.

Existing 256 GB RAM is deemed sufficient for the current workload.

Outdated HBAs are replaced with LSI 9200 series cards (mix of genuine and counterfeit).

A Mellanox Nvidia ConnectX-6 dual-port 100 gig network card and a 2.5-inch Dell Gen 4 NVMe SSD are added.

A Noctua industrial fan is installed for cooling.

TrueNAS SCALE is chosen for its free nature and ZFS file system suitability.

Alternative NAS software options (Hexos, Unraid, Open Media Vault) are mentioned.

Drives are installed and detected; 61 out of 62 drives are working.

A short SMART test is performed; all drives pass.

Drives are grouped into six 10-drive RAIDZ2 vdevs for data protection.

The resulting storage capacity is 960 terabytes.

Investigation into the origin of the recertified drives is attempted, with limited success.

Total cost of $1160 for 62 drives is compared to the estimated $235,000 cost of new drives.

ZFS resilience and Server Parts Deals warranty are highlighted as mitigating factors against potential drive failures.

Personal experience with recertified drives is shared, emphasizing long-term reliability.

The availability of recertified enterprise IT equipment is discussed.