Oh, that’s right. Nice catch! I can probably repurpose a 256GB SSD I have, can’t I? Should be enough for OS + utilities
The plan is to get more down the road, this is a starter setup! 4TB are enough for all my data at the moment, and the second drive is for backup; I will add a third drive for redundancy and that should be enough as a starter
This may be a silly question, but why get a whole extra machine just to make 4tb available on the network? I have an old Linksys router with a USB port that allows you add USB storage to your network, that may be a good place to look if your needs are basic.
Additionally, it looks like you are going for a relatively powerful machine to be able to access a minimal amount of data at a slow speed. Have you considered just getting a couple of external hard drives and just hooking them up to a Raspberry Pi?
Going the Raspberry Pi route would cost $288-$333 depending on accessories. This could be a really good route depending on what all you want to do.
Another way to save more would be to just add the drives to an existing desktop, and just setup samba/nfs to run on it. You’d just need to leave the desktop on all the time, or turn it on when you need to access it elsewhere.
Also, for reference, I have a NAS with 40tb of storage that I use as a VPS host. I have several virtual machines running 24/7, including a Plex server. I stream many videos locally, and have many users that access my content regularly. From what I’ve learned over the years of running that machine, I know that as long as you are just directly streaming the files, and not trying to transcode them, you don’t need hardly any processing power. I don’t have a GPU in my server, and don’t see a need to put one in. If your needs are really as simple as your post suggests, the raspberry pi route would be the way to go. It would also allow you to dip your toes into running a NAS, and see where your original build was lacking, and give you a better idea for what you want your next evolution to look like.
It boils down to two things: inexperience (apparently 8GB of RAM might be enough? Just to name one issue wih my build haha) and I’ve already maxed out my main PC’s expansion slots.
I have a compact case, which is already housing two HDDs (a “landing” HDD, where I store and keep all the… Linux ISOs… until they reach a satisfactory ratio of upload, which has a 2y uptime and just recently encountered its first uncorrectable sector error, and a 2TB HDD where I keep my stuff, including the aforementioned… ISOs…, my GOG games and other media. This 2TB is backed up on a 2TB external drive, and it’s already full). Since this 2TB internal is full, I plan on moving some files to a different external HDD so I can stuff more things in it, but that leaves me with no backups for this stuff I want to move; this second external HDD is very small (650GB) so moving that stuff will make it full and I’ll have no chance to add onto that collection.
It’s a pretty unorganized situation and if I could get rid of all internal HDDs I could get rid of the HDD harness and fit a fan instead, also reduce noise.
In addition to this, I have yet to understand whether or not external drives can sustain “high” rates of writes (when I download GOG games, for example, it can easily add up to 100GB in a day) and reads (I usually seed… Linux ISOs… for tens to hundreds of GB a day). Of course these numbers arent’t for everyday, some days I download nothing for example, other days I don’t even turn on the PC.
powerful machine
And I even went for the cheapest parts I could get haha! Only way to make it less powerful is getting an Intel CPU, like the datahoarder wiki suggests, based on LGA 1150/1151, I can get one for like $20, but I can’t find used motherboards for a decent price, all around 130. That’s why I picked an AM4 platform: parts more widely available.
raspberry pi
Checked it out, unfortunately it’s out of stock in all the licensed retailers in my country. It does sound like a nice starting point, though, so I will keep an eye out for it to be restocked, but since it uses external drives I’m back to the previous question: are external drives sturdy enough to sustain the amount of data I write and read daily?
no GPU
Wait, how can you turn it on and configure eveything necessary on it without graphics? Is it all done remotely via a main machine?
Sorry for the wall of text! And thanks again for the help!
Okay, I feel like I have a better idea of what you’re looking for now.
Unless you’re using some sort of shingled drive, reads from the magnetic platter should be non-destructive, so reading files for seeding shouldn’t do much to the drives in terms of wear and tear.
External drives are fine for durability, and if you get them from a reputable company I’m sure you’ll be happy with them. Buying cheap, high capacity external drives and shucking them is a common tactic that some data hoarders will use to get the most bang for their buck. Remember that external HDDs are going to be subject to more stresses than a normal drive (people moving them while they’re on, etc), so it wouldn’t be in the best interest of the manufacturer to make them less durable than regular HDDs. You just have to keep in mind that HDDs are mechanical in nature. Anything with moving parts will wear down over time and eventually break. Typically you should expect to cycle through your drives every 4 years or so (I say this while never having cycled any of my drives and they are all working fine).
I see you also keep picking drives that have NAS in the name, just so you know, that’s mostly marketing bullshit. They name the drives with different use cases so they can charge more for them and get that little bit of extra profit. There’s very little that can be done to tune the drives firmware for those different use cases. So, just get the most capacity your can afford from the company you want to use, and don’t worry about it it has NAS in the name.
You’re going to want more storage capacity, so don’t lock yourself down with a tiny case like the Node 304. Get a case that can hold a lot of HDDs. I went with the Node 804, which has mounting spots for 8 HDDs and two 2.5" SSDs (more spots than that when you realize that SSDs have no moving parts, so it doesn’t matter where you put them). I ended buying a new bigger case within a year of building my NAS, so definitely leave yourself room to grow, it may cost more up front, but will save you money in the long term.
As far as the GPU goes, I initially got a Ryzen 3400G, which is an APU, so I just used the integrated graphics. Since then, (remember how I bought a new case within a year?) I have upgraded the CPU to a Ryzen 3800x without onboard graphics, and I am now using this motherboard, which has onboard graphics, as well as IPMI. The IPMI is amazing, and I highly recommend it; it’s allowed me to have the computer in a remote location far away from any monitors, and it only has Ethernet cables and a power cable connected.
As far as other options goes, you could also get a bigger case for the computer you already have, that would allow you to add more storage. You could also get a very large SATA SSD (I just saw an 8tb one for ~$340), and you could just shove that into your current case anywhere to hold you over until you save up to get a more dedicated build going.
I researched again for parts and, whichever way I go, it’s all very expensive.
The “cheaper” options are still expensive considered the lack of expandability: I can get, for example, two 3.5" 4TB drives for around 90-100 each, or, slightly better, two 2.5" 4TB drives for around 120-130 each; the 2.5" drives would allow me to get rid of the HDD cage in my PC and mount a fan in its place. Either way, 8TB (actually 4TB + mirrored backup) for 200-250, and I can’t expand it further.
Slightly more expensive: a one drive Synology NAS, for around 300 (including a 4TB HDD, bought separately); again, locked with no further scaling possible.
Then, a two drives Synology NAS, for 450-500 (including two 4TB drives, bought separately), no scaling. But it’s getting closer to my needs.
Finally, a four drives Synology NAS, definitely fitting for my needs, which is 500 not including drives; once I add storage, for example just two 8TB drives (to which I’d add another two down the road), I’m close to 800 (and this is by getting the cheapest 8TB drive I can find, nothing with “NAS” in the name haha).
Shucking isn’t really a financially good option, it looks like, since external drives are actually more expensive in my country.
tiny case
Well, the Node 304 can hold six drives. Realistically, that should be enough for me even if I decide to have two backups (so using two drives for storage and then the other four just to back up those two) and even if I get 8TB drives (at the moment I have just shy of 3TB of data and I think i can easily add another 2TB onto it, but further than that… not sure). Worst case scenario changing only the case is pretty painless, especially if I can sell the old one to buy the larger one.
gpu
Yeah, I know I can get an APU, issue is, with that MoBo I picked only PRO APUs support ECC RAM. For what I understand, ECC RAM, while not mandatory, is highly recommended. The data I have is not vital, but if I can avoid corruption and having to download it again, I’d rather do so. To be fair, I’m currently not using ECC RAM and I haven’t encountered data corruption in the past two or three years… so I’m a bit torn on this point now
your config
That’s a very expensive motherboard, wow! I can’t find it for less than 350! Definitely over budget for me at the moment
larger case for my current setup
This is definitely something I’ll consider, at least as a stopgap until I can build a proper NAS. This way would allow me to have many drives and I could also repurpose the ones I already have.
I think I’ll have to put this project on hold for now haha but thanks for the huge help, I’ll definitely watch that video in the meantime!
Thanks again!
edit: looks like I can shave off another 100 from the planned build, since I can get a CPU (Ryzen 5 2600) and a 550W PSU for free. Which is actually not bad. Remove one stick of RAM, since 8GB should be enough, and I can save another 30, bringing the total to around 500. Not that bad as a start! Actually, bring that back to 600, since it would make more sense to get 8TB drives since I’m already around 3TB. Still, saved money is money saved haha
I’m really happy you’re looking at all your options. Running a homelab/nas can be a lot of fun, but, you’re right, it’s expensive, but you can turn what you learn from it into potential career advancements, so it could pay for itself.
Personally, I’ve had terrible luck with Seagate drives, and the subsequent warranty process. I sent a 12TB Ironwolf back for bad sectors in July and I still don’t have my replacement. They shipped me one once, but UPS sent it back because it didn’t have clearance to cross the border into Canada. I spent hours on the chat getting bounced around, I could find no phone number to call. The experience has been awful. I’ve had other bad experiences with Seagate in the past but the price on the Ironwolfs was so good, but now I really wish I had just got some WD reds or something. I got 3 left in my QNAP and i am praying another doesn’t fail before I get this replacement or my array is toast.
I’m sorry about your experience! I actually have an IronWolf, a small 2TB one, and it’s been a year without issues. I don’t write big amounts of data daily on it though, so my experience might be different.
It looks like DownThemAll doesn't do recursive search (it's not looking at any deeper levels of the site, so if you run it on the page you linked, it'll only grab the thumbnails, because it's not navigating to the subpage where each full image is stored).
A quick search turned up this tool, which might do what you're looking for? HTTrack
100TB, external enclosures, no backups 🤔 I would pickup a used storage server with a lot of drive bays from eBay. The external bays seem attractive but I’ve never heard good stories about performance or reliability.
You’ll be better off trying to get a proper network attached storage (NAS) rather than an enclosure. Either buy a pre-made one or make one out of parts. That way you can use the network speeds. Or you could get a usb3 pcie card, they are very cheap these days.
If you’re in it for the long haul buy a “cheap” used server off of eBay and upgrade it.
If you want something more inexpensive buy the cheapest case you can find with the most HD mounting points. Then get yourself a SAS controller from eBay and connect everything up.
I got an HP DL380 with 16 drive bays, and I basically just dump any old hard drives in it whenever I upgrade. I have 24TB in it, had it for years, and I’ve only ever lost one drive at a time, and I just shrink out the dead drive, and then toss another one in if I get a new one. “I’ll move your files to your new computer if I can keep the old one…” I even 3d-printed a couple of 2.5"-3.5" adapters to stuff old laptop drives in there. Caddies? Uhh… I think it was 120$ from the local electronics recycler. It’s old, it’s slow, it’s basically a giant samba share.
I have a couple of servers (all 2.5" drives) and a disk shelf (for the much easer to get large volume 3.5" drives) attached to one of them with an external sas PCIe card. I could push that to 300+TB if I had the cash
If I where you I would just buy a regular case that can fit a decent amount of HDDs like a fractal define 7 or one of its older versions and transplant your current computer into that with some new drives. 100tb is 5 20tb drives so you don’t need that many.
USB enclosures are not a great way to handle storage as USB tends to be unreliable.
For 100TB it’s worth looking into a dedicated storage server – there are tons of them available for cheap. labgopher makes it easy to track sales on ebay by price/storage/ram/whatever.
To summarize a few details, the PM1633A is a SAS3 (aka SAS 12 Gbps) SSD drive, which accepts an SFF-8482 plug. This SFF-8482 plug is the one named in the SAS3 standard for use on drives. You mention the LSI 9311 HBA, which does support SAS3 and has a pair of SFF-8643 receptacles, which is specified in the SAS3 standard for use on backplane aggregators. That is to say, when multiple drives are bundled up onto a single cable.
When used for SAS3, SFF-8643 supports up to four drives. And so you will find forward-breakout cables online that go from SFF-8643 to 4x SFF-8482.
The cable you mentioned – an SFF-8643 to SFF-8639 – is meant for U.2 drives. Because of the 4x PCIe lanes used for U.2, a single drive uses all the pins in an SFF-8643 plug, which is why this cable can only attach to a single drive. Because SFF-8639 is backwards compatible with SFF-8482, this could still be used for SAS3 drives, but it would waste the other three “lanes” in the cable.
With all that said, I would not recommend the cable you listed, and instead replace it with the aforementioned forward-breakout to 4x SFF-8482. This way, you can later buy three more SAS3 drives. I presume you’re not planning to ever use U.2 here.
Also, regarding the choice of HBA, was there a reason you chose the 9311? I have both the venerable 9300-8i and a newer 9305-16i. Both work great for me and support SAS3. It’s notable that power and heat is lower on the 9305. The 9300, 9305, and 9311 all have the same pair of SFF-8643 connectors.
Thank you for the much needed help. I have been clueless and trying to find a decent inexpensive solution.
QA: Q “was there a reason you chose the 9311?” A: no, only that it looked nice.
The 9300-8i is 1/4 the price so I will go with it.
I took a gamble on a cheap used drive purchase at $500 for local game installs. The drive arrives in 2 days. I couldn’t find out what connection type the drive had or power delivery needs
Here are the products I have selected based on your recommendation. Do these look like good choices?
The first link is an SFF-8087 to 4x SFF-8482. While this cable could technically support SAS3 speeds, the SFF-8087 connector was specified for SAS2, not SAS3. As a result, you won’t really find any HBAs that have an SFF-8087 connector and do SAS3 over it. This cable is incompatible with the 9300-8i from your second link. I would choose something more like this: www.amazon.com/dp/B01GPD5KFK . Also be advised that if your SSD isn’t recognized with this cable, the reviews mention that the 3.3v power pin – if you have one at all – might need to be disabled, to avoid PWDIS issues.
For the second link, that Inspur 9300-8i appears identical to the HBA I have, and it’s worked fine for me, although I only have SAS2 drives hooked up to it right now. The nice thing is that the listing advertises “IT mode”, which was important to me, because burning firmware to switch to IT mode is a sad experience.
EDIT: BTW, when you receive this drive, you should probably dump the SMART data to see how much lifetime is left on this SSD. This is an enterprise SSD, so it’s possible that it came from several years of use as a caching drive in a server somewhere. That could do a number to its remaining lifetime, but I would imagine its performance would fit well for your use-case.
An aside: one-to-many breakout cables have a forward and backward variety, and care must be taken to avoid buying the wrong one. This link explains the difference, which is still applicable to SAS3: …unraid.net/…/6723-sas-to-sata-cables-forward-or-…
Note that some combinations of backwards breakout cables simply don’t exist, so there might be only one version available for sale. Still, read the product descriptions carefully for which end is meant for the HBA and which end goes to drives or the backplane.
BTW OP: when you get this set up, please run some benchmarks and tell this community how it performs. I’ve had a free PM1633A sitting on my desk for 1.5 years, just because I haven’t gotten around to it. I’m keen to know how it behaves.
Sure, I would love too. Do you know what software to run in Windows to provide good results? I can temporarily attach it to my linux machine if necessary
I’m afraid I don’t have much experience with benchmarking in Windows. I think I’ve seen CrystalDiskMark mentioned in a few places, and it’s FOSS under MIT License.
Cool, I will give it a try. I may install it to my linux machine and access over samba. I need to test if game installs will work over network like this first
Some of it may be, but the fact that the LTS versions (20.04 & 22.04) are downloaded overwhelmingly more than the others seem to indicate it’s more intentional.
Yes, Ubuntu 20 isn’t EOL yet. A lot of those downloads are probably IT staff or developers that are running Ubuntu servers or developing on those versions.
ETA: We still have some RHEL 7 and clones at my day job
datahoarder
Oldest
This magazine is from a federated server and may be incomplete. Browse more on the original instance.