Oh, that’s right. Nice catch! I can probably repurpose a 256GB SSD I have, can’t I? Should be enough for OS + utilities
The plan is to get more down the road, this is a starter setup! 4TB are enough for all my data at the moment, and the second drive is for backup; I will add a third drive for redundancy and that should be enough as a starter
This may be a silly question, but why get a whole extra machine just to make 4tb available on the network? I have an old Linksys router with a USB port that allows you add USB storage to your network, that may be a good place to look if your needs are basic.
Additionally, it looks like you are going for a relatively powerful machine to be able to access a minimal amount of data at a slow speed. Have you considered just getting a couple of external hard drives and just hooking them up to a Raspberry Pi?
Going the Raspberry Pi route would cost $288-$333 depending on accessories. This could be a really good route depending on what all you want to do.
Another way to save more would be to just add the drives to an existing desktop, and just setup samba/nfs to run on it. You’d just need to leave the desktop on all the time, or turn it on when you need to access it elsewhere.
Also, for reference, I have a NAS with 40tb of storage that I use as a VPS host. I have several virtual machines running 24/7, including a Plex server. I stream many videos locally, and have many users that access my content regularly. From what I’ve learned over the years of running that machine, I know that as long as you are just directly streaming the files, and not trying to transcode them, you don’t need hardly any processing power. I don’t have a GPU in my server, and don’t see a need to put one in. If your needs are really as simple as your post suggests, the raspberry pi route would be the way to go. It would also allow you to dip your toes into running a NAS, and see where your original build was lacking, and give you a better idea for what you want your next evolution to look like.
It boils down to two things: inexperience (apparently 8GB of RAM might be enough? Just to name one issue wih my build haha) and I’ve already maxed out my main PC’s expansion slots.
I have a compact case, which is already housing two HDDs (a “landing” HDD, where I store and keep all the… Linux ISOs… until they reach a satisfactory ratio of upload, which has a 2y uptime and just recently encountered its first uncorrectable sector error, and a 2TB HDD where I keep my stuff, including the aforementioned… ISOs…, my GOG games and other media. This 2TB is backed up on a 2TB external drive, and it’s already full). Since this 2TB internal is full, I plan on moving some files to a different external HDD so I can stuff more things in it, but that leaves me with no backups for this stuff I want to move; this second external HDD is very small (650GB) so moving that stuff will make it full and I’ll have no chance to add onto that collection.
It’s a pretty unorganized situation and if I could get rid of all internal HDDs I could get rid of the HDD harness and fit a fan instead, also reduce noise.
In addition to this, I have yet to understand whether or not external drives can sustain “high” rates of writes (when I download GOG games, for example, it can easily add up to 100GB in a day) and reads (I usually seed… Linux ISOs… for tens to hundreds of GB a day). Of course these numbers arent’t for everyday, some days I download nothing for example, other days I don’t even turn on the PC.
powerful machine
And I even went for the cheapest parts I could get haha! Only way to make it less powerful is getting an Intel CPU, like the datahoarder wiki suggests, based on LGA 1150/1151, I can get one for like $20, but I can’t find used motherboards for a decent price, all around 130. That’s why I picked an AM4 platform: parts more widely available.
raspberry pi
Checked it out, unfortunately it’s out of stock in all the licensed retailers in my country. It does sound like a nice starting point, though, so I will keep an eye out for it to be restocked, but since it uses external drives I’m back to the previous question: are external drives sturdy enough to sustain the amount of data I write and read daily?
no GPU
Wait, how can you turn it on and configure eveything necessary on it without graphics? Is it all done remotely via a main machine?
Sorry for the wall of text! And thanks again for the help!
Okay, I feel like I have a better idea of what you’re looking for now.
Unless you’re using some sort of shingled drive, reads from the magnetic platter should be non-destructive, so reading files for seeding shouldn’t do much to the drives in terms of wear and tear.
External drives are fine for durability, and if you get them from a reputable company I’m sure you’ll be happy with them. Buying cheap, high capacity external drives and shucking them is a common tactic that some data hoarders will use to get the most bang for their buck. Remember that external HDDs are going to be subject to more stresses than a normal drive (people moving them while they’re on, etc), so it wouldn’t be in the best interest of the manufacturer to make them less durable than regular HDDs. You just have to keep in mind that HDDs are mechanical in nature. Anything with moving parts will wear down over time and eventually break. Typically you should expect to cycle through your drives every 4 years or so (I say this while never having cycled any of my drives and they are all working fine).
I see you also keep picking drives that have NAS in the name, just so you know, that’s mostly marketing bullshit. They name the drives with different use cases so they can charge more for them and get that little bit of extra profit. There’s very little that can be done to tune the drives firmware for those different use cases. So, just get the most capacity your can afford from the company you want to use, and don’t worry about it it has NAS in the name.
You’re going to want more storage capacity, so don’t lock yourself down with a tiny case like the Node 304. Get a case that can hold a lot of HDDs. I went with the Node 804, which has mounting spots for 8 HDDs and two 2.5" SSDs (more spots than that when you realize that SSDs have no moving parts, so it doesn’t matter where you put them). I ended buying a new bigger case within a year of building my NAS, so definitely leave yourself room to grow, it may cost more up front, but will save you money in the long term.
As far as the GPU goes, I initially got a Ryzen 3400G, which is an APU, so I just used the integrated graphics. Since then, (remember how I bought a new case within a year?) I have upgraded the CPU to a Ryzen 3800x without onboard graphics, and I am now using this motherboard, which has onboard graphics, as well as IPMI. The IPMI is amazing, and I highly recommend it; it’s allowed me to have the computer in a remote location far away from any monitors, and it only has Ethernet cables and a power cable connected.
As far as other options goes, you could also get a bigger case for the computer you already have, that would allow you to add more storage. You could also get a very large SATA SSD (I just saw an 8tb one for ~$340), and you could just shove that into your current case anywhere to hold you over until you save up to get a more dedicated build going.
I researched again for parts and, whichever way I go, it’s all very expensive.
The “cheaper” options are still expensive considered the lack of expandability: I can get, for example, two 3.5" 4TB drives for around 90-100 each, or, slightly better, two 2.5" 4TB drives for around 120-130 each; the 2.5" drives would allow me to get rid of the HDD cage in my PC and mount a fan in its place. Either way, 8TB (actually 4TB + mirrored backup) for 200-250, and I can’t expand it further.
Slightly more expensive: a one drive Synology NAS, for around 300 (including a 4TB HDD, bought separately); again, locked with no further scaling possible.
Then, a two drives Synology NAS, for 450-500 (including two 4TB drives, bought separately), no scaling. But it’s getting closer to my needs.
Finally, a four drives Synology NAS, definitely fitting for my needs, which is 500 not including drives; once I add storage, for example just two 8TB drives (to which I’d add another two down the road), I’m close to 800 (and this is by getting the cheapest 8TB drive I can find, nothing with “NAS” in the name haha).
Shucking isn’t really a financially good option, it looks like, since external drives are actually more expensive in my country.
tiny case
Well, the Node 304 can hold six drives. Realistically, that should be enough for me even if I decide to have two backups (so using two drives for storage and then the other four just to back up those two) and even if I get 8TB drives (at the moment I have just shy of 3TB of data and I think i can easily add another 2TB onto it, but further than that… not sure). Worst case scenario changing only the case is pretty painless, especially if I can sell the old one to buy the larger one.
gpu
Yeah, I know I can get an APU, issue is, with that MoBo I picked only PRO APUs support ECC RAM. For what I understand, ECC RAM, while not mandatory, is highly recommended. The data I have is not vital, but if I can avoid corruption and having to download it again, I’d rather do so. To be fair, I’m currently not using ECC RAM and I haven’t encountered data corruption in the past two or three years… so I’m a bit torn on this point now
your config
That’s a very expensive motherboard, wow! I can’t find it for less than 350! Definitely over budget for me at the moment
larger case for my current setup
This is definitely something I’ll consider, at least as a stopgap until I can build a proper NAS. This way would allow me to have many drives and I could also repurpose the ones I already have.
I think I’ll have to put this project on hold for now haha but thanks for the huge help, I’ll definitely watch that video in the meantime!
Thanks again!
edit: looks like I can shave off another 100 from the planned build, since I can get a CPU (Ryzen 5 2600) and a 550W PSU for free. Which is actually not bad. Remove one stick of RAM, since 8GB should be enough, and I can save another 30, bringing the total to around 500. Not that bad as a start! Actually, bring that back to 600, since it would make more sense to get 8TB drives since I’m already around 3TB. Still, saved money is money saved haha
I’m really happy you’re looking at all your options. Running a homelab/nas can be a lot of fun, but, you’re right, it’s expensive, but you can turn what you learn from it into potential career advancements, so it could pay for itself.
I think you could do better with your storage drives. Go to pcpartpicker and look at storage then sort by price/GB ascending. The very first entry is a 6tb drive for $57
edit: forgot to mention that I’m not USA-based, sorry! Wow, US prices are amazing. In my country that one is just a little less expensive, adjusted for capacity (the ones I picked are 4TB and 120, the Barracuda is 8TB and 205)! That’s a bummer haha but thanks for the suggestion! edit: let me check the 4TB though edit2: now that’s better, the 4TB is only 89 compared to 120 needed for the IronWolf
Sorry for the double post, I don’t see how to edit a post with Memmy, but at some point I’d like to use Jellyfin or Plex. Is that something that needs to be separate, or can it be combined with the rest of this?
Curcial and WD havea much higher rate on average across all their models.
The 800% is only because they had a single drive for a certain model, and it failed within 2 months. They have a lot of other Seagate models that are much older on average without any failures.
There’s stuff you can tweak of course if it don’t quite work for you. Worked fine on me tests.
If you notice a page missing, you should be able to just scroll back to it and then download again to get everything. The first script just keeps collecting pages till you refresh the site. Which also means you should refresh once you are done downloading, as it eats CPU for breakfast.
Oh and NEVER RUN ANY JAVASCRIPT CODE SOMEONE ON THE INTERNET TELLS YOU TO RUN
Well, I may be technologically semi-literate and I may have felt a bit dizzy when I saw actual code in your comment, but I sure as hell will find a way to put it to use, no matter the cost.
I think for me, it’s because it’s sort-of (though not!) self-archiving.
If a site has a reasonable amount of popularity and subscriptions, it would take that site going down, plus all the sites that communicated with it, for the data to be fully lost.
Not to mention that a lot of the sites are run by reasonably altruistic people, who are more likely to hand over than just shutter.
I’ve come across it once with a laptop power supply. I believe it correlated to a voltage issue of some sort. We just replaced it with a new one and the problems disappeared. It definitely wasn’t coil whine, but I didn’t believe the person complaining about it until I held it up to my head myself.
My monitor status LED makes a faint beeping sound when in standby.
Also the panel makes an audible whine when the content changes to a specific amount of white.
As another user already said: Probably low quality electronics.
The truth of academia is that it is extremely slow. there are less than 20 minds total on all of earth working on this idea, separately, in different countries. And these 20 people are in their 20’s, severely underpaid, don’t necessarily have all the resources they want, and science may not be their #1 life priority.
anyways:
reading and writing DNA is the main driver of evolution, and it does so because it is error prone (causing mutations). You can imagine this is bad if you want to preserve the integrity of the data.
DNA storage would be okay if you were to… say archive the entire internet for future generations, or geneology records, etc. things that do not need to be written and accessed quickly or often.
What I do is on the originator drive, I create new subdirectories and start categorizing items by content; like I'll put all the ebooks into one directory, and all the television into another. It just makes it easier for me to find things later if I can just head to the drive with all the television on it.
If there's a particular directory with a lot of content, I might create further divisions - maybe shows that are finished vs those who are still getting new episodes, or sitcoms vs drama, that kind of thing.
Then I make a list of how big each master directory is, and I start copying them over to the most appropriate-sized drive. I usually find that I can fit in one large directory, and a couple of smaller ones, and then the last drive gets all the leftovers. I also tape a post-it note to each drive saying something like "2022-23 television" or "science fiction audiobooks" or whatever.
I also create a new directory on the originating drive called something like ++COPIED and, once I've copied content to a new drive, I move the original directory to ++COPIED: I'll still have access if I need it, but I don't have to keep track of it any longer. Once everything is successfully copied over, I can just delete that one directory.
It's a manual process, yes, but it does make it easier for me to find stuff when I want to look at it again later.
sounds like your main limitation is attaching the drives – if you can attach them all to a single system (ex. a separate computer or a NAS case) then at least it becomes somewhat easier to access them all at once
I was thinking JBOD but Wikipedia points out the same issue you mention with RAID 0, failure of one drive can mess up the logical volume which leads to a whole host of new issues to deal with during recovery
Not that big of a deal when it is a backup. Raid is not a backup solution, it is a 24/7 uptime solution. If the main drive dies with JBOD, then you have the backup. If a backup drive fails, then you still have the main. Trick is to ensure any drive issues are dealt with immediately and no backup runs if there is a smart error or similar on any drive.
So having software that monitors drive health and email/notifies you is necessary.
Secondary benefit of JBOD is all drives in a pool are still readable separately.
datahoarder
Top
This magazine is from a federated server and may be incomplete. Browse more on the original instance.