datahoarder

This magazine is from a federated server and may be incomplete. Browse more on the original instance.

tetris11, in Whatever happened to DNA-based storage research?
@tetris11@lemmy.ml avatar

I remember there being a water based storage solution for music that was under development, though it was said to drain entire ecosystems by doing so. Sad, as it seemed promising.

Atemu, in What's an elegant way of automatically backing up the contents of a large drive to multiple smaller drives that add up to the capacity of the large drive? (on Linux)
@Atemu@lemmy.ml avatar

I don’t want to do any sort of RAID 0 or striping because the hard drives are old and I don’t want a single one of them failing to make the entire backup unrecoverable.

This will happen in any case unless you had enough capacity for redundancy.

What is in this 4TB drive? A Linux installation? A bunch of user data? Both? What kind of data?

The first step to this is to separate your concerns. If you had e.g. a 20GiB Linux install, 10GiB of loose home files, 1TiB of Movies, 500GiB of photos, 1TiB of games and 500GiB of Music for example, you could back each of those up separately onto separate drives.

Now, it’s likely that you’d still have more data of one category than what fits on your largest external drive (movies are a likely candidate).

For this purpose, I use git-annex.branchable.com. It’s a beast to get into and set up properly with plenty of footguns attached but it was designed to solve issues like this elegantly.
One of the most important things it does is separate file content from file metadata; making metadata available in all locations (“repos”) while data can be present in only a subset, thereby achieving distributed storage. I.e. you could have 4TiB of file contents distributed over a bunch of 500GiB drives but in each one of those repos you’d have the full file tree available (metadata of all files + content of present files) allowing you to manage your files in any place without having all the contents present (or even any). It’s quite magical.

Once configured properly, you can simply attach a drive, clone the git repo onto it and then run a git annex sync --content and it’ll fill that drive up with as much content as it can or until each “file”'s numcopies or other configured constraints are reached.

Lemmchen, (edited ) in A dozen or two TB of storage for media on the cheap?

Unfortunately I’m not familiar with the US market, but in Europe we have sites like Geizhals (“Skinflint” in the UK) that are excellent at listing electronics, so you could source them for cheap: internal drives, external drives

If whatever you end up doesn’t have enough 3.5" slots, you can get these things on ebay to use them as a hacked together external housing:
https://feddit.de/pictrs/image/d34e7bd0-5f78-4e3c-a5b7-56430e8482fb.png
You’d need to get the SATA cables out of your case though.

As for what drives to use: If you don’t need redundancy/parity, then a single 12 (14, 16, 18, whatever you need) TB SATA drive will probably beat everything else pricewise. I’d say that leaves you with roughly $300 for the system itself, if you need to buy a new one.

but doesn’t have any way of adding a bunch of drives

Well, you only really need one or two drives. Are you sure it doesn’t offer any SATA connections?
What about PCIe? You could use a cheap HBA card then.

GlitzyArmrest, (edited ) in A dozen or two TB of storage for media on the cheap?
@GlitzyArmrest@lemmy.world avatar

For drives, Shuckstop has a table of current shuckable drive prices. Shucking is usually the cheapest way to get new drives, you just have to get them out of their external case (or in your case, leave them in and plug them into your pi, ideally with some sort of fan). There’s also ServerPartDeals for refurbished enterprise drives.

7Sea_Sailor, in A dozen or two TB of storage for media on the cheap?

May I ask: are you sure you need a media center with transcoding? Because it may be totally sufficient for you to access files through a file explorer and play them with VLC/mpv or whatever else. Having a media center is only really useful if you need external access to your media. I set all that stuff up once, then realized i never watch shows/movies on the go. And if I do, i know beforehand and can copy the raw files to the device i plan to watch on.

bdonvr,

On my PC direct play is possible 99% of the time.

I also watch a lot on my Apple TV which requires transcoding for some codecs.

But the big thing is my SO who does watch on the go often.

7heo, in Mechanical or SSD for offsite storage that’s updated at least yearly?
@7heo@lemmy.ml avatar

It depends how critical the data is.

SSDs and flash are great for speed, but as other comments have mentioned, they can eventually lose data more than other mediums. However this reportedly isn’t likely to happen over the normal rated timespan of the devices, unless there is a critical defect.

Magnetic storage will likely last longer, and as it is a much older tech, is less likely to have firmware bugs and other problematic surprises. Plus, as you can see on diskprices.com, the cheapest medium per TB remains magnetic storage.

Then there are tapes. The drives sure cost a hefty sum, but if you have loads of data to backup, this is likely the cheapest option.

Finally, optical. Optical is great in the sense that is is physically a ROM, so data cannot get compromised by mishandling or other staff mistakes; but it still can have issues with the reflective layers peeling away from disks.

So, in the end, I would personally not recommend using SSDs for data backups, out of precaution. Sure, SSDs will likely retain all data just fine for years to come, but I want to be able to store data for as long as possible, with the peace of mind that only magnetic storage will afford me. Plus, if your data is worth backing up, it is worth whatever extra price or effort you will have to do with.

As for the other options, well, they all have their use case, but I don’t see much advantage for them in the general use cases. Just make backup copies of your data on magnetic drives, in a few physically different locations, with proper access control.

M500,

Thanks! I don’t mind spending some money, what I was trying to communicate is that I don’t want to spend $1000 on some solution.

I think I’ll get a mechanical drive and a few 256gb usb drive or maybe an ssd and have a few off site copies in case one medium fails.

I’ve never had an ssd go bad, but I’ve had external mechanical drives fail over time, so I’ve been hesitant to trust them.

7heo, (edited )
@7heo@lemmy.ml avatar

I had SSDs go bad, and mechanical hard drives too. The major plus for me is that with HDDs, it is somewhat predictable, while with SSDs it has always been sudden (in my experience, at least).

However, there are more parameters to consider. The storage temperature, the relative humidity, to backup frequency, etc. In the end, if you want a 100% time proof solution without caring for the costs, engraving a crystal, storing it underground in a lead-lined container, is probably the surest way to go. Everything else is a compromise.

comador, in Mechanical or SSD for offsite storage that’s updated at least yearly?
@comador@lemmy.world avatar

As NateNate60 mentioned: USB Flash. I second this as a cost effective alternative to anything else. Corsair Survivor, Sandisk Exteme Pro and Kingston DataTraveler Flash drives to 256GB are cheaper than anything else and just as reliable.

Should you want to go the SSD route, the Corsair MX500 drives purchased with any external esata or usb chassis is the most reliable option for the price.

M500,

Thanks! I have a spare enclosure and an ssd is about the same cost as a usb drive is similar cost.

Nogami, in Bought a couple of Exos 7E10 8TB drives, warranty status says "initially sold as part of a larger system, contact seller". It means I can't take advantage of their five years warranty, doesn't it?

I’ve been buying used 8TB HGST SAS drives on eBay for $50USD each, so far no issues but I really don’t care if they die. Basically disposable at that price point.

mumei,
@mumei@lemmy.world avatar

That’s fair, I paid three times as much though so I do care a bit more haha

will_a113, in Bought a couple of Exos 7E10 8TB drives, warranty status says "initially sold as part of a larger system, contact seller". It means I can't take advantage of their five years warranty, doesn't it?

Hook them up, run a SMART test and see what their powered-on hours and error-rates look like. If it’s not a significant fraction of the MTBF, chances are you’ll be fine – assuming you’re using them in a RAID or ZFS array where a drive loss is not necessarily catastrophic.

mumei,
@mumei@lemmy.world avatar

Sorry for late reply, app kept saying “this account is being verified” and I coulnd’t comment or anything else.

Anyways, I kept them. One drive loss won’t be catastrophic (got two mirrored and also spare backup), so I decided to go for it!

breakingcups, in Lost Doctor Who episodes found – but owner is reluctant to hand them to BBC

But, I thought an amnesty was already in place?

wolfshadowheart, in Large external HDD noise levels
@wolfshadowheart@kbin.social avatar

I can't speak to what you're looking for specifically, however I can give you an idea of the level and type of noise you could expect.

With a local server I built with 3 standard 8TB HDDs, the "noisiest" part of the whole computer is the fans itself - it's very quiet compared to say, a fanned air purifier on full or maybe even half power. I've never actually heard my hard drives from any of the computers I've built in the last 6-8 years, and I'm usually right by them. If I listen very closely I may hear some ticking/spinning if they happen to be ramped up.

So under load, there may be a chance of hearing some of the spinning disks in a quiet room, however I haven't actively heard an HDD since 2012 - though the size you're going for could be different as you mentioned. I can't speak for the 18tb sizes. As for the smaller external HDD's, I have 3 6TB ones running in tandem and it's pretty much the same, I've heard them ramp up slightly but it's nearly silent even under load it's more of a gentle hum. If there were any noise sources like music it would be unnoticeable. But again, the size may play a factor.

You mentioned it's right by your bed but would your active hours align with the backups? I.e. if you run them overnight from 2a to 8a, you'd already be asleep for the backup. If it's noise that prevents you from sleeping that may help.

nossaquesapao, in That many people need old Ubuntu installations?

I once downloaded a really old (like 10 years old) ubuntu iso, because I had an app in deb format made for that version, that needed older libraries to run. Perhaps, there were other ways to run it, but running the older iso in a vm worked fine.

GnomeComedy, in That many people need old Ubuntu installations?

ITT: speculation by people that clearly don’t use/understand Ubuntu.

ms264556, (edited ) in That many people need old Ubuntu installations?

I occasionally have to download and run old versions in a VM to build poorly supported software.

E.g. step 1 of the build instructions here

Install the following packages in an ubuntu - 14.04.6 LTS machine

BlueEther, in Need some advice to increase my storage capacity
@BlueEther@no.lastname.nz avatar

I have a couple of servers (all 2.5" drives) and a disk shelf (for the much easer to get large volume 3.5" drives) attached to one of them with an external sas PCIe card. I could push that to 300+TB if I had the cash

  • All
  • Subscribed
  • Moderated
  • Favorites
  • datahoarder@lemmy.ml
  • localhost
  • All magazines
  • Loading…
    Loading the web debug toolbar…
    Attempt #