datahoarder

This magazine is from a federated server and may be incomplete. Browse more on the original instance.

bela, in Google Books - colour images

I just spent a bit too much time making this (it was fun), so don’t even tell me if you’re not going to use it.

You can open up a desired book’s page, start this first script in the console, and then scroll through the book:


<span style="color:#323232;">let imgs = new Set();
</span><span style="color:#323232;">
</span><span style="color:#323232;">function cheese() {    
</span><span style="color:#323232;">  for(let img of document.getElementsByTagName("img")) {
</span><span style="color:#323232;">    if(img.parentElement.parentElement.className == "pageImageDisplay") imgs.add(img.attributes["src"].value);
</span><span style="color:#323232;">  }
</span><span style="color:#323232;">}
</span><span style="color:#323232;">
</span><span style="color:#323232;">setInterval(cheese, 5);
</span>

And once you’re done you may run this script to download each image:


<span style="color:#323232;">function toDataURL(url) {
</span><span style="color:#323232;">  return fetch(url).then((response) => {
</span><span style="color:#323232;">    return response.blob();
</span><span style="color:#323232;">  }).then(blob => {
</span><span style="color:#323232;">    return URL.createObjectURL(blob);
</span><span style="color:#323232;">  });
</span><span style="color:#323232;">}
</span><span style="color:#323232;">
</span><span style="color:#323232;">async function asd() {
</span><span style="color:#323232;">  for(let img of imgs) {
</span><span style="color:#323232;">    const a = document.createElement("a");
</span><span style="color:#323232;">    a.href = await toDataURL(img);
</span><span style="color:#323232;">    let name;
</span><span style="color:#323232;">    for(let thing of img.split("&amp;")) {
</span><span style="color:#323232;">      if(thing.startsWith("pg=")) {
</span><span style="color:#323232;">        name = thing.split("=")[1];
</span><span style="color:#323232;">        console.log(name);
</span><span style="color:#323232;">        break;
</span><span style="color:#323232;">      }
</span><span style="color:#323232;">    }
</span><span style="color:#323232;">    a.download = name;
</span><span style="color:#323232;">    document.body.appendChild(a);
</span><span style="color:#323232;">    a.click();
</span><span style="color:#323232;">    document.body.removeChild(a);
</span><span style="color:#323232;">  }
</span><span style="color:#323232;">}
</span><span style="color:#323232;">
</span><span style="color:#323232;">asd();
</span>

Alternatively you may simply run something like this to get the links:


<span style="color:#323232;">for(let img of imgs) {
</span><span style="color:#323232;">	console.log(img)
</span><span style="color:#323232;">}
</span>

There’s stuff you can tweak of course if it don’t quite work for you. Worked fine on me tests.

If you notice a page missing, you should be able to just scroll back to it and then download again to get everything. The first script just keeps collecting pages till you refresh the site. Which also means you should refresh once you are done downloading, as it eats CPU for breakfast.

Oh and NEVER RUN ANY JAVASCRIPT CODE SOMEONE ON THE INTERNET TELLS YOU TO RUN

antonim,

Well, I may be technologically semi-literate and I may have felt a bit dizzy when I saw actual code in your comment, but I sure as hell will find a way to put it to use, no matter the cost.

You’re terrific, man. No idea what else to say.

GreatAlbatross, in Thesaurizant, ergo sum
@GreatAlbatross@feddit.uk avatar

I think for me, it’s because it’s sort-of (though not!) self-archiving.

If a site has a reasonable amount of popularity and subscriptions, it would take that site going down, plus all the sites that communicated with it, for the data to be fully lost.

Not to mention that a lot of the sites are run by reasonably altruistic people, who are more likely to hand over than just shutter.

person,

A fair point!

Helix, in Mechanical or SSD for offsite storage that’s updated at least yearly?

SSDs lose charge over time. Within a year, bitflips might occur. I recommend going with an HDD with a shockproof case.

This ineo t2580 looks good but I can’t seem to find the maximum capacity. Try not to use HDDs over 12TB and you should be fine.

cmnybo,

If you avoid QLC drives, keep it at a reasonable temperature, and don’t use a drive that’s had a large number of write cycles, it will be fine for a year.

I would definitely read everything on the disk yearly so the controller can detect any weak blocks and rewrite them though. A good way to read everything would be to take a checksum. You can then compare that to the previous checksum to make sure the files haven’t changed too.

Mechanical drives have issues with long term storage as well. When hard drives get older, sometimes they will just refuse to spin up after sitting for a long time.

M500,

Thanks, I was thinking about a 1tb my passport from western digital.

I live in a third world country so my options are a bit limited.

I’m not worried about her dropping the drive, I just don’t want to ask her to lug around a 3.5” drive.

Are mechanical drive resistant to bit flips?

synapse1278, in Seagate Expansion external HDD power supply making a very faint alarm sound
@synapse1278@lemmy.world avatar

You say the sound comes from the power supply and the HDD is not plugged into the computer. My diagnistic: the power-supply makes a noise when it operates at very low load (almost 0mA of current), it is probaly making the cyclic noise because of some blinking LED or another very small variation of the loaf somewhere. This is a very common symptom of cheap power-supplies, but it doesn’t necessarly mean it isn’t working normally, just an annoyance.

otp,

It’s weird – the sound happens when the power supply is plugged into the outlet and the HDD, but it turns off when the HDD is plugged into the PC.

I could look for a blinking LED, but it really does sound like a noise maker rather than something electrical. But I’m not an expert.

synapse1278,
@synapse1278@lemmy.world avatar

I suppose, when you plug the HDD to the computer, it spins up and start drawing more current from the power-supply and the noise disapears. This is because the PSU has a buck-converter, the switching frequency increase proportionally with the current drawn by the load. When the current it almost 0mA, the switching frequency can be audible (electromagnetic forces can make some components vibrate, e.g: coils). When the current is nominal, the frequency will be ultrasound and you won’t hear it. I have observed this with many electronic devices. If you are worried, you can try another power-supply, after checking it has the same voltage and polarity on the plug, and can deliver at least as much current as the original one.

otp,

Thank you for the reassurance. I guess I’ll just either let it be noisy or leave it fully unplugged when not in use.

First world problems… having to choose between barely audible noise, or having to plug TWO things in to access my 14TB, lmao

moosqueak, in Seagate Expansion external HDD power supply making a very faint alarm sound

I’ve come across it once with a laptop power supply. I believe it correlated to a voltage issue of some sort. We just replaced it with a new one and the problems disappeared. It definitely wasn’t coil whine, but I didn’t believe the person complaining about it until I held it up to my head myself.

otp,

Was it like a little tiny weewoo alarm? Because that’s all I can describe it as, Lmao

ArbiterXero, in Seagate Expansion external HDD power supply making a very faint alarm sound

It’s not an intentional noise, it’ll be a shitty bridge rectifier.

It’s just a cheap shitty power supply.

otp,

What’s a bridge rectifier?

The power supply is “universal” (comes with different attachments for different regions’ outlets), so I could see it doing something weird like that.

But it only happens when it’s plugged into the outlet, but the HDD is NOT plugged into the computer.

Could it be expecting some extra power from the PC or something?

ArbiterXero,

No, it’s nothing to worry about, it’ll be just a handful of super cheap parts in the power supply. Essentially when the power supply converts ac to dc, it has a bunch of standard parts, and if you cheap out on them, sometimes they make high pitched noises. The noises can vary in pitch too.

SheeEttin, in Seagate Expansion external HDD power supply making a very faint alarm sound

Can you record the sound? I’ve never heard of a power supply having any kind of noisemaker. It’s probably just electrical interference or coil whine or something, where the waveform happens to produce that periodic sound. (You might even look around and find a nearby device changing its power draw with the same periodicity.)

otp,

Yeah, interesting. I’ll see if I can record it tomorrow!

otp,

Haven’t been able to record it yet, but the sound only happens when the power supply is plugged into the HDD and the outlet (not one or the other), and the HDD is NOT plugged into the computer.

Plugging the HDD into the computer stops the sound. I was unplugging it from the PC to keep it offline when I didn’t need it. (After safely ejecting and waiting etc. of course)

I guess I’ll just need to unplug the power when I’m not using it, too.

scottmeme, in Seagate Expansion external HDD power supply making a very faint alarm sound

Check the SMART data with crystaldiskinfo

otp,

Will do, thanks. Did lots of thorough checks when I got it, so I didn’t expect there to be any issues now.

hazeebabee, in Whatever happened to DNA-based storage research?

Im most excited about the potential for crystal based storage. Right now there is work being done to etch silica glass internally, allowing for incredibly long term preservation and durability. It can even be rewritten, though the tech is definetly best for achival purposes and is being pursued primarily by movie companies wanting high quality storage.

Heres a video discussing it more in deapth: m.youtube.com/watch?v=6CzHsibqpIs

DNA also sounds interesting, though it doesnt seem like a good way of preserving data long term. DNA is very fragile, and seems like an odd route to take for long term archiving.

TCB13,
@TCB13@lemmy.world avatar

DNA also sounds interesting, though it doesnt seem like a good way of preserving data long term. DNA is very fragile, and seems like an odd route to take for long term archiving.

Yeah the 5D quartz disk is very cool.

Anyways if you think about storage density DNA isn’t that “odd”. With DNA you can store dozens of copies of the data and parity checks in a very small space so even if some gets corrupted you can still get it. I get that organic stuff has its limits but the density is just mind blowing.

hazeebabee,

Density is defintly amazing in DNA, its just so fragile. Even our own bodies have a constant degridation of our DNA… I wonder if they could take that concept and make something sturdier by using slightly different molecules to make up the chains.

Maybe shorter chains with stronger cross bonding & a gentle method of reading the chain could also help?

Its definetly an interesting route & itll be cool to see what happens with it over the next 10-15 years.

Vex_Detrause, in Help me acquire discontinued sound effect libraries please

Why? It sounds like you need it for a project. Why not get it from the source? Like from source audio.

Shady_Shiroe,
@Shady_Shiroe@lemmy.world avatar

OP said that a lot of those are not on source audio

Shadow, in How to datahoard Lemmy
@Shadow@lemmy.ca avatar

Run your own instance, subscribe to every community, then backup the db.

ImJustInTimeForThis,

My Canadian king 😍

Deckweiss, (edited )

As far as I have searched, I didn’t find a way to find “every” instance. Most of the popular ones - yes, but far from all.

Also I did not know that each instance copies all pf the data of all other instances into their own db.

But thanks, I will try it out next year.

andrew,
@andrew@lemmy.stuart.fun avatar

There’s no authoritative list of instances since federation isn’t required, but tools like lemmyverse.net will give you a solid list of the ones discoverable from the most well known federations.

person, in scraped media links from instagram and threads

What is your goal with this?

I know, stupid question to a datahoarder. My point is that archiving all of instagram or threads would be impossible even for ArchiveTeam, much less a single person. Are these random posts, or ones you care about?

cerement, in What's an elegant way of automatically backing up the contents of a large drive to multiple smaller drives that add up to the capacity of the large drive? (on Linux)
@cerement@slrpnk.net avatar

sounds like your main limitation is attaching the drives – if you can attach them all to a single system (ex. a separate computer or a NAS case) then at least it becomes somewhat easier to access them all at once

I was thinking JBOD but Wikipedia points out the same issue you mention with RAID 0, failure of one drive can mess up the logical volume which leads to a whole host of new issues to deal with during recovery

ColdCreasent, (edited )

Not that big of a deal when it is a backup. Raid is not a backup solution, it is a 24/7 uptime solution. If the main drive dies with JBOD, then you have the backup. If a backup drive fails, then you still have the main. Trick is to ensure any drive issues are dealt with immediately and no backup runs if there is a smart error or similar on any drive.

So having software that monitors drive health and email/notifies you is necessary.

Secondary benefit of JBOD is all drives in a pool are still readable separately.

bela, (edited ) in How do I archive a podcast I've already downloaded in Playapod (iOS) that has since gone offline?

According to this you can copy stuff onto it. I assume you’ve tried to see if you can copy off? If not, then there is probably no way to extract the audio, short of recording it as it plays.

I was only able to find a couple episodes online. One on someone’s google drive and a few in the wayback machine, all from this thread

edit: if playapod has an option to store files on an sd card, that’s usually stored unencrypted. (on android anyway…)

edit: wait iphones don’t even have sd card slots, right? lol

bela, (edited )

Actually there might be a whole bunch more in the wayback machine. (though still not that many) here

Just sort by mime type and check out audio/* for items of interest

yo_scottie_oh, (edited )

This helped me, thank you so much! I filtered by .mp3 and I’m using DownThemAll to download all the episodes in batches.

ETA at first, all the downloads failed because I was attempting to download multiple files simultaneously. After configuring DownThemAll to allow only one simultaneous download, it’s been smooth sailing.

SplicedBrainwrap, in How do I archive a podcast I've already downloaded in Playapod (iOS) that has since gone offline?

The Playapod website mentions a web player, does the web player happen to sync with your app on the phone? If so maybe you can extract it through the web player on a desktop.

yo_scottie_oh,

Thanks for the idea. I figured out how to get what I wanted and edited the solution into my post. Thanks again!

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • datahoarder@lemmy.ml
  • localhost
  • All magazines
    •
    Loading…
    Loading the web debug toolbar…
    Attempt #