I wish there was a good way to store a backup of my media. I recently suffered a terrible hard drive issue. I lost a terabyte of media. Fortunately, the pirate ship has saved me and has me rethinking some of my backup methodologies.

Outside of periodically backing up onto an external hard drive, I haven't been able to find a reasonably priced online backup solution that isn't going to fuck me when I have to pull data out. Egress fees are killer.

[–] WhatAmLemmy@lemmy.world 0 points 5 months ago (1 children)

All you need is to use ZFS or BTRFS locally to prevent master version bitrot and provide failover/redundancy, manually sync that to a separate "offline" HDD periodically, then setup a simple pi with tailscale + HDD at a family member or friends house, and rclone all your data to it (encrypted) as a cron job every night or week. This performs the function of a cloud provider (offsite backup); alternately, just manually sync the offline HDD once a month.

With this approach you're covered for accidental deletion, hard drive failures, bitrot, ransomware, and fire; possibly many natural disasters, depending how far away the offsite is.

Then you can just keep your most important data E2E encrypted in 1 or 2 cloud storage providers.

[–] far_university1990@feddit.de 0 points 5 months ago (1 children)

Note: zfs/btrfs cannot repair bitrot without redundancy, only detect it. But if redudancy, is repaired automatically (self-healing).

Raid 1/5/6 cannot repair, only detect bitrot, cannot decide which copy good.

[–] Nithanim@programming.dev 0 points 5 months ago* (last edited 5 months ago) (1 children)

I have never seen an implementation of e.g. a mirror that gives up on disagreements of both disks. Repairing/redundnancy is what raid is there for.

Edit: maybe old hardware raid does not check?

[–] far_university1990@feddit.de 0 points 5 months ago

No, raid for disk failure: https://en.wikipedia.org/wiki/RAID

Raid has no checksum, see argument here: https://superuser.com/questions/769104/is-raid1-or-similar-needed-for-btrfs-zfs-to-protect-against-bit-rot

But look like some raid implementation (and lot of drive) have error correction now: https://en.wikipedia.org/wiki/Error_detection_and_correction#Data_storage https://serverfault.com/questions/77710/is-bit-rot-on-hard-drives-a-real-problem-what-can-be-done-about-it

[–] SnotFlickerman@lemmy.blahaj.zone 0 points 5 months ago (1 children)

It's a lot of up front cost, but a NAS that is RAIDed with parity puts you in a pretty safe spot.

The short explanation is you have at minimum three drives, and you "stripe and span" them. This is a setup called RAID 5 where, if any one of the three drives fails, it can be replaced with a similar-sized drive and the "parity bits" from the other two drives can rebuild the data on the third drive. Yes, this means you only have the effective space of only two out of the three drives. So say you had 3x4TB drives, you'd have a total of 8TB to work with, and one drive is the "parity" drive (although this is actually split among the drives, so if any one fails, it can be revived by the other two).

However, in practice, the space lost is worth it for redundancy. It does mean an up-front cost in buying drives, a NAS enclosure (or using something like TrueNAS plus off the shelf parts to build your own), and includes the cost of physical maintenance and support (a Uninterruptible Power Supply to keep the hardware safe, for instance, on top of eventual maintenance of physical parts).

The offers the cloud solutions seem cheap up-front, but they don't buy you as much time as the one-time up-front cost of building your own NAS and maintaining it. I understand why people choose the cloud solutions, it's much easier. But if you're dedicated to this lifestyle, it's something worth looking into, at the very least.

[–] far_university1990@feddit.de 0 points 5 months ago

raid cannot repair corruption/bitrot. zfs/btrfs with redundancy can.

load more comments (4 replies)