Storage situation advice re physical/cloud storage solutions

[u/pocketobservations]

Hi all, I've done a lot of research and am still unsure about what direction to take for my data backup system and was wondering if you had any word of wisdom based on your diverse experiences.

I'm a beginner amateur photographer. I've been accumulating a many more photos over the last year since practicing more frequently and consistently. At the moment I'm actually trying to consolidate all my photos from over the years (from phones, previous cameras) and securely back them up with a 3-2-1 system.

After gathering all my photos I've reached over 1T. It's going to grow faster as I'm getting more into photography, though I don't plan on becoming that serious.

I don't own an SSD yet, but I really think I need at least one. But oh my it is expensive. Currently, all my photos are in:

1. Two HDDs (1T + an older 300GB one), one from which I edit, but need to switch to SSD.

   1. Lightroom Cloud : I have a subscription to Adobe Creative through my work that I'm using as a back up for now but need to find a personal cloud solution.

I also have access to a Google Workspace Business Standard subscription (this is my own personal one), and an Amazon Prime subscription, which I could use for storage.

Ive also looked into NAS, Blackblaze as physical/ cloud options, but it's a bit overwhelming and I don't know if it's worth the investment for my situation.

Considering that I only have 1T at this stage and don't see myself becoming that serious with photography, would it be a good option to get one SSD + one other HDD, and back up everything to my Amazon and/or Google Workspace? Or should I be future proof and get a NAS and other more secured cloud solutions?

[u/qtx]

Just remember that 90% of people do not need a NAS. A simple DAS (with Backblaze) would suffice for most people. It's a lot cheaper and you aren't beholden to some OEM deciding they don't want to support your NAS anymore (which happens quite a lot).

It's best to buy second hand server drives, they're a lot cheaper and most are recertified.

Go for the largest size you can afford, don't buy 1/2TB drives. Storage will run out fast the more photos you take. An SSD should only be used for editing, not long term storage. SSDs will stop working without any warning, an HDD will give you plenty of warning before it stops working.

[u/Sudden_Welcome_1026 → u/qtx]

aren't beholden to some OEM deciding they don't want to support your NAS anymore (which happens quite a lot).

Hence why a DIY NAS built on UNRAID or TrueNAS is a great alternative!.

[u/pocketobservations → u/Sudden_Welcome_1026]

I'm not a tech expert at all and don't understand this but am interested. Do you mind explaining what a NAS built on UNRAID or TrueNAS is, and whether it's accessible to DYI for a non tech savy person like me?

[u/Sudden_Welcome_1026 → u/pocketobservations]

So a NAS is just a computer with a NAS OS and software running on it. UNRAID and TrueNAS are the software that you can buy/download and install to run yourself on your own computer. This differs from the appliances Ugreen/Synology/Qnap etc offer which are what can go end of life and no longer be supported. Because UNRAID and TrueNAS are just software, you can continue updating and patching them as you go. Technically either could result in end of life, but likely you will still have stable running software and hardware that can be replaced over time. If you self claim to not be tech savvy, this requires a heavier DIY approach but are by no means hard to do. 

For your use case, an external hard drive and cloud backup are probably all you need right now. NAS is really about making files accessible across the network and remotely. For instance, I make it so my wife can access all of our family photos from her phone and laptop and we can access our photos from outside the home too. These are the reasons a NAS exists. Nut purely as a storage solution. For your use case, a DAS in RAID 1 might be the way to go with 2x 4TB drives. That would give you good runway to grow with 4TB of usable storage and they just back the whole thing up to Amazon Prime Photos which is free. 

Now. If you want to be a little extra careful, get a second external drive and occasionally mirror your main external drive to it. This is your insurance in case your main drives go bad and you can’t recover data from it. Your multiple terabytes of photos are backed up to Amazon prime but it’s gonna take a long time and be a complete pain in the ass to recover that data from the cloud. If you have a offsite backup of your data that you maintain say every 4 to 6 months then you can recover most of your data there and only need to reach for the cloud to recover the delta from the last synchronization. 

This is basically the solution I came up with, and it works really well for me. Because I’m a nerd and I work in tech I wrote up the whole solution brief here if you want to read. It’s way more than you need to consider, but you might see some ideas that are useful. 

https://docs.google.com/document/d/1kopMp7tLQlT4c9tlnhvMQmISGIB20b-Ze7SxRuj4gVU/edit?usp=drivesdk

[u/pocketobservations → u/Sudden_Welcome_1026]

Woahhh this is amazing. Thank you so much. I'm a nerd too but def not in tech, so I appreciate the Google doc and detailed explanation!

[u/kag0]

Grab a USB 4 external drive array, backblaze desktop, done. That gives you 3-2-1 and you can buy new drives for the array as needed. 

But honestly even that is overkill for you. You could just hook mountainduck up to backblaze b2 and be good to go (although I'll admit that does grey my hair a bit when I do it).

[u/pocketobservations → u/kag0]

Thank you. I'm going to look into this. Why does this set up greys your hair ? (I know nothing about it!).

[u/kag0 → u/pocketobservations]

Only full copy is remote so you're somewhat at the mercy of the software (mountainduck) to make sure things behave. 

It's not hard to verify that everything got uploaded after a shoot. But simply having a full local copy would be nice if you can rig the drives.

[u/pocketobservations → u/kag0]

Thank you. I do have a bunch of follow-up questions. I hope it's okay - I'm a bit lost in the technical jargon (as a non tech person).

I see all comments include backblaze, clearly I need this. I was trying to maximise my access to Amazon Photos and Google Workspace, is the reason everyone recommends backblaze related to security and long term cost of growing storage?

I see you and other mentioning different things: backblaze desktop/ backblaze personal/ backblazeB2. I need look into it, but to start, are these different names for the same service or are these different services that backblaze offers?

When I google USB 4 external drive array, it does look a lot more affordable/ manageable than NAS. I see there are quite a few brands. Do you have any recommendations for brands to stick with or even characteristics the product must have that are non negotiable ?

I was also wondering about the SSD, is something you recommend to have as a portable drive in addition to the Drive array + backblaze?

Finally for the 321 system, there's one thing I don't understand with the USB 4 drive array + backblaze set up. Where's the 3rd copy?

Thank you. Really appreciate your help.

[u/kag0 → u/pocketobservations]

By all means

Backblaze has good prices, good products, and by most accounts just a great company. Their engineering blog is a great read and (as an engineer) seeing the engineering that goes into their service is confidence inspiring.

Backblaze has two services. Personal/desktop is a fixed price monthly subscription to back up everything on your local drives (removable drives included, NAS not included). You can't retrieve individual files, only full recovery. B2 is an object storage product where you can upload and download individual files. Pricing is usage based. It has a web UI but it's not meant to be used like google drive/dropbox, you have some other software interacting with the B2 API.

I haven't shopped recently for removable storage. Last time I looked (pre-AI price boom) my preference was to buy a drive and enclosure separately. Depends a lot if you want to use HDDs or SSDs.

If you have a USB4 SSD, that's plenty fast to use directly for editing. If you have USB HDD then you'll want to either copy things to your computer's local SSD to work on them, or use something like Lightroom smart copies so that you only need to fetch from the HDD when needed.

321 here means one copy on your computer, one copy on the removable drive, one copy backed up to Backblaze.
It could also look like one copy on your SD card, one copy on the removable drive, one copy backed up in Backblaze.
Caveats there:

• Your computer likely has less storage than your HDD, so it might be a subset of your data ("working copy"). That's fine for personal use like this.
• Files on SD cards will get corrupted if the card just sits in a drawer for a few years. This is also fine. I'm delivering something from a shoot within a year so if some backup goof goes undetected for five years I'm unlikely to care.
• SD card prices are through the roof thanks to AI, it's probably not reasonable to always be buying new cards right now.

also https://www.backblaze.com/blog/the-3-2-1-backup-strategy

Some parting shots at 321: the rationale for having two separate local copies of the data is so that if one goes down you can quickly recover using the other and operations aren't interrupted. For personal use - and even many professional use cases - it's not necessary to have rapidly resume operations. In fact unless you've designed around this use case, it's unlikely you'll be able to operate even with two local copies (what happens when the drive in your computer dies and you have a NAS? Your OS install is toast and you likely need to buy a new drive anyway. How soon you can get those files downloaded off your NAS is the least of your concerns). For that reason 2 copies of your data, 1 local, 1 offsite is likely appropriate for most people. Hence the setups I outlined above are more like 2.5-1.5-1 and more than adequate.

[u/flabmeister]

I have the following if it helps:

1. Working system. Mac Mini M4 + OWC M2 Express with 1TB W Black SSD. Stores the current month’s working files.

2. Home storage system. OWC Thunderbay 8 with 8 x Toshiba Enterprise 8TB drives in RAID 1+0. Stores all files from the day’s newly shot RAW files to my first ever shoots many years ago.

3. Backblaze off-site cloud backup of entire system.

[u/pocketobservations → u/flabmeister]

Thank you. I'll have a look into this. What backblaze plan do you have? Are you also an amateur photographer? How much data do you have?

[u/flabmeister → u/pocketobservations]

I have the regular Backblaze computer backup. It’s unlimited so by far the cheapest cloud backup service I’m aware of at around $9 per month. I’m a professional photographer and have around 23TB of data.
I’ll just add this at you may find it useful. For my business email I use Zoho. Very reasonable pricing and far better than Gmail that I used to use. They also offer WorkDrive which is like Dropbox. I use this to share files with clients. With an email account this works out pennies for 35GB. I’m very very happy with Zoho.

[u/Low_Pass_Philter]

Just as easy to leave copy of your backup with a trusted friend or family member. It’s cheap and reliable and you don’t have to worry about a rouge ai agent scrambling your files in a server farm somewhere. That’s the way I’ve done it for decades. 3-2-1. Working, local backup and remote backup. It’s a brute force system but it works. Every few weeks I swap the local for the remote backup and start over. Plus I get to visit my family member more often that I might otherwise.

[u/pocketobservations → u/Low_Pass_Philter]

Thanks. Can I ask what age the physical drive you use, brand/characterics etc?

[u/Low_Pass_Philter → u/pocketobservations]

I been using the same three drives for a while now. My old back up worked the same way but it was a collection of smaller usb powered drives so it was hard to stay organized and they weren't as reliable. I added three 8 GB WD Easystore (Best Buy) as I found them on sale. Those are desktop spinner HDDs that are known to have decent internals. A lot of people actually buy these and "shuck" them for the enterprise class drives they sometimes inlcude and then stuff them into a NAS or whatever.

I leave them in their original USB desktop cases. Even though I've had them for years they mostly just sit so they don't get much wear and they should last a while. They used to go for crazy low prices around black Friday but I haven't bought one in a while so not sure about cost. I also have a couple of the 14TB's but 8 TB's covers all the important stuff for me. I use carbon copy cloner on the Mac to synch them. I'm not a huge fan of that software but it works and I've used it forever. It took me along time to settle into that routine but I'm very comfortable with it now.

Edit.... Sorry I think I sent you in the wrong direction. I just went and looked at the prices of those drives and it's nuts. At today's prices my HDD's would cost thousands. Can't really recommend at these prices. 3-4 times the highest price I ever paid for one of those drives. Maybe try and get along with an online back up until the crazy gets a little less crazy.

[u/pocketobservations → u/Low_Pass_Philter]

Thank you so much. Yes the costs are absolutely unreal. Which is why I'm trying to pool the crowd to see what are financially realistic backup options for a hobbyist. I'm based in Australia but we also have black friday sales now. I doubt there will be crazy deals, especially for SSDs. But I'll be on the lookout.

[u/typesett]

Same

i M JUST DOING 16 terabytes on a external to stall for a few years

[u/Ambitious_Pipe_3032]

Before anything else: get backblaze personal.

Then get some sort of setup that replicates your images over two drives at least. This will give you a three replication setup which is ideal

[u/pocketobservations → u/[deleted]]

Thank you. Yes I found 10k duplicate I need to deal with. But that still leaves me with A LOT. But this was one of the reasons I consolidated!

[u/micahpmtn]

As a beginner, you don't need storage advice, you need to how to cull your photos. Professional photographers and serious amateurs keep around 10% of all their photos from any given shoot. Learning how to cull now will serve you well in the future.

[u/pocketobservations → u/micahpmtn]

Interesting. I see lots of people on forums and online tutorial and courses that say they keep everything, and they recommend to do so if we can. Though this may change as the price of storage is increasing so much.