I feel cheated by Backblaze

[u/--Arete]

I need to restore a bunch of files from my personal Backblaze backup.

Instead of restoring the whole backup (which is many terabytes), I only need files that are a specific file type (.xyz files). Reducing the restore from terabytes to some megabytes.

​

• If I use the "View/Restore Files" interface lets me search for .xyz files, but there is no way to select all the XYZ files from the search result. I will have to manually click on thousands of files like this. 🤯
• If I decide to restore the root folder instead (including a bunch of files I don't need to restore), I get the message saying:

The total size of the selected files exceeded the limit of restore method for zip file.

The stupid message does not even say what the limit is. So I will have to figure this out myself. 🤦‍♂️ Backblaze does not have any info on this. They have not answered the same question from five years ago. But it seems 500 GB is the limit.

This means I now have two options:

• Either restore a whole bunch of root folders in 500 GB chunks with a lot of files I don't even need.
• Or order a 8TB drive from Backblaze (which is not free because I will have to pay for taxes, duties, customs and fees which are not refunded). I will also have to wait for a 1 week shipping at least.

The shitty user interface for selecting files to restore is something that makes me want to switch provider. 😡 The interface has not been improved for years, and I assume Backblaze gives 0 fucks about it even though it is extremely important to the customer experience.

If anyone from the Backblaze team reads this I would love to hear how you would go about this specific problem. To me it's a clusterfuck.

"At least I have backup I guess..." 😓

[u/brianwski]

Disclaimer: I work on the Backblaze client team as a programmer.

If anyone from the Backblaze team reads this I would love to hear how you would go about this specific problem.

First, the biggest most important answer: this is our second biggest complaint from customers, so you are FAR from all alone, and we have a project half way finished to build the solution to most of your issues. It's called "bzrestore" and it's a local app you run on your computer that skips the ZIP step, has no size of restore limits, and downloads the files to where you want them even over-writing the originals in place if you want. This solves a particular problem where to restore a 500 GByte ZIP worth of files you need 1 TByte of free disk space currently. 500 GBytes to download the ZIP, and an ADDITIONAL 500 GBytes to unzip it. The bzrestore app doesn't have the temporary ZIP copy so it would only take 500 GBytes to download it to a "new" folder, and if you were overwriting a bunch of existing files it wouldn't take any space at all. Well, maybe 1 or 2 GBytes of temporary space to reconstruct files, but that's it.

This new bzrestore "native restore client" won't take away any of the previous methods of restoring, so as we get the bugs and kinks worked out of it the service is not degraded in any way.

None of this helps you or anybody else for at least another month or two while we finish bzrestore. But we are aware this is a shortcoming and attempting to address it.

They have not answered the same question from five years ago. But it seems 500 GB is the limit.

It started as a 1 TByte limit in 2008, and personally I assumed it would have gone UP by this time. However, the ZIP format has been problematic. We originally chose ZIP because it was the only cross platform format that did not require ANY extra software to install on your system. ZIP unpacks on Windows and Macintosh both, without any additional software required.

Along the way, we discovered that there are very annoying limitations of ZIP, and those have been solved in different degrees by Windows vs Macintosh vs 3rd party solutions like 7-zip. The support organization requested we lower the limit to 500 GBytes because customers would get themselves wedged where they had PREPARED a really large restore but couldn't download it or unpack it, so by artificially limiting it earlier in the process we have fewer support issues.

Or order a 8TB drive from Backblaze (which is not free because I will have to pay for taxes, duties, customs and fees which are not refunded).

That part really bums me out, where the fees are not refunded. Here is the analogy: if a large 18 wheel truck delivers a TV or video game console to your home, then the truck drives back to the country of origin, you do NOT have to pay taxes, duties, customs, and fees on the 18 wheel truck!! The hard drive we send you is simply the transport mechanism, and in a sane world when you return that hard drive CLEARLY the item you have truly had delivered is your data. So while you should owe duties, customs, and fees on the DATA, you shouldn't owe those things on the stupid transport mechanism of the hard drive. But we realize this is a real issue and that's part of why we are building bzrestore.

[u/--Arete → u/brianwski]

This is a very reassuring and satisfying answer. The fact that you have taken the time to write such a thorough answer makes me very happy.

I really hope that bzrestore will address these issues. I am especially excited about how it will handle searches and how to select multiple files. Hopefully, it will work as close to Windows Explorer as possible, to begin with, but I realize that this is a long journey.

Thanks for letting us know. It makes it easier to hang in there until help arrives 😉

[u/brianwski → u/--Arete]

I am especially excited about how it will handle searches and how to select multiple files.

I'm not sure version 1.0 will be perfect in that area yet. But we all think bzrestore has a solid chance to become the PRIMARY way files are downloaded and restored eventually, and the way we are architecting it might very well make it faster than either ZIP or a USB drive delivery depending on the customer's download speeds of course. The reason it could very well be faster than ZIP (or USB drive) is that one ZIP restore (or USB restore) is handed to one Backblaze server to prepare. That one server is responsible for decryption, then uncompression (each file is compressed before being encrypted), then that one server takes a considerable amount of time ZIPping up the contents. The way we architected bzrestore, it could have 100 different servers decrypting and uncompressing for it, and it skips the ZIP step entirely. If a customer has a 1 Gbit/sec or faster download speed (which is getting quite common in many areas, like 80% of the USA has that available now) then it's very plausible bzrestore can beat a hard drive on a FedEx truck end-to-end to completion of a 2 or 4 or even 8 GByte restore.

So anyway, I believe it is where we will put a lot more effort in to make things like search and selection better.

But I am going to forward this whole thread on to the team working on bzrestore, and point out the search and select issue.

[u/Bakgrund → u/brianwski]

Regarding this part

>Or order a 8TB drive from Backblaze (which is not free because I will have to pay for taxes, duties, customs and fees which are not refunded).

If you allowed to transfer backup from the US site to the EU site, you could just ship the physical backup from EU so your customers within the EU wouldn't have that problem.

I really hope you will allow us to transfer to the EU site in the future.

[u/brianwski → u/Bakgrund]

I really hope you will allow us to transfer to the EU site in the future.

It is most definitely on the roadmap to provide USB restores that originate in the EU. We have a datacenter in the Netherlands, and our plan is to provide USB restores from there to help alleviate some of these issues.

Of course, that won't help people in other regions such as Japan, Korea, etc. Until we have the same service there. I really hope that within 2 years we have a much more comprehensive restore for all customers. We sell Backblaze Personal Backup in over 175 countries, it would be nice if we could restore for free equally across at least 80% of those countries.

[u/Ok-Bass-5368 → u/brianwski]

How come you don't work at backblaze anymore? (tag says former) - they should have kept you on.

[u/brianwski → u/Ok-Bass-5368]

How come you don't work at backblaze anymore? (tag says former) - they should have kept you on.

I got very old and retired. Startups are a young person's game.

I'm still around on reddit answering questions for anybody I can help. I'm still a shareholder and want the very best for Backblaze, and answering reddit questions gives me something to feel useful in my retirement.

[u/Ok-Bass-5368 → u/brianwski]

Well, that serves you right for doing something silly like getting old. Thanks Brian.

[u/ZivH08ioBbXQ2PGI → u/brianwski]

and we have a project half way finished to build the solution to most of your issues. It's called "bzrestore" and it's a local app you run on your computer that skips the ZIP step, has no size of restore limits, and downloads the files to where you want them even over-writing the originals in place if you want.

For the love of God, just build it into the backup client.

Seriously. Brian, why is this such a thing with you guys, for like a freaking decade now? Crashplan had this completely streamlined pretty much perfectly like a decade ago. It was beautiful, and it worked perfectly. No extra software, no extra logins. Just hit the Restore tab, make your selection, pick a date, and a restore location. Done.

Don't make this harder than it CAN be.

Edit: I'm going to edit this to say the obvious: The backup client already knows the encryption key and it can easily decompress and assemble the incoming (restoring) files much faster than enslaving some random little server in your datacenter with decrypting and zipping everything, after sharing my encryption key, to boot. I know you said this is what the new (COMPLETELY SEPARATE) restore application is going to do, but this is moronically obviously how it should have been done in the first place, but with the backup client; just let the client download the raw data from the destination backup servers and decrypt it all on the fly.

I'm sure it's not quite that simple, but in the end, the concept sure it, and it beats the pants off of what you've come up with in better part of the last decade.

I apologize if I'm coming off like a jerk, and you've been very supportive to users with troubles here on reddit, but I feel like a giant ball of chastising is due here. It's 2022 and your backup client can't restore a simple file that it backed up.

[u/brianwski → u/ZivH08ioBbXQ2PGI]

For the love of God, just build it into the backup client.

It is, that's part of the design. Or put differently, there are already 4 distinct executable programs that are part of the backup client: bzserv, bzfilelist, bztransmit, and bzbui. This will become the 5th: bzrestore.

There won't be any way to install the backup client without installing this, just like there isn't any way to install the backup client without installing bzbui or bztransmit.

It will launch from a button in the main GUI of the existing bzbui (backup client).

Seriously. Brian, why is this such a thing with you guys, for like a freaking decade now?

We are attempting to fix it now. It's been 15 years, so technically a decade and a half. We did offer a variety of restore techniques up until now, one of the more popular is the free USB drive restore which works better than this for all non-technical customers inside the USA. Also, the USB restore hard drive works better for low bandwidth situations. The weakness (which I fully admit) is that non-technical customers outside the USA are hit by tariffs and customs charges for the USB hard drive, thus it has always been worth fixing at some point.

The difficulty wasn't the will to do it, or stubbornness on our part, or any philosophical issue. The difficulty was lack of resources and a limited staff. That limitation was removed (for a few years) when we raised money in an IPO, so post IPO we hired up in the client team and want to get this done with that new staff.

[u/vadalus911 → u/ZivH08ioBbXQ2PGI]

+1 for crashplan. I looked at moving to BackBlazeB2 (and have an active account) but getting the restore interface right s sooo important. Feels like BackBlaze treat actually getting your data back as an Amazon S2 task (at least for B2 users). The issue i was facing with C\rashplan is that i run it in a non supported configuration (Docker etc...)..

Whenever i have really been needing to restore i have invariably overwritten or correupted files both locally and then backed them remotely. The ability to download an image "at a point in time" is crucial. This is not properly supported on B2 at all and i was told to look for 3rd party providers (and no, selecting versions on a per file basis is useless).

[u/brianwski → u/vadalus911]

The ability to download an image "at a point in time" is crucial. This is not properly supported on B2 at all

The server side of B2 has more support than Backblaze Personal Backup for "a point in time" restore. B2 offers APIs to do it, check out: https://www.backblaze.com/b2/docs/file_versions.html for a high level description, and you can list all versions of all files inside your backup in B2 with the API described here: https://www.backblaze.com/b2/docs/b2_list_file_versions.html and then (programmers) can write software to restore those file versions within a date range if they simply write it. (I'm not saying you should have to write it, I'm saying SOME programmer should have already written it for you.)

With the b2.py command line tool described here: https://www.backblaze.com/b2/docs/quick_command_line.html you can see all versions of every file you have backed up into B2 with the command "b2.py ls --long --versions". With a few lines of script you could combine that with "b2.py download-file-by-id" to restore from a date range.

It is a tragedy if none of the 3rd party tools have EVER been developed to have this ability. That's over 200+ software tools, and none of them have ever pulled it off? Gah, that sucks. Since B2 has an Amazon S3 compatibility layer that also means Amazon S3 lacks a tool to do this also.

This was one of the main reasons we built Backblaze B2: to stop having our lack of resources be the crucial bottleneck to progress. To allow 100% of all programmers world-wide to craft solutions our customers are asking for in any language the particular programmer wants to use. Again, that doesn't mean "you" have to roll your own, that isn't scalable or plausible for all customers, but Backblaze B2 was supposed to unlock this type of functionality for everybody. That was the point!

[u/vadalus911 → u/brianwski]

I get you. But if you look at backblaze marketing it kinda says it’s supported but it’s only for the personal version not the b2 and it’s not exactly made clear. I don’t really want to cobble together a bunch of scripts to do this. I think it’s basic functionality which ideally is simply supported (I realize the amazon back end supports it). I dislike having to rely on another 3rd party to complete a recovery of what is usually quite important data and it’s alll a bit stressy recovering it !

[u/brianwski → u/vadalus911]

look at backblaze marketing it kinda says it’s supported but it’s only for the personal version not the b2 and it’s not exactly made clear

The closest thing we have built in for B2 is found under the “Snapshot” functionality in the web UI login. You first select up to 10 TBytes and place into a “Snapshot” which behind the scenes is just a separate B2 bucket in your same account. From there, you can order a USB drive FedEx’ed to your home with the contents, or use the standard command line tool to download it, or use a third party tool to download it. There is a little more info here: https://www.backblaze.com/blog/b2-snapshot-return-refund-program/

[u/mcholbe2 → u/brianwski]

Will this include B2 customers? I've spent several weeks attempting to download a single ~1TB file. The download failed with the command line tool and Cyberduck.

[u/brianwski → u/mcholbe2]

fails I don’t know if this will work, but I always had luck with rclone. I used Transmit, but it crashed with large files or when queuing multiple files - rclone always did the trick. Never attempted 1TB

We would like to eventually include "B2" to replace the "B2 Snapshots" in this functionality. But it isn't the very highest priority. The pain point here is Backblaze Personal Backup, but we are totally open to expanding it to B2.

Maybe this is too much detail, but we quietly will be using B2 in the background as part of this service (bzrestore). All customers have all product lines available to them, and we have "reserved buckets" in B2 for Backblaze Personal Backup customers already who do things like "Share file" with friends. We are using some of that same technology to implement this bzrestore native client, so it isn't beyond the realm of imagination that it would also be provided to B2 customers.

[u/Cardsharp007 → u/mcholbe2]

If the b2-tools fails I don’t know if this will work, but I always had luck with rclone. I used Transmit, but it crashed with large files or when queuing multiple files - rclone always did the trick. Never attempted 1TB though.

[u/farstukvist → u/brianwski]

bzrestore

This is 5 months ago, is the bzrestore app available now? I didn't find it when trying to restore my broken drive and I can't use the zip option because it's too large.

[u/brianwski → u/farstukvist]

is the bzrestore app available now? I

Not yet. But it is close. Many parts now work, we're down to just one last request protocol the server team needs to implement and we can finish it.

You should order an external hard drive restore. It is free (if you return the drive afterwards), and 8 TBytes per restore drive. You can have all your files back in a few days. It takes zero of your own hard drive space because it comes on it's own drive. It's the best option until we finish bzrestore.

[u/brianwski → u/[deleted]]

Well here we ate 7 months later and no bzrestore

It is well underway and is supposed to ship a "beta" version in Q1, which means end of March. These things take time, and other stuff rises in priority and pushes delivery dates back a little. We'll get it to you.

[u/BlackHorseUK → u/brianwski]

Backblaze! Where is BZRESTORE? How much longer? I need it really urgently.

[u/brianwski → u/BlackHorseUK]

I need it really urgently.

If you need a restore, you should be thinking ZIP restore today, possibly USB restore. Don't delay.

The earliest bzrestore will be available in beta form is probably late March or even further like mid-April. That is too late if you need to restore data right now. I have less and less context about the bzrestore project, but I tried my best to get the last hurdles cleared before I retired. And it kind of went a little sideways on me, they started changing things about the protocol I disagreed with, adding quite a bit of complexity to the original design we had agreed to over a year ago. Complexity and uncertainty and design changes add time to schedules. The changes they were discussing as I left are also going to hurt the performance. The design was the design for a reason.

[u/TheAspiringFarmer]

yeah the restore process is easily the achille's heel of Backblaze. it hasn't tangibly improved over the years at all. i believe one of the employees mentioned to me a bit back that they have a new system or windows app or something that will improve it quite a bit but for right now it is what it is.

[u/scorpiona]

Be extra careful even if you decide to order a drive from them, and try to recover as much as you can through the web interface. I lost an entire system's backups after Backblaze sent me a dead drive.

[u/ErebusBat → u/scorpiona]

Did they delete it after they dent you the drove or what?

[u/scorpiona → u/ErebusBat]

Yes, they did not retain the restored data for 30 days like they do for web restores.

[u/TheAspiringFarmer → u/scorpiona]

that's disconcerting. ugh.

[u/ErebusBat → u/scorpiona]

That is very troubling

[u/--Arete → u/scorpiona]

Do they delete the backup after? Or did you delete it, thinking it was safe because the drive was shipped?

[u/scorpiona → u/--Arete]

I've written about it before here.

Basically, I had a single personal subscription and I was transferring files across an OS reinstall. Normally when you request a restore from the web interface, it copies your files to an available restore that you can then download for 30 days even if you delete the original files. I backed up my entire PC, reinstalled the OS and wiped my local files, and tried to download my files again.

I ran into the same problem as you, where the web interface was too clunky and slow to restore everything, so I ordered a drive from Backblaze. While I was waiting for it to arrive, I installed Backblaze on my newly wiped PC to backup my files while I was using it.

What I didn't know is that:

• Backblaze does NOT keep a restore of your files when doing a disk restore. They just copy them onto the disk and do not keep a restore available for 30 days, like the web restore.

• If you have a personal subscription, installing Backblaze onto a new OS (even if it's the same machine) will count as one of your license "slots" and you will lose ALL the data associated with the "old" machine.

So at this point I was waiting for my drive to arrive, but Backblaze had deleted all my backed up files after reinstalling Backblaze, and they did not keep a restore for 30 days after requesting the drive backup -- so there was no other copy of my data except the drive.

And the drive was dead-on-arrival. I contacted Backblaze support, and they were 100% antagonistic and unhelpful the entire time, they basically ran me through "USB 101" while I was sending them diagnostic data, badblocks, etc showing that the drive was unreadable.

After we spent days going back-and-forth diagnosing it, they gave up and said "sorry, your data is gone". :)

[u/--Arete → u/scorpiona]

I am not sure if I understood this correctly and I still don't understand how and when your Backblaze backup was deleted.

Normally when you request a restore from the web interface, it copies your files to an available restore that you can then download for 30 days even if you delete the original files.

Is this official information or is that just your assumption?

I ran into the same problem as you, where the web interface was too clunky and slow to restore everything

This is not the same problem I had. If I were going to restore the whole shebang I would not have any problem with the interface really.

Backblaze does NOT keep a restore of your files when doing a disk restore.

What do you mean by "disk restore" in this sentence? Do you mean shipping a drive with the backup to you? A.k.a Backblaze "USB hard drive restore"?

Sorry, I feel like I am the only one not getting this, but I am curious.

[u/scorpiona → u/--Arete]

This is not the same problem I had. If I were going to restore the whole shebang I would not have any problem with the interface really.

Except if you try to restore a large backup (e.g., many TiB) and select everything, it will take a minute or so to process before erroring out and saying that you've exceeded the backup limit. It also doesn't give you any easy way to split the download, so you have to go folder-by-folder and individually select just under the 500GiB cap, prepare the download, and repeat until you've got your whole system prepared into a series of zip restores. For a large whole-system backup, this is incredibly tedious and unwieldy.

What do you mean by "disk restore" in this sentence? Do you mean shipping a drive with the backup to you? A.k.a Backblaze "USB hard drive restore"?

Yes, exactly. The one where you pay ~$200 and they Fedex you a drive with your data on it.

This is what I bought to restore my data, instead of trying to deal with the web interface for zip restores.

The difference with disk restores is that they are not retained for a while after the restore is prepared, unlike the zip/web restores which are.

Is this official information or is that just your assumption?

It's 30 7 days, but yes, it's official. Once the zip restore is prepared you can retrieve it no matter what you do with the originals. 30 days is for the default rolling version history, i.e., you can still access live files up to 30 days back.

I still don't understand how and when your Backblaze backup was deleted.

That's the same fun and surprise I had when Backblaze told me they didn't have my data. 🙃

Backblaze deleted my data after I reinstalled the client to the wiped PC. They will wipe all data associated with the "old" license when that happens.

After that, the data is gone and only prepared restores still exist with a copy. Because they sent me a dead disk for the USB hard drive restore, I never got my data.

I also found out from this that they don't retain a copy of the restore disk data (even though it shows up in "My Restores"), so there's no way to prepare and resend another restore drive from the data associated with the old license slot. It also took more than a week for them to get the drive to me, so even the zip/web restores that I had already prepared were expired by the time I found out the disk was dead.

[u/--Arete → u/scorpiona]

When you set up the new computer, did you by any chance activate "Inherit Backup State"?

Edit: I am asking because this article says:

If a file is in the inherited backup, but not on the new computer, it will be purged from the backup

Maybe this is why you lost your files?

[u/brianwski → u/scorpiona]

Disclaimer: I work at Backblaze.

Be extra careful even if you decide to order a drive from them, ... I lost an entire system's backups after Backblaze sent me a dead drive.

I very much agree with the "careful" part, but if done carefully you can depend on the 8 TByte drive. Mail delivery isn't particularly kind to drives, so we get FAIRLY REGULAR reports from customers that the drive arrived DOA. We ship something close to 100 restore USB drives a day, so I believe this is well under 1%, but at 1/10 of 1% this occurs practically every month at least once.

We can re-prepare the same restore that we shipped customers in this case, and get that going even before the customer returns their USB drive, so it's usually only a delay of 1 or 2 days until the customer gets their data. Probably a little longer if the customer is in Europe or elsewhere out of North America.

[u/scorpiona → u/brianwski]

We can re-prepare the same restore that we shipped customers in this case, and get that going even before the customer returns their USB drive, so it's usually only a delay of 1 or 2 days until the customer gets their data. Probably a little longer if the customer is in Europe or elsewhere out of North America.

This is NOT true. Backblaze sent me a DOA drive and did not have a prepared restore to resend.

[u/brianwski → u/scorpiona]

me a DOA drive and did not have a prepared restore to resend.

It really should be the case that we can re-prepare the drive for you. The only exceptions I can think of is if you are up against the 30 day deadline (which is the default version history) or have not paid for the 1 year version history (at which point there really shouldn't be any issue at all, even 11 months into the process).

So again, if you are CAREFUL and prepare the restore PROMPTLY and check the results within 15 days (so that we have plenty of time to prepare a second drive for you) then it shouldn't be an issue at all. Even up to 25 days we can place a "do not delete files" on your account so that we can get you a clean restore! The most important thing is to be clear with the customer support and explain your situation. Literally NOBODY (not Backblaze, not you) wants to have data we can cleanly restore and not hand it over to you.

[u/av-IT-privacy-fun → u/scorpiona]

Oof that sucks. Can you elaborate? In what way was it dead? And why couldn’t Backblaze Support send you another drive with the same data you requested?

[u/scorpiona → u/av-IT-privacy-fun]

I believe it was a failure of the USB controller and/or hardware encryption on the specific model of WD Passport they sent me.

I asked Backblaze to resend my restore, but they told me they did not retain disk restores for 30 days like they do with web restores -- the only copy is on the disk.

[u/Ener_Ji → u/scorpiona]

I asked Backblaze to resend my restore, but they told me they did not retain disk restores for 30 days like they do with web restores -- the only copy is on the disk.

What! That blows my mind. 🤯 Why would they do it that way?!

[u/milennium972]

You can maybe do a gateway for backblaze with minio and use their interface to interact with it

[u/av-IT-privacy-fun]

I hear you OP. This does sound like a frustrating situation, and I agree that there really should be a better search interface. In the meantime, have you considered using an AWS EC2 (or similar from one of their competitors) as a way around the problem? If I were in your situation, I might think about spinning up a virtual computer in the cloud which has enough bandwidth and processing power to easily download and unzip all of those 500 GB chunks, and then find the files you need and download only those. Yes, it is no substitute for Backblaze being able to do this themselves, but consider their “all you can eat “pricing model. They charge a very low price and it would seem that not offering the computational feature of search by parameter is one way by which they keep their expenses low to match.

[u/--Arete → u/av-IT-privacy-fun]

Thanks for the tip. I have no idea what AWS EC2 is or how to use it, but I will check it out.

[u/av-IT-privacy-fun → u/--Arete]

Oof it has more than a bit of a learning curve. Can anyone else recommend a virtual desktop environment that will let OP get their data faster than the one week via hard drive shipping And is more user-friendly to start?

[u/mcholbe2]

Thanks for the suggestion. It's worth a try. B2 is currently our only off-site data storage so restore times are critically important for us should a disaster affect our local copies. It was interesting....with Cyberduck the download appeared to work correctly then suddenly reported it was 128% complete and kept downloading. With the command line tool there was a several hour I/O intensive process before download traffic started.