Hyper Backup Extremely Slow

Discussion room for the Hyper Backup package
Forum rules
We've moved! Head over to Synology Community (community.synology.com) to meet up with our team and other Synology enthusiasts!
the1lemming
I'm New!
I'm New!
Posts: 7
Joined: Fri Jan 26, 2018 8:43 pm

Is Hyper Backup really this slow?

Unread post by the1lemming » Tue Aug 07, 2018 8:42 pm

I've had my DS214 for 3-4 years however I never gave much thought to actually backing up the NAS box.

Of the couple of times I actually backed up the NAS, I simply used my desktop computer to do a drag and drop into an external USB drive.

a few days ago, I had a stab at automating the process of backing up by connecting a USB external hard drive into a USB3 port and using Hyper Backup to do the task. The external drive was formatted with NTFS.

I only had about 1Tb of data to transfer over. I was shocked to discover that 20 hours later I was only up to 80% of completing the task. I then reformatted the external hard drive to Ext4 and tried Hyper Backup again.

Things went quicker in that I actually completed the task in just under 20 hours. At one point I was getting write speeds to the external hard drive of 16kb/s.
I'm an amateur with no IT qualifications but I'm guessing kilobits per seconds speeds is pitiful.

I then had another go but this time I connected the USB drive to my desktop computer and fired up Acronis and used that to back up the data on my Synology DS214. Over the LAN connection, I was getting between 70Mb/s to 90Mb/s. Things got even faster with the validation process with read speeds of 155mb/s.

Does anybody know why and how backing up my DS214 to an external USB drive connected straight into the box takes days to backup and validate?

Doing the same task from a desktop computer over a LAN connection with Acronis to a USB external drive takes 5-7 hours.

Am I doing something wrong?
The DS214 is using DSM 6.2

tb123
Knowledgeable
Knowledgeable
Posts: 362
Joined: Sun Sep 03, 2017 10:55 am

Re: Is Hyper Backup really this slow?

Unread post by tb123 » Tue Aug 07, 2018 11:08 pm

What was the USB drive, 1,2 or 3?
DS916+, 3 x 4TB WD Red, 1 x 4TB WD Purple
UniFi USG, 16 port PoE switch and 4 x AP’s, some Windows, Mac, Android and iDevices

User avatar
sciurius
Sharp
Sharp
Posts: 160
Joined: Thu Sep 05, 2013 9:35 am

Re: Is Hyper Backup really this slow?

Unread post by sciurius » Wed Aug 08, 2018 7:18 am

And was it connected to the USB3 port at the back, not the USB2 port at the front?
DS418 / DSM 6.2-23739 Update 2 / 2 x WD40EFRX-68WT0N0 (SHR) / MailServer / MailStation / ownCloud / Transmission / ResilioSync / Syncthing / spotweb / LMS+Spotify
DS413 / DSM 6.2-23739 Update 2 / 2 x ST3000DM001-1CH166 (SHR) / Testing

the1lemming
I'm New!
I'm New!
Posts: 7
Joined: Fri Jan 26, 2018 8:43 pm

Re: Is Hyper Backup really this slow?

Unread post by the1lemming » Wed Aug 08, 2018 9:00 pm

a few days ago, I had a stab at automating the process of backing up by connecting a USB external hard drive into a USB3 port and using Hyper Backup to do the task. The external drive was formatted with NTFS.

Good evening. Yep, my USB3 hard drive was plugged into a USB3 port.

Cheers

At this very moment I am using FreeFileSync to backup my NAS drive to an external hard drive which is plugged into my desktop computer. The NAS and computer are hard wired to my router and the task is expected to take just over two hours.

the1lemming
I'm New!
I'm New!
Posts: 7
Joined: Fri Jan 26, 2018 8:43 pm

Re: Is Hyper Backup really this slow?

Unread post by the1lemming » Thu Aug 09, 2018 9:15 pm

At the same time I posted this subject on the forums, I also contacted Synology with a ticket.

Here is the initial reply to the same question.

Spoiler, Synology expects the 1Gb task to take 57 hours.



Thank you for contacting Synology.

Sorry to hear of the issue you have encountered.

From the figures provided (33% in 19 hours), your backup would complete in ~57 hours if this pace is maintained.

This works out to ~5.11MB/s which can be normal on the first runtime of a Hyper Backup task.

The initial speed varies greatly depending on the kind of data being backed up, for example lots of little files take much longer to place in the database format. Very large files can also take a significant amount of time because our database format slices files into smaller chunks for greater deduplication and faster incremental changes.

I sincerely apologise for the slow initial backup, however any subsequent backups should be vastly improved.

Please let me know if you encounter any errors, I would be happy to investigate further.

Kind Regards,

Filip

Technical Support

This is an automated message, please log in to your Synology Account for further actions or replies.

tb123
Knowledgeable
Knowledgeable
Posts: 362
Joined: Sun Sep 03, 2017 10:55 am

Re: Is Hyper Backup really this slow?

Unread post by tb123 » Thu Aug 09, 2018 10:48 pm

Not sure if it depends on what type of files you are backing up, but my initial HyperBackup run on around 1.5TB of data to a USB3 drive was approx 6 hours on a DS916+ with 8 gig of ram.
DS916+, 3 x 4TB WD Red, 1 x 4TB WD Purple
UniFi USG, 16 port PoE switch and 4 x AP’s, some Windows, Mac, Android and iDevices

the1lemming
I'm New!
I'm New!
Posts: 7
Joined: Fri Jan 26, 2018 8:43 pm

Re: Is Hyper Backup really this slow?

Unread post by the1lemming » Fri Aug 10, 2018 1:58 pm

tb123 wrote:
Thu Aug 09, 2018 10:48 pm
Not sure if it depends on what type of files you are backing up, but my initial HyperBackup run on around 1.5TB of data to a USB3 drive was approx 6 hours on a DS916+ with 8 gig of ram.
Hello

If I got speeds of 6 hours for 1Tb to 1.5Tb then I'd be happy with that especially as I'm backing up domestic stuff like word docs, music, photos and home videos. But anything in excess of 24 and possibly 50+hours is not very good in my opinion.

I have now been asked by "Synology Support" to submit a log after initially telling me that 50+hours was acceptable.

Cheers

the1lemming
I'm New!
I'm New!
Posts: 7
Joined: Fri Jan 26, 2018 8:43 pm

Re: Is Hyper Backup really this slow?

Unread post by the1lemming » Mon Aug 13, 2018 9:04 pm

The problem has been solved and confirmed by Synology.

I created a ticket and got in touch with the correct Synology Support Team/Section.

I created and gave Synology the relevant logs created by my DS214 NAS box.
The reply was that my DS214 was working correctly and that there were no errors between my NAS box and my external USB3 hard drive.

It was confirmed by Synology Support that 1Tb of data would take in excess of 50 hours to create and then validate that data.

Its a good job that 3rd party software can deal with backing up 1Tb of data to a domestic Synology NAS box within a couple of hours because, like almost everybody on here, I have considerably more than 1Tb of data to protect.

Here is a copy of Synology's reply:

Hi the1lemming,

Thanks for getting back to me and providing a debug file.

From investigating your system logs, I cannot isolate any problems with the processes running on your NAS and there does not appear to be any evidence that hardware problems exist which would contribute to the slow backup experienced.

Unfortunately, I must conclude that the data being transmitted via Hyper Backup is performing normally and that the speeds you are encountering are expected.

It is often the case that the initial backup is significantly slower than all subsequent backups. I cannot speak for third party alternatives, but our own database format has many integrity checks in place and placing files into this format (especially many very small or very large files) takes significant processing time and this varies greatly depending on the kind of data being backup up.

Usually you can see the transfer rate by hovering over the progress bar on a running task. This will often appear to show adequate speeds but generally stop / start because there are pauses for database calculation.

Again I am sorry for any inconvenience caused, I would recommend proceeding with the initial backup task through Hyper Backup. Subsequent backups will be vastly improved.

Kind Regards,

ivyanch
Novice
Novice
Posts: 49
Joined: Thu Dec 15, 2016 8:41 am

Re: Hyper Backup To USB HDD Slow

Unread post by ivyanch » Fri Aug 17, 2018 5:37 am

corematrix wrote:
Fri Jul 27, 2018 10:32 am
So when i copy and paste from the NAS to the USB HDD it runs at about 60MB/s, but hyper backup is running between 6MB/s and 25 MB/s. Why the speed loss?
Are you using encryption?

Also, lots of small files transfer much more slowly than a few large files.

25 MB/s is not super terrible, but 6 MB/s is.

chuck@on-high.com
I'm New!
I'm New!
Posts: 3
Joined: Sun Aug 19, 2018 9:34 pm

Re: Hyper Backup Extremely Slow

Unread post by chuck@on-high.com » Sun Aug 19, 2018 9:40 pm

I'm backing up just at 1.5TB using a 218J with a connected USB(3) external drive and HyperBackup (really ought to consider changing that name! LOL) is only a 6% after 20.5 hours.

I have one drive in my mirrored pair that's failing which I guess I could see slowing things down (even though I specified the source as the remaining good drive) but this is ridiculous. If it takes 5-6 DAYS to back up, I'll be looking at non-Synology solutions for a near-term RAID 5 solution need. I've backed up to TAPE in far less time than this.

Absurd...

Cheers,
C

User avatar
sciurius
Sharp
Sharp
Posts: 160
Joined: Thu Sep 05, 2013 9:35 am

Re: Hyper Backup Extremely Slow

Unread post by sciurius » Mon Aug 20, 2018 8:58 am

chuck@on-high.com wrote:
Sun Aug 19, 2018 9:40 pm
I have one drive in my mirrored pair that's failing which I guess I could see slowing things down
Hardly. Read speed will be a bit lower, but the limiting factor for a backup is (should be) write speed.
(even though I specified the source as the remaining good drive)
Interesting. AFAIK HyperBackup backs up a shared folder, not a physical drive.

Also, if a drive of a RAID fails, it is automatically deactivated.

In any case, HyperBackup has to do a lot of work (calculating block checksums and so) which may be slow.

Can you try an rsync backup instead?
DS418 / DSM 6.2-23739 Update 2 / 2 x WD40EFRX-68WT0N0 (SHR) / MailServer / MailStation / ownCloud / Transmission / ResilioSync / Syncthing / spotweb / LMS+Spotify
DS413 / DSM 6.2-23739 Update 2 / 2 x ST3000DM001-1CH166 (SHR) / Testing

chuck@on-high.com
I'm New!
I'm New!
Posts: 3
Joined: Sun Aug 19, 2018 9:34 pm

Re: Hyper Backup Extremely Slow

Unread post by chuck@on-high.com » Mon Aug 20, 2018 4:37 pm

Interesting. AFAIK HyperBackup backs up a shared folder, not a physical drive.
Yeah, to avoid being confusing - when I created the b/u task, it allowed me to select the source drive (perhaps due to the status of the failing drive? But it did look like a normal dialog presentation...)

I might give rsync a go after this pass is completed. Can rsync be performed to a USB connected drive?

I used USB thinking it would take less time than over ethernet. I'm using a WD EasyStore USB3 capable external drive. But I'm not so sure now... I'm 39 hours in and still only 21% done. I didn't know the backup was going to take a week to complete. This truly is absurd.

The good news is I leave town for almost a week tomorrow (which based on this, I know the b/u probably won't even be at 50% by the time I leave) so it will have plenty of time to complete without me sweating over status.

The bad news could be if the other drive fails while the b/u is running, in which I case I will be mostly screwed. I'm tempted to just take the darn Synology box apart, clone the good drive (which will only take HOURS not DAYS!) and then replace the bad drive and move on with life - choosing QNAP or others for future solutions.

Granted, frustration aside and to be fair... it is a 212j so its processor is weak by even cell phone standards these days and it's been serving me for probably 4+ years with only one other prior failure (in which I cloned the good drive to perform b/u), so once my frustration cools off I will likely give Synology a more fair shake - but DAYUM! I'm serious, I used to b/u data to tape way faster than this; with compression and initial backups. Crazy. And I am currently on the hunt for a RAID 5 16TB solution for multiple purchases and this experience HAS to weigh in my decision.

John H
I'm New!
I'm New!
Posts: 4
Joined: Fri Jul 18, 2014 5:49 am

Re: Hyper Backup Extremely Slow

Unread post by John H » Fri Aug 24, 2018 8:10 pm

I'm using Hyperbackup to back up around 5TB from a DS1815+ (6 drives in BTRFS RAID and 2 SSD cache) to an ioSafe 214 (2-bay fireproof NAS that runs Synology OS). I've been using this hardware with Hyperbackup for a couple of years now and while it usually works day-to-day, any significant change to files on the source, like purging a lot of older PC backups, will cause Hyperbackup to fail. The logging is completely useless - only saying that the job failed with no details what-so-ever. I've learned from the first occurrence of the issue in which I opened a ticket with support that the only solution is to delete all the backups one at a time, oldest first, and start from scratch. This process can take a day or two. The next original backup has always worked, though, and daily incremental backups work again afterwards until I have to add or delete a lot of data on the source.

I've been going through this whole process yet again due to purging some old source data, and it appears something has changed in the latest version of Hyperbackup that's making it much slower than it was previously. It was never fast before, typically taking around a day for an initial backup, but it's now been 3 days, 9 hours and the job is less than 1/3 done. Not only that, the load on the source is so high that I can't open its web interface, browse shares, or ssh to it. The only reason I know the job is still running is because I can ssh into the target and see the size of /volume1 slowly change when I execute df. This is really pathetic for what should be considered business-grade hardware. I'd really like to find an alternative to Hyperbackup. Unfortunately cloud solutions aren't currently viable due to limited broadband alternatives, so I'm stuck with whatever I can use to backup from on Synology to the other.

TeeDee
I'm New!
I'm New!
Posts: 4
Joined: Sat Aug 25, 2018 6:05 pm

Re: Hyper Backup Extremely Slow

Unread post by TeeDee » Sat Aug 25, 2018 6:10 pm

I have a DS415+ and my encrypted HyperDrive back up to Google Drive is really slow. I have a Gigabit network but I am only able to achieve approx 5.7MB/s (uploaded about 500GB in 24h). I’ve heard that other people are able to achieve much higher speeds (e.g. 25+ MB/s) with Google Drive.

Could it be the encryption? I thought that since DS415+ has hardware encryption, encryption shouldn’t really affect performance.

tb123
Knowledgeable
Knowledgeable
Posts: 362
Joined: Sun Sep 03, 2017 10:55 am

Re: Hyper Backup Extremely Slow

Unread post by tb123 » Sat Aug 25, 2018 11:57 pm

You have a Gigabit network on your LAN, or your ISP is Gigabit?
DS916+, 3 x 4TB WD Red, 1 x 4TB WD Purple
UniFi USG, 16 port PoE switch and 4 x AP’s, some Windows, Mac, Android and iDevices

Locked

Return to “Hyper Backup”