Time backup very slow

Discussion room for the Synology Time Backup Package
Forum rules
1) This is a user forum for Synology users to share experience/help out each other: if you need direct assistance from the Synology technical support team, please use the following form:
https://myds.synology.com/support/suppo ... p?lang=enu
2) To avoid putting users' DiskStation at risk, please don't paste links to any patches provided by our Support team as we will systematically remove them. Our Support team will provide the correct patch for your DiskStation model.

Time backup very slow

Postby ufq » Fri Dec 23, 2011 12:40 pm

Hi,

We have two DS1511+ (called A and B) and time backup running on on the first DS1511+, A.

When I set up a task with hourly backup and smart recycle it starts and is at "Processing: 10%" for ages. The backup job is 2 TB big. I can see in the resource monitor that it does copy data to station B - but at very slow speed (0-6000 kb/s). We have Gigabit network and I have no problems copying from both disk stations to my own PC at 80 MB/s which is near theoretical speed.
CPU and RAM is at around 20% for both disk stations.

What slows this process down? Does Time Backup run at a lower network speed?

BTW: I have changed the schedule to every two hours as I was getting a lot of "Time Backup on C-NAS has failed. Please check the package log for information".

The package log states "...[failure to create snapshot]"

I still get the errors however only a couple of times per 24 hours.

EDIT: I changed the frequency to 6 hours but still get slow performance and backup errors as described above...:-(

Running: DS1511+, DSM 3.2-1955, TimeBackup 1.1-1922

Best regards
Ulrik
ufq
I'm New!
I'm New!
 
Posts: 2
Joined: Fri Dec 23, 2011 12:15 pm

Re: Time backup very slow

Postby hajuq » Wed Jan 11, 2012 1:46 pm

Hi Ulrik,

not an answer or solution, but a "you're not alone" from me.

Last weekend I've installed my new DS110j to backup a DS209 with only 300GB data. I configured time backup remote DS209 -> DS110j using a perfect GigabitLAN connection exclusively. I started the backup job yesterday evening and this morning (8 hours later) it has processed 10%. The DS209 status monitor shows 100% CPU load. BTW 20% are use by ssh (why ssh? - I don't need ssh in my private network!).

I've DSM 3.2-1955 and Time Backup 1.1-1922 on both Synos.

Did anyone make a speed comparison between backing up local/remote? Does only the remote method waste so much time?

Regards
Juergen
hajuq
I'm New!
I'm New!
 
Posts: 1
Joined: Wed Jan 11, 2012 1:15 pm

Re: Time backup very slow

Postby jdf123 » Sat Feb 04, 2012 5:40 pm

I am experiencing slowness as well. I am backing up from my DS1010 with 3.2 to my DS411j with beta 4 DSM. I average about 30-25 mbps transfer speed even though both are connected at gigabit with mtu set to 9000. I have been running my backup for the several days and it has yet to finish seeing how it is 1.5TB in size. Running standard backup to the remote DS411j gives me about 300+ mbps transfer speed. The other thing I notice is that the CPU is pegged on the DS411j so I am wondering if this is due to software glitch taking up too much CPU for time backup transfers that is limiting the speed.

-Jesse
jdf123
I'm New!
I'm New!
 
Posts: 1
Joined: Sat Feb 04, 2012 5:29 pm

Re: Time backup very slow

Postby duffykins » Sun Feb 12, 2012 9:13 am

I am new to NAS but I am having similar problems. My plan was to buy a small NAS to use as a file share and back up device then later this year buy a "performance" NAS for file serving and "demote" the small NAS to a back up machine to the new, primary NAS.

So the first, current, NAS is a DS212j and two WD drives. First thing that I thought would be sensible to do was to create a backup of my system drive - 60Gb held on a WD Raptor. (I keep all data files on other drives so this is all Windows workfiles, programs etc).

Twenty seven hours later I'm still waiting for the backup to finish. My network speed isn't the best, granted, but this makes backing up somewhat pointless and negates the purpose of buying the box. What is going on?
DS412+, 4 x Seagate ST2000DM001 in RAID 10 backed up to 4TB HDD in Akasa X31 Lokstor case.
Billion 7800DXL Modem/Router; TP-Link TL-SG1008D 8-Port Gigabit Switch; cat6
PCs running Win 7 Pro (mix of 32 and 64 bit)
Squeezebox Touch and Squeezebox Boom via wireless
Various Samsung TVs via Powerline
duffykins
Seasoned
Seasoned
 
Posts: 525
Joined: Tue Feb 07, 2012 10:28 am

Re: Time backup very slow

Postby robertz » Sun Feb 12, 2012 10:42 pm

Same problem here between two DS411's, the backup has been running for 2 days now. I can see files being replicated so i know it's working.

Didn't take me anywhere near that long to copy all my data onto the primary nas..

CPU maxed out too rsync taking 75% of it, running gigabit.
robertz
I'm New!
I'm New!
 
Posts: 2
Joined: Sun Feb 12, 2012 6:40 pm

Re: Time backup very slow

Postby MrFr33zE » Sun Mar 18, 2012 6:37 am

Same issue here. DS1512+ going to a DS209. I have just over a terrabyte of data. I can see data going to the DS209, but the upload speed on the 1512 is ~300KBps. I have a gigabit network, why is this so slow? I was hoping to do an initial backup with both devices onsite, and then do a remote delta backups over WAN. Is rsync any faster? Any recommendations?
MrFr33zE
Trainee
Trainee
 
Posts: 11
Joined: Tue Nov 23, 2010 7:43 am

Re: Time backup very slow

Postby Stuckster » Mon Jun 18, 2012 1:21 am

We have been having the same poor performance issues with Time Backup. Having just invested over $1000 on two new Synology DS with the express purpose of performing Network backup between the two, we are very disappointed with the results to date.

These forums seem to be full of problems with very few posts offering any solutions for desperate customers. Most likely because the solutions are not known or are not forthcoming.

As such, I sent the following ticket to Synology tech support. I'll let you know what they say.

------------------
<start>
We recently purchased a DS412+ to use as our primary NAS and a DS411j to use as a remote backup unit.

We are in the process of seeding the initial backup using Time Backup over our Gigabit network. Both DS are directly connected to our gigabit router.

The problem it is transferring data at very slow speeds. Only 3.5 MB/s! We have several TB of data to backup and at these speeds it will take several days which is simply not acceptable.

We have had no problem transferring TB's of data manually (not using Time Backup) between both machines at 30-40 MB/s.

The theoretical speed of both machines is between 20-80 MB/s and the theoretical speed of our Gigabit network is 125 MB/s! Yet we are only getting 3.5 MB/s.

Time Backup is also maxing out the CPU on the DS411j, reading 70% for sshd and 30% for Rsync.

At this point we feel very disappointed having just invested over $1000 in Synology products to be receiving such poor performance.

Questions to answer please
---------=--------------
1. Why are transfer speeds using Time Backup SO much slower than manual data transfers?

2. What is sshd and why is it taking up so much CPU?

3. Where is the bottleneck when using Time Backup? Can we make Time Backup perform any faster?

4. If not, please tell us which method of Network Backup will produce the fastest transfer speeds between these two DS over a local gigabit network.
Stuckster
Trainee
Trainee
 
Posts: 16
Joined: Sat Apr 21, 2012 7:03 am

Re: Time backup very slow

Postby Stuckster » Wed Jun 20, 2012 2:27 am

Still waiting to hear back from tech support...

In the meantime since Time Backup was chewing up 100% of the CPU on the DS411j we figured that this was the bottleneck (suspecting poor program design).

So next we tried a straight Network Backup.
With no other apps running on either DS CPU usage on the 411j is only 40%. So that's fixed the CPU bottleneck. Data transfers speeds started out at an agonisingly slow 1.5-3MB/s for the first 5 minutes but then all of a sudden jumped up to around 15MB/s and have been holding steady. Perhaps the first part of the backup is slower dues to indexing of files etc.

So although this is only half the speed of our manual transfers at 30-40MB/s it is still markedly improved over the Time Backup solution.

We will be trying the Shared Folder Sync option next and will report back with transfer speeds.
Stuckster
Trainee
Trainee
 
Posts: 16
Joined: Sat Apr 21, 2012 7:03 am

Re: Time backup very slow

Postby jpit » Wed Jun 20, 2012 5:21 pm

Hi,

I'm seeing also very slow Time Backups. I'm using a DS1812+ with a DX510 expansion unit (connected via e-sata)

Backing up 290 GB of data takes about 3 hours. The hourly 'Smart Recycle' takes 45 minutes, with no changes to any file (this unit is still in test).

The DX510 is configured as RAID 10. DSM version is 4.0-2228

Regards,

Jpit
jpit
I'm New!
I'm New!
 
Posts: 5
Joined: Wed Jun 20, 2012 5:12 pm

Does Time Backup work well after the first slow backup?

Postby mcgillca » Thu Jun 21, 2012 6:29 am

Hi - I have just started using Time Backup to a second Synology server, and was surprised to find out how slow it was.

I was pleased to find this thread and find out that this is normal. My transfer rates are about 2 MB/sec, so its going to take about 3 days to complete the first backup of my ~ 500GB of data.

Once this has been done, I assume Time Back up snapshots will be a lot faster. Have people generally found that this works well, once the pain of the first back up is completed?
mcgillca
I'm New!
I'm New!
 
Posts: 2
Joined: Fri Feb 11, 2011 6:55 pm

Re: Time backup very slow

Postby vortix » Fri Jun 22, 2012 12:28 am

Are all of you that are reporting this problem attempting to start using Time Backup on a source NAS that already has data on it? This is just a theory at this point, but perhaps the slowness is happening because the Time Backup service needs to scan any pre-existing data so it is able to keep track of changed blocks or files in the future. Maybe this initial scan is triggered when the initial Time Backup task is kicked off. If so, I wonder if there's a way to run the initial scan first, and then run the network sync piece later (instead of trying to do both at the same time).

I should be able to prove it one way or another fairly soon. I just purchased two brand new Synology devices. My goal is to set up the Time Backup service on both devices first, before placing any of my data on the devices. I will update this thread when I have more information.
vortix
I'm New!
I'm New!
 
Posts: 2
Joined: Tue Jan 10, 2012 8:26 pm

Re: Time backup very slow

Postby Stuckster » Thu Jun 28, 2012 1:35 pm

Finally got some answers out of tech support once they had consulted the Time Backup dev team.

Here's their response with answers to my original questions...

<start>
We have obtain feedback from developer.

Time Backup uses SSH tunneling, iSCSI device, and rsync to perform snapshot and version backup.
Hence the underlying mechanism is quite different from the normal network backup, so the performance could not be compared.

Why are transfer speeds using Time Backup SO much slower than manual data transfers?
Time Backup uses encrypted tunnel, that is, all data will be transferred through the tunnel with encryption and will be much slower. So if you wish to compare the performance, compare with encrypted network backup instead. This is why sshd consumes a lot of CPU.

What is sshd and why is it taking up so much CPU?
Time Backup uses iSCSI device to do volume level snapshot. The snapshot layer will introduce overhead to the volume read/write, and hence the backup (which is volume read) speed will drop comparing to the normal network backup.
But note that if the backup data set were modified during the backup period, normal backup will fail but Time Backup won't (thanks to the snapshot).

Where is the bottleneck when using Time Backup? Can we make Time Backup perform any faster?
For this case, I believe the bottleneck of the performance would be the CPU of the destination server (i.e. 411j).
The reason is that both SSH tunneling and iSCSI device consumes a lot of CPU resources and the performance of 411j is much lower than 412+, and due to high use of CPU on entry model like DS411J with less powerful CPU and RMA, this will reduce greatly the performance of backup also.


So there you have it. Disappointing to say the least.
Remote backups using Time Backup in particular was the main reason we shelled out over $1000 buying two DiskStations to make the whole thing as easy as possible. Synology can argue that Time Backup DOES work on the DS411j, but at such slow speeds as to make it completely impractical. Yet the DS411j (along with many older, less powerful models) is listed as Time Backup compatible on the slick webpage http://www.synology.com/dsm/home_backup ... backup.php. NO mention of significant performance issues for less powerful models.

Please tell us which method of Network Backup will produce the fastest transfer speeds between these two DS over a local gigabit network.
It's hard to say which backup method is the fastest. For normal network backup, it simply copies the entire data set to the destination. Although you can increase the data transfer speed by disabling encryption, the larger the data set is, the more time it takes to complete the backup. In addition, each backup yields a backup of the entire data set. If there's only a little change to the data set, duplicated copies were made.


Not sure if this is entirely accurate since Network Backup is supposed to perform incremental backups as you can see at the bottom of this page http://forum.synology.com/wiki/index.ph ... rver#Notes. Either way, once the initial backup has been done you're going to be limited by the bandwidth and congestion of your internet connection which for us will top out at about 1MB/s. So really, the main issue is completing the initial backup in an acceptable timeframe.

In contrast, for Time Backup, although the data transfer is slow, but Time Backup preserves "versions."
That is, if a file is not modified between two backups, Time Backup will be copy the file but instead build a had link.
In this way, if there's only a little change to the data set, only the modified part is transferred.


<end>

So it seems that either way Time Machine is going to be very SLOW performing the initial backup, even over a gigabit LAN, because of the built in encryption (unless BOTH of your DiskStations have a powerful CPU).

My Proposed Solution

1. Forget Time Backup (unless you're prepared to wait days/weeks backing up TB's of data at 3MB/s). It seems our DS411j simply isn't powerful enough to run it, which is very disappointing considering it's a 2011 model!

2. We'll try seeding the initial backup using Network Backup directly over our LAN with NO encryption (to keep the CPU down and speed high).

3. Move the backup unit to the remote location.

4. Forward the recommended ports 873 and 22 for encrypted backups http://forum.synology.com/wiki/index.ph ... ync_Server

5. Reconnect the backup set to the remote DiskStation's new IP address.

6. To get Network Backup to perform similarly to Time Backup, edit the backup set and check the following options
"Enable Transfer Encryption"
"Enable Block-level Backup"
"Reserve the Backed up files at the destination"

7. Schedule backups for times when we won't be changing the data. Network backups main disadvantage vs Time Backup appears to be that it cannot handle changes to the data WHILE a backup is in progress. Scheduling backups during down times should negate this issue.

-----------------------------------

So we'll give this a go. The saga of network backups continues...

I hope this will help some of you with similar issues.

But still pretty disappointing that a "Synology approved" app will only run effectively on a handful of the more powerful DiskStations. I really feel this should be mentioned in the documentation. :cry:
Stuckster
Trainee
Trainee
 
Posts: 16
Joined: Sat Apr 21, 2012 7:03 am

Re: Time backup very slow

Postby Rancid » Sun Jul 01, 2012 8:48 am

i think the comments from synology about CPU being an issue are misleading.

Im running time backup from a RS3411 to a DS1511 on the same GBE switch and get shockingly poor performance. The CPU on both barley registers any load.

last nights incremental backup took 10 hours and our initial data set is only 2.5tb.
and this was on a Saturday where the incremental differeance from the backup on Friday was probably less then 200mb's

So 10 hours to backup 200mb of data ?

Shocking.

i know you have spent $1000+ to do this and it should do exactly as it says its supposed to do.

Ive spent nearly £10,000 out of my budget on 1x RS3411(20tb) 1xDS511(10tb) 1xDS1512(10tb).
All with extended warrenties and onsite service.
Support from Synology is shameful.
i think the main reason no support is offered in the forums is because it would very quickly expose the contradiction and misleading advice they give.
Rancid
Sharp
Sharp
 
Posts: 168
Joined: Mon Mar 28, 2011 1:32 pm

Re: Time backup very slow

Postby Stuckster » Mon Jul 02, 2012 2:56 am

@Rancid

That's terrible. Have you submitted a support ticket? I would love to hear their response in your case with the CPU bottleneck removed from question.

We recently chose to go with Synology based on rumours of easy to use products and excellent tech suport. But for a company that makes NAS units aimed at the SMB and consumer market we have found the support and documentation to be extremely lacking. I consider us to be quite competent when it comes to anything tech or internet related, but we are certainly NOT Network Administrators. I have lost count of the number of tickets we have had to submit simply because processes and features have not been documented clearly or at all.

I for one am not impressed so far. Synology needs to know that this is not good enough. If they don't start listening to their customers and addressing our concerns they're going to find their "good" reputation going down the toilet overnight.

What are the other thousands of Synology users using for remote backups?? We can't be the only handful of users who have experienced this problem in setting up remote backups. What is everyone else using for remote backups. Have they got them to run effectively?
Stuckster
Trainee
Trainee
 
Posts: 16
Joined: Sat Apr 21, 2012 7:03 am

Re: Time backup very slow

Postby Rancid » Mon Jul 02, 2012 8:29 am

i tried that logging a ticket thing a couple of times, end result was an overwhelming urge to hit something.

Unlike yourself i have 20 years plus of IT experience.
I have certification from Microsoft/Vmware/Cisco/ITIL.
I'm very comfortable around IT and always have been.

However i still find the quality of support and documentation to be woefully poor.
So rest assured its not your lack of being an experienced IT pro that is the problem here.
Rancid
Sharp
Sharp
 
Posts: 168
Joined: Mon Mar 28, 2011 1:32 pm

Next

Return to Time Backup

Who is online

Users browsing this forum: No registered users and 0 guests