Amazon S3: better log info during backups?

Discuss backup and restore functions of the DiskStation with respect other DiskStations, USB/eSATA, Amazon S3, network backup, or other rsync clients.

Amazon S3: better log info during backups?

Postby glc650 » Sun Mar 25, 2012 7:51 am

Hi,

Anyway to get more log info when a backup to Amazon is running? "Backing up..." is a pretty useless status indication. Anyway to see more log info like which file/directory is currently being uploaded? How much data is slated to be transferred and how much is left to transfer? etc.

Thanks,

->g.
glc650
Novice
Novice
 
Posts: 53
Joined: Sun Jan 23, 2011 10:29 pm

Re: Amazon S3: better log info during backups?

Postby leahcimp » Mon Mar 26, 2012 11:55 am

When I backup, it does show the current backed up amount vs total amount of data (at least it did in DSM 3.2; I haven't observed it since upgrading to DSM 4). However, in DSM 3.2 there was a bug and the calculation was incorrect, so it was pretty useless.

In general, S3 backup deserves better logging support from Synology. In the summary email that is sent after a backup, and in the event log, I'd like to see basic stats such as:

    - Total backup size
    - New files backed up
    - Changed files backed up
leahcimp
Trainee
Trainee
 
Posts: 13
Joined: Tue Sep 14, 2010 10:28 pm
Location: Atlanta, GA

Re: Amazon S3: better log info during backups?

Postby glc650 » Mon Mar 26, 2012 5:04 pm

leahcimp wrote:When I backup, it does show the current backed up amount vs total amount of data
I'm not seeing that in DSM 4.0. Only status I see once backup starts is "Backing up...". DSM 4.0 is the first time I used Amazon so I have nothing to compare to.
glc650
Novice
Novice
 
Posts: 53
Joined: Sun Jan 23, 2011 10:29 pm

Re: Amazon S3: better log info during backups?

Postby leahcimp » Tue Mar 27, 2012 12:41 am

glc650 wrote:
leahcimp wrote:When I backup, it does show the current backed up amount vs total amount of data
I'm not seeing that in DSM 4.0. Only status I see once backup starts is "Backing up...". DSM 4.0 is the first time I used Amazon so I have nothing to compare to.


I just checked, and sure enough they removed the small amount of backup detail in DSM 4.0. It just says "Backing up...". I guess DSM 3.2 was better for something, even if the calculation wasn't quite right.

Synology: If you're reading this, PLEASE add some detail to S3 backups! Can't possibly be that hard to add this info to a backup log.
leahcimp
Trainee
Trainee
 
Posts: 13
Joined: Tue Sep 14, 2010 10:28 pm
Location: Atlanta, GA

Re: Amazon S3: better log info during backups?

Postby glc650 » Sat Mar 31, 2012 5:00 am

Well this is all a moot point anyway since the Amazon backup feature is useless. Every time it runs all my existing files (even ones that haven't changed) on Amazon get re-uploaded and replaced.
glc650
Novice
Novice
 
Posts: 53
Joined: Sun Jan 23, 2011 10:29 pm

Re: Amazon S3: better log info during backups?

Postby leahcimp » Sat Mar 31, 2012 10:26 pm

glc650 wrote:Well this is all a moot point anyway since the Amazon backup feature is useless. Every time it runs all my existing files (even ones that haven't changed) on Amazon get re-uploaded and replaced.


There is a long thread about this (http://forum.synology.com/enu/viewtopic.php?f=160&t=46315). People seem to have different results. For me, only the changed files are backed up.
leahcimp
Trainee
Trainee
 
Posts: 13
Joined: Tue Sep 14, 2010 10:28 pm
Location: Atlanta, GA

Re: Amazon S3: better log info during backups?

Postby glc650 » Fri Apr 13, 2012 8:55 am

leahcimp wrote:
glc650 wrote:Well this is all a moot point anyway since the Amazon backup feature is useless. Every time it runs all my existing files (even ones that haven't changed) on Amazon get re-uploaded and replaced.


There is a long thread about this (http://forum.synology.com/enu/viewtopic.php?f=160&t=46315). People seem to have different results. For me, only the changed files are backed up.
Yea I noticed that thread among others. Support also told me to check the same box.

What I ended up doing is just starting over. I manually deleted all my data on Amazon including the bucket (I wanted to use a different name anyway), deleted the backup task and created a new one with the same options as before (including *not* checking the "Reserve Backed Up Files at Destination" option).

But this time instead of selecting all the data I wanted to backup to Amazon right off the bat I backed up my data in stages by either selecting a share or a few directories within a share, backing that up, then selecting more folders/shares to backup or by using File Station to move my data out of my selected shares/folders to limit the amount of data uploaded each time (the method depended on whether or not I wanted newly added files/folders to be automatically included in the next backup). I ran the backup right after each portion of data was backed up just to verify it worked and to re-upload any files that were added/changed during the previous backup.

Today I finished uploading all desired data to Amazon and just finished running my last backup after enabling server side encryption and standard storage on all my data. Will see how it goes from here on out.
glc650
Novice
Novice
 
Posts: 53
Joined: Sun Jan 23, 2011 10:29 pm


Return to Backup/Restore for DiskStation to DiskStation/USB/eSATA/Off-site backups

Who is online

Users browsing this forum: No registered users and 1 guest