How often should you scrub and or defrag?

Questions pertaining to Power settings, Auto Block, Permissions, User Quotas and Email alerts may be posted here.
Forum rules
1) This is a user forum for Synology users to share experience/help out each other: if you need direct assistance from the Synology technical support team, please use the following form:
https://myds.synology.com/support/suppo ... p?lang=enu
2) To avoid putting users' DiskStation at risk, please don't paste links to any patches provided by our Support team as we will systematically remove them. Our Support team will provide the correct patch for your DiskStation model.
User avatar
Cheney Coker
Beginner
Beginner
Posts: 20
Joined: Tue Apr 25, 2017 7:13 pm

How often should you scrub and or defrag?

Postby Cheney Coker » Tue May 02, 2017 1:00 am

Currently it takes six days to so a data scrub. I don't now how long a defrag takes, it is still running after 2.6 million files. For you folks that have been doing this longer than I have, is there a good rule of thumb on how frequently one should do these things? I can't scrub weekly or it would be the only thing it did. Pretty mich the same for monthly. Every other month? If there a way to know what the scrubbing did? Does it keep logs? How about defrag?
DS2413+ 12-bay with seven 5TB drives, 4gig ram. Less than 20tb in use, mostly for media. This is being streamed by PLEX from an older Dell server with six Xeon 2.66ghz, 18 gig ram.
dima
Trainee
Trainee
Posts: 10
Joined: Fri Feb 24, 2017 11:02 pm

Re: How often should you scrub and or defrag?

Postby dima » Wed May 17, 2017 4:18 pm

Did you ever figure this out? I just got a DS916+ and set it up with SHR-2/Btrfs and am also trying to figure out how often to scrub, defrag, and run the quick and full SMART checks.
FordGuy61
Experienced
Experienced
Posts: 128
Joined: Thu Oct 16, 2014 1:47 am

Re: How often should you scrub and or defrag?

Postby FordGuy61 » Wed May 17, 2017 7:16 pm

I've a DS414 with 4x4TB drive in SHR1.

I run quick SMART test every night. They take just a minute or so per drive.

I run the extended SMART tests once a month. The tests are scheduled to run on a different day for each drive. Given the CPU in my NAS and the size of the drive, the extended test can take ~24 hours. SMB transfer speed is also reduced by 1/3 to 1/2 during an extended test. YMMV, especially with a different CPU & filesystem.

I scrub the disks once per month as well. A scrub takes 4 - 5 days and reduces SMB transfer speed, otherwise I'd probably run it more often. The scrubbing task hits the CPU quite a bit, so with a newer/faster processor, more RAM, etc., it may not take as long.

Extended SMART checks and RAID scrubbing have both found problems in the past and let me replace drives before they completely failed. The ability to schedule scrubbing was a welcome addition to the latest version of DSM.

I don't remember the quick SMART test ever failing or finding an issue. However, it has zero impact on my system, so there is no downside to running it every night.
dima
Trainee
Trainee
Posts: 10
Joined: Fri Feb 24, 2017 11:02 pm

Re: How often should you scrub and or defrag?

Postby dima » Wed May 17, 2017 7:24 pm

Thanks for the insight!
Thor68
Apprentice
Apprentice
Posts: 95
Joined: Sun Oct 09, 2011 8:09 pm
Location: Oslo, Norway

Re: How often should you scrub and or defrag?

Postby Thor68 » Fri May 19, 2017 7:06 pm

I just tried the new scheduled raid scrubbing, and it finished in roughly 20 hours on my system. The scheduled raid scrubbing have no options to configure, so I don't know if it differs from manuell scrubbing, but I seem to remember that manuell scrubbing took longer time.

20 hours once a month is something I can accept, so I set my scheduler to this interval.
DS1815+ [16GB RAM / SHR-1 (6x4TB HGST, ext4) / DSM 6.1.2-15132] Primary server
DS1812+ [3GB RAM / SHR-1 (btrfs) / DSM 6.1.2-15132] - Backup for the 1815

DS1511+ [3GB RAM / SHR-1 / DSM 6.0.2-8451-9] - Retired
ubronan
I'm New!
I'm New!
Posts: 9
Joined: Fri Dec 04, 2015 9:43 pm

Re: How often should you scrub and or defrag?

Postby ubronan » Sun May 21, 2017 11:55 am

I have a RS2414+ running 8 x 4 Tb in raid 6 with 2 60 gb cache intel ssd's ( i know they are kinda overkill 2 x 30 would have been plenty but i got them very cheap).
Thinking about switching to btrfs when i replace the 4 Tb drives for 6 x 6 TB ones ofcourse i will use HGST drives.
Scrubbing takes about 2 days not sure if the cache helps, but thinking about scrubbing my my old 411, 411j, 412, 414+ with 4 x 4 TB in raid 5 it seems to help alot the 414+ took 5 days to scrub. Ofcourse the older ones it took much longer because they where sata1 - 150 Mbps the 414+ has sata 2 - 300 Mbps
I have read somewhere that you could speed this process up by changing some value in you system, but it comes with a risk if the system fails or crashes you loose data.
Those who want to know this trick should search the forum about it, but i think you should avoid it if not having a UPS to protect against power failure.
But i learned that even this kinda trustworthy synology product once in a while could crash/stop ...
Anyway it seems to make scrubbing and rebuilding happen in hours instead of days. But ofcourse the risk of loosing data is not something to take lightly.
I do not scrub constant because i kinda know exact how much data has been changed/moved/deleted or added so when alot has been changed i start a scrub no matter how long ago it was. In reality this happens about 1 to 2 times in 3 months.
The 2414+ shows exactly if drives start to degrade in the storage manager and in the dock when i login which i actually do at least 1 time in 2 weeks
The simple smart checks are for me done auto magical by the nas, i see no added security in a full check too often because the simple ones are enough.
The advanced smart check are done by me about every 6 months, however i think most people have no clue what they are looking at with all these numbers.
I forgot if my smaller nas models did the simple smart checks auto, but i would recommend to do them if not.
However there is really no need to do them daily, with the exception that if you have older drives it becomes a must.
I did a simple smart check every 3 to 4 days in the older nas models
They really can warn in time for disaster strikes, i am pretty happy that most of these tasks are done auto on this rs2414+
Every disk fail i had in any of my synology's was detected by the simple smart test.
I have been running servers and nas systems for about 30 years and i guess i have been pretty lucky for a long time
In all these years i had only 6 drive disasters ( except the early ssd rampage ;) ) and had much luck running drives up to 18 years without failures.
To be honest the old WD ide drives i had was 18 years old when i brought them to be scrapped because nobody wanted to use these oldies anymore.
6 months ago 3 year old WD 2 TB drives which i used for short term storage in raid 0 was detected to become a risk... dsm messaged to replace them urgently
Because these where non important drives i kept them running to see what happened, well 1 died all of a sudden 2 months later.
Crashing the volume so trust the warnings from the nas .. they are right no matter when but disaster will strike when it warns you :).

For example the old intel cache ssd's i placed changed a week back from 100% reliable to 99% reliable and yes thats shows in the storage manager when i open it up.
As soon as it changes into a more serious error i know i have to remove them. Btw these intel ssd's are the first 60 Gb models intel released yes indeed AGES old.
User avatar
Cheney Coker
Beginner
Beginner
Posts: 20
Joined: Tue Apr 25, 2017 7:13 pm

Re: How often should you scrub and or defrag?

Postby Cheney Coker » Fri May 26, 2017 5:53 pm

FordGuy61 wrote:I've a DS414 with 4x4TB drive in SHR1.

I run quick SMART test every night. They take just a minute or so per drive.

I run the extended SMART tests once a month. The tests are scheduled to run on a different day for each drive. Given the CPU in my NAS and the size of the drive, the extended test can take ~24 hours. SMB transfer speed is also reduced by 1/3 to 1/2 during an extended test. YMMV, especially with a different CPU & filesystem.

I scrub the disks once per month as well. A scrub takes 4 - 5 days and reduces SMB transfer speed, otherwise I'd probably run it more often. The scrubbing task hits the CPU quite a bit, so with a newer/faster processor, more RAM, etc., it may not take as long.

Extended SMART checks and RAID scrubbing have both found problems in the past and let me replace drives before they completely failed. The ability to schedule scrubbing was a welcome addition to the latest version of DSM.

I don't remember the quick SMART test ever failing or finding an issue. However, it has zero impact on my system, so there is no downside to running it every night.



Thanks for this. Made excellent sense.
DS2413+ 12-bay with seven 5TB drives, 4gig ram. Less than 20tb in use, mostly for media. This is being streamed by PLEX from an older Dell server with six Xeon 2.66ghz, 18 gig ram.
cfuttrup
Specialist
Specialist
Posts: 1198
Joined: Fri May 09, 2008 8:01 pm
Location: Denmark
Contact:

Re: How often should you scrub and or defrag?

Postby cfuttrup » Fri May 26, 2017 6:09 pm

Cheney Coker wrote:Currently it takes six days to so a data scrub. I don't now how long a defrag takes, it is still running after 2.6 million files. For you folks that have been doing this longer than I have, is there a good rule of thumb on how frequently one should do these things? I can't scrub weekly or it would be the only thing it did. Pretty mich the same for monthly. Every other month? If there a way to know what the scrubbing did? Does it keep logs? How about defrag?


I have never scrubbed (I'm running RAID1 and RAID10 on my machines) or defragged my drives.

Consider this - if you mess around with files a lot, then it makes sense to do it more frequently ... but if not, once a year can be good enough. The more often you do it, the less effect it has ...

Best regards,
Claus

Return to “System Management”

Who is online

Users browsing this forum: No registered users and 3 guests