SHR (RAID5) reshape speed extremely slow

Questions pertaining to Power settings, Auto Block, Permissions, User Quotas and Email alerts may be posted here.
Forum rules
1) This is a user forum for Synology users to share experience/help out each other: if you need direct assistance from the Synology technical support team, please use the following form:
https://myds.synology.com/support/suppo ... p?lang=enu
2) To avoid putting users' DiskStation at risk, please don't paste links to any patches provided by our Support team as we will systematically remove them. Our Support team will provide the correct patch for your DiskStation model.
User avatar
sirdharma
Beginner
Beginner
Posts: 26
Joined: Wed Oct 13, 2010 7:30 am
Location: Paris, France

SHR (RAID5) reshape speed extremely slow

Postby sirdharma » Sat Jan 18, 2014 1:54 pm

Hello,

I have recently swapped one of the hard drives in my DS413. The system has been repairing the volume for 15 days now, which was not an issue as I have been away from home but now that I am back I would want to investigate what is going wrong and speed things up.

All the services are disabled so there should be very little — if not at all — I/O on the volume and all the hard drives have normal S.M.A.R.T. status (as reported in the DSM).

Server load is around 3.0, and I/O wait constantly above 50%.

mdstat reports a reshaping speed around 320K/sec, and another 7 days before it finishes.

Code: Select all

$ cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md7 : active raid1 sdb10[0] sdc10[1]
      976742784 blocks super 1.2 [2/2] [UU]
     
md6 : active raid5 sdd9[3] sdb9[0] sdc9[2] sda9[1]
      3906989568 blocks super 1.2 level 5, 64k chunk, algorithm 2 [4/4] [UUUU]
      [=================>...]  reshape = 89.2% (1743130724/1953494784) finish=11442.6min speed=306K/sec
     
md5 : active raid5 sdd8[4] sdc8[3] sda8[2] sdb8[1]
      976733568 blocks super 1.2 level 5, 64k chunk, algorithm 2 [4/4] [UUUU]
         resync=DELAYED
     
md4 : active raid5 sdd7[5] sda7[3] sdb7[2] sdc7[4]
      527418624 blocks super 1.2 level 5, 64k chunk, algorithm 2 [4/4] [UUUU]
     
md3 : active raid5 sdd6[6] sdb6[3] sdc6[5] sda6[4]
      205090368 blocks super 1.2 level 5, 64k chunk, algorithm 2 [4/4] [UUUU]
     
md2 : active raid5 sdd5[7] sda5[6] sdc5[4] sdb5[5]
      718391040 blocks super 1.2 level 5, 64k chunk, algorithm 2 [4/4] [UUUU]
     
md1 : active raid1 sdd2[3] sda2[0] sdb2[1] sdc2[2]
      2097088 blocks [4/4] [UUUU]
     
md0 : active raid1 sdd1[3] sda1[0] sdb1[1] sdc1[2]
      2490176 blocks [4/4] [UUUU]
     
unused devices: <none>

I have tried tinkering with the values in /proc/sys/dev/raid/speed_limit_min, /proc/sys/dev/raid/speed_limit_max and /sys/block/md6/md/stripe_cache_size but none of the changes seem to have any effect at all on the reshaping speed. Current values are the following:

Code: Select all

$ cat /proc/sys/dev/raid/speed_limit_min
100000

Code: Select all

$ cat /proc/sys/dev/raid/speed_limit_max
0

Code: Select all

$ cat /sys/block/md6/md/stripe_cache_size
16384

Would you have any idea what is going wrong and how I can increase the reshaping speed?

Thanks!
Ate_Bit
Seasoned
Seasoned
Posts: 578
Joined: Mon Feb 11, 2013 1:55 am

Re: SHR (RAID5) reshape speed extremely slow

Postby Ate_Bit » Mon Jan 20, 2014 6:49 pm

Had the volume been scrubbed recently before you swapped the disk?

Two weeks ago I did the same thing on my DS413. I replaced a 1.5TB drive with a 3TB drive and the SHR rebuild and expansion took between 24-36 hours.

However, I scrubbed the parity right before I swapped the drive. I read another thread where the rebuild slowed down. The only thing that I can think of is that it is somehow trying to compensate for possibly bad parity data. But in an SHR-1 scenario, this should result in data loss, so I don't know for sure.
NAS: DS413 running DSM 6.0
Configuration: SHR, with 1 disk data protection
HDD: 3x4TB
User avatar
sirdharma
Beginner
Beginner
Posts: 26
Joined: Wed Oct 13, 2010 7:30 am
Location: Paris, France

Re: SHR (RAID5) reshape speed extremely slow

Postby sirdharma » Mon Jan 20, 2014 10:59 pm

Ate_Bit wrote:Had the volume been scrubbed recently before you swapped the disk?

Two weeks ago I did the same thing on my DS413. I replaced a 1.5TB drive with a 3TB drive and the SHR rebuild and expansion took between 24-36 hours.

However, I scrubbed the parity right before I swapped the drive. I read another thread where the rebuild slowed down. The only thing that I can think of is that it is somehow trying to compensate for possibly bad parity data. But in an SHR-1 scenario, this should result in data loss, so I don't know for sure.


Hi Ate_Bit,

Thanks for your answer. I do not think the volume had been scrubbed prior to changing the disk. I have about 5 days left to go so I guess I will just wait and see. If it does finish, repairing and expanding the volume would have taken 24 days!

Return to “System Management”

Who is online

Users browsing this forum: No registered users and 2 guests