data verification after migration?

[lgm]

I just recently copied all my data from my old data disk (now attached to the DS via eSATA) to the new two disk SHR setup. The copy took about half a day since there's almost 2T of data.

Is there a way to verify that the copied data is consistent with the original copy?

Thank you.

[LocoDelColor]

You could run a diff.

[lgm]

ok, this may seem silly but how do i run a diff?

[LocoDelColor]

I thought about this for a couple more minutes. Diff would work but rsync would probably work better. That said, I'm a little reticent to give specific instructions on how to do it, since you'd be better off learning how to do it yourself and I don't know where DS mounts an external drive. And I'm just doing this off the top of my head. So, the command would be something like this, assuming that the external drive is mounted as /volume2:

rsync -a -W -n -v --exclude=.DS_Store --exclude=@eadir /volume2/ /volume1/

The -n means "dry run." This would generate a list of files that are different between volume1 and volume2. If you want to want to actually run the sync, remove the -n.

[SirMaster]

To really determine if the data is the same you need to hash each file with a good enough hash like MD5.

I've used this before with great success.
http://eggdrop.ch/blog/2008/02/17/compare-directories/

[lgm]

thanks for the detailed reply, LocoDeiColor.

i was looking at diff and thought it would probably take forever the compare 2T of data line by line, especially since it does not seem to take dirs as inputs.

as to resync, can i specify a dir under volume1 or 2 instead of the whole volume?

i was really just looking to do a parity check on all files copied. a rsync dry run would only give a list of missing files and a rsync, it seems to me, would do the copy all over again? which still leaves me with a unverified copy.

[lgm]

To really determine if the data is the same you need to hash each file with a good enough hash like MD5.

I've used this before with great success.
http://eggdrop.ch/blog/2008/02/17/compare-directories/

Thanks, SirMaster. I will give it a try. Sounds a lot like what I wanted.

[LocoDelColor]

thanks for the detailed reply, LocoDeiColor.

i was looking at diff and thought it would probably take forever the compare 2T of data line by line, especially since it does not seem to take dirs as inputs.

Diff does accept directories as inputs. I just did

Code: Select all

cd /volume1
diff -r music photo

and it's still giving me output. So you could do [code]diff -r source target > /volume1/admin/filename.txt to have the results piped to a file.

as to resync, can i specify a dir under volume1 or 2 instead of the whole volume?

You can specify whatever you want.

i was really just looking to do a parity check on all files copied. a rsync dry run would only give a list of missing files and a rsync, it seems to me, would do the copy all over again? which still leaves me with a unverified copy.

That's why I suggested using rsync instead. Rsync uses MD5 to hash the files and make sure they match. It then only copies what doesn't match. A little known caveat of rsync, though, is that you need to use a slash after the source directory, or it's going to include the directory in the sync.