How to access iSCSI targets temporarily
When first migrating from my existing OpenFiler based NAS to Synology, I took the opportunity to do some maintenance on my volumes.
In particular, I had an iSCSI target LUN that was partitioned with several LVM physical volumes in a single volume group but only one logical volume (an artifact of my disk usage growing over the years). Given that there is already logical volume management at the server level, this seemed unnecessary and I wanted to migrate to having the LUN be unpartitioned with the filesystem image directly on the target.
This was done using DSM 4.1.
To make this happen I would need to do the following:
- Attach the old iSCSI target to the DiskStation (e.g. DiskStation as the iSCSI initiator)
- Get LVM to recognize the volume group (vg) on the old target
- Bit copy (using dd) the logical volume (lv) to the new iSCSI LUN on the DiskStation
I'm going to skip the steps where I created the iSCSI LUNs in DSM, since those were done using the normal mechanism. I'm also going to skip the steps where I actually did the dd, since once we have the device files, there's nothing special about this use of dd.
DiskStation as iSCSI Initiator
All the necessary kernel modules and tools to attach an iSCSI target to the DiskStation are present in DSM 4.1, but require some setup to get access to.
You need to load some additional kernel modules out of /lib/modules:
insmod libiscsi.ko insmod libiscsi_tcp.ko insmod iscsi_tcp.ko
Set the Initiator Name for your DiskStation (e.g. the name used by the DiskStation's iSCSI initiator to identify it to your old target):
All you need is one line in the file:
Now you can use iscsiadm to attach to your existing iSCSI target (if you use CHAP authentication, read up on iscsiadm in more detail, I didn't try it):
iscsiadm -m discovery -t sendtargets -p <IP address of old server> iscsiadm -m node -t <target IQN> --login
In dmesg you should see something like:
[56619.232296] got iSCSI disk [56619.236209] sd 21:0:0:0: Attached scsi generic sg9 type 0 [56619.241755] sd 21:0:0:0: [isdb] 838860800 512-byte logical blocks: (429 GB/400 GiB) [56619.254585] sd 21:0:0:0: [isdb] Write Protect is off [56619.259560] sd 21:0:0:0: [isdb] Mode Sense: 3b 00 00 00 [56619.261146] sd 21:0:0:0: [isdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA [56619.308130] isdb: isdb1 isdb2 < isdb5 > [56619.314946] sd 21:0:0:0: [isdb] Attached SCSI disk
This tells you that the disk got attached to /dev/isdb and we're good to go
Accessing LVM volume groups off the newly attached disk
By default, the DiskStation does not have the vgscan tool and has its lvm.conf set to only look at its own md devices. Again, all the tools we need to access a foreign vg are present, but require some setup.
In reality, the various lvm tools like vgchange, vgscan, vgdisplay, etc... are all just symlinks to a master binary called lvm. This binary is present, it's only the symlink for vgscan that is missing. So just add it:
cd /sbin ln -s lvm vgscan
Then we need to change the default filter in /etc/lvm/lvm.conf to include /dev/isd devices. To do this, open /etc/lvm/lvm.conf in vi and look for the following line:
filter = [ "r|^/dev/md0$|", "r|^/dev/md1$|","a|/dev/md|", "r/.*/" ]
You'll need to add an accept rule for /dev/isd, which means changing that line to look like:
filter = [ "r|^/dev/md0$|", "r|^/dev/md1$|","a|/dev/md|", "a|/dev/isd|", "r/.*/" ]
After saving this file, vgscan will now search your /dev/isd* devices for vg's and pv's. So let's find the vg on the iSCSI target and then activate it with:
vgscan vgchange -ay <VG name>
This should activate the VG and the logical volumes in the VG will show up in /dev/<vg name>/ and in /dev/mapper/
Cleanup is easy, the main things are to deactivate the vg and then log out (e.g. unattach) from the iSCSI target:
vgchange -an <VG name> iscsiadm -m node -U all
If you want, you can undo your changes to the config files as well. Delete /etc/iscsi/initiatorname.iscsi and restore the original filter configuration in /etc/lvm/lvm.conf.
If that partitioned LUN containing my LVM physical volumes were local rather than remote, in theory I could've done the same kind of transfer using kpartx to map the individual partitions inside of the device file to device files of their own and then used vgscan to detect them.
However, I was unable to find a good way to get kpartx onto the DiskStation. Even though I have an Intel based DiskStation, there seems to be a mix of 32-bit and 64-bit libraries and binaries on the box and a particular version of libdevicemapper.so (in 32-bit) that makes it hard to find a matching binary for.
There is an alternative though! If you can get that file transfered over to an iSCSI LUN on the Diskstation's own iSCSI target, you can then use the above directions to attach to that LUN over localhost. The partitions inside that file will then show up as /dev/isd* nodes when you login.
Access to this LUN will be done over the loopback (lo0) interface than one of the eth* interfaces, which makes this access faster than the remote case, although probably not as fast as if I could get a working kpartx. Still, I was able to get 90 megabytes/second throughput doing a simultaneous read/write to the same disk group (copying from one file on the disk group to another file on the same disk group) this way.