Low perf in NFS and ISCSI vs CIFS in DSM 2.2 and RS407

[vm.franklin]

Greetings all

Just as a thought to use the DiskStation with ESXi Hypervisor - you may install the the OS on a datastore of ESXi - which can be hosted by the DiskStation. Once you have an OS operating, within this virtual world, if it access a samba share, or NFS, or iSCSI, the performance would be as normal/expected within this virtual world. Case in point, a Virtual Linux accessing an iSCSI Target of the DS1010+ achieves around 40MB/Sec writing.

To backup contents of this Virtual computer, using the traditional physical desktop methods of conducting backups within the Virtual world still apply - where you may use DR3 or Arconis TrueImage or Symantec BackupExec to conduct your backups.

Hope this helps.

[rs407]

Since Synology have announced the VMware compability on 1010+ (and coming RS810) with FW 2.3.1161 and the only "offical" change between 2.3.1161 and 2.3.1157 are "Fixed the abnormal startup issue of scheduled power on function" I got the Idea to test if there was any performance improvements between 2.3.1141 and 2.3.1157 for the RS407.

Unfortunately I didnt find any improvments in performance regarding NFS and ISCSI in RS407 to ESXI 3.5U5 between 2.3.1141 and 2.3.1157 , the big performance bottleneck are writes to disk.

But with the new "vmware" certification Its hopfully safe to use 1010+ and the new (not yet released) RS810, maybe we can hope for a 2.3.1161 for other products.

My test was made with 3.5U5 beacuse I didnt have any Vshere 4 Vmservers avalible for easy testing

[jgallagher]

We are now well into the second half of 2010. Any news on a update to address this issue? Any Beta firmware we can test?

[rs407]

FYI
I did some simple tests with and without disk aligment on the vm server (both windows and linux vm:s).
The setup was the RS407 (DSM 2.3-1157) and ESXI 3.5 (Update 4) on a old HP server DL380 G3 with dual CPU with dual core and 12 GB of RAM
The change in % are the intresting part with and without disk aligment (not the actual performance numbers)
VM Windows server 2003 (test tool nbench)- 1024KB offset
Disk via ESXI<->ISCSI<->RS407 Improvments with disk aligment (Read/Write) = -5%/+5% (RS407 CPU load decrease from 90% to <70%)
Disk via ESXI<->NFS<->RS407 Improvments with disk aligment (Read/Write) = +14%/+126% (write performance x2!,RS407 CPU load decrease from 60% to 30%)
In pure performance NFS have better performance then ISCSI in this tests (60% in read and 80% in write)

VM Linux Novell SLES 11 (test tool iozone) - 16384 KB offset
Disk via ESXI<->ISCSI<->RS407 Improvments with disk aligment (Read/Write) = +5%/+415% ( write performance x4!,no change in RS407 CPU load)
Disk via ESXI<->NFS<->RS407 Improvments with disk aligment (Read/Write) = -1%/-3% (RS407 CPU load increase from 30% to 55%)
In pure performance NFS have better performance then ISCSI in this tests (11% in read and equal in write)
In this test its seems that the (big) offset have a negative impact on the NFS performance.

Conclusion
Disk aligment seems to improve performance, however the offest used in Linux test should probably been lower (equal to windows offset) and then the same performance increase whould be expected since the vm:s dont access the RS407 direct, its all done via ESXI.

Disk aligment must be done when partition are created before format.

Commands to make disk aligment
*Windows 2003 (basic disks)
-Use diskpart in command prompt
list disk
select disk (your disk nr)
create partition primary align=1024
list partition

*Linux
fdisk /dev/(your disk, example /dev/sdg)
u (change display units to sectors)
p (list disk)
n (create new partition)
p (primary)
1 (number 1)
2048 (offset= aligment 1024 KB)
<enter> (use defaults)
p (list disk)
w (save change and exit)

Edit 2010-09-02
I made new test with linux (compare disk aligment and non aligment)
VM Linux Novell SLES 11 (test tool iozone) - 1024 KB offset
Disk via ESXI<->ISCSI<->RS407 Improvments with disk aligment (Read/Write) = +20%/+304% ( write performance x3!,no change in RS407 CPU load)
Disk via ESXI<->NFS<->RS407 Improvments with disk aligment (Read/Write) = +6%/-8% (RS407 CPU load increase from 30% to 80%)
In pure performance NFS have better performance then ISCSI on write (+20%) and ISCSI have better in read +7%
The aligment dosent seem to have any positive impact on Linux-ESXI-NFS but on ISCSI and 1024KB offset seems to be better than 16384 KB.
The strange part is that the test shows positive impact on vm-windows 2003. (W+126%,R +14%)
I have no answer why disk aligment seems to have a negative impact on ESXI NFS for virtual Linux host but not for windows,maybe its something in the setup/testing tools on the different platforms.

In test to local disk on the ESXI (hardware raid1) I got better performance from a VM-Linux
16384 Write +10% Read +8%
1024 Write +5% Read +24%

[jgallagher]

Anyone care to benchmark with the new 3.0 that was just released?

[rs407]

Hi, I tested DSM 3.0 in my simple testbed, with and without asyncron setting in RS407.
Unfortunable I didnt see any improvments but it might be my testbed.

Edit 2010-11-04
I did some more testing today with the same testbed (VM windows 2003 server,ESXI 3.5U5,non aligned discs and NBench) but with new resaults.
Bottom line is that DSM 3.0 Rocks when it comes to NFS and ESXI 3.5.
I got 10 times better throughput on writes (no change in read) with DSM 3.0 and asyncron settings in NFS priviliges compared to older DSM versions.
The NFS performance was equal (sometimes better) to CIFS.
ISCSI still have low performance on writes in RS407.(no change in DSM 3.0)

Conclusion
The performance of ESXI 3.5 and RS407 with DSM 3.0 in NFS are equal to CIFS when asyncron NFS settings is used.
RS407 are a old NAS but performs well in my tests.
The new RS810 with a lot more processing power and memory will probably only improve the performance.
I expect ISCSI also to improve in RS810 vs RS407.
However I prefeer NFS when used with ESXI than ISCSI.(simplicity and "no" vm per volume restrictions compared to FC/ISCSI LUNS)

Now we feel that we can use synology as storage for ESXI.
The next challange is how to handle redundancy for the storage but thats an other topic.

I have no answer to why I didnt get this good resaults the first time I tested.

Edit 2010-12-01
After running more test (lager files and more threads) I have to change my statment above.
In DSM 3.0 the NFS have improved but its not as good as the CIFS performance, when I increase the load CIFS can perform 60% better in write than NFS and 10% better in Read, NFS is still useful but CIFS performs better.
When I increase the load on ISCSI the resource monitor in DSM 3.0 stops uppdating (Memory and/or CPU load?).
ISCSI in RS407 and ESXI 3.5U5 isnt usable.

But im not so sure that the RS810 will be our next low-end ESXI storage.

/R

[jgallagher]

jumbo frames on our off in the network config? You ever do your benchmarking from the esx service console directly to the NFS share to cut out win/linux issues. For example transferring an ISO from ESX local storage directly to NFS SYN?

[rs407]

Jumboframes = off
Only from Vm server, not from the ESXI service console.
Both my and vm.franklin tests showed that it was a ESXI issue, not a win/linux issue.
The tests points to that it was the NFS config combination between ESXI and synology that wasent optimized in DSM 2.2, in DSM 3.0 with Async setting on the NFS share, NFS have performance that are accepable compared to CIFS.
The low ISCSI performance is not intresting for me since we have other low-end storage with good ISCSI performance plus that Synology only have ESXI support/certification for NFS.
We prefeer NFS over LUN regardless if it is FC or ISCSI for the simplicity,we have today about 50 LUN FC/ISCSI and ONE NFS share (non synology) in our wmware cluster, thats a 50/1 in regards to complexity in the favor of NFS.
With 100 GB free space on every LUN we have a "LUN waste" of 5 TB, with 1 TB free space on the NFS share we have a 5 to 1 reducation of waste in favor for NFS.
Plus that NFS dosent have the "disklock" issue when we do administrative work (snapshots/clones etc)

I guess that synology in DSM 3.0 just made it possible to set the "standard ASYNC flag" via the web interface.
For more info about async performance chapter 5.9
http://nfs.sourceforge.net/nfs-howto/ar01s05.html
/R

[jgallagher]

Thanks..looking good so far. Just that checkbox alone is now giving us around 45-50MBps of sustained transfer for something like storage vmotion to our rs810+ with a 200GB vm on a heavy traffic lun that it came from. Thats with just plugging it in and not really segmenting traffic as well as other tweaks. Im going to assume that I can get probably close to 70MBps for which I am very happy with for what you pay for for the rs810+.

Thanks again.

[EmcNetappF5mckenzie]

Question on what are the supported Read/Write Mount sizes for NFS and Versions being utilize with Either TCP or UDP from the client/server to the Synology NAS?
Mount -o tcp read=32 write=16 192.168.2.44:/folder /mnt/used
Question is there changeable sizing for NFS exports and client mount support? this will directly affect TCP window sizing and performance... if your file is 128k and it is defaulting to 8k then your going to exchange packets 16 times versus if your wind size is 32k or 64k which would be 4 packets or 2 packets... for data respectfully