Very poor NFS server performance

Kernels & Hardware, configuring network, installing services

Very poor NFS server performance

Postby debdoode » 2011-11-13 21:53

I installed a new debian 6.0 squeeze system to see if I could find a resolve to an nfs problem I am having.
The prior system was centos. I've have been using RHEL non stop for many years and wanted to give debian a shot since it seems to have solved another issue I was facing with a media server.

Anyhow, the server is only going to be used for NFS/Samba and nothing else. It is running on a rackable system platform and am wondering if someone knows of a doc or url which might offer some NFS server tuning tips. Searching has not lead me to anything specific yet so thought I would ask here.

Thanks.
debdoode
 
Posts: 6
Joined: 2011-11-13 21:22

Re: Very poor NFS server performance

Postby sej7278 » 2011-11-13 22:37

maybe if you told us what your problem was, how your system is setup etc?

i've found nfs is just slow, plain and simple. tried v3 and v4 servers on centos, fedora, ubuntu and debian. funnily enough the fastest client i've got is a mac!
sej7278
 
Posts: 220
Joined: 2011-06-11 17:03

Re: Very poor NFS server performance

Postby debdoode » 2011-11-13 23:43

You're right, I didn't include much info.

It's a dual xeon 2.8 with 4GB of memory and 16 SATA slots. The server is a default debian 6.0 install and I picked 'file server' while installing, nothing else. I don't need to run anything else on this server. I only want it to server up NFS/CIFS shares. I wanted to try debian because I seem to be getting better performance after switching to a debian system for serving up red5 apps. Looking around the net, it sounds like debian is a better choice for NFS, MEDIA and various other services, perhaps even web server.

For testing, I've tried a variety of share and mount options, none of which make much difference.

Examples;
# mount -t nfs -o rw,bg,hard,nointr,rsize=32768,wsize=32768,udp,vers=3,timeo=600 fi50:/shared/nfs /mnt
# mount arc220:/nfs /mnt -o rsize=8192,wsize=8192,noatime (trying various combinations)

Tried using options such as with and without noatime and not much difference.

A simple nfs test shows;

# time dd if=/dev/zero of=/mnt/testfile bs=16k count=16384
16384+0 records in
16384+0 records out
268435456 bytes (268 MB) copied, 28.2116 seconds, 9.5 MB/s

real 0m29.626s
user 0m0.032s
sys 0m1.315s

On the centos and debian NFS shares, I'm seeing on average about 8MB to 12MB or so.
On my fibre channel heads serving up NFS shares, I see on average 50MB/s.

I think I need a better method of testing. Not sure what else to add but happy to if you ask.
I doubt this is a useful test but I also tend to use mc to copy a file to a remote share to see what the transfer rate is. On filer heads, it remains consistent at what ever rate it hits but on centos/debian, it starts high then goes down to the low rates above. Guessing that's just an average however so nothing to use as a statistic.
debdoode
 
Posts: 6
Joined: 2011-11-13 21:22

Re: Very poor NFS server performance

Postby sej7278 » 2011-11-19 18:41

something is very wrong there, i just tried it from my wheezy client to my squeeze server and got:
Code: Select all
time dd if=/dev/zero of=/media/data/testfile bs=16k count=16384
16384+0 records in
16384+0 records out
268435456 bytes (268 MB) copied, 4.86692 s, 55.2 MB/s

real   0m4.954s
user   0m0.000s
sys   0m0.180s

locally on the server i get:
Code: Select all
time dd if=/dev/zero of=/export/data/testfile2 bs=16k count=16384
16384+0 records in
16384+0 records out
268435456 bytes (268 MB) copied, 2.3234 s, 116 MB/s

real   0m2.325s
user   0m0.004s
sys   0m0.272s

so the disk (sata2/7200rpm) is capable of over twice the network speed (gigabit) i'm getting. here's my configs:

client /etc/fstab:
Code: Select all
squeeze:/data      /media/data            nfs4    noauto,rw,user,hard,intr,timeo=14,rsize=32768,wsize=32768,noatime  0 0

server /etc/exports:
Code: Select all
/export/data          192.168.0.0/24(rw,nohide,insecure,no_subtree_check,sync)


as i recall through various testing i've done, noatime helps, r/w-size do nothing at all, neither does async.
sej7278
 
Posts: 220
Joined: 2011-06-11 17:03

Re: Very poor NFS server performance

Postby debdoode » 2011-11-20 17:08

I tried a couple of the options you mention in my 'exports' file and that helped by about 3MB/s.

Is there any way of making NFS and SMB top priority services on the system? That's all it's being used for.
Also, is there a CLI method of seeing the actual throughput of a file being read or written to an NFS share?

I know about 'time dd if=/dev/zero of=/mnt/testfile bs=16k count=16384' for example but that's not showing me the actual throughput in real time.
debdoode
 
Posts: 6
Joined: 2011-11-13 21:22

Re: Very poor NFS server performance

Postby sej7278 » 2011-11-20 17:37

debdoode wrote:I tried a couple of the options you mention in my 'exports' file and that helped by about 3MB/s.

Is there any way of making NFS and SMB top priority services on the system? That's all it's being used for.
Also, is there a CLI method of seeing the actual throughput of a file being read or written to an NFS share?

I know about 'time dd if=/dev/zero of=/mnt/testfile bs=16k count=16384' for example but that's not showing me the actual throughput in real time.


have you tried samba, in theory it should be faster as doesn't have the overhead of permissions and file attributes, ok for a media centre but not much else.

i'm completely lost with nfs4, performance seems to be totally random, i remember back in my fedora days i was getting speed drops in files over a gigabyte, and sshfs was just as fast (despite encryption overhead).
sej7278
 
Posts: 220
Joined: 2011-06-11 17:03

Re: Very poor NFS server performance

Postby debdoode » 2011-11-21 01:42

Do you mean testing transfer speeds? No, I haven't yet.
I need to find a tool that will allow me to see transfer speeds.
debdoode
 
Posts: 6
Joined: 2011-11-13 21:22


Return to System configuration

Who is online

Users browsing this forum: No registered users and 9 guests

fashionable