I installed a new debian 6.0 squeeze system to see if I could find a resolve to an nfs problem I am having.
The prior system was centos. I've have been using RHEL non stop for many years and wanted to give debian a shot since it seems to have solved another issue I was facing with a media server.
Anyhow, the server is only going to be used for NFS/Samba and nothing else. It is running on a rackable system platform and am wondering if someone knows of a doc or url which might offer some NFS server tuning tips. Searching has not lead me to anything specific yet so thought I would ask here.
Thanks.
Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230
Very poor NFS server performance
Re: Very poor NFS server performance
maybe if you told us what your problem was, how your system is setup etc?
i've found nfs is just slow, plain and simple. tried v3 and v4 servers on centos, fedora, ubuntu and debian. funnily enough the fastest client i've got is a mac!
i've found nfs is just slow, plain and simple. tried v3 and v4 servers on centos, fedora, ubuntu and debian. funnily enough the fastest client i've got is a mac!
Re: Very poor NFS server performance
You're right, I didn't include much info.
It's a dual xeon 2.8 with 4GB of memory and 16 SATA slots. The server is a default debian 6.0 install and I picked 'file server' while installing, nothing else. I don't need to run anything else on this server. I only want it to server up NFS/CIFS shares. I wanted to try debian because I seem to be getting better performance after switching to a debian system for serving up red5 apps. Looking around the net, it sounds like debian is a better choice for NFS, MEDIA and various other services, perhaps even web server.
For testing, I've tried a variety of share and mount options, none of which make much difference.
Examples;
# mount -t nfs -o rw,bg,hard,nointr,rsize=32768,wsize=32768,udp,vers=3,timeo=600 fi50:/shared/nfs /mnt
# mount arc220:/nfs /mnt -o rsize=8192,wsize=8192,noatime (trying various combinations)
Tried using options such as with and without noatime and not much difference.
A simple nfs test shows;
# time dd if=/dev/zero of=/mnt/testfile bs=16k count=16384
16384+0 records in
16384+0 records out
268435456 bytes (268 MB) copied, 28.2116 seconds, 9.5 MB/s
real 0m29.626s
user 0m0.032s
sys 0m1.315s
On the centos and debian NFS shares, I'm seeing on average about 8MB to 12MB or so.
On my fibre channel heads serving up NFS shares, I see on average 50MB/s.
I think I need a better method of testing. Not sure what else to add but happy to if you ask.
I doubt this is a useful test but I also tend to use mc to copy a file to a remote share to see what the transfer rate is. On filer heads, it remains consistent at what ever rate it hits but on centos/debian, it starts high then goes down to the low rates above. Guessing that's just an average however so nothing to use as a statistic.
It's a dual xeon 2.8 with 4GB of memory and 16 SATA slots. The server is a default debian 6.0 install and I picked 'file server' while installing, nothing else. I don't need to run anything else on this server. I only want it to server up NFS/CIFS shares. I wanted to try debian because I seem to be getting better performance after switching to a debian system for serving up red5 apps. Looking around the net, it sounds like debian is a better choice for NFS, MEDIA and various other services, perhaps even web server.
For testing, I've tried a variety of share and mount options, none of which make much difference.
Examples;
# mount -t nfs -o rw,bg,hard,nointr,rsize=32768,wsize=32768,udp,vers=3,timeo=600 fi50:/shared/nfs /mnt
# mount arc220:/nfs /mnt -o rsize=8192,wsize=8192,noatime (trying various combinations)
Tried using options such as with and without noatime and not much difference.
A simple nfs test shows;
# time dd if=/dev/zero of=/mnt/testfile bs=16k count=16384
16384+0 records in
16384+0 records out
268435456 bytes (268 MB) copied, 28.2116 seconds, 9.5 MB/s
real 0m29.626s
user 0m0.032s
sys 0m1.315s
On the centos and debian NFS shares, I'm seeing on average about 8MB to 12MB or so.
On my fibre channel heads serving up NFS shares, I see on average 50MB/s.
I think I need a better method of testing. Not sure what else to add but happy to if you ask.
I doubt this is a useful test but I also tend to use mc to copy a file to a remote share to see what the transfer rate is. On filer heads, it remains consistent at what ever rate it hits but on centos/debian, it starts high then goes down to the low rates above. Guessing that's just an average however so nothing to use as a statistic.
Re: Very poor NFS server performance
something is very wrong there, i just tried it from my wheezy client to my squeeze server and got:
locally on the server i get:
so the disk (sata2/7200rpm) is capable of over twice the network speed (gigabit) i'm getting. here's my configs:
client /etc/fstab:
server /etc/exports:
as i recall through various testing i've done, noatime helps, r/w-size do nothing at all, neither does async.
Code: Select all
time dd if=/dev/zero of=/media/data/testfile bs=16k count=16384
16384+0 records in
16384+0 records out
268435456 bytes (268 MB) copied, 4.86692 s, 55.2 MB/s
real 0m4.954s
user 0m0.000s
sys 0m0.180s
Code: Select all
time dd if=/dev/zero of=/export/data/testfile2 bs=16k count=16384
16384+0 records in
16384+0 records out
268435456 bytes (268 MB) copied, 2.3234 s, 116 MB/s
real 0m2.325s
user 0m0.004s
sys 0m0.272s
client /etc/fstab:
Code: Select all
squeeze:/data /media/data nfs4 noauto,rw,user,hard,intr,timeo=14,rsize=32768,wsize=32768,noatime 0 0
Code: Select all
/export/data 192.168.0.0/24(rw,nohide,insecure,no_subtree_check,sync)
Re: Very poor NFS server performance
I tried a couple of the options you mention in my 'exports' file and that helped by about 3MB/s.
Is there any way of making NFS and SMB top priority services on the system? That's all it's being used for.
Also, is there a CLI method of seeing the actual throughput of a file being read or written to an NFS share?
I know about 'time dd if=/dev/zero of=/mnt/testfile bs=16k count=16384' for example but that's not showing me the actual throughput in real time.
Is there any way of making NFS and SMB top priority services on the system? That's all it's being used for.
Also, is there a CLI method of seeing the actual throughput of a file being read or written to an NFS share?
I know about 'time dd if=/dev/zero of=/mnt/testfile bs=16k count=16384' for example but that's not showing me the actual throughput in real time.
Re: Very poor NFS server performance
have you tried samba, in theory it should be faster as doesn't have the overhead of permissions and file attributes, ok for a media centre but not much else.debdoode wrote:I tried a couple of the options you mention in my 'exports' file and that helped by about 3MB/s.
Is there any way of making NFS and SMB top priority services on the system? That's all it's being used for.
Also, is there a CLI method of seeing the actual throughput of a file being read or written to an NFS share?
I know about 'time dd if=/dev/zero of=/mnt/testfile bs=16k count=16384' for example but that's not showing me the actual throughput in real time.
i'm completely lost with nfs4, performance seems to be totally random, i remember back in my fedora days i was getting speed drops in files over a gigabyte, and sshfs was just as fast (despite encryption overhead).
Re: Very poor NFS server performance
Do you mean testing transfer speeds? No, I haven't yet.
I need to find a tool that will allow me to see transfer speeds.
I need to find a tool that will allow me to see transfer speeds.