Page 1 of 1

Disk IO Degradation over long periods of high IO

Posted: 2017-02-03 01:00
by cjq
Hey, I have had this issue for years when copying large amounts of data between disks.

Initially you get high io speeds and the disk cache will max out from there and then the system will become sluggish and eventually unresponsive and the copying slows down to almost zero, this happens on any debian system(although it may happen on other distros but I generally exclusively use debian when doing this task).

I do have a work around which I run in a bash root shell when doing large copies which maintains the the hi throughput copying which constantly does a sync and drops the caches:

Code: Select all

while true ; sync ; echo 3 > /proc/sys/vm/drop_caches ; done
This has to be started before the copying as once it becomes sluggish it never seems to recover until after a reboot.

Just wondering if anyone has encountered this and has found a better solution to resolving this? I have tried tuning the kernel parameters around caching but it doesn't seem to make a difference.

Thanks

Re: Disk IO Degradation over long periods of high IO

Posted: 2017-02-03 09:18
by Head_on_a_Stick
Try a newer kernel, perhaps?

https://lwn.net/Articles/682582/

Re: Disk IO Degradation over long periods of high IO

Posted: 2017-02-06 22:37
by cjq
I'm already running the backport 4.8 Kernel on these where I am experiencing the issue although as I have said have experienced this through the 3.* series kernels too. Just trying out the 4.9 from today.

Thanks for the link, that appears to be the same issue... however no resolution.