Disk IO Degradation over long periods of high IO

Here you can discuss every aspect of Debian. Note: not for support requests!

Disk IO Degradation over long periods of high IO

Postby cjq » 2017-02-03 01:00

Hey, I have had this issue for years when copying large amounts of data between disks.

Initially you get high io speeds and the disk cache will max out from there and then the system will become sluggish and eventually unresponsive and the copying slows down to almost zero, this happens on any debian system(although it may happen on other distros but I generally exclusively use debian when doing this task).

I do have a work around which I run in a bash root shell when doing large copies which maintains the the hi throughput copying which constantly does a sync and drops the caches:
Code: Select all
while true ; sync ; echo 3 > /proc/sys/vm/drop_caches ; done


This has to be started before the copying as once it becomes sluggish it never seems to recover until after a reboot.

Just wondering if anyone has encountered this and has found a better solution to resolving this? I have tried tuning the kernel parameters around caching but it doesn't seem to make a difference.

Thanks
cjq
 
Posts: 2
Joined: 2017-02-03 00:46

Re: Disk IO Degradation over long periods of high IO

Postby Head_on_a_Stick » 2017-02-03 09:18

Try a newer kernel, perhaps?

https://lwn.net/Articles/682582/
No code is faster than no code.

Please read before posting How to report a problem
User avatar
Head_on_a_Stick
 
Posts: 6531
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: Disk IO Degradation over long periods of high IO

Postby cjq » 2017-02-06 22:37

I'm already running the backport 4.8 Kernel on these where I am experiencing the issue although as I have said have experienced this through the 3.* series kernels too. Just trying out the 4.9 from today.

Thanks for the link, that appears to be the same issue... however no resolution.
cjq
 
Posts: 2
Joined: 2017-02-03 00:46


Return to General Discussion

Who is online

Users browsing this forum: No registered users and 3 guests

fashionable