Disk IO Degradation over long periods of high IO
Posted: 2017-02-03 01:00
Hey, I have had this issue for years when copying large amounts of data between disks.
Initially you get high io speeds and the disk cache will max out from there and then the system will become sluggish and eventually unresponsive and the copying slows down to almost zero, this happens on any debian system(although it may happen on other distros but I generally exclusively use debian when doing this task).
I do have a work around which I run in a bash root shell when doing large copies which maintains the the hi throughput copying which constantly does a sync and drops the caches:
This has to be started before the copying as once it becomes sluggish it never seems to recover until after a reboot.
Just wondering if anyone has encountered this and has found a better solution to resolving this? I have tried tuning the kernel parameters around caching but it doesn't seem to make a difference.
Thanks
Initially you get high io speeds and the disk cache will max out from there and then the system will become sluggish and eventually unresponsive and the copying slows down to almost zero, this happens on any debian system(although it may happen on other distros but I generally exclusively use debian when doing this task).
I do have a work around which I run in a bash root shell when doing large copies which maintains the the hi throughput copying which constantly does a sync and drops the caches:
Code: Select all
while true ; sync ; echo 3 > /proc/sys/vm/drop_caches ; done
Just wondering if anyone has encountered this and has found a better solution to resolving this? I have tried tuning the kernel parameters around caching but it doesn't seem to make a difference.
Thanks