Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

KVM/QEMU-guest, very slow disk performance?

If none of the specific sub-forums seem right for your thread, ask here.
Post Reply
Message
Author
slavka
Posts: 1
Joined: 2017-09-07 08:31

KVM/QEMU-guest, very slow disk performance?

#1 Post by slavka »

I'm really confused of QEMU/KVM behavior. I have a server - about 100GB RAM, 2 CPU - 6 core + HT, and three HDDs - one for root, and two for BTRFS-mirror for virtual machines (HGST HDN724030ALE640 if it's matter). OS - Debian 9.1, QEMU (disk type - raw, IO - native).

So, server is idle - it's run only one VM. This virtual machine (Debian 9.1, 16GB RAM, 4 VCPU), which runs PostgreSQL and MySQL databases. Queries receive only one - PostgreSQL (few insert per second). And... it takes lot of time, to do this inserts. I wrote simple script, which insert data to very simple table, whitout any constraints or indexes (three varchar fields). Script performance - about 1-2 inserts per second. It's extremely slow!

At first, I thought it's was Postgres trouble. But I rewrote my script for MySQL, and it shows same result. Then, I create new VM on FreeBSD, install PostgreSQL, create same database and table from Linux VM, and check performance again - 1-2 inserts per second.

Okay. Let's see on iostat and iotop. iotop shows, that Postgres process, which serves my insert queries, consume about 60-70% of IO (for simple inserts!) on VM. iostat -x says, that now uses about 40-45 writes/sec on VM, and about 80-90 w/s in host machine.

I tried to rate write speed with dd utility, like this:

Code: Select all

dd if=/dev/urandom of=/tmp/fooo.bar bs=1M count=1000
and it's done about 5-6 secs.

Next, while script execute about 100 inserts, I check file:

Code: Select all

/proc/[Postgres_PID]/io

rchar: 138544
wchar: 114689
syscr: 39
syscw: 15
read_bytes: 0
write_bytes: 540672
cancelled_write_bytes: 0
What I'm doing wrong? Why I have so slow disk performance?

UPD: I'm copied remote file (about 4,5 GB) to this VM via SFTP. It was copied on 15-20 MB/s speed (about 3-4 minutes). I really don't understand it. I thought, that it was KVM problem (2 different database have same troubles, different VM on FreeBSD have same troubled behavior, wich Linux VM), but remote file was copied fast, iotop shows 20-30% load, and iostat shows w/s about 150-200 (sometimes about 300). What's happening? Please, help someone...

User avatar
pylkko
Posts: 1802
Joined: 2014-11-06 19:02

Re: KVM/QEMU-guest, very slow disk performance?

#2 Post by pylkko »

I don not know, and I am not sure if I understand what you are doing..

But btrfs is a copy-on-write (COW) file system and often the guides say that do not do COW on COW. So do not run virtual machine images that are written into the middle of on a COW system and do not run databases either. So, could it be that you have a COW btrfs partition with a COW vm image in it with a database in it and are getting massive i/o overhead?

Post Reply