Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

swap space usage increases

Linux Kernel, Network, and Services configuration.
Message
Author
vbrummond
Posts: 4432
Joined: 2010-03-02 01:42

Re: swap space usage increases

#21 Post by vbrummond »

jlinkels wrote:The screenshot is here: http://www.linkels.net/~jlinkels/links/kmonitor.png
Thanks.
When I add up all processes > 1 MB I get 681 MB. There are 68 processes < 1 MB (which you can't see) ranging from 0.8 MB down to 0.1 MB. That should account for no more than ~30 MB. Still used memory show 770 MB and I can account for only 710 MB.
I added up everything I could see on the screenshot and got a total of around 700mb. It is too close for the math to be off. If it were the kernel bug, the system would not allow you to use more than 50% of the RAM before it went to all swap. You are well over half in normal memory. You just need to find a way to trim down on what is running to save memory. KDE can disable certain services such as searching and indexing to save memory. You might have answered this, please forgive me if so. Are you using 64-bit? 64-bit uses around 2x more ram than 32-bit on the same hardware.
But this is the 1GB system, which is somewhat limited on RAM (more underway, waiting for the order to arrive)
Adding another 1gb will probably be enough for the current load on the 1gb system. Obviously the more the merrier.
I can't access the 4 GB system GUI from here but I make a screenshot of that one as well Monday.
Sounds good. I do understand your frustration. I am sorry to say, nothing points to there being anything unusual. It is normal usage for 64-bit and only slightly high for 32-bit. And if there IS a problem, I can only think to switch from KDE to something else because KDE is likely the problem
Always on Debian Testing

jlinkels
Posts: 48
Joined: 2008-07-12 14:34
Been thanked: 1 time

Re: swap space usage increases

#22 Post by jlinkels »

Thanks for your insights. Yes, all systems are running 64-bits versions.

This is the picture for the 4 GB system.
Image
When I add up all processes' memory I end up with 1.8 GB, not 3.1 as the status line tells me. And I am not even sure what means "Shared" memory, how to count for it

I'll follow your advice and try a lightweight desktop, but that might take some time. Once in a while I have to do some real work as well.

jlinkels

vbrummond
Posts: 4432
Joined: 2010-03-02 01:42

Re: swap space usage increases

#23 Post by vbrummond »

If you quit all virtualbox processes are you still using a lot of memory? Also, it shows Xorg there using 500mb of memory, which is way too much. This is a feature of KDE I noticed that the Xorg process goes out of control. The 3gb system seems to be more suspicious than the 1gb one, which is why I am thinking of virtualbox.

As for using a lighter desktop, I just wanted to confirm it was not going out of control for no reason. You do not have to use a light desktop just something that is 'not kde' might fix the issue. Though of course the goal is to get KDE working right.

Perhaps try 32-bit on a live cd and see if the environment works well? http://cdimage.debian.org/debian-cd/cur ... esktop.iso
Always on Debian Testing

jlinkels
Posts: 48
Joined: 2008-07-12 14:34
Been thanked: 1 time

Re: swap space usage increases

#24 Post by jlinkels »

I am seeing reallly strange things going around here!

First, I have switched off the VM. That cleaned up the RAM usage. The status line showed 3.1G - 500 MB = 2.6 GB. Correct.

Then I closed that memory hog Opera, which was close to 400MB. Status line showed 2.6GB - 400MB - 2.2GB. Also correct.

That is, the decrease was correct. With Xorg accounting for 500 MB and a handful of other processes, I never reach 2.2 GB when I add up all those.

But now the next thing: I read the link in dtl131's post: https://bugs.launchpad.net/ubuntu/+sour ... +bug/98783. This is reporting a bug of Xorg using more memory when Okular was used.
BTW my machine uses the ATI driver, not NVidia

So I tried this:
Started Okular and loaded a PDF of 43 MB. That increased the memory use of Xorg from 550MB to 600MB (give or take a few MB). After closing Okular, memory was not reclaimed.

So I opened Okular again, with again that file of 43 MB. But this time I browsed slowly through almost all pages, giving Okular time to render. Now the memory increased to 842MB for Xorg. But the strangest thing of all is that wiht each page I flipped, the swap space was DECREASED with an almost equal amount as Xorg memory use was increased

When closing Okular again, memory was not reclaimed either, neither did the swap usage change.
This is the screen dump. Note that Opera and Virtualbox are gone here. Also see the decrease in swap usage from this post as compared to the previous post.


Image

I am using xorg 1.7.7-13

jlinkels

vbrummond
Posts: 4432
Joined: 2010-03-02 01:42

Re: swap space usage increases

#25 Post by vbrummond »

This makes me think it might be a bit out of my hands. Considering how much memory Xorg is using the core issue has to be related to it somehow. Basically as far as I know a few options for going forward are:

1. Try a different display driver. I do not have this memory leak issue using KDE under virtualbox, so perhaps using another ATI driver (perhaps the proprietary one) might fix this. Vesa may work as well but vesa has no 3d acceleration.
2. Use a different desktop environment. My gnome laptop has been on over a week and Xorg is using under 50mb. KDE on it easily uses 300mb for xorg.

Edit: I will do more testing of KDE under virtualbox.

My system has many processes running but I am not using up much memory at all:
http://i.minus.com/ibvcjNM9Ds1bPu.png
http://i.minus.com/icF6GFCu2XmhV.png
Always on Debian Testing

dtl131
Posts: 449
Joined: 2011-07-11 19:56

Re: swap space usage increases

#26 Post by dtl131 »

After closing Okular, memory was not reclaimed.
The virtual memory manager keeps the old data cached (unless running out of virtual memory) in case you need it again. It may swap it out to disk swap depending on different factors, but just closing an app won't automatically remove all of its virtual memory pages (which is beneficial, even if some people don't think so).
jlinkels wrote:So I opened Okular again, with again that file of 43 MB. But this time I browsed slowly through almost all pages, giving Okular time to render. Now the memory increased to 842MB for Xorg. But the strangest thing of all is that with each page I flipped, the swap space was DECREASED with an almost equal amount as Xorg memory use was increased
It makes sense. You closed Okular and the kernel swapped out the unused data to disk cache. As you reload the pages, the data is moved back to RAM.

vbrummond
Posts: 4432
Joined: 2010-03-02 01:42

Re: swap space usage increases

#27 Post by vbrummond »

I do not think the kde system monitor program shows that though.
Always on Debian Testing

jlinkels
Posts: 48
Joined: 2008-07-12 14:34
Been thanked: 1 time

Re: swap space usage increases

#28 Post by jlinkels »

dtl131 wrote: The virtual memory manager keeps the old data cached (unless running out of virtual memory) in case you need it again. It may swap it out to disk swap depending on different factors, but just closing an app won't automatically remove all of its virtual memory pages (which is beneficial, even if some people don't think so).
dtl131 wrote:It makes sense. You closed Okular and the kernel swapped out the unused data to disk cache. As you reload the pages, the data is moved back to RAM.
That sounds quite contradictionary, doesn't it?

Then again, this is what happened:

- swap space had been constant for at least a number of hours at 770 MB
- when I opened Okular the first time I did not browse slow enough to render all pages. Real memory use for xorg increased from 550 to 600 MB
- swap space was untouched: no disk activity
- When I closed Okular, swap space was untouched, xorg did not release the memory.
- Then I opened Okular for the second time, rendering many more pages, xorg memory use increased with over 200 MB
- Swap space was decreased. Which pages the swap area could have been holding if xorg has only been increasing its memory use, nothing had been written to the swap area during the previous hours, and the document had never been viewed before?

jlinkels

dtl131
Posts: 449
Joined: 2011-07-11 19:56

Re: swap space usage increases

#29 Post by dtl131 »

It sounds like you're expecting virtual memory to be completely deleted/freed just by closing a program. That's not how it works because systems tend to use data they just used and virtual memory managers are written accordingly (the theoretical term is "temporal locality"). However, Xorg shuold give back memory after logging out. If you're using the open-source radeon driver, I'm guessing you're suffering from this bug: https://bugzilla.redhat.com/show_bug.cgi?id=529721
It's been fixed in newer xorg versions (or you could disable kernel modesetting).

vbrummond
Posts: 4432
Joined: 2010-03-02 01:42

Re: swap space usage increases

#30 Post by vbrummond »

That might be it. You can add the parameter "nomodeset" to grub to disable kernel modesetting.
Always on Debian Testing

jlinkels
Posts: 48
Joined: 2008-07-12 14:34
Been thanked: 1 time

Re: swap space usage increases

#31 Post by jlinkels »

Hell no! I rebooted this machine 22 days ago, and that was already the second time in 3 months! :lol:

I have read up about this bug, and frankly I doubt it. Debian Stable is outdated, but I am running Squeeze now and this bug should have been resolved some time ago. OTOH I admit that the description matches very much what I see.

By your advice I already planned to reboot the machine, maybe even this week :?
I'll report back when I did.

Thanks so far.
jlinkels

vbrummond
Posts: 4432
Joined: 2010-03-02 01:42

Re: swap space usage increases

#32 Post by vbrummond »

I tested KDE in virtualbox using the vesa driver. If I open a huge pdf my xorg process inflates to 160mb used. If I close okular it returns to using 30-40 like on a fresh boot. So the memory does de-allocate using Debian Squeeze, and it is not a 'general bug' but a more specific one.
Always on Debian Testing

dtl131
Posts: 449
Joined: 2011-07-11 19:56

Re: swap space usage increases

#33 Post by dtl131 »

jlinkels wrote:Debian Stable is outdated, but I am running Squeeze now and this bug should have been resolved some time ago. OTOH I admit that the description matches very much what I see.
stable/squeeze has a relatively old xorg-server (1.7.x).

jlinkels
Posts: 48
Joined: 2008-07-12 14:34
Been thanked: 1 time

Re: swap space usage increases

#34 Post by jlinkels »

This is what I did:
In /etc/modprobe.d/radeon-kms.conf
I changed this line:

Code: Select all

options radeon modeset=1
to this:

Code: Select all

options radeon modeset=0
And in addition I added the boot parameter nomodeset while booting. The parameter was seen during booting according to
/proc/cmdline. Any way I can check kms was actually disabled?

Right after booting xorg memory use was 101 MB.
When I opened a large PDF file, during browsing and rendering, memory increased to > 500 MB.
AFter closing the file, memory use dropped to 211 MB
Then I opened another PDF and browsed until memory use was > 600 MB
(Those are PDFs 35-50 MB in size and contain a lot of bitmaps)
After closing this PDF memeory was released until the 211 MB mark

Swap space was untouched.

Let me see what happens the next few hours/days.

jlinkels

jlinkels
Posts: 48
Joined: 2008-07-12 14:34
Been thanked: 1 time

Re: swap space usage increases

#35 Post by jlinkels »

The memory hogging monster is back! :(

It went fine for a few hours. Then I opened LibreOffice Impress with a fairly large presentation and Inkscape with a drawing with several thousands of objects, did some work and closed them. When I looked again in KSysMonitor, Xorg had increased to 500+ MB. No swap space in use yet.

I know, I should have paid better attention, but I was concentrating on my work. Contrary to popular belief you can become quite productive using Linux on the Desktop, even for GUI orientend tasks :wink:

jlinkels

vbrummond
Posts: 4432
Joined: 2010-03-02 01:42

Re: swap space usage increases

#36 Post by vbrummond »

Well, not sure what could solve it then other than a different driver. My virtual machine with KDE has been running for a day and half now and it is still using 40mb for Xorg.
Always on Debian Testing

jlinkels
Posts: 48
Joined: 2008-07-12 14:34
Been thanked: 1 time

Re: swap space usage increases

#37 Post by jlinkels »

Good, an update:

The 1 GB machine got upgraded to 2 GB. Behaviour is much better. However, this morning I checked memory usage. Swapping has begun again. Xorg is using 550 MB. THe machine has been on for 6 days.

Then the 4 GB machine. Yesterday I made the stupendous error to open Inkscape on this machine. Uptime must have been 12 days. Then it started swapping. Nothing could be done anymore. Not even the mouse would move. Clock displays were frozen. I tried to log in remotely to check memory usage. Login never got beyond MOTD. Swapping went on continuously for 30 minutes , then I switched off the power. I can't find any other explanation as that an active part of xorg was swapped in and out all the time. What else could have been used all the time if all programs were inactive and could not accept any input. Unless the swap algorithm is flawed of course.

Now the 2 GB machine runs the NVidia display driver. The 4 GB machine runs the ATI display driver. Can we now rule out the the display driver is the culprit?

jlinkels

Post Reply