Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

HOWTO: Using tmpfs for /tmp, /var/{log,run,lock...}

Share your HowTo, Documentation, Tips and Tricks. Not for support questions!.
Post Reply
Message
Author
synss
Posts: 28
Joined: 2007-06-22 15:35
Location: Japan

HOWTO: Using tmpfs for /tmp, /var/{log,run,lock...}

#1 Post by synss »

Note: this tip is not intended to the beginner as it will change things early in the boot process. I expect you know how Debian boots using sys v and that you understand what you are doing.

I have been happily using Gentoo for a long time and now got Debian on my laptop. The idea is not mine but comes from the Gentoo forums http://forums.gentoo.org/viewtopic-t-37 ... tmpfs.html

So, tmpfs allows you to use part of your RAM as a normal filesystem. Unlike a ramdisk, the unused space is not wasted. Provided your kernel is recent enough, you have it already set at /dev/shm. You can find and set how big it should be (maximum size) in

Code: Select all

cat /etc/default/tmpfs
It is a very interesting to know and can be safely used for /tmp. You can mount as many tmpfs as you want, so that a line in /etc/fstab such as:

Code: Select all

tmpfs /tmp tmpfs defaults,noexec,nosuid 0 0
actually does the job.

Now, I find it cleaner not to clutter /etc/fstab with that kind of things and I prefer to mount --bind stuff in /dev/shm, I achieved that with the following initscripts:

/etc/default/my-tmpfs.sh

Code: Select all

# Where to mount the DIRS
# if empty, defaults to /dev/shm
MNT_TMPFS="/dev/shm"

# Set directories to be mounted in tmpfs
# /var/log is not recommended!
# /etc/default/rcS for /var/run and /var/lock
DIRS="/tmp /var/log /var/run /var/lock"

# Mount options for tmpfs
# if empty, defaults to rw,nodev,nosuid
# if you want to set the size= option, you need to do it in
#/etc/default/tmpfs
OPTS_TMPFS="nodev,noexec"

# Size of the ramdisk, note that the unoccupied space in tmpfs is not
# wasted so that this parameter can be made artificially high without
# impairing performance
# if empty, defaults to the value in /etc/default/tmpfs
#SHM_SIZE=64

# "quiet mount" will pass the -n option to the mount command
# mount(8) for more information
# Leave empty to unset
#QUIET=""
And /etc/init.d/my-tmpfs.sh

Code: Select all

#! /bin/sh
### BEGIN INIT INFO
# Provides:          
# Required-Start:    mountall
# Required-Stop: 
# Should-Start:      
# Should-Stop:
# Default-Start:     S
# Default-Stop:
# Short-Description: Mount directories in tmpfs
# Description: This initscript can bind directories in a tmpfs ramdisk, the
#              primary goal is to allow laptop users to let the hard-disk
#              drive spin up most of the time, it can also be used by people
#              having their system on a USB disk or a CF card.
# Author: Mathias Laurin
# License: GNU General Public License version 2 or higher
# Version: 1.0.4
### END INIT INFO

PATH=/sbin:/bin
NAME="my-tmpfs.sh"

. /lib/init/vars.sh
. /lib/lsb/init-functions

# Source tmpfs before $NAME so that SHM_SIZE gets overwritten
[ -e /etc/default/tmpfs ] && . /etc/default/tmpfs
[ -e /etc/default/$NAME ] && . /etc/default/$NAME

# no directory to mount, exit graciously
[ -z "$DIRS" ] && exit 0
[ -n "$QUIET" ] && OPTS_TMPFS_QUIET="-n"
[ -z "MNT_TMPFS" ] && MNT_TMPFS="/dev/shm"

# no /dev/shm, error and exit
[ ! -d "/dev/shm" ] && echo "Enable tmpfs in the kernel" >&2 && exit 1

do_start() {
	# Prepare mount point
	[ ! -d "$MNT_TMPFS" ] && mkdir "$MNT_TMPFS"
	
	# Mount options
	MNT_OPTS=${MNT_OPTS:-"rw,nodev,nosuid"}
	[ -n "$SHM_SIZE" ] && MNT_OPTS="$MNT_OPTS",size=$SHM_SIZE

	[ -n "$(grep $MNT_TMPFS /proc/mounts)" ] && umount "$MNT_TMPFS"
	
	mount -t tmpfs -o "$MNT_OPTS" tmpfs "$MNT_TMPFS"
	
	for MY_DIR in $DIRS; do
		MY_TMPFS="$MNT_TMPFS/$MY_DIR"
		[ ! -d "$MY_TMPFS" ] && mkdir -p "$MY_TMPFS"

		mount --bind $OPTS_TMPFS_QUIET "$MY_TMPFS" "$MY_DIR" -o "$OPTS_TMPFS"

		# special cases handled here, using "case" allows more flexibility
		case $MY_DIR in
			/tmp|/var/tmp)
			mount -o,remount,nodev,nosuid $MY_DIR
			chmod 1777 $MY_DIR
			;;
			/var/log)
			touch /var/log/lastlog
			touch /var/log/wtmp
			chgrp utmp /var/log/lastlog
			chmod 0644 /var/log/lastlog
			chmod 0644 /var/log/wtmp
			chmod 0600 /var/log/btmp
			mkdir /var/log/news
			chown news:news /var/log/news
			chmod g+s /var/log/news
			[ -f /etc/init.d/mpd ] &&
			mkdir /var/log/mpd &&
			chown mpd:audio /var/log/mpd
			[-f /etc/init.d/ipw3945d ] &&
			mkdir /var/log/ipw3945d &&
			chown Debian-ipw3945d:Debian-ipw3945d /var/log/ipw3945d
			;;
			/var/run)
			mount -o,remount,nodev,nosuid,noexec /var/run
			chmod 1777 /var/run
			[ ! -d /var/run/screen ] && mkdir -m 0777 /var/run/screen
			touch /var/run/utmp
			chgrp utmp /var/run/utmp
			chmod 0644 /var/run/utmp
			;;
		esac
	done
}

case "$1" in
  start|"")
	do_start
	;;
  restart|reload|force-reload)
	echo "Error: argument '$1' not supported" >&2
	exit 3
	;;
  stop)
	# No-op
	;;
  *)
	echo "Usage: my-tmpfs.sh [start|stop]" >&2
	exit 3
	;;
esac

:
I call it my- because I like to put my- in front of my own scripts to recognize them, put if you do not like it change the name.

Then

Code: Select all

chmod 755 /etc/init.d/my-tmpfs.sh
cd /etc/rcS.d
ln -s ../init.d/my-tmpfs.sh S37my-tmpfs.sh
and reboot.

That should do it.

Consider my script "(c) Mathias Laurin, Tokyo, 2007, GPL2+" after an idea from the Gentoo forums see the link above and modify as you see fits.


Now there will probably be a few (not fatal) error messages due to missing directories if you use the same list as me.

mounting /var/log is not recommended because your log will not be available at the next reboot, but if on a laptop with suspend working, for example, you probably do not care much about the logs anyway, so you can tweak /etc/syslog.conf not to log "news" at all (so that it does not complain /var/log/news is missing)
You can also add "case"s in my script for special... cases you think of. Follow what I have done for /tmp.

Also read http://www.debian-administration.org/articles/57 and following discussion about making /tmp noexec.


And note that options in /etc/default/rcS can put /var/run and /var/lock in tmpfs (but no other dir)
Last edited by synss on 2007-07-31 04:32, edited 7 times in total.

synss
Posts: 28
Joined: 2007-06-22 15:35
Location: Japan

#2 Post by synss »

Updates in the first post:
  • A working QUIET option for people using read-only media
    MNT_TMPFS allows to chose another directory than /dev/shm as mount point
    SHM_SIZE sets the size in ram used for tmpfs
and a bit more comments for the documentation.

synss
Posts: 28
Joined: 2007-06-22 15:35
Location: Japan

#3 Post by synss »

Update:
  • Fixed a stupid bug
Thanks for the feedback anyway.

User avatar
Dargor
Posts: 653
Joined: 2006-08-14 08:54
Location: New Zealand, Hamilton

#4 Post by Dargor »

Thanks for the howto, ill definitively be giving this a go.

synss
Posts: 28
Joined: 2007-06-22 15:35
Location: Japan

#5 Post by synss »

Dargor wrote:Thanks for the howto, ill definitively be giving this a go.
Thank you for your feedback! and here I want to repeat that mounting /var/log on tmpfs, although in principal, it should be OK on laptops, is not for the beginner! but it is fun!

I update a couple of "cases" in the original post.

/var/log on tmpfs: /etc/init.d/ipw3945d needs to be updated as well as it will not create /var/log/ipw3945d, so in case you need it, you need to create that directory and give it the expected permission. The script does it for /var/run/ipw3945d so it is trivial to adapt. If anyone is interested, I can put the required changes to /etc/init.d/ipw3945d in full here.

User avatar
perlhacker14
Posts: 464
Joined: 2007-06-19 20:19
Location: 127.0.0.1

#6 Post by perlhacker14 »

Just curious, but how much RAM is reccommended for this? It seems useful and interesting, but I am a bit short on RAM.
Arven bids you a good day...

My Laptop: Toshiba Satellite A25-S3072; 3.06 GHz Pentium 4; 473 MiB RAM; Debian Testing/Unstable/Experimental / Slackware 12; Whatever WM/DE I feel like at the moment

synss
Posts: 28
Joined: 2007-06-22 15:35
Location: Japan

#7 Post by synss »

perlhacker14 wrote:Just curious, but how much RAM is reccommended for this? It seems useful and interesting, but I am a bit short on RAM.
Well, depends on what you want to include there, if you only go for /var/run and /var/lock, you can do it in /etc/default/rcS (and that is probably <12Mb), if you do include more, then it depends on how often you reboot. You can always use logrotate, too. But anyway, it should not be much, I used to do it on my late computer with 512Mb (running openbox). You can always check the size of the directories you would include with
du -sh <path/to/directory>
anyway, for /tmp /var/log /var/run /var/lock, it is not much but for some reason I do not know, df does not show me how much is used in this way.

Also note that tmpfs does not waste the unused space à la ramdisk.

But I would say that 64Mb are enough and will prevent RAM to fill to one half if the logger goes crazy.

User avatar
Dargor
Posts: 653
Joined: 2006-08-14 08:54
Location: New Zealand, Hamilton

#8 Post by Dargor »

So how are you guys monitoring the writes to disk, to see if its being effective.

synss
Posts: 28
Joined: 2007-06-22 15:35
Location: Japan

#9 Post by synss »

Dargor wrote:So how are you guys monitoring the writes to disk, to see if its being effective.
Not sure what you mean.

It should not make a big difference anyway, just make the tmp dirs *really* temporary and avoid some unnecessary spinning up of your hard drive. If the dirs content's gone after you reboot, then it's working.

User avatar
Dargor
Posts: 653
Joined: 2006-08-14 08:54
Location: New Zealand, Hamilton

#10 Post by Dargor »

How do you monitor the number of writes to the disk the frequency of writes, so i can compare my gateways day with/without doing this tmpfs mod.

synss
Posts: 28
Joined: 2007-06-22 15:35
Location: Japan

#11 Post by synss »

Dargor wrote:How do you monitor the number of writes to the disk the frequency of writes, so i can compare my gateways day with/without doing this tmpfs mod.
Well, actually I don't. Performance will depend on the dirs you put on tmpfs, of course, but this is not so important to me. I see it more as a convenience trick than a performance one. I even do not really think the overall performance of the computer will change significantly.
However, if you want to mount your filesystem read-only, tmpfs will help as well.
I have a few more exceptions for the /var/log section I will add later today. EDIT: done! but I think the cases should be taken into account into the corresponding /etc/init.d/* and not in my script... but it looks like the Debian ppl never thought anyone would dish /var/log/*!

camason
Posts: 1
Joined: 2008-12-28 10:35

Thanks

#12 Post by camason »

I know this is an old topic, but this script has been very useful in setting up the correct log folders on my Asus Eee 901 (Ubuntu) with an Apache server.

So I decided to take the time and register to say thanks!

bill_baroud
Posts: 1
Joined: 2009-10-03 19:56

Thanks too

#13 Post by bill_baroud »

More and more older topic, but still usefull, thanks :)

Used with a mini atx box, Via 1GHz 512 SDRAM CF 1Go Lenny 2.6-26-2-486

bsod
Posts: 1
Joined: 2004-09-15 10:06

Re: HOWTO: Using tmpfs for /tmp, /var/{log,run,lock...}

#14 Post by bsod »

Hi,

This year I'm the one who say thanks, I'm about to use your great script to save the hard drive of an old laptop that will run 24/24 (I know it's not a very good idea).

I plan to add more directory in tmpfs and maybe rsync them daily on the hd (it wouldn't be a big loss if I loose the last day of data/log/etc.).

So thank you for this work!

bsod

friese
Posts: 7
Joined: 2011-07-06 18:26

Re: HOWTO: Using tmpfs for /tmp, /var/{log,run,lock...}

#15 Post by friese »

Hi,

I also just registered to say thanks. We will use this on our self-build ThinClients to save the CF they boot from. I made a small addition and added a "Provides"-info at the top:

Code: Select all

### BEGIN INIT INFO
# Provides:          var_to_ram
# Required-Start:    mountall
This way it can be activated by insserv or even rcconf and you can use the info in "Provides" to prevent other services from beeing started before this script ran by using

Code: Select all

# Required-Start:    var_to_ram
in their scripts.

Thanks again, Jens

kevmitch
Posts: 498
Joined: 2008-01-29 20:43

Re: HOWTO: Using tmpfs for /tmp, /var/{log,run,lock...}

#16 Post by kevmitch »

This should probably be mounting to /run now instead of /dev/shm. There are now options in /etc/default/rcS to make /tmp and /dev/shm as symlinks to /run.

Code: Select all

RAMRUN=yes
RAMLOCK=yes
RAMSHM=yes
RAMTMP=yes

friese
Posts: 7
Joined: 2011-07-06 18:26

Re: HOWTO: Using tmpfs for /tmp, /var/{log,run,lock...}

#17 Post by friese »

Are you sure that will work in debian squeeze? I just had a look in the rcS manpage an I only found RAMRUN and RAMLOCK. Another question I'm having ist if there are any more directories than /tmp /var/log /var/run /var/lock /var/tmp and /media (which is where I auto-create and delete directories for mounting USB-Drives when they are connected) that have to be moved to a tmpfs in order to prevent our poor little CF-cards from beeing destroyed too soon? /swap is of course turned off.

cu, Jens

promitheus
Posts: 1
Joined: 2011-09-19 10:54

Re: HOWTO: Using tmpfs for /tmp, /var/{log,run,lock...}

#18 Post by promitheus »

Thanks for the fine article!

in order for the script to work in Uubtu 10.04 LTS i added in /etc/init.d/my-tmpfs.sh after "touch /var/log/wtmp the line"
touch /var/log/btmp

also in line "[-f /etc/init.d/ipw3945d ]" && there should be a space between [ and - so the line should be
[ -f /etc/init.d/ipw3945d ] &&

Also apt is protesting with "E: Directory '/var/log/apt/' missing"

Also from the whole /var/log directory with entries like above
root@ubuntu:/var/log# ls -la
total 2472
drwxr-xr-x 9 root root 4096 2011-09-19 14:17 .
drwxr-xr-x 13 root root 4096 2011-09-19 11:54 ..
drwxr-xr-x 2 root root 4096 2011-01-20 19:43 apparmor
drwxr-xr-x 2 root root 4096 2011-09-19 11:57 apt
-rw-r--r-- 1 root root 3660 2011-09-19 12:03 aptitude
-rw-r----- 1 syslog adm 19874 2011-09-19 14:19 auth.log
-rw-r----- 1 root adm 31 2011-09-19 11:55 boot
-rw-r--r-- 1 root root 256 2011-09-19 14:17 boot.log
-rw-rw-r-- 1 root utmp 0 2011-09-19 11:54 btmp
-rw-r----- 1 syslog adm 15008 2011-09-19 14:17 daemon.log
-rw-r----- 1 syslog adm 73775 2011-09-19 14:17 debug
drwxr-xr-x 2 root root 4096 2010-10-14 09:48 dist-upgrade
-rw-r----- 1 root adm 24008 2011-09-19 14:17 dmesg
-rw-r----- 1 root adm 23831 2011-09-19 14:12 dmesg.0
-rw-r----- 1 root adm 8305 2011-09-19 12:37 dmesg.1.gz
-rw-r----- 1 root adm 8387 2011-09-19 12:25 dmesg.2.gz
-rw-r----- 1 root adm 8400 2011-09-19 12:18 dmesg.3.gz
-rw-r----- 1 root adm 59 2011-09-19 11:55 dmesg.4.gz
-rw-r--r-- 1 root root 292867 2011-09-19 12:24 dpkg.log
-rw-r--r-- 1 root root 24024 2011-09-19 12:10 faillog
drwxr-xr-x 2 root root 4096 2011-09-19 11:55 fsck
drwxr-xr-x 3 root root 4096 2011-09-19 12:18 installer
-rw-r----- 1 syslog adm 604864 2011-09-19 14:17 kern.log
-rw-rw-r-- 1 root utmp 292292 2011-09-19 14:19 lastlog
-rw-r----- 1 syslog adm 0 2011-09-19 12:18 lpr.log
-rw-r----- 1 syslog adm 0 2011-09-19 12:18 mail.err
-rw-r----- 1 syslog adm 0 2011-09-19 12:18 mail.info
-rw-r----- 1 syslog adm 0 2011-09-19 12:18 mail.log
-rw-r----- 1 syslog adm 0 2011-09-19 12:18 mail.warn
-rw-r----- 1 syslog adm 534331 2011-09-19 14:17 messages
drwxr-xr-x 2 root root 4096 2011-09-19 12:18 news
-rw-r--r-- 1 root root 0 2011-09-19 11:57 pycentral.log
-rw-r----- 1 syslog adm 630909 2011-09-19 14:17 syslog
-rw-r--r-- 1 root root 134042 2011-09-19 14:17 udev
-rw-r----- 1 syslog adm 0 2011-09-19 12:18 ufw.log
drwxr-xr-x 2 root root 4096 2011-01-15 02:48 unattended-upgrades
-rw-r----- 1 syslog adm 0 2011-09-19 12:18 user.log
-rw-rw-r-- 1 root utmp 21888 2011-09-19 14:19 wtmp

after the script there are only

root@ubuntu:/var/log# ls -la
total 4
drwxr-xr-x 3 root root 120 2011-09-19 14:23 .
drwxr-xr-x 13 root root 4096 2011-09-19 11:54 ..
-rw------- 1 root root 0 2011-09-19 14:23 btmp
-rw-r--r-- 1 root utmp 0 2011-09-19 14:23 lastlog
drwxr-sr-x 2 news news 40 2011-09-19 14:23 news
-rw-r--r-- 1 root root 0 2011-09-19 14:23 wtmp

Finally because the script possibly(?) runs after ssh for a reason, i do not have ssh working (netstat shows listening on 22) so i tried to overcome the problem with Provides and Required-start as friese noticed with no luck. I overcomed the problem by restarting ssh script in /etc/rc.local

Is there a way for the script to run before all other scripts?

Thanks again

Post Reply