HOW-TO: New and Improved RAMboot for speed and silence

[IsaacKuo]

How to RAMboot

This details a method of loading your entire OS into an uncompressed ramdisk. The result is lightning fast performance, and elimination of hard drive noise and power consumption (if swap is not used and the hard drive is spun down).

The basic steps are:

1. Install Debian 4.0 on the hard drive

2. Create a modified /etc/fstab which has tmpfs for the root partition

3. Optionally create a startup script to park hard drives

4. Create a script which makes a stripped down OS image

5. Create a custom initrd.img which loads the OS image into a tmpfs ramdisk

6. Modify /boot/grub/menu.lst with an entry for the custom initrd.img

After completing these steps, you will have a dual boot system with the following boot options:

A) Boot normally, where you install new software or change settings

B) Boot into the "ramboot" OS image, for high speed silent computing

-------------------------------------------------
Step 1. Installing Debian 4.0

a) Install Debian 4.0 onto a partition of least 500megs. For purposes of the rest of these instructions, I'll assume you're have installing into the hda1 partition.

b) In the software selection step, deselect the so-called Base software suite. You will manually install only what you need later on.

c) After doing the install, log in as root and edit /etc/apt/sources.list to comment out the CD-ROM entry (use the command "nano /etc/apt/sources.list"). Then run the following commands:

Code: Select all

apt-get update
apt-get install hdparm localepurge debconf-english
apt-get remove --purge aptitude tasksel tasksel-data laptop-detect
apt-get clean

Note that installing debconf-english will remove debconf-i18n. This is normal.

-------------------------------------------------
Step 2. Create a modified /etc/fstab

Create and edit a new fstab using these commands:

Code: Select all

cd /etc/
cp fstab fstab.ramboot
nano fstab.ramboot

Comment out the / entry. Create a new / line like this:

Code: Select all

none / tmpfs defaults 0 0

-------------------------------------------------
Step 3. Optionally create a startup script to park hard drives

If you want, create a startup script with these commands:

Code: Select all

nano /etc/init.d/ijkijkijk
chmod 755 /etc/init.d/ijkijkijk
update-rc.d ijkijkijk defaults 20

The contents of ijkijkijk should be something like this:

Code: Select all

#! /bin/sh
# /etc/init.d/ijkijkijk

# Some things that run always
touch /var/lock/ijkijkijk

# Carry out specific functions when asked to by the system
case "$1" in
  start)
    echo "Starting script ijkijkijk"

        echo "Isaac Kuo script parking drives"
        hdparm -S 6 /dev/hda
        hdparm -y /dev/hda
        #hdparm -S 6 /dev/hdb
        #hdparm -y /dev/hdb
        #hdparm -S 6 /dev/hdc
        #hdparm -y /dev/hdc
        #hdparm -S 6 /dev/hdd
        #hdparm -y /dev/hdd

    ;;
  stop)
    echo "Stopping script ijkijkijk"



    ;;
  *)
    echo "Usage: /etc/init.d/blah {start|stop}"
    exit 1
    ;;
esac

exit 0

-------------------------------------------------
Step 4. Create a script which makes a stripped down OS image

Login as root. Then create a basic script like this:

Code: Select all

#!/bin/sh
#
# Takes an OS snapshot, strips it down, and wraps it up into /snapstrip.tar

# Clean up anything previous
touch /snapstrip.tar
touch /snapstrip
rm -fvr /snapstrip.tar
rm -fvr /snapstrip

# Create temporary ramdisk and copy files over
mkdir /snapstrip
mount -t tmpfs -o size=100% none /snapstrip
cp -vax /. /snapstrip/.
cp -vax /dev/. /snapstrip/dev/.

# Move over the modified fstab
cd /snapstrip/etc/
cp -vax fstab.ramboot fstab

# Strip down unnecessary stuff
cd /snapstrip/
rm -fvr /snapstrip/boot/*
rm -fvr /snapstrip/var/lib/apt/lists/*
rm -fvr /snapstrip/usr/share/doc-base/*
rm -fvr /snapstrip/usr/share/doc/*
rm -fvr /snapstrip/usr/share/man/*

rm -fvr /snapstrip/lib/modules/*/kernel/drivers/bluetooth
rm -fvr /snapstrip/lib/modules/*/kernel/drivers/ieee1394
rm -fvr /snapstrip/lib/modules/*/kernel/drivers/parport
rm -fvr /snapstrip/lib/modules/*/kernel/drivers/pcmcia
rm -fvr /snapstrip/lib/modules/*/kernel/drivers/telephony
rm -fvr /snapstrip/lib/modules/*/kernel/drivers/isdn
rm -fvr /snapstrip/lib/modules/*/kernel/drivers/md

rm -fvr /snapstrip/lib/modules/*/kernel/fs/ntfs
rm -fvr /snapstrip/lib/modules/*/kernel/fs/reiserfs
rm -fvr /snapstrip/lib/modules/*/kernel/fs/hfs
rm -fvr /snapstrip/lib/modules/*/kernel/fs/hfsplus
rm -fvr /snapstrip/lib/modules/*/kernel/fs/xfs

rm -fvr /snapstrip/lib/modules/*/kernel/net/appletalk
rm -fvr /snapstrip/lib/modules/*/kernel/net/bluetooth
rm -fvr /snapstrip/lib/modules/*/kernel/net/irda

### ADD IN MORE STUFF TO STRIP HERE ###

# Create the tar archive
cd /snapstrip/
tar cf /snapstrip.tar *

Run the script to create the tar archive. You'll run this script after making changes to the main OS to create a new snapshot file.

-------------------------------------------------
Step 5. Create a custom initrd.img which loads the OS image into a tmpfs ramdisk

This is step is a hack. It works with Debian 4.0. There's probably a less "hackish" way of doing this.

Use the following commands:

Code: Select all

cd /usr/share/initramfs-tools/scripts/
cp -vax local local.bak
nano local
cp -vax local local.ramboot


In nano, you'll want to modify the portion where the actual "mount" command is done. Comment it out and insert something like this:

Code: Select all

[...]
########### ramboot
        # FIXME This has no error checking
        # Mount root
###     mount ${roflag} -t ${FSTYPE} ${ROOTFLAGS} ${ROOT} ${rootmnt}

########### mount the filesystem
        mkdir /ijkijk
        mount ${roflag} -t ${FSTYPE} ${ROOTFLAGS} ${ROOT} /ijkijk

########### create root ramdisk
        mount -t tmpfs -o size=100% none ${rootmnt}

########### copy the files over to the ramdisk
        cd ${rootmnt}
        tar xf /ijkijk/snapstrip.tar

########### umount the filesystem and set to spin down
        umount /ijkijk
   hdparm -S 1 /dev/hda
[...]

After making these modifications, create the initrd.img with this command:

Code: Select all

mkinitramfs -o /boot/initrd.img.ramboot

After creating this ramdisk make sure to copy back the backup file with:

Code: Select all

cp -vax local.bak local

This is important! If you forget to do this, then your system will be screwed up if your kernel is upgraded!

-------------------------------------------------
Step 6. Modify /boot/grub/menu.lst with an entry for the custom initrd.img

Modify /boot/grub/menu.lst with a new entry. Copy existing OS's entry. Then modify the initrd to use your new initrd.img. It will look something like this:

Code: Select all

title   RAMdisk Debian GNU/Linux
root    (hd0,0)
kernel  /boot/vmlinuz-2.6.18-6-486 root=/dev/hda1 ro
initrd  /boot/initrd.img.ramboot

-------------------------------------------------

After following these steps, you'll have a very basic working system. Now you can boot into the "main" OS and install things like X (only install the xserver you need) and other programs like icewm and iceweasel. For example:

apt-get install xserver-xorg-video-vesa xserver-xorg-video-ati xfonts-base alsa-base alsa-utils icewm menu iceweasel xfe aterm

The default icewm theme is rather ugly, so you can copy over a nice theme like /usr/share/icewm/themes/IceCrack2 from another install. Obviously, you don't want to install all of the themes in icewm-themes since they'll be consuming RAM just sitting there.

[DsOft]

OMG, no comments in more than a month?

I just found that Howto, that way owns, the improved performance will be godly, just like "Tin Hat" or "Puppy Linux" but with the power of debian.

Anyone have tryed it?


IsaacKuo, thank you so much for that guide.
Ill try it after backup

[DsOft]

Ok, i did it on my laptop, but, i dont know if it worked well, since i've a pretty fast laptop.
How can i truly check it?

those are vmstats:

Normal boot:
dsoft@Goliat:~$ vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 0 0 3296960 12068 180780 0 0 692 176 183 576 10 3 74 13

Ramboot:
dsoft@Goliat:~$ vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 0 1820228 852 1625940 0 0 8401 1 199 798 8 5 74 13


And in normal boot i can use apt, ramboot dont allow me to use it (error)

Is it working?
i think its, cause of the cache memory difference betwen normal and ramboot.

[IsaacKuo]

The instructions I give above delete almost everything related to apt, to minimize the size of the OS filesystem. If you want to install anything new, you first reboot into the normal hard drive install, and then install new software. After that, you run the snapstrip script again to create the new image.

Depending on how much RAM you have, you can choose to not delete so much stuff in the snapstrip script.

I'm not familiar with vmstat, but the way I determine how I'm doing for RAM usage is a combination of "free -m" and "df". Also, when I'm booted into the normal hard drive install, I can look at the size of /snapstrip.tar to see how much space the OS will use up when booted up into RAM.

[roooz]

Could this be useful for an 'multimedia pc' (e.g. to record music, safe cpu and noise?)
And is it also usable for debian testing and sid?

[IsaacKuo]

I use it for my silent HTPCs, as well as my workstations. Currently, I'm running Lenny on all of them (I upgraded because I wanted aufs). You need to go through a bit of effort when you want to upgrade the kernel. This is less of an issue with Debian Stable than Testing or Unstable.

[roooz]

I use it for my silent HTPCs, as well as my workstations. Currently, I'm running Lenny on all of them (I upgraded because I wanted aufs). You need to go through a bit of effort when you want to upgrade the kernel. This is less of an issue with Debian Stable than Testing or Unstable.

What kind of effort specifically (could you describe howto do it)? And is it possible to use this howto for an existing debian testing installation?
thanks

edit: and what if I just want to use this with an 'almost never changing' realtime kernel? So I've the default debian testing kernels, but one realtime kernel which I use for making music...

[lindylex]

DsOft, roooz I did this on a file server because I did not want the hard drives to spin when nothing was being sent to it. I also want to make the computer a little greener, so it save electricity. And I wanted it to be quiet.

I recommend it everyone give this a try for all computing situations since ram is so cheap and new mother boards support so much.

Thanks, IsaacKuo for this great how to.

[IsaacKuo]

I use it for my silent HTPCs, as well as my workstations. Currently, I'm running Lenny on all of them (I upgraded because I wanted aufs). You need to go through a bit of effort when you want to upgrade the kernel. This is less of an issue with Debian Stable than Testing or Unstable.

What kind of effort specifically (could you describe howto do it)? And is it possible to use this howto for an existing debian testing installation?
thanks

Sorry, I forgot to reply to this. If your kernel changes, then you need to make a new initrd.img.ramboot. Here are the steps:

cd /usr/share/initramfs-tools/scripts
cp -vax local.ramboot local
mkinitramfs -o /boot/initrd.img.ramboot
cp -vax local.bak local

This will use the files "local.bak" and "local.ramboot" which you created during the initial HowTo. I hope the idea is self explanatory.

[didi]

Would this work if you have configured your drives in a RAID (1) array?
Any additional/other steps to take in order to get this working?
Does it matter that I use LVM (Logical Volume Manager) over RAID?

I would like to apply this to my server, which has 3 drives
- 1 IDE drive (80GB)
- 2 SATA drives (320GB) in RAID1 with LVM on top of it

hdparm does support SATA drives (hdparm -I /dev/sda produces output) as stated in the manpage, but not in the package description.

[superstubby]

Thanks Mr. Isaac, my brother from another mother!

You've really peaked my interest on this one. I'm a non-virginal noob, been messing with Linux the last 2 years or so.

I would love to implement this on my laptop. I've been experimenting with minimal installs for the last few days. I mostly followed your advice by installing the base Debian system with the addition of the laptop components (and no desktop). I then installed X11, Xfce4, and Xdm with apt.

I mostly use my laptop for working on LaTeX docs, so texlive came next along with Emacs. I know I could save some space with different alternatives here, but I'd rather stick with what I know, for now. The only other things I installed were Xfmedia, Iceweasel (gotta love those addons), and the battery monitor plugin for Xfce. Before removing the files you suggest, my root partition is about 1.2GB. After, I'm a little under 900MB (haven't tried tarring it yet). I have 1GB of mem in my laptop. So, I don't think I should even try actually using this yet. Another 1GB mem chip is running about $15 right now (add to cart...).

I have some questions: will I save any space using Icewm instead of Xfce? While I was copying the root dir I noticed that there seems to be a fair amount of locale data sitting around, is it worth while to remove it? I read somewhere that aptitude can complain if you do, but that shouldn't be a problem as your method removes most of apt anyway.

Thanks again!

PS. I intend on continuing to use the harddrive for data. How does that change any of the scripts you've posted? I read your other thread on this and I understand you use an NAS for /home, but I can't.

[IsaacKuo]

I don't know how much space is consumed by XFCE4, but IceWM is extremely lightweight, so it would save space. Another thing that saves space is to only install the xorg video driver you need (described in the how-to at the end).

Using the hard drive for data has no effect on any of the scripts. Just include your desired data partition mounts in /etc/fstab and /etc/fstab.ramboot. Include the "noatime" option to minimize/eliminate the need for the OS to access the drive when not actively using it. That way, the hard drive can spin down to save on battery power when you're, say, browsing the Internet or other activity that doesn't involve reading/writing saved files.

My ramboot workstations have a local data partition mounted, although it is only sporadically used. I use the local data partition to backup data from my network drive (using rsync). Even though this partition is left mounted all the time, the hard drive doesn't need to spin up unless I'm running my rsync script.

[Jackiebrown]

Just curious, but does this burn out your ram quicker?

I run a lot of CPU intensive tasks. I assume, that that would not effect this since they use more CPU than ram?

Has anyone tried this on a rolling distribution (say Sid?)

[superstubby]

How to RAMboot

Code: Select all

apt-get install hdparm localepurge debconf-english

Jeez... I must have been tired. Completely skipped some of what you were saying...

Anyway, for posterity's sake, I tried IceWM and seemed to save 300MB+. However, (no offense intended) I didn't quite like it as much as Xfce. I had previously tried Fluxbox and Enlightenment. But I'm not quite that masochistic. I think I'll stick with Xfce.

[superstubby]

Hey Prof. Isaac,

I have a few (more) questions. When I said I was non-virginal, I meant that I've been screwed before by not understanding what I was doing. I've read up a bit on some of the commands here, but I'm not sure exactly how Debian (or linux in general) works in this case.

I'm still a little foggy on how the initrd / initramfs works, but most of the scripts seem pretty easy to understand. However, when you run

Code: Select all

mount -t tmpfs -o size=100% none ${rootmnt}

are you telling it to use 100% of mem for the RAM disk? Or is that 100% of what's left after setting aside enough RAM to run the system? Or, do I completely not understand....