RAM boot - improved (named "make-me-live")

Share your own howto's etc. Not for support questions!

RAM boot - improved (named "make-me-live")

Postby thewanderer » 2010-02-07 16:17

Hello,
This is a howto on my new and improved RAM booting method. The original, by IsaacKuo, is located here:
viewtopic.php?f=16&t=29774&start=0
It has been a base for building this solution to a huge extent. However, as this is meant to be a standalone how to, let's see what you can accomplish by this method:
What this is about
1. The solution allows you to load your whole Debian system into RAM memory so that you can minimize your disk usage after it has been fully loaded and receive a performance gain, eliminating the I/O bottleneck
2. An addinational feature is an union filesystem which allows you to still read your system from the storage (disk) but write the changed files to RAM. This way, you achieve a COW (Copy-On-Write) setup.

Use cases
1. Building a snappy desktop machine to deliver blazing performance for amazing your friends
2. Running a Debian system from only a single medium (e.g. a pendrive) on multiple workstations - you boot on one, then pull the storage and plug it into another and so on
3. Building a kiosk system where all changes should be lost between reboots
4. Enhancing security by only booting into 'read-write' mode when upgrades are needed - to minimize disk activity, one would need some kind of network storage (this is out of the scope of this document)

Goals
  • To provide a convenient way of allowing the system to boot into RAM / to boot in a volatile mode
  • To be non-intrusive and to impose no unnecessary difficulties with management tasks on the system administrator
  • To actually work

Limitations
Currently, this solution has several drawbacks. I am going to target them in the future or find someone who will do it for me. These are:
  • Mounting only 1 filesystem/partition is supported by the provided scripts - you are supposed to keep everything on one partition, otherwise your files from other partitions will not be there when you boot. You can still mount your own filesystems during runtime, though.
  • There is no cleanup when booting to RAM - the whole system gets copied over, so with a standard desktop install, the memory can be filled very easily. Therefore, it is important to carefully choose your packages when planning a ram-booting system. It could probably be improved by using some of IsaacKuo's advice, like only copying what is necessary - this is a TODO.

The actual HOWTO
All the scripts are packaged and available as a deb file for installation with dpkg.
The package uses dpkg-divert during installation and removal and replaces initramfs scripts with its custom ones to accomplish its goals.
There is no source deb because all files in the actual package are human-readable script files anyway. You are free to examine their content and modify them as you wish etc.

1. Get the package (along with dependencies) and install it
Code: Select all
# The dependency:
apt-get install aufs-tools
# The package file
wget http://rkaw.pl/make-me-live_0.2.deb
# at this point, if you want to review what's being installed - a good reading for educational purposes:
# mkdir make-me-live && cd make-me-live &&  dpkg-deb --extract make-me-live_0.2.deb . && dpkg-deb --control make-me-live_0.2.deb
dpkg --install make-me-live_0.2.deb


Huge fat note: This package requires your kernel to support the aufs filesystem. If you are using kernel 2.6.32 or any of the recent kernels, it should already be there. However, for Debian Lenny, you have to install it by:
Code: Select all
apt-get install aufs-modules-$(uname -r)

It is not included in the dependencies as newer kernels do not need those modules.

Now, you need to regenerate your initramfs:
Code: Select all
update-initramfs -k all -u


2. How to make it run
The newly installed initramfs scripts support two new boot options. That means, you can just pass options to the kernel and your system will boot accordingly.
The boot options are:
toram - loads your whole system into RAM (option 1 as described at the top of this document)
live - mounts your root filesystem as aufs with read-write branch being tmpfs, so all changes done to the filesystem will be lost after reboot

For testing purposes, you may just append one of those options to your kernel command line in your boot loader.
When you are satisfied with how it works, you can, for example, set one of those options as a default boot option (should be the case when you are building a kiosk machine).

3. Notes on removal / upgrade
Removal should be straightforward if ever needed. However, it helps to check if all the files are back on their places (for list of files, see the package file).

Please report bugs and, possibly, patches. Thanks for your attention.
thewanderer
 
Posts: 416
Joined: 2007-03-19 18:11
Location: my desk, Warsaw, Poland

Re: RAM boot - improved (named "make-me-live")

Postby Jackiebrown » 2010-02-08 06:24

You might also lookk at debirf
User avatar
Jackiebrown
 
Posts: 1276
Joined: 2007-01-02 04:46
Location: San Antonio, TX

Re: RAM boot - improved (named "make-me-live")

Postby thewanderer » 2010-02-08 15:42

Thanks for the link. However, my approach is different in that it does not try to build a custom initrd - it just boots your system in a "live" mode, so it is easy to modify it without a complete rebuild. I already knew of Debian Live, but it simply did not fit my needs and I think there will be more uses for my solution.
thewanderer
 
Posts: 416
Joined: 2007-03-19 18:11
Location: my desk, Warsaw, Poland

Re: RAM boot - improved (named "make-me-live")

Postby Jackiebrown » 2010-02-09 10:13

thewanderer wrote:Thanks for the link. However, my approach is different in that it does not try to build a custom initrd - it just boots your system in a "live" mode, so it is easy to modify it without a complete rebuild. I already knew of Debian Live, but it simply did not fit my needs and I think there will be more uses for my solution.

That is what debrif does. I think you are thinking of something else :)
debirf (DEBian on Initial Ram Filesystem) is a set of tools designed to create and prepare a kernel and initial ram filesystem that can run a full-blown Debian environment entirely from RAM.
User avatar
Jackiebrown
 
Posts: 1276
Joined: 2007-01-02 04:46
Location: San Antonio, TX

improved

Postby llivv » 2010-02-09 13:08

if there were a way for you to sign your deb I would be less reluctant to check the scripts out of your repo..
I have HOW-TO: New and Improved RAMboot for speed and silence bookmarked and use it quite often as a reference...
I'm not a big fan of diverts either, but just because your package uses a divert wouldn't necessarily stop me from trying it, if it were signed
in the kitchen with Julia
The Past, Christmas Present and Future
Get on the Dbus to Bcan
User avatar
llivv
 
Posts: 5614
Joined: 2007-02-14 18:10
Location: cold storage

Re: RAM boot - improved (named "make-me-live")

Postby thewanderer » 2010-02-09 21:13

Firstly, this package is different in that it modifies the standard initrd script, so there is no 'special' generation phase - you just install it and forget it exists, for all your next kernel initrd's will already be live-enabled. It also does not create a separate runtime environment, but uses your currently installed one, so if you install a package in read-write mode on your Debian system - it goes straight into the 'live' version, too. It does so while staying very close to core Debian, so all your other modifications of initrd, hooks etc. are preserved.
Well, I guess I could sign the package, but I don't have a repo nor a widely recognized digital signature - just a personal website. For your convenience, though, here is the SHA1 checksum:
Code: Select all
60ba34929f651cd8cededfc9e522bcc723d22167  make-me-live_0.2.deb

I've never had an own (real) repository nor do I know how to create one with signed packages. I advise you to read the source modifications, anyway, before installing - you might want to tweak something (even though the code is extraordinarily simple).
thewanderer
 
Posts: 416
Joined: 2007-03-19 18:11
Location: my desk, Warsaw, Poland

Re: RAM boot - improved (named "make-me-live")

Postby Jackiebrown » 2010-02-10 09:47

I missed that yours has a deb already made to do the heavy work. Sweet!
User avatar
Jackiebrown
 
Posts: 1276
Joined: 2007-01-02 04:46
Location: San Antonio, TX

Re: RAM boot - improved (named "make-me-live")

Postby llivv » 2010-02-10 11:30

thewanderer wrote: I advise you to read the source modifications, anyway, before installing - you might want to tweak something (even though the code is extraordinarily simple).
Can you talk about the reasoning for the divert?
Are both initramfs-tools and live-initramfs being used to build the system initrd image?
If yes, are there any potential issues I might see?
And is the divert listed in the filesystem?
etc...
in the kitchen with Julia
The Past, Christmas Present and Future
Get on the Dbus to Bcan
User avatar
llivv
 
Posts: 5614
Joined: 2007-02-14 18:10
Location: cold storage

Re: RAM boot - improved (named "make-me-live")

Postby thewanderer » 2010-02-10 19:57

I consider the divert as needed because there is no other way I could replace or change a stock Debian file delivered with another package.
There are two diverts: initramfs-tools/scripts/local and mkinitramfs. The one regarding mkinitramfs should not generally be needed (as you can see by a diff between this one and the original). Therefore, I see the diversion of mkinitramfs script itself as unnecessary as of now. It used to perform some task (don't exactly remember which, perhaps the aufs mount helper hook), but the functionality has been implemented using more Debian-compliant way. So the diversion of mkinitramfs can be thought of as a bug (though not impacting any functionality), but still, the "local" init script of initramfs-tools needs to be changed (diversion #0 as mentioned above). It is crucial for the functionality to alter the local boot process and this is the most appropriate place. It does not modify the original code flow in any way, provided that no live-related flags are passed to the kernel, and so, it does not alter one's system' behaviour in any noticeable way unless they enable the extra functionality. Also, as far as I know, the init script is changed by the original developers once in a blue moon, so no functionality should be broken in the nearest future. That is my rationale for the diversion of /usr/share/initramfs-tools/scripts/local.

Actually, live-initramfs is not used in any way. Is it listed in the dependencies? It shouldn't be. This method is completely independent of live-initramfs. However, I think a kernel parameter option conflict might appear when the two are run in parallel. Therefore, if running live-initramfs next to my package, expect strange behaviour when booting in live mode - it could be avoided by changing the names of boot parameters in my package, for sure (to some less intuitive names). For now, the package conflicts with live-initramfs - Conflicts field will be added to the next release of the package or the conflict must get fixed before.
Live-initramfs seems to handle this process using hooks - I am going to look into that soon and see if it is possible to port my little package to hooks to avoid diversions. But still, they are harmless. The basic reason to use my boot method over live-initramfs is that it supports booting into RAM whereas such an option is not mentioned in live-initramfs (it might be supported, though - I have no idea). The code is also extremely simple, and Debian-Live's scripts tend to look complex. Of course, dpkg-diverts look quite ugly, but that does not stop me from using it on several virtual machines and one physical PC - works fine.

The divert is not "listed" in the filesystem, but the pre-divert (original) file and the new file are both there and they share the original directory. Moreover, `dpkg-divert` can list current dpkg diversions in effect - the same standard mechanism as with packages providing aliases (vim: vim-tiny, awk: mawk; etc).
thewanderer
 
Posts: 416
Joined: 2007-03-19 18:11
Location: my desk, Warsaw, Poland

Re: RAM boot - improved (named "make-me-live")

Postby llivv » 2010-02-12 11:20

Thanks for explaining...
I'll take a look when I get some free time
in the kitchen with Julia
The Past, Christmas Present and Future
Get on the Dbus to Bcan
User avatar
llivv
 
Posts: 5614
Joined: 2007-02-14 18:10
Location: cold storage

Re: RAM boot - improved (named "make-me-live")

Postby llivv » 2010-02-20 13:50

I noticed that loop-aes-utils install as a diversion as well. I need to make time to check these out....
thanks again wander and Daniel b
in the kitchen with Julia
The Past, Christmas Present and Future
Get on the Dbus to Bcan
User avatar
llivv
 
Posts: 5614
Joined: 2007-02-14 18:10
Location: cold storage

Re: RAM boot - improved (named "make-me-live")

Postby thewanderer » 2010-04-21 17:29

I'm now using this on my ThinkPad X60 to make it even more portable. With the right system configuration (my installed Debian is less than 1GB in size) and my tiny set of scripts, I've managed to cut its power usage to the bare minimum, running entirely from RAM. Plus, should I need to install anything, I just mount my root partition, chroot onto it, perform the required operations, and exit. That's it - the mobility and silence of a netbook (is probably more silent due to the hard disk not spinning and the second CPU core powered off => less heat dissipation) with the power of a laptop.
thewanderer
 
Posts: 416
Joined: 2007-03-19 18:11
Location: my desk, Warsaw, Poland


Return to Docs, Howtos, Tips & Tricks

Who is online

Users browsing this forum: No registered users and 4 guests

fashionable