Upcoming how-to...RAMboot to a ramdisk for blazing speed!

Here you can discuss every aspect of Debian. Note: not for support requests!

Re: Upcoming how-to...RAMboot to a ramdisk for blazing speed

Postby Aubrey_Snoddie » 2013-03-15 10:18

Perhaps Isaac has not read the questions in my last post...
User avatar
Aubrey_Snoddie
 
Posts: 11
Joined: 2013-03-14 11:22
Location: EU

Re: Upcoming how-to...RAMboot to a ramdisk for blazing speed

Postby Pick2 » 2013-03-15 14:20

This thread had been dead since 10 Jun 2008 ,that's 4 3/4 years.
I would not get to impatient or get your hope set to high :D
User avatar
Pick2
 
Posts: 797
Joined: 2007-07-07 13:31
Location: Decatur Il

Re: Upcoming how-to...RAMboot to a ramdisk for blazing speed

Postby Aubrey_Snoddie » 2013-03-15 14:42

Strange, Isaac replied yesterday.

I woul have thought that with the normal man in the street being able to afford 16GB RAM nowadays, a RAMboot (persistent) would have aroused more interest.

Silly me.
User avatar
Aubrey_Snoddie
 
Posts: 11
Joined: 2013-03-14 11:22
Location: EU

Re: Upcoming how-to...RAMboot to a ramdisk for blazing speed

Postby IsaacKuo » 2013-03-15 18:47

Sorry, I was a bit busy. I'm not sure exactly what you mean by "checkpointing". Periodic use of rsync could be good enough for what you want...but it sounds like you want to do periodic differential backups while running. I haven't experimented with it, but perhaps you could find what you're looking for with rsnapshot (which uses rsynce on the back end).

How about this:

1) Main OS image is normal partition on SSD.
2) Main OS is copied to tmpfs ramdisk on boot.
3) One or more "snapshot" images use aufs, attached to other partitions on SSD. (These can be smaller.) Alternatively, if you have plenty of space on the SSD, these can also be normal partitions--equal in size to the main OS image. I would recommend the latter, at least to get things going.
4) Periodically use rsync to backup tmpfs ramdisk to "snapshot" image.
5) On shutdown, rsync to copy tmpfs ramdisk back to main OS image.

I would recommend another backup image that you manually copy over periodically in case the main OS image gets messed up.
Isaac Kuo
User avatar
IsaacKuo
 
Posts: 316
Joined: 2008-04-24 20:06

Re: Upcoming how-to...RAMboot to a ramdisk for blazing speed

Postby IsaacKuo » 2013-03-15 18:51

Thinking about it, I'd recommend skipping aufs and just using two or more normal partitions. The big disadvantage of aufs, besides the added complexity and inefficiency, is that if your main OS image gets messed up all aufs images based on it get messed up also. So you need another full size image anyway as a backup.

So...I'd just go with two or more normal partitions sized for your OS. The main one is what the boot copies from and rsyncs to on startup/shutdown. The secondary ones are the snapshots rsynced periodically while running. The danger of corruption of a secondary one is not such a big deal. It's really mostly the same effect as accidental loss of power, if something goes wrong in the middle of an rsync.
Isaac Kuo
User avatar
IsaacKuo
 
Posts: 316
Joined: 2008-04-24 20:06

Re: Upcoming how-to...RAMboot to a ramdisk for blazing speed

Postby Aubrey_Snoddie » 2013-03-16 15:34

Isaac,

again, thank you for your time and the brain cells (if any) expended on thinking about my wacky idea.

I have spent 35 years working with IBM mainframes and am not a complete *nix novice but "copying my main OS partition into a tmpfs RAMdisk at boot" and everything else that that entails is beyond me.

Is there a cook book / howto on how to do same somewhere?

Would it be advisable to have a sandbox / test environment / chroot available to do all of this?

Thanks again.


Aubrey
User avatar
Aubrey_Snoddie
 
Posts: 11
Joined: 2013-03-14 11:22
Location: EU

Re: Upcoming how-to...RAMboot to a ramdisk for blazing speed

Postby llivv » 2013-03-16 18:49

Aubrey_Snoddie wrote:Isaac,
I have spent 35 years working with IBM mainframes and am not a complete *nix novice but "copying my main OS partition into a tmpfs RAMdisk at boot" and everything else that that entails is beyond me.

Is there a cook book / howto on how to do same somewhere?

ibm should have a few,
Also, a short description of what you have done so far while implementing your " wacky idea"
would help the rest of us a lot in figgering out where to point you.
Aubrey_Snoddie wrote:Would it be advisable to have a sandbox / test environment / chroot available to do all of this?
in my mind that is kinda a redundant question because the ramboot build processs
kinda parallels all three to some degree.

original hint:
have you looked at the debian debirf package yet?

I don't know about anyone else, but

I'm eagerly awaiting the upcoming howto

and hoping for more stripping hints.
In memory of Ian Ashley Murdock (1973 - 2015) founder of the Debian project.
User avatar
llivv
 
Posts: 5488
Joined: 2007-02-14 18:10
Location: cold storage

Re: Upcoming how-to...RAMboot to a ramdisk for blazing speed

Postby AnInkedSoul » 2013-03-16 19:17

is this the new and improved version?
viewtopic.php?f=16&t=29774
AnInkedSoul
 
Posts: 461
Joined: 2010-06-11 05:05

Re: Upcoming how-to...RAMboot to a ramdisk for blazing speed

Postby Aubrey_Snoddie » 2013-03-16 19:19

In one sentence have memory speed when doing I/O or anything type of fetching, just like with the good ole Mainframe.

In layman's terms -

Boot to Grub, copy OS to memory, checkpoint (write changes to SSD) every 5 mins for example - run as normal / daily stuff etc. etc. At shutdown time write OS-in-memory back to SSD and power off - simple or not?

Have a gander at these figures -

root@Carcassonne ~: df -ah
Filesystem Size Used Avail Use% Mounted on
rootfs 12G 5.8G 5.1G 54% /
sysfs 0 0 0 - /sys
proc 0 0 0 - /proc
udev 10M 10M 0 100% /dev
devpts 0 0 0 - /dev/pts
tmpfs 1.6G 712K 1.6G 1% /run
/dev/disk/by-uuid/fd78685e-ea1b-4ad3-8583-0aff167de1fe 12G 5.8G 5.1G 54% /
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 3.1G 0 3.1G 0% /run/shm
tmpfs 9.0G 20K 9.0G 1% /tmp
tmpfs 7.8G 0 7.8G 0% /var/tmp
tmpfs 7.8G 340K 7.8G 1% /var/log
binfmt_misc 0 0 0 - /proc/sys/fs/binfmt_misc
root@Carcassonne ~: dd if=/dev/zero of=/tmp/tt bs=1M count=8192
8192+0 records in
8192+0 records out
8589934592 bytes (8.6 GB) copied, 1.97469 s, 4.4 GB/s
root@Carcassonne ~: df -ah
Filesystem Size Used Avail Use% Mounted on
rootfs 12G 5.8G 5.1G 54% /
sysfs 0 0 0 - /sys
proc 0 0 0 - /proc
udev 10M 10M 0 100% /dev
devpts 0 0 0 - /dev/pts
tmpfs 1.6G 712K 1.6G 1% /run
/dev/disk/by-uuid/fd78685e-ea1b-4ad3-8583-0aff167de1fe 12G 5.8G 5.1G 54% /
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 3.1G 0 3.1G 0% /run/shm
tmpfs 9.0G 8.1G 1.0G 89% /tmp
tmpfs 7.8G 0 7.8G 0% /var/tmp
tmpfs 7.8G 344K 7.8G 1% /var/log
binfmt_misc 0 0 0 - /proc/sys/fs/binfmt_misc
root@Carcassonne ~: free -g
total used free shared buffers cached
Mem: 15 8 6 0 0 8
-/+ buffers/cache: 0 14
Swap: 0 0 0
root@Carcassonne ~:

As you can see I have a supposed throughput of 4.4 GB/sec

I must state here that with having a fast SSD (Samsung 840 Pro) I will probably never feel the "difference" between the relatively slow SSD transfers speeds compared to RAM-to-RAM.

I am running on a miniscule Thinkpad X230 with 16GB RAM, am not a gamer, overclocker or media weirdo. I have a few Iceweasel browser windows open, VPN perhaps also, otherwise no heavy load.

It would be nevertheless nice to see the memory (RAM) being used more efficiently though...

Pray tell what is "figgering"??


Aubrey
User avatar
Aubrey_Snoddie
 
Posts: 11
Joined: 2013-03-14 11:22
Location: EU

Re: Upcoming how-to...RAMboot to a ramdisk for blazing speed

Postby Aubrey_Snoddie » 2013-03-16 19:21

I have had a look at debirf and was not impressed - too convoluted and not intuitive,
User avatar
Aubrey_Snoddie
 
Posts: 11
Joined: 2013-03-14 11:22
Location: EU

Re: Upcoming how-to...RAMboot to a ramdisk for blazing speed

Postby Aubrey_Snoddie » 2013-03-17 13:06

Quote from Isaac "Thinking about it, I'd recommend skipping aufs and just using two or more normal partitions...."

I notice that my Debian 3.8.x needs AUFS(3) patches.

I received this message -

"mounting aufs on /root/ failed: no such device.." using this information I found on my google travels - http://mce.commsbyte.com/index.php/arti ... isk-how-to

In the great (Live) scheme of things, how does one substitute a "normal" (tmpfs?) partition instead of AUFS.

Perhaps Daniel Baumann is listening?

Would be most appreciative of any enlightening pointers / ideas.




Regards,


Aubrey
User avatar
Aubrey_Snoddie
 
Posts: 11
Joined: 2013-03-14 11:22
Location: EU

Re: Upcoming how-to...RAMboot to a ramdisk for blazing speed

Postby IsaacKuo » 2013-03-17 15:16

AnInkedSoul wrote:is this the new and improved version?
viewtopic.php?f=16&t=29774


Ah, yes--that was indeed the new and improved version. Of course, I wrote that ages ago--but still, I never made any significant improvements over that.

If it's not clear how to use it, maybe it would be best to ask questions about it on the other thread.
Isaac Kuo
User avatar
IsaacKuo
 
Posts: 316
Joined: 2008-04-24 20:06

Re: Upcoming how-to...RAMboot to a ramdisk for blazing speed

Postby IsaacKuo » 2013-03-17 15:20

Aubrey_Snoddie wrote:Quote from Isaac "Thinking about it, I'd recommend skipping aufs and just using two or more normal partitions...."

In the great (Live) scheme of things, how does one substitute a "normal" (tmpfs?) partition instead of AUFS.


I'm sorry for not being clear--the "normal partitions" I refer to would be normal hard drive partitions using whatever normal file system you prefer--like ext3. So, you have two or more ext3 partitions on your SSD. In RAM, you just have one tmpfs file system for the OS.
Isaac Kuo
User avatar
IsaacKuo
 
Posts: 316
Joined: 2008-04-24 20:06

Re: Upcoming how-to...RAMboot to a ramdisk for blazing speed

Postby Aubrey_Snoddie » 2013-03-17 16:00

Thank you Isaac.

Seeing as the original article was written in 2008 before SSD and cheap memory became mainstream, is there any chance of an update to your masterpiece to reflect today's state of things?

I would like to achieve all of this without installing multiple OSes on the machine as I have enough space to copy / rsync things around...

Again, apologies for the bother.


Aubrey
User avatar
Aubrey_Snoddie
 
Posts: 11
Joined: 2013-03-14 11:22
Location: EU

Re: Upcoming how-to...RAMboot to a ramdisk for blazing speed

Postby IsaacKuo » 2013-03-17 16:24

Aubrey_Snoddie wrote:Thank you Isaac.

Seeing as the original article was written in 2008 before SSD and cheap memory became mainstream, is there any chance of an update to your masterpiece to reflect today's state of things?

I'm actually using the same hardware I was using back then. My main computers are still the same Socket 754 Sempron builds. The motherboards are limited to 2gigs of RAM, so I really haven't had the chance to do much more.

In the years since, basic web browser requirements have ballooned, so 2gigs just isn't enough. I could turn off flash in the old days, and still have something usable for the vast majority of websites. But that's not possible with javascript. So, I eventually abandoned RAMboot in favor of normal installs, until I got more capable hardware. So far, I haven't.
I would like to achieve all of this without installing multiple OSes on the machine as I have enough space to copy / rsync things around...

I'm not sure what your goals are. If you want to have multiple "snapshots", they have to go somewhere. I don't see any point to putting them in RAM.

If you just want to periodically save changes back to the SSD, periodically running rsync to incrementally backup from the tmpfs OS partition to the SSD partition will work.
Again, apologies for the bother.

No problem! It's my pleasure to assist!
Isaac Kuo
User avatar
IsaacKuo
 
Posts: 316
Joined: 2008-04-24 20:06

PreviousNext

Return to General Discussion

Who is online

Users browsing this forum: No registered users and 8 guests

fashionable