Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Can I put /var on a RAMDISK

Ask for help with issues regarding the Installations of the Debian O/S.
Message
Author
larienna
Posts: 106
Joined: 2014-09-27 20:54

Can I put /var on a RAMDISK

#1 Post by larienna »

I made a new installation of Debian 11 on my old HP mini 110 1030 and since it now runs on a SSD, I am trying to see how I could limit useless writes.

I enabled tmp.mount service to set /tmp on a ramdisk. But now I am wondering if I could put the entire /var directory, or some sub directories on a RAMDISK.

Can it be done? If yes, how to I proceed?

Do I simply create an entry in fstab and delete the /var folder on the disk?

Currently my /var folder is 836M (682M comes from the /var/lib folder), since I only have 1 Gig of RAM and willing to upgrade to 2G, putting the entire /var directory might not be the best solution. Which subfolders are mostly written in the /var folder:

/var/tmp
/var/log

Seems like good candidates

User avatar
canci
Global Moderator
Global Moderator
Posts: 2502
Joined: 2006-09-24 11:28
Has thanked: 136 times
Been thanked: 136 times

Re: Can I put /var on a RAMDISK

#2 Post by canci »

I'd be hesitant to put this into RAM, as I've read that e.g. databases tend to store things there. See here:
https://superuser.com/questions/20653/w ... -directory
Image Stable / Asus VivoBook X421DA / AMD Ryzen 7 3700U / Radeon Vega Mobile Gfx (Picasso) / 8 GB RAM / 512GB NVMe

READ THIS:

* How to Post a Thread Here
* Other Tips and Great Resources

Bulkley
Posts: 6383
Joined: 2006-02-11 18:35
Has thanked: 2 times
Been thanked: 39 times

Re: Can I put /var on a RAMDISK

#3 Post by Bulkley »

larienna wrote: 2022-07-26 13:20 . . . I am trying to see how I could limit useless writes.
Do a little research on setting swappiness.

Tinkering with swappiness might help or it might not depending on your machine and how you use it. At least you will know it is there and accessible.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: Can I put /var on a RAMDISK

#4 Post by Head_on_a_Stick »

larienna wrote: 2022-07-26 13:20 since it now runs on a SSD, I am trying to see how I could limit useless writes
Any advice to restrict writes for SSDs has been obsolete for quite some time now. Modern SSDs can outlast spinning rust devices.
deadbang

p.H
Global Moderator
Global Moderator
Posts: 3049
Joined: 2017-09-17 07:12
Has thanked: 5 times
Been thanked: 132 times

Re: Can I put /var on a RAMDISK

#5 Post by p.H »

larienna wrote: 2022-07-26 13:20 I am wondering if I could put the entire /var directory
No. Or at least you should not unless you know exactly what you are doing. But the fact that you are asking indicates that you do not.
Parts of /var such as /var/lib are used to store persistent state data thus should not use volatile storage. Even not all /var/cache should be volatile as /var/cache/debconf contains debconf database (package configuration).
According to the FHS, /var/tmp should not be deleted on shutdown thus cannot be volatile either.

IMO, it is not worth it.

User avatar
sunrat
Administrator
Administrator
Posts: 6412
Joined: 2006-08-29 09:12
Location: Melbourne, Australia
Has thanked: 116 times
Been thanked: 462 times

Re: Can I put /var on a RAMDISK

#6 Post by sunrat »

Bulkley wrote: 2022-07-26 16:04
larienna wrote: 2022-07-26 13:20 . . . I am trying to see how I could limit useless writes.
Do a little research on setting swappiness.

Tinkering with swappiness might help or it might not depending on your machine and how you use it. At least you will know it is there and accessible.
Good reading that article. It debunks the common myth that swappiness alters the threshold of writes to swap, but rather it affects the ratio between file pages and anonymous pages. It concludes that it's generally not worth changing.
So you can see that you cannot use the Linux swappiness value to influence swap’s behavior with respect to RAM usage. It just doesn’t work like that.
Follow the advice of Head_on_a_Stick. Forget about RAMDISK, and trying to minimise writes to SSD is pointless. 1GB or even 2GB is a rather small amount of RAM for modern systems. Even opening a browser will tax it.
“ computer users can be divided into 2 categories:
Those who have lost data
...and those who have not lost data YET ”
Remember to BACKUP!

larienna
Posts: 106
Joined: 2014-09-27 20:54

Re: Can I put /var on a RAMDISK

#7 Post by larienna »

I still want to extend it's life as much as possible. With shortage of supplies coming in, finding replacement will always be more difficult and expensive.

Maybe /var is a bad idea. I intend to use my computer mostly for doing C programming. It might be too slow to do web browsing even if it's ironically a netbook.

On another thread, somebody suggested putting /home/username/.cache on a ram disk. Else he put on ram disk :

/var/spool
/var/tmp
/var/log

I used Zram which creates a swap disk in ram that is compressed.

p.H
Global Moderator
Global Moderator
Posts: 3049
Joined: 2017-09-17 07:12
Has thanked: 5 times
Been thanked: 132 times

Re: Can I put /var on a RAMDISK

#8 Post by p.H »

As I wrote previously, /var/tmp should not be volatile. I think /var/spool shouldn't either, but not sure.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: Can I put /var on a RAMDISK

#9 Post by Head_on_a_Stick »

If the OP is using systemd the canonical reference would be file-hierarchy(7), which states:
PERSISTENT VARIABLE SYSTEM DATA

/var/
Persistent, variable system data. Must be writable. This directory might be pre-populated with vendor-supplied data, but applications should be able to reconstruct necessary files and directories in this subhierarchy should they be missing, as the system might start up without this directory being populated. Persistency is recommended, but optional, to support ephemeral systems. This directory might become available or writable only very late during boot. Components that are required to operate during early boot hence shall not unconditionally rely on this directory.

/var/cache/
Persistent system cache data. System components may place non-essential data in this directory. Flushing this directory should have no effect on operation of programs, except for increased runtimes necessary to rebuild these caches.

/var/lib/
Persistent system data. System components may place private data in this directory.

/var/log/
Persistent system logs. System components may place private logs in this directory, though it is recommended to do most logging via the syslog(3) and sd_journal_print(3) calls.

/var/spool/
Persistent system spool data, such as printer or mail queues.

/var/tmp/
The place for larger and persistent temporary files. In contrast to /tmp/, this directory is usually mounted from a persistent physical file system and can thus accept larger files. (Use /tmp/ for small ephemeral files.) This directory is generally not flushed at boot-up, but time-based cleanup of files that have not been accessed for a certain time is applied.

If applications find the environment variable $TMPDIR set, they should use the directory specified in it instead of /var/tmp/ (see environ(7) for details).

The same security restrictions as with /tmp/ apply: mkstemp(3), mkdtemp(3), and similar calls should be used. For further details about this directory, see Using /tmp/ and /var/tmp/ Safely[5].
https://manpages.debian.org/bullseye/sy ... YSTEM_DATA

So no, /var/ should not be on a RAMDISK.
deadbang

larienna
Posts: 106
Joined: 2014-09-27 20:54

Re: Can I put /var on a RAMDISK

#10 Post by larienna »

Thanks for the doc, I did not know they documented every folder. According to the text above:
Persistency is recommended, but optional, to support ephemeral systems.
I think that means to me that /var could be on a RAM disk.

Still it could be safer to use individual sub folder. One interesting remark for /var/tmp is that it is used for larger files.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: Can I put /var on a RAMDISK

#11 Post by Head_on_a_Stick »

larienna wrote: 2022-07-31 09:00
Persistency is recommended, but optional, to support ephemeral systems.
I think that means to me that /var could be on a RAM disk.
Only for stateless systems. Which yours is not.
deadbang

jmgibson1981
Posts: 295
Joined: 2015-06-07 14:38
Has thanked: 11 times
Been thanked: 32 times

Re: Can I put /var on a RAMDISK

#12 Post by jmgibson1981 »

You need to look at current information. The way SSD's are made these days it has a strong potential of outliving even you. They aren't like the first gen ones. That has mostly been solved. Everything dies I suppose but they are so durable and long lasting these days that they give regular HDD's a run for their money on duration. Avoiding writes is almost a needless exercise the way they are made now.

Just for a personal anecdote. I've had the same SSD in my desktop for the last 6 years. I had it in my laptop for 4 or so before that. I was a perpetual distro hopper and have installed / reinstalled.... I can safely assume hundreds of times. Along with running all kinds of stuff in between.

It really has become a non issue under all but the most insane load levels.

LE_746F6D617A7A69
Posts: 932
Joined: 2020-05-03 14:16
Has thanked: 7 times
Been thanked: 65 times

Re: Can I put /var on a RAMDISK

#13 Post by LE_746F6D617A7A69 »

jmgibson1981 wrote: 2022-07-31 17:52 You need to look at current information. The way SSD's are made these days it has a strong potential of outliving even you.
Nope - the newer the SSD technology, the shorter life time. To protect that crap from wearing out within a month or two, the manufacturers are using a trick - an SSD write cache, which is constructed by dynamically reprogramming QLC cells (a complete crap) to MLC operating mode. Writing or reading directly to/from QLC is deadly slow. QLC has write endurance far lower than a 40-years old floppy disk and millions of times lower than any HDD (aka "spinning rust").

FYI: https://www.ionos.com/digitalguide/serv ... life-span/
Single-level cell SSDs (SLC) have a particularly long life, although they can only store 1 bit per memory cell. They can withstand up to 100,000 write cycles per cell and are particularly fast, durable, and fail-safe.
Multi-level cell SSDs (MLC) have a higher storage density and can store 2 bits per flash cell. They are more cost-effective than the SLC type but can only tolerate up to 10,000 write cycles per cell.
Triple-level cell SSDs (TLC) can hold 3 information bits per memory cell. However, at the same time, life expectancy can drop to 3,000 memory cycles per cell.
Quad-level cell SSDs (QLC) accommodate 4 information bits per cell. Reduced costs, more storage capacity, and higher storage density are also associated with a shorter service life with this type of device. Manufacturers usually only guarantee 1,000 write or erase cycles per cell.
See also:
viewtopic.php?p=752768#p752768

It's very important to limit the amount of writes to SSD to keep it running at reasonable speed for 2-3 years.

Programming involves a lot of writes during compiling of the program and usually it is repeated tens of times per day - so instead of moving some system dirs to a ramdisk, You should build your programs in the RAM, because this is the main source of disk writes.

BTW, if HDDs are "spinning rust" then SSD must be a "solid sh.it drive" ;)
Bill Gates: "(...) In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating system."
The_full_story and Nothing_have_changed

jmgibson1981
Posts: 295
Joined: 2015-06-07 14:38
Has thanked: 11 times
Been thanked: 32 times

Re: Can I put /var on a RAMDISK

#14 Post by jmgibson1981 »

Bastages always have to lie to us :( My machine is fast still but I should probably bench it...

Thank you for the article.

*EDIT* I've spent a long time believing that. Results of an hdparm run.

Code: Select all

$ sudo hdparm -Tt /dev/main/arch-home

/dev/main/arch-home:
 Timing cached reads:   29124 MB in  1.99 seconds = 14620.72 MB/sec
 Timing buffered disk reads: 814 MB in  3.01 seconds = 270.88 MB/sec
In this case im logged into my Arch install on LVM. Not sure how much the lvm adds to overhead, it is spanned across 2 drives. Regardless this is not close to what either drive was ever advertised at. Yet again I am corrected. Thank you very much.

p.H
Global Moderator
Global Moderator
Posts: 3049
Joined: 2017-09-17 07:12
Has thanked: 5 times
Been thanked: 132 times

Re: Can I put /var on a RAMDISK

#15 Post by p.H »

jmgibson1981 wrote: 2022-07-31 21:16 Not sure how much the lvm adds to overhead
Why don't you run hdparm on the raw drive ?
jmgibson1981 wrote: 2022-07-31 21:16 it is spanned across 2 drives
With RAID0 or linear allocation ?

jmgibson1981
Posts: 295
Joined: 2015-06-07 14:38
Has thanked: 11 times
Been thanked: 32 times

Re: Can I put /var on a RAMDISK

#16 Post by jmgibson1981 »

Just 2 drives in the same vg.

Code: Select all

vgcreate main /dev/sda2 /dev/sdb2
is roughly the command I used.

larienna
Posts: 106
Joined: 2014-09-27 20:54

Re: Can I put /var on a RAMDISK

#17 Post by larienna »

Programming involves a lot of writes during compiling of the program and usually it is repeated tens of times per day - so instead of moving some system dirs to a ramdisk, You should build your programs in the RAM, because this is the main source of disk writes.
Indeed, that way the first thing I wanted to do. I create a build folder on the ramdisk where all project will build their stuff. The source would remain on the SSD, but I setup a commit delay of 5 minutes. So in case of failure, I will lose 5 minutes of work.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: Can I put /var on a RAMDISK

#18 Post by Head_on_a_Stick »

LE_746F6D617A7A69 wrote: 2022-07-31 19:25It's very important to limit the amount of writes to SSD to keep it running at reasonable speed for 2-3 years.
I have three SSDs older than that which have never been restricted in writes. Their speed has decreased over time but only by about 12% and using blkdiscard gets the speed back, even for the oldest drive.

The OP should read https://techreport.com/review/27909/the ... -all-dead/ & https://www.guru3d.com/news-story/endur ... rites.html as well as your links.
jmgibson1981[/quote wrote:this is not close to what either drive was ever advertised at
That measurement is meaningless in isolation. Did you perform the same test when the drives were new? That would provide a base point for comparison. In my experience the specs *never* match the hdparm output.
deadbang

jmgibson1981
Posts: 295
Joined: 2015-06-07 14:38
Has thanked: 11 times
Been thanked: 32 times

Re: Can I put /var on a RAMDISK

#19 Post by jmgibson1981 »

Fair enough. I never did do that, never thought to track it. For what it's worth though between the 2 ssds. When i ran on the lv I got the above result. When I ran on the partition though it dropped on the older drive for sure. Know it's just a snapshot but I can't help but see it. sdb is the older of the 2 drives.

Code: Select all

$ sudo hdparm -Tt /dev/main/arch-home
[sudo] password for jason: 

/dev/main/arch-home:
 Timing cached reads:   28366 MB in  1.99 seconds = 14240.56 MB/sec
 Timing buffered disk reads: 814 MB in  3.00 seconds = 271.08 MB/sec
[jason@archwrecker ~]$ sudo hdparm -Tt /dev/sd[ab]2

/dev/sda2:
 Timing cached reads:   28846 MB in  1.99 seconds = 14481.56 MB/sec
 Timing buffered disk reads: 802 MB in  3.01 seconds = 266.83 MB/sec

/dev/sdb2:
 Timing cached reads:   28798 MB in  1.99 seconds = 14457.67 MB/sec
 Timing buffered disk reads: 752 MB in  3.01 seconds = 250.23 MB/sec

LE_746F6D617A7A69
Posts: 932
Joined: 2020-05-03 14:16
Has thanked: 7 times
Been thanked: 65 times

Re: Can I put /var on a RAMDISK

#20 Post by LE_746F6D617A7A69 »

Head_on_a_Stick wrote: 2022-08-01 14:12 I have three SSDs older than that which have never been restricted in writes. Their speed has decreased over time but only by about 12% and using blkdiscard gets the speed back, even for the oldest drive.
Discarding unused blocks does not stop the wearing-out, nor regenerate the memory cells - it only allows to speed up some write operations, when the data chunks are not matching the erase block size. The problem is, that even *completely new* TLC/QLC drives are producing tons of read errors - so they can't work without strong error correction. For ageing drive the error correction takes more and more time, causing the device to slow down or even freeze.
The advertised "nominal" speed is achievable only when operating within MLC cache memory.

Anyone interested can test his SSD using the following command:

Code: Select all

fio --filename=/dev/sd<x> --direct=1 --rw=read --bs=2M --ioengine=libaio --iodepth=64 --runtime=600 --numjobs=1 --time_based --group_reporting --name=seq-ro-test --eta-newline=1 --readonly
It performs a non-destructive (read-only) sequential read test for --runtime=<seconds>.
The results will show the true condition of the drive, eliminating effects of system-level cashes.
The most interesting part of the result are the latency stats - they are showing how long it takes to correct the read errors, and thus the level of damage of the memory cells.
The --eta-newline=1 allows to see how the device is behaving during the test - e.g. the moments when the device is "freezing" for a long time (typical for used SSDs)
Those are sponsored tests - they are cheating:
1. Long before the device *completely* fails the data are lost due to unrecoverable read errors - they are not checking this.
2. The data retention time is inversely proportional to the level of cell damage - who cares that it's possible to write PB of data within a week if it's impossible to read that data back after 6 months?
3. Would You be interested to read the data at 10KB/s long before the device *completely* fails? Test your SSD - even new SSDs can have damaged, very slow cells - and the older the device, the more damaged cells.
.
Bill Gates: "(...) In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating system."
The_full_story and Nothing_have_changed

Post Reply