/dev/shm vs a ramdisk

[jcizzo]

hello all, i'm new so go easy.

I'm currently running a headless deb 12 plex server with 64Gigs of ram. i created a 32G ramdisk under /mnt (so it's /mnt/ramdisk/ tell me if this is wrong, vs placing it in /tmp).

if /dev/shm is backed by swap than why wouldn't i just direct plex's transcoder to use the /dev/shm as its transcoder cache directory? if it runs outta space (which looks like it could when it's transcoding a tv stream) it would just put it to swap instead of the regular filesystem.

if i should NOT be running the transcoder to cache to /dev/shm, is there a way to safely limit it's size? it really won't be used as this is ONLY for plex so it just sits there, meanwhile i could increase the size of the ramdisk that I have to 48, or more, gigs.

thanks!!

[CwF]

@jcizzo
viewtopic.php?p=807166
lots of extra reading, but this includes your use case too. Somewhere in there I mention using large tmpfs's for holding video, and edit and handbrake all in ~/Videos/vram which currently is 85GiB of ram as tmpfs. I use this for recording all tv, so I'd assume pointing Plex to a similar destination should work fine, but I don't use plex. Somewhere in there I mention the size is a quota and only uses what is actually needed.
The method does pretty much throws out prior best common practice. food for thought...

[wizard10000]

I've transcoded to /dev/shm for years.

That said, disk space required for transcoding depends mainly on the client(s) being served. If you're serving to a mobile device I believe the recommendation is size of largest video plus 100MB. For TV, desktop and web clients Plex only transcodes enough to keep the client's buffer full.

Hope this helps -

[jcizzo]

My confusion stems from creating the ramdisk in fstab and originally giving it 48GB.. however because of the /dev/shm file taking half available ram (so 32GB) by default, the system only allowed my ramdisk to be 32.. so despite fstab RAMDISK=48GB, running df -h shows RAMDISK=32 instead of 48.

transcoding 1080p movies/shows down to 720P won't use much ram.. the most i've ever seen it on the current plexserver (win10 based, with a ramdisk) is several gigs (maybe 10) with several remote streams occurring at once. however, i noticed that when i started a live tv stream, the transocder directory on this linux box ate up ram a lot quicker and i'm trying to anticipate a condition where mulitple people are watching something on espn like the superbowl or the masters that lasts for hours and that buffer fills up with a 1080P (or 4k) video stream..

i don't have DVR activated so it's not an issue. i've read that i can edit a file like fstab or something that can limit the /dev/shm, but again, if /dev/shm writes out to swap, that works.. i don't care if an old video stream becomes 'dirty' as they call it (and if someone could explain that to me in layman's terms i'd appreciate it).

again, coming from a windows environment, i'm trying to migrate to a better world of debian so i can minimize my windows reliance.

thanks for you advice and help!

[wizard10000]

fstab should use a size= option. Something like

Code: Select all

tmpfs /dev/shm tmpfs defaults,size=48G 0 0

Hope this helps -

[jcizzo]

there aren't any entries in fstab for /dev/shm

so, is there a problem with me using /dev/shm or is it safest to create my own separate ramdisk?

[wizard10000]

Kernel documentation suggests swap is enabled by default in tmpfs so either should work fine. You can use /dev/shm and it will swap but if you want it without swapping you'd need to create the fstab entry.

https://www.kernel.org/doc/html/latest/ ... tmpfs.html

Hope this helps -

[CwF]

Well I tried, I guess it didn't take! My solution would end up something like this, in ~/.profile.

Code: Select all

systemd-mount -t tmpfs -o size=48G vram ~/Videos/.plex

This type of 'ramdisk' will not swap anything out and will report as 'shared'. It will only consume the memory needed, and will run out of space if ask to. Sometimes I forget to delete what I watched, and it just passively fails. Most is continuously overwritten, like each days news programs.

I am curious on something though. I assume ‘Live TV streaming’ means ip based and not antenna based. I've been wondering what the streams are made of? Perhaps a report from ‘mediainfo’!? My Debian based antenna system works wonderfully and takes very little horsepower.

[jcizzo]

i have a silicondust homerun prime (cablcard tuner) and a homerun antenna tuner.. haven't used the cablecard tuner yet but working on that because it'll save my family a bunch of money. i'm sure it'll be the same as the antenna situation, with 1080i (or P) coming in.. whether or not you have dvr activated in plex, plex 'records' the stream so you can pause and back up, so the incoming video stream gets written to the cache directory specified under the transcoder. in my case as of now it's /mnt/ramdisk

movies seem to write to the ramdisk at however fast the gpu/cpu can run the transoder (so, significantly faster than the 24fps of the playout), but video streams from antenna or cable can only write out at the speed of the incoming media stream because it's all in realtime.

i was reading through kernel.org on tmpfs the other day. it seems that while it operates the same, it's parameters change depending on which directory or partition it's acting upon... so, to create a tmpfs in /tmp or /mnt (like in my case) it doesn't page out when full. however the tmpfs of /dev/shm DOES write out to swap when full. at least, that what i gathered from it..

[CwF]

at however fast the gpu/cpu can run the transoder (so, significantly faster than the 24fps of the playout), but video streams from antenna or cable can only write out at the speed of the incoming media stream because it's all in realtime.

I do it the hard way, but with a custom ‘Free-to-Air’ tcl/tk program and here's a typical stream as shown by mediainfo for a file based type.

Code: Select all

Format                                   : MPEG-TS
File size                                : 7.48 GiB
Duration                                 : 1 h 34 min
Overall bit rate mode                    : Variable
Overall bit rate                         : 11.3 Mb/s

Video
ID                                       : 49 (0x31)
Menu ID                                  : 1 (0x1)
Format                                   : MPEG Video
Format version                           : Version 2
Format profile                           : Main@High
Format settings, BVOP                    : Yes
Format settings, Matrix                  : Custom
Format settings, GOP                     : Variable
Format settings, picture structure       : Frame
Codec ID                                 : 2
Duration                                 : 1 h 34 min
Bit rate mode                            : Variable
Bit rate                                 : 10.2 Mb/s
Maximum bit rate                         : 14.7 Mb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Active Format Description                : Full frame 16:9 image
Frame rate                               : 29.970 (30000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Interlaced
Scan order                               : Top Field First
Compression mode                         : Lossy
Bits/(Pixel*Frame)                       : 0.164
Time code of first frame                 : 11:32:56;06
Time code source                         : Group of pictures header
GOP, Open/Closed                         : Open
Stream size                              : 6.72 GiB (90%)

Audio #1
ID                                       : 52 (0x34)
Menu ID                                  : 1 (0x1)
Format                                   : AC-3
Format/Info                              : Audio Coding 3
Mode extension                           : CM (complete main)
Format settings, Endianness              : Big
Codec ID                                 : 129
Duration                                 : 1 h 34 min
Bit rate mode                            : Constant
Bit rate                                 : 384 kb/s
Channel(s)                               : 6 channels
Channel positions                        : Front: L C R, Side: L R, LFE
Sampling rate                            : 48.0 kHz
Frame rate                               : 31.250 FPS (1536 spf)
Bit depth                                : 16 bits
Compression mode                         : Lossy
Delay relative to video                  : -848 ms
Stream size                              : 259 MiB (3%)
Language                                 : English

Audio #2
ID                                       : 53 (0x35)
Menu ID                                  : 1 (0x1)
Format                                   : AC-3
Format/Info                              : Audio Coding 3
Mode extension                           : VI (visually impaired)
Format settings, Endianness              : Big
Codec ID                                 : 129
Duration                                 : 1 h 34 min
Bit rate mode                            : Constant
Bit rate                                 : 192 kb/s
Channel(s)                               : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 31.250 FPS (1536 spf)
Bit depth                                : 16 bits
Compression mode                         : Lossy
Delay relative to video                  : -656 ms
Stream size                              : 130 MiB (2%)
Language                                 : Spanish

Text #1
ID                                       : 49 (0x31)-CC1
Menu ID                                  : 1 (0x1)
Format                                   : EIA-608
Muxing mode                              : A/53 / DTVCC Transport
Muxing mode, more info                   : Muxed in Video #1
Duration                                 : 1 h 34 min
Bit rate mode                            : Constant
Stream size                              : 0.00 Byte (0%)

Text #2
ID                                       : 49 (0x31)-1
Menu ID                                  : 1 (0x1)
Format                                   : EIA-708
Muxing mode                              : A/53 / DTVCC Transport
Muxing mode, more info                   : Muxed in Video #1
Duration                                 : 1 h 34 min
Bit rate mode                            : Constant
Stream size                              : 0.00 Byte (0%)

Menu
ID                                       : 48 (0x30)
Menu ID                                  : 1 (0x1)
Duration                                 : 1 h 34 min
List                                     : 49 (0x31) (MPEG Video) / 52 (0x34) (AC-3, English) / 53 (0x35) (AC-3, Spanish)
Language                                 :  / English / Spanish

I use both mpv cache for some, file based cache for others. All go to the ‘ramdrive’ ~/Videos/vram. Here there is no transcoding unless I want to save it as smaller. I use hauppage and recording 4 while watching a few takes very little effort. I tried plex a few years back and thought it was too much. The current gnutv I use in the background is broken in bookworm still, the prior version still works.

Is transcoding required? With gnutv it is not, all the play,pause,seek functionality works on the native recording, as shown above.

[jcizzo]

yeah it's required.. my problem is i'm stuck with $hitty comcast/xfinity cable and unless i move my connection to business class, i'm stuck with a 24Mb/s upload (and that's with overprovisioning... and while business class would just about double my upload, it would cut my download in half, all while doubling my monthly price).

all in all, plex is fantastic! i have my max remote stream bitrate set to 720p/4mbps. it runs on a stripped and gutted win10 iot enterprise install on a newer ryzen. i've had several remote streams running at once (like, 5-7) and it's been great.. and i don't use it just because it's cool, i use it because it helps my family save a LOT of money.

as i said though, it's currently on a win10 box and i'd like to install it on a headless debian install, just for reliability and more of a "set it and forget it" system.. i figure with linux it'll handle it all on far less power and i won't have to reboot it nearly as often.

[CwF]

i'd like to install it on a headless debian install, just for reliability and more of a "set it and forget it" system.. i figure with linux it'll handle it all on far less power and i won't have to reboot it nearly as often.

Yes, I've had that conversation. People think I have big money invested, 3 quad tuners, ~105GB ram, hours of time, etc. I then ask them to multiply their cable bill x12 months x7 years...bargain.

As it turns out my tv.tk takes very little power so it eventually became one of the background things the ‘house’ computer does. Once one of the vms, now a minor host extra. Fabien put up some screenshots, and it has changed much since then.
viewtopic.php?p=772062#p772062
You can run this remotely from headless, mine isn't headless but I do ssh in and launch tv.tk. It is not a program I will release or support, but a demonstration of how we can glue together the native capability of Debian exactly how we want, without any branded all encompassing apps.

Now that I think about it, the homerun is a network device and not a dvb device? So maybe a front end like plex is required, I don't know. But with a little effort (lots), I think you can transition to a pure Debian. Feels good!

Overall I rethought it from the ground up. One of those ingredients was how to manage the tmpfs (ramdrive) and mount extra storage. I now recommend the systemd-mount ~/.profile for both.

And my challenge - broadcast TV can be much higher quality than ip streamed! We'll see how long that last...

[jcizzo]

i dunno what tv.tk is, i've never heard of it.

i need simplicity in operation and understanding because my parents are both 80 and my father is paying $200/month to comcast for 2 cable boxes and another 260/month to verizon.. it's ridiculous.. if i can swing this than all that can go away, being replaced by one cablecard in a hdhomerun prime. get rid of all the cable boxes and have their tv wherever they are.