Persistence on debian-live distro

If none of the more specific forums is the right place to ask

Persistence on debian-live distro

Postby Giygas » 2019-11-28 21:39

Hello everyone,

To begin with, I'll sum up the context :

I needed to create a distro based on Debian 10 for a personal project. I used live-build to generate the distro since I needed to put specific files in the architecture of the iso.

Since it's a live-distro, it has obviously no persistence in its initial state. So in an attempt to fix this, I executed the classic install at the boot time. But the thing is it overrides all the files and packages I wanted to put inside it.

Is there any way to have persistence on the distro while being able to keep all of the files and packages installed?

Thank you in advance!

EDIT : I forgot to add that I would like to be able to set at least the username/password when booting the distro for the first time.
Last edited by Giygas on 2019-11-29 08:36, edited 2 times in total.
Giygas
 
Posts: 7
Joined: 2019-11-28 19:47

Re: Persistence on debian-live distro

Postby Dai_trying » 2019-11-28 21:46

Not sure if I understand the problem correctly but are you saying you want the packages that are installed (and available) in your live-cd to be kept in place after you install from that image?

And when you say you used "debian-live" do you mean live-build?
Dai_trying
 
Posts: 836
Joined: 2016-01-07 12:25

Re: Persistence on debian-live distro

Postby arochester » 2019-11-28 23:13

in an attempt to fix this, I executed the classic install at the boot time. But the thing is it overrides all the files and packages I wanted to put inside it.

Is there any way to have persistence on the distro while being able to keep all of the files and packages installed?


persistence: It will NOT do some things. You cannot upgrade the Debian system (kernel and other core components). You cannot install drivers. Although you can install/update some software... there are probably others that you cannot. It is a very handy feature, but it has limitations that may make it unsuitable for some uses.
Source - https://www.linux.org/threads/debian-li ... post-54169
arochester
 
Posts: 1579
Joined: 2010-12-07 19:55

Re: Persistence on debian-live distro

Postby Giygas » 2019-11-29 08:36

Dai_trying wrote:Not sure if I understand the problem correctly but are you saying you want the packages that are installed (and available) in your live-cd to be kept in place after you install from that image?

And when you say you used "debian-live" do you mean live-build?


Yes, exactly, I would like to be able to keep the files and packages I have put in the architecture after I launch the install for the first time.

Yes, I'm talking about live-build, sorry if I didn't make myself very clear, I will edit!
Giygas
 
Posts: 7
Joined: 2019-11-28 19:47

Re: Persistence on debian-live distro

Postby Giygas » 2019-11-29 08:39

arochester wrote:
in an attempt to fix this, I executed the classic install at the boot time. But the thing is it overrides all the files and packages I wanted to put inside it.

Is there any way to have persistence on the distro while being able to keep all of the files and packages installed?


persistence: It will NOT do some things. You cannot upgrade the Debian system (kernel and other core components). You cannot install drivers. Although you can install/update some software... there are probably others that you cannot. It is a very handy feature, but it has limitations that may make it unsuitable for some uses.
Source - https://www.linux.org/threads/debian-li ... post-54169


The problem is, in my case, that we have no access to the device we will install the OS on, so I can't allocate space to the distro beforehand.
Giygas
 
Posts: 7
Joined: 2019-11-28 19:47

Re: Persistence on debian-live distro

Postby Dai_trying » 2019-11-29 09:23

Giygas wrote:
Dai_trying wrote:Not sure if I understand the problem correctly but are you saying you want the packages that are installed (and available) in your live-cd to be kept in place after you install from that image?


Yes, exactly, I would like to be able to keep the files and packages I have put in the architecture after I launch the install for the first time.


This will depend on a few things,

1.) whether everything you installed to the Live-Cd is from the debian repositories or not, I wanted to install Pycharm in my LiveCd and had to implement a few workarounds but got it all working in the end

2.) Installation method, I used calamares installer as installing directly from the boot menu didn't seem to give me the same packages although I changed this quite early on in my build and so there could be a simple way to make it work properly, but I didn't find it.

Are you installing from boot menu or with calamares?

Also if it is not classified could you post your build instruction (I use a script to create mine) which might help guide you better.
Dai_trying
 
Posts: 836
Joined: 2016-01-07 12:25

Re: Persistence on debian-live distro

Postby stevepusser » 2019-11-30 17:34

Though I'm not deeply involved in that part of the MX project, the Debian-derived MX Linux does have the most advanced Live Persistence options out there. It does require that you create a Live USB using the MX Live USB tool, but you can also use that in a non-persistent Live session.

MX also includes another killer app, the MX Snapshot tool for remastering the distro, though if you want to have any customized config files for apps or desktop environments in a redistributable ISO, you'll have to manually put those in /etc/skel before remastering.
The MX Linux repositories: Backports galore! If we don't have something, just ask and we'll try--we like challenges. New packages: Clipgrab 3.8.6, Hedgewars 1.0.0, PulseEffects 4.6.9, Telegram-desktop 1.8.15, Pale Moon 28.8.0, KeepassXC 2.5.1
User avatar
stevepusser
 
Posts: 11321
Joined: 2009-10-06 05:53

Re: Persistence on debian-live distro

Postby Head_on_a_Stick » 2019-12-01 09:50

Giygas wrote:I would like to be able to keep the files and packages I have put in the architecture after I launch the install for the first time.

Yes, I'm talking about live-build

So why not just use live-build to make the changes you want before building the image? Grml have a nice framework for live-build that simplifies the process: https://grml.org/grml-live/

The images produced by live-build have a "hybrid" partition table structure and so cannot be made to work with persistence. Look at Refracta & MX's live system for alternatives.
Don't break DebianHow to report bugs

SharpBang GNU/Linux® — a pre-configured Openbox/Tint2 desktop running on Debian stable
User avatar
Head_on_a_Stick
 
Posts: 10695
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: Persistence on debian-live distro

Postby pcalvert » 2019-12-01 14:20

So, you have made a customized live CD/DVD, and you want to be able to install it and have your modifications carried over to the installed OS. Is that correct?

Phil
“Property is the fruit of labor; property is desirable; it is a positive good
in the world. That some should be rich shows that others may become
rich, and hence is just encouragement to industry and enterprise.”
— Abraham Lincoln
pcalvert
 
Posts: 1864
Joined: 2006-04-21 11:19
Location: Sol Sector

Re: Persistence on debian-live distro

Postby Giygas » 2019-12-02 17:09

This will depend on a few things,

1.) whether everything you installed to the Live-Cd is from the debian repositories or not, I wanted to install Pycharm in my LiveCd and had to implement a few workarounds but got it all working in the end

2.) Installation method, I used calamares installer as installing directly from the boot menu didn't seem to give me the same packages although I changed this quite early on in my build and so there could be a simple way to make it work properly, but I didn't find it.

Are you installing from boot menu or with calamares?

Also if it is not classified could you post your build instruction (I use a script to create mine) which might help guide you better.


1) I have put a few shared libraries and binaries in the includes.chroot of the repository and everything I installed comes from Debian repositories beside curlpp, which I'm downloading from its github.

2) I'm installing from boot menu, but it overrides the files I have implemented. Do calamares allow to create an user and a root?

I have only one script who is installing the dependencies, here you go :

apt-get update
apt-get install -y g++ cmake tcl libncurses5-dev liblouis-dev libgtkmm-3.0-dev brltty libspeechd-dev libalut-dev libbrlapi-dev libboost-all-dev libcurl4-openssl-dev libssl-dev libgmp-dev sqlite3 libsqlite3-dev
cd /bin
git clone https://github.com/jpbarrette/curlpp.git
cd curlpp
mkdir build && cd build && cmake .. && make
sudo make install
Giygas
 
Posts: 7
Joined: 2019-11-28 19:47

Re: Persistence on debian-live distro

Postby Giygas » 2019-12-02 17:26

Head_on_a_Stick wrote:
Giygas wrote:I would like to be able to keep the files and packages I have put in the architecture after I launch the install for the first time.

Yes, I'm talking about live-build

So why not just use live-build to make the changes you want before building the image? Grml have a nice framework for live-build that simplifies the process: https://grml.org/grml-live/

The images produced by live-build have a "hybrid" partition table structure and so cannot be made to work with persistence. Look at Refracta & MX's live system for alternatives.


The thing is the installation from the boot menu overrides everything that have been put inside before.

Do you know any software or framework allowing to create a fully functionnal OS? Because that's what I really need. The user needs to be able to configure whatever he wants but the OS will already have some files in his architecture as well.
Giygas
 
Posts: 7
Joined: 2019-11-28 19:47

Re: Persistence on debian-live distro

Postby Giygas » 2019-12-02 17:28

pcalvert wrote:So, you have made a customized live CD/DVD, and you want to be able to install it and have your modifications carried over to the installed OS. Is that correct?

Phil


That's exactly what I want. A fully functional OS where the user is able to configure whatever he wants in the install, while having some packages and files installed already.
Giygas
 
Posts: 7
Joined: 2019-11-28 19:47

Re: Persistence on debian-live distro

Postby Dai_trying » 2019-12-02 17:46

I had some initial problems with installing from the boot menu and discounted that early on in favour of using calamares which I found to be much more configurable and easier too.

The script I was referring to was for creating your iso with live-build, I will post mine below so you can take a look at it to see if it can be of any use, I have customized the image to default to UK location and UK English as language but obviously that can be easily changed.

Please bear in mind that I am not a professional and have done this in my spare time and there is likely to be much that could be done easier/better or just differently, this is just what I have got to work for me so far.

Code: Select all
#!/bin/bash

starttime=$(date)

if [ ! -d "MyLive" ]; then mkdir MyLive; fi

cd MyLive

lb config --binary-images iso-hybrid --mode debian -a amd64 -k amd64 -d buster --archive-areas "main contrib non-free" --updates true --security true --cache true --apt-recommends true --debian-installer live --debian-installer-gui true --win32-loader false --bootappend-live "boot=live components live-config.locales=en_GB.UTF-8 live-config.timezone=Europe/London live-config.keyboard-layouts=gb live-config.user-default-groups=cdrom,floppy,audio,video,plugdev,netdev,libvirt,libvirt-qemu,dialout" --iso-application "Debian Buster" --iso-preparer "Dai Trying" --iso-publisher "Dai Trying" --iso-volume "DaiOs Genesis"

if [ ! -d "config/includes.chroot/usr" ]; then mkdir config/includes.chroot/usr; fi
if [ ! -d "config/includes.chroot/usr/local" ]; then mkdir config/includes.chroot/usr/local; fi
if [ ! -d "config/includes.chroot/usr/local/bin" ]; then mkdir config/includes.chroot/usr/local/bin; fi
if [ ! -d "config/includes.chroot/usr/share" ]; then mkdir config/includes.chroot/usr/share; fi
if [ ! -d "config/includes.chroot/usr/share/applications" ]; then mkdir config/includes.chroot/usr/share/applications; fi
if [ ! -d "config/includes.chroot/etc" ]; then mkdir config/includes.chroot/etc; fi
if [ ! -d "config/includes.chroot/etc/calamares" ]; then mkdir config/includes.chroot/etc/calamares; fi
if [ ! -d "config/includes.chroot/etc/calamares/modules" ]; then mkdir config/includes.chroot/etc/calamares/modules; fi
if [ ! -d "config/includes.chroot/usr/share/hardinfo" ]; then mkdir config/includes.chroot/usr/share/hardinfo; fi


echo xorg xserver-xorg-video-nouveau xserver-xorg-video-intel lightdm xfce4 xfce4-panel xfdesktop4 xfwm4 xfce4-settings xfce4-session terminator xfce4-appfinder thunar xfce4-power-manager ristretto thunar-volman policykit-1 geany xfce4-mount-plugin xfce4-notifyd xfce4-screenshooter xfce4-taskmanager xfce4-whiskermenu-plugin xfce4-xkb-plugin gvfs conky-all command-not-found mlocate mediainfo lm-sensors bash bash-completion firmware-linux-free grub-firmware-qemu bsdmainutils ghostscript gimp haveged galculator gparted flac faad faac mjpegtools x265 x264 mpg321 ffmpeg sox pulseaudio pavucontrol twolame lame aisleriot filezilla libxvidcore4 mpv htop jfsutils xfsprogs ntfs-3g p7zip-full p7zip-rar inxi simplescreenrecorder dos2unix audacity rar unrar ssh cifs-utils fuse gvfs-fuse gvfs-backends gvfs-bin pciutils dosfstools lsb-release menu dkms curl wget calamares calamares-settings-debian neofetch ncdu qemu-kvm libvirt-clients libvirt-daemon-system bridge-utils libguestfs-tools genisoimage virtinst libosinfo-bin virt-manager sqlitebrowser palemoon network-manager-gnome user-setup wbritish python3 python3-pyqt5 python3-apt meld hardinfo> config/package-lists/desktop.list.chroot

echo "live-boot" > config/package-lists/live.list.chroot
echo "live-config" >> config/package-lists/live.list.chroot
echo "live-config-systemd" >> config/package-lists/live.list.chroot

if [ ! -f config/packages.chroot/palemoon_28.7.2+repack-1_amd64.deb ]; then
  cp ~/Downloads/palemoon_28.7.2+repack-1_amd64.deb config/packages.chroot/
fi

if [ ! -f config/includes.chroot/usr/local/bin/ps_mem ]; then
  unzip -p ~/Downloads/ps_mem-master.zip ps_mem-master/ps_mem.py > config/includes.chroot/usr/local/bin/ps_mem
fi

if [ ! -d "config/includes.chroot/usr/share/pycharm-community-2019.2" ]; then
  tar -xvzf ~/Downloads/pycharm-community-2019.2.5.tar.gz -C config/includes.chroot/usr/share/
fi

if [ ! -d "config/includes.chroot/usr/share/arduino-1.8.10-linux64" ]; then
  tar -xvf ~/Downloads/arduino-1.8.10-linux64.tar.xz -C config/includes.chroot/usr/share/
fi

cp ~/Documents/PyCharm.desktop config/includes.chroot/usr/share/applications/
cp ~/Documents/Arduino.desktop config/includes.chroot/usr/share/applications/
cp ~/Documents/bash.bashrc config/includes.chroot/etc/
cp ~/Documents/benchmark.conf.new config/includes.chroot/usr/share/hardinfo/benchmark.conf

cp ~/users.conf config/includes.chroot/etc/calamares/modules/
cp ~/locale.conf config/includes.chroot/etc/calamares/modules/
cp ~/lsb-release config/includes.chroot/etc/

sudo lb build

endtime=$(date)

echo "Script started $starttime"
echo "Script finished $endtime"


users.conf
Code: Select all
---
userGroup:       users
defaultGroups:
    - cdrom
    - floppy
    - sudo
    - audio
    - dip
    - video
    - plugdev
    - netdev
    - lpadmin
    - scanner
    - bluetooth
    - libvirt
    - libvirt-qemu
    - dialout
doAutologin: false
sudoersGroup:    sudo
setRootPassword: true
doReusePassword: true


locale.conf
Code: Select all
---
region:                     "Europe"
zone:                       "London"

geoip:
    style:  "json"
    url:    "https://geoip.kde.org/v1/calamares"
selector: "" # leave blank for the default


lsb-release
Code: Select all
PRETTY_NAME="Dai's Buster"
DISTRIB_ID=DaiOs
DISTRIB_RELEASE=1.0
DISTRIB_CODENAME=Genesis
DISTRIB_DESCRIPTION="DaiOs V1.0 Genesis"


The files Pycharm.desktop, Arduino.desktop, bash.bashrc and benchmark.conf are just startup files and personal preferences and you could either replace or remove them as you see fit. Also you would need to change the lb config to whatever your image wants to have.

I have been using this setup for a few months or so now and simply run lb clean and then run the script again when I want to update the image with the latest updates.

Of course You would need to replace a lot of stuff I have for the things that you want but this might give you some pointers, I got a lot of my information from eznixos (find him on youtube/sourceforge or elsewhere on the net) and then a few searches on calamares configuration and plenty of trial and error! And obviously you can always just pick any bits of the script you like and do whatever suits you best, many choices. :D

One last thing, I added the start and end time to let me know how long it took as I would normally leave it running while doing something else, these are printed to the terminal after everything else has finished.
Dai_trying
 
Posts: 836
Joined: 2016-01-07 12:25

Re: Persistence on debian-live distro

Postby Head_on_a_Stick » 2019-12-02 17:48

Giygas wrote:The thing is the installation from the boot menu overrides everything that have been put inside before.well.

Use the "--debian-installer live" option in auto/config, see https://live-team.pages.debian.net/live ... n.html#626 and lb_config(1) for more on this.

The GitHub page linked from the title of my website has an example of live-build using the "live" installer.

EDIT: ninja'd by Dai :)
Don't break DebianHow to report bugs

SharpBang GNU/Linux® — a pre-configured Openbox/Tint2 desktop running on Debian stable
User avatar
Head_on_a_Stick
 
Posts: 10695
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: Persistence on debian-live distro

Postby Giygas » 2019-12-02 18:02

Head_on_a_Stick wrote:
Giygas wrote:The thing is the installation from the boot menu overrides everything that have been put inside before.well.

Use the "--debian-installer live" option in auto/config, see https://live-team.pages.debian.net/live ... n.html#626 and lb_config(1) for more on this.

The GitHub page linked from the title of my website has an example of live-build using the "live" installer.

EDIT: ninja'd by Dai :)


Oh my, how did I miss this? You might have probably solved my entire issue, mate.

I'll keep you updated!
Giygas
 
Posts: 7
Joined: 2019-11-28 19:47

Next

Return to General Questions

Who is online

Users browsing this forum: No registered users and 13 guests

fashionable