Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Persistence on debian-live distro

If none of the specific sub-forums seem right for your thread, ask here.
Post Reply
Message
Author
Giygas
Posts: 7
Joined: 2019-11-28 19:47

Persistence on debian-live distro

#1 Post by Giygas »

Hello everyone,

To begin with, I'll sum up the context :

I needed to create a distro based on Debian 10 for a personal project. I used live-build to generate the distro since I needed to put specific files in the architecture of the iso.

Since it's a live-distro, it has obviously no persistence in its initial state. So in an attempt to fix this, I executed the classic install at the boot time. But the thing is it overrides all the files and packages I wanted to put inside it.

Is there any way to have persistence on the distro while being able to keep all of the files and packages installed?

Thank you in advance!

EDIT : I forgot to add that I would like to be able to set at least the username/password when booting the distro for the first time.
Last edited by Giygas on 2019-11-29 08:36, edited 2 times in total.

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: Persistence on debian-live distro

#2 Post by Dai_trying »

Not sure if I understand the problem correctly but are you saying you want the packages that are installed (and available) in your live-cd to be kept in place after you install from that image?

And when you say you used "debian-live" do you mean live-build?

arochester
Emeritus
Emeritus
Posts: 2435
Joined: 2010-12-07 19:55
Has thanked: 14 times
Been thanked: 54 times

Re: Persistence on debian-live distro

#3 Post by arochester »

in an attempt to fix this, I executed the classic install at the boot time. But the thing is it overrides all the files and packages I wanted to put inside it.

Is there any way to have persistence on the distro while being able to keep all of the files and packages installed?
persistence: It will NOT do some things. You cannot upgrade the Debian system (kernel and other core components). You cannot install drivers. Although you can install/update some software... there are probably others that you cannot. It is a very handy feature, but it has limitations that may make it unsuitable for some uses.
Source - https://www.linux.org/threads/debian-li ... post-54169

Giygas
Posts: 7
Joined: 2019-11-28 19:47

Re: Persistence on debian-live distro

#4 Post by Giygas »

Dai_trying wrote:Not sure if I understand the problem correctly but are you saying you want the packages that are installed (and available) in your live-cd to be kept in place after you install from that image?

And when you say you used "debian-live" do you mean live-build?
Yes, exactly, I would like to be able to keep the files and packages I have put in the architecture after I launch the install for the first time.

Yes, I'm talking about live-build, sorry if I didn't make myself very clear, I will edit!

Giygas
Posts: 7
Joined: 2019-11-28 19:47

Re: Persistence on debian-live distro

#5 Post by Giygas »

arochester wrote:
in an attempt to fix this, I executed the classic install at the boot time. But the thing is it overrides all the files and packages I wanted to put inside it.

Is there any way to have persistence on the distro while being able to keep all of the files and packages installed?
persistence: It will NOT do some things. You cannot upgrade the Debian system (kernel and other core components). You cannot install drivers. Although you can install/update some software... there are probably others that you cannot. It is a very handy feature, but it has limitations that may make it unsuitable for some uses.
Source - https://www.linux.org/threads/debian-li ... post-54169
The problem is, in my case, that we have no access to the device we will install the OS on, so I can't allocate space to the distro beforehand.

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: Persistence on debian-live distro

#6 Post by Dai_trying »

Giygas wrote:
Dai_trying wrote:Not sure if I understand the problem correctly but are you saying you want the packages that are installed (and available) in your live-cd to be kept in place after you install from that image?
Yes, exactly, I would like to be able to keep the files and packages I have put in the architecture after I launch the install for the first time.
This will depend on a few things,

1.) whether everything you installed to the Live-Cd is from the debian repositories or not, I wanted to install Pycharm in my LiveCd and had to implement a few workarounds but got it all working in the end

2.) Installation method, I used calamares installer as installing directly from the boot menu didn't seem to give me the same packages although I changed this quite early on in my build and so there could be a simple way to make it work properly, but I didn't find it.

Are you installing from boot menu or with calamares?

Also if it is not classified could you post your build instruction (I use a script to create mine) which might help guide you better.

User avatar
stevepusser
Posts: 12930
Joined: 2009-10-06 05:53
Has thanked: 41 times
Been thanked: 71 times

Re: Persistence on debian-live distro

#7 Post by stevepusser »

Though I'm not deeply involved in that part of the MX project, the Debian-derived MX Linux does have the most advanced Live Persistence options out there. It does require that you create a Live USB using the MX Live USB tool, but you can also use that in a non-persistent Live session.

MX also includes another killer app, the MX Snapshot tool for remastering the distro, though if you want to have any customized config files for apps or desktop environments in a redistributable ISO, you'll have to manually put those in /etc/skel before remastering.
MX Linux packager and developer

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: Persistence on debian-live distro

#8 Post by Head_on_a_Stick »

Giygas wrote:I would like to be able to keep the files and packages I have put in the architecture after I launch the install for the first time.

Yes, I'm talking about live-build
So why not just use live-build to make the changes you want before building the image? Grml have a nice framework for live-build that simplifies the process: https://grml.org/grml-live/

The images produced by live-build have a "hybrid" partition table structure and so cannot be made to work with persistence. Look at Refracta & MX's live system for alternatives.
deadbang

pcalvert
Posts: 1939
Joined: 2006-04-21 11:19
Location: Sol Sector
Has thanked: 1 time
Been thanked: 2 times

Re: Persistence on debian-live distro

#9 Post by pcalvert »

So, you have made a customized live CD/DVD, and you want to be able to install it and have your modifications carried over to the installed OS. Is that correct?

Phil
Freespoke is a new search engine that respects user privacy and does not engage in censorship.

Giygas
Posts: 7
Joined: 2019-11-28 19:47

Re: Persistence on debian-live distro

#10 Post by Giygas »

This will depend on a few things,

1.) whether everything you installed to the Live-Cd is from the debian repositories or not, I wanted to install Pycharm in my LiveCd and had to implement a few workarounds but got it all working in the end

2.) Installation method, I used calamares installer as installing directly from the boot menu didn't seem to give me the same packages although I changed this quite early on in my build and so there could be a simple way to make it work properly, but I didn't find it.

Are you installing from boot menu or with calamares?

Also if it is not classified could you post your build instruction (I use a script to create mine) which might help guide you better.
1) I have put a few shared libraries and binaries in the includes.chroot of the repository and everything I installed comes from Debian repositories beside curlpp, which I'm downloading from its github.

2) I'm installing from boot menu, but it overrides the files I have implemented. Do calamares allow to create an user and a root?

I have only one script who is installing the dependencies, here you go :

apt-get update
apt-get install -y g++ cmake tcl libncurses5-dev liblouis-dev libgtkmm-3.0-dev brltty libspeechd-dev libalut-dev libbrlapi-dev libboost-all-dev libcurl4-openssl-dev libssl-dev libgmp-dev sqlite3 libsqlite3-dev
cd /bin
git clone https://github.com/jpbarrette/curlpp.git
cd curlpp
mkdir build && cd build && cmake .. && make
sudo make install

Giygas
Posts: 7
Joined: 2019-11-28 19:47

Re: Persistence on debian-live distro

#11 Post by Giygas »

Head_on_a_Stick wrote:
Giygas wrote:I would like to be able to keep the files and packages I have put in the architecture after I launch the install for the first time.

Yes, I'm talking about live-build
So why not just use live-build to make the changes you want before building the image? Grml have a nice framework for live-build that simplifies the process: https://grml.org/grml-live/

The images produced by live-build have a "hybrid" partition table structure and so cannot be made to work with persistence. Look at Refracta & MX's live system for alternatives.
The thing is the installation from the boot menu overrides everything that have been put inside before.

Do you know any software or framework allowing to create a fully functionnal OS? Because that's what I really need. The user needs to be able to configure whatever he wants but the OS will already have some files in his architecture as well.

Giygas
Posts: 7
Joined: 2019-11-28 19:47

Re: Persistence on debian-live distro

#12 Post by Giygas »

pcalvert wrote:So, you have made a customized live CD/DVD, and you want to be able to install it and have your modifications carried over to the installed OS. Is that correct?

Phil
That's exactly what I want. A fully functional OS where the user is able to configure whatever he wants in the install, while having some packages and files installed already.

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: Persistence on debian-live distro

#13 Post by Dai_trying »

I had some initial problems with installing from the boot menu and discounted that early on in favour of using calamares which I found to be much more configurable and easier too.

The script I was referring to was for creating your iso with live-build, I will post mine below so you can take a look at it to see if it can be of any use, I have customized the image to default to UK location and UK English as language but obviously that can be easily changed.

Please bear in mind that I am not a professional and have done this in my spare time and there is likely to be much that could be done easier/better or just differently, this is just what I have got to work for me so far.

Code: Select all

#!/bin/bash

starttime=$(date)

if [ ! -d "MyLive" ]; then mkdir MyLive; fi

cd MyLive

lb config --binary-images iso-hybrid --mode debian -a amd64 -k amd64 -d buster --archive-areas "main contrib non-free" --updates true --security true --cache true --apt-recommends true --debian-installer live --debian-installer-gui true --win32-loader false --bootappend-live "boot=live components live-config.locales=en_GB.UTF-8 live-config.timezone=Europe/London live-config.keyboard-layouts=gb live-config.user-default-groups=cdrom,floppy,audio,video,plugdev,netdev,libvirt,libvirt-qemu,dialout" --iso-application "Debian Buster" --iso-preparer "Dai Trying" --iso-publisher "Dai Trying" --iso-volume "DaiOs Genesis"

if [ ! -d "config/includes.chroot/usr" ]; then mkdir config/includes.chroot/usr; fi
if [ ! -d "config/includes.chroot/usr/local" ]; then mkdir config/includes.chroot/usr/local; fi
if [ ! -d "config/includes.chroot/usr/local/bin" ]; then mkdir config/includes.chroot/usr/local/bin; fi
if [ ! -d "config/includes.chroot/usr/share" ]; then mkdir config/includes.chroot/usr/share; fi
if [ ! -d "config/includes.chroot/usr/share/applications" ]; then mkdir config/includes.chroot/usr/share/applications; fi
if [ ! -d "config/includes.chroot/etc" ]; then mkdir config/includes.chroot/etc; fi
if [ ! -d "config/includes.chroot/etc/calamares" ]; then mkdir config/includes.chroot/etc/calamares; fi
if [ ! -d "config/includes.chroot/etc/calamares/modules" ]; then mkdir config/includes.chroot/etc/calamares/modules; fi
if [ ! -d "config/includes.chroot/usr/share/hardinfo" ]; then mkdir config/includes.chroot/usr/share/hardinfo; fi


echo xorg xserver-xorg-video-nouveau xserver-xorg-video-intel lightdm xfce4 xfce4-panel xfdesktop4 xfwm4 xfce4-settings xfce4-session terminator xfce4-appfinder thunar xfce4-power-manager ristretto thunar-volman policykit-1 geany xfce4-mount-plugin xfce4-notifyd xfce4-screenshooter xfce4-taskmanager xfce4-whiskermenu-plugin xfce4-xkb-plugin gvfs conky-all command-not-found mlocate mediainfo lm-sensors bash bash-completion firmware-linux-free grub-firmware-qemu bsdmainutils ghostscript gimp haveged galculator gparted flac faad faac mjpegtools x265 x264 mpg321 ffmpeg sox pulseaudio pavucontrol twolame lame aisleriot filezilla libxvidcore4 mpv htop jfsutils xfsprogs ntfs-3g p7zip-full p7zip-rar inxi simplescreenrecorder dos2unix audacity rar unrar ssh cifs-utils fuse gvfs-fuse gvfs-backends gvfs-bin pciutils dosfstools lsb-release menu dkms curl wget calamares calamares-settings-debian neofetch ncdu qemu-kvm libvirt-clients libvirt-daemon-system bridge-utils libguestfs-tools genisoimage virtinst libosinfo-bin virt-manager sqlitebrowser palemoon network-manager-gnome user-setup wbritish python3 python3-pyqt5 python3-apt meld hardinfo> config/package-lists/desktop.list.chroot

echo "live-boot" > config/package-lists/live.list.chroot
echo "live-config" >> config/package-lists/live.list.chroot
echo "live-config-systemd" >> config/package-lists/live.list.chroot

if [ ! -f config/packages.chroot/palemoon_28.7.2+repack-1_amd64.deb ]; then
  cp ~/Downloads/palemoon_28.7.2+repack-1_amd64.deb config/packages.chroot/
fi

if [ ! -f config/includes.chroot/usr/local/bin/ps_mem ]; then
  unzip -p ~/Downloads/ps_mem-master.zip ps_mem-master/ps_mem.py > config/includes.chroot/usr/local/bin/ps_mem
fi

if [ ! -d "config/includes.chroot/usr/share/pycharm-community-2019.2" ]; then
  tar -xvzf ~/Downloads/pycharm-community-2019.2.5.tar.gz -C config/includes.chroot/usr/share/
fi

if [ ! -d "config/includes.chroot/usr/share/arduino-1.8.10-linux64" ]; then
  tar -xvf ~/Downloads/arduino-1.8.10-linux64.tar.xz -C config/includes.chroot/usr/share/
fi

cp ~/Documents/PyCharm.desktop config/includes.chroot/usr/share/applications/
cp ~/Documents/Arduino.desktop config/includes.chroot/usr/share/applications/
cp ~/Documents/bash.bashrc config/includes.chroot/etc/
cp ~/Documents/benchmark.conf.new config/includes.chroot/usr/share/hardinfo/benchmark.conf

cp ~/users.conf config/includes.chroot/etc/calamares/modules/
cp ~/locale.conf config/includes.chroot/etc/calamares/modules/
cp ~/lsb-release config/includes.chroot/etc/

sudo lb build

endtime=$(date)

echo "Script started $starttime"
echo "Script finished $endtime"
users.conf

Code: Select all

---
userGroup:       users
defaultGroups:
    - cdrom
    - floppy
    - sudo
    - audio
    - dip
    - video
    - plugdev
    - netdev
    - lpadmin
    - scanner
    - bluetooth
    - libvirt
    - libvirt-qemu
    - dialout
doAutologin: false
sudoersGroup:    sudo
setRootPassword: true
doReusePassword: true
locale.conf

Code: Select all

---
region:                     "Europe"
zone:                       "London"

geoip:
    style:  "json"
    url:    "https://geoip.kde.org/v1/calamares"
selector: "" # leave blank for the default
lsb-release

Code: Select all

PRETTY_NAME="Dai's Buster"
DISTRIB_ID=DaiOs
DISTRIB_RELEASE=1.0
DISTRIB_CODENAME=Genesis
DISTRIB_DESCRIPTION="DaiOs V1.0 Genesis"
The files Pycharm.desktop, Arduino.desktop, bash.bashrc and benchmark.conf are just startup files and personal preferences and you could either replace or remove them as you see fit. Also you would need to change the lb config to whatever your image wants to have.

I have been using this setup for a few months or so now and simply run lb clean and then run the script again when I want to update the image with the latest updates.

Of course You would need to replace a lot of stuff I have for the things that you want but this might give you some pointers, I got a lot of my information from eznixos (find him on youtube/sourceforge or elsewhere on the net) and then a few searches on calamares configuration and plenty of trial and error! And obviously you can always just pick any bits of the script you like and do whatever suits you best, many choices. :D

One last thing, I added the start and end time to let me know how long it took as I would normally leave it running while doing something else, these are printed to the terminal after everything else has finished.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: Persistence on debian-live distro

#14 Post by Head_on_a_Stick »

Giygas wrote:The thing is the installation from the boot menu overrides everything that have been put inside before.well.
Use the "--debian-installer live" option in auto/config, see https://live-team.pages.debian.net/live ... n.html#626 and lb_config(1) for more on this.

The GitHub page linked from the title of my website has an example of live-build using the "live" installer.

EDIT: ninja'd by Dai :)
deadbang

Giygas
Posts: 7
Joined: 2019-11-28 19:47

Re: Persistence on debian-live distro

#15 Post by Giygas »

Head_on_a_Stick wrote:
Giygas wrote:The thing is the installation from the boot menu overrides everything that have been put inside before.well.
Use the "--debian-installer live" option in auto/config, see https://live-team.pages.debian.net/live ... n.html#626 and lb_config(1) for more on this.

The GitHub page linked from the title of my website has an example of live-build using the "live" installer.

EDIT: ninja'd by Dai :)
Oh my, how did I miss this? You might have probably solved my entire issue, mate.

I'll keep you updated!

pcalvert
Posts: 1939
Joined: 2006-04-21 11:19
Location: Sol Sector
Has thanked: 1 time
Been thanked: 2 times

Re: Persistence on debian-live distro

#16 Post by pcalvert »

Giygas wrote: That's exactly what I want. A fully functional OS where the user is able to configure whatever he wants in the install, while having some packages and files installed already.
I accomplished that a few years ago using Debian Jessie. I included Refracta Installer on the live CD, and that's what I used to install the OS. I could not use the Debian installer for my application, nor did I want to.

I obtained the Refracta Installer packages from here:
https://sourceforge.net/projects/refracta/files/tools/

Refracta Installer isn't as complex and full-featured as the Debian installer, but I see its simplicity as an advantage (if it does what you want).

Phil
Freespoke is a new search engine that respects user privacy and does not engage in censorship.

pcalvert
Posts: 1939
Joined: 2006-04-21 11:19
Location: Sol Sector
Has thanked: 1 time
Been thanked: 2 times

Re: Persistence on debian-live distro

#17 Post by pcalvert »

Dai_trying wrote:I had some initial problems with installing from the boot menu and discounted that early on in favour of using calamares which I found to be much more configurable and easier too.

Code: Select all

#!/bin/bash

starttime=$(date)

if [ ! -d "MyLive" ]; then mkdir MyLive; fi

cd MyLive

lb config --binary-images iso-hybrid --mode debian -a amd64 -k amd64 -d buster --archive-areas "main contrib non-free" --updates true --security true --cache true --apt-recommends true --debian-installer live --debian-installer-gui true --win32-loader false 
[snip]
If you are using Calamares, why do you include the Debian installer?

Phil
Freespoke is a new search engine that respects user privacy and does not engage in censorship.

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: Persistence on debian-live distro

#18 Post by Dai_trying »

I must have thought i needed it at some point, I will remove it and see if everything still works ok.

Post Reply