Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Debian Live encrypted persistence

New to Debian (Or Linux in general)? Ask your questions here!
Message
Author
haylo
Posts: 3
Joined: 2013-02-12 14:49

Re: Debian Live encrypted persistence

#16 Post by haylo »

here i just posted the code. That way there is some online documentation of this.


### LUKS PERSISTENCE ###
## for debian live ##

# Copyright (c) 2012-2013 cgraff , haylo
# LIVE BUILD SYNTAX AND PROGRAM FROM: Baumann, Armstrong and Lamb
# THIS IS FREE CODE AS LONG AS THESE COMMENTS ARE INCLUDED :-)

# WORK IN PROGRESS ... CHANGING until perfected


# BASIC LOW DOWN IS:
# 1 format the partition,
# 2 encrypt it with luks,
# 3 open it with luks,
# 4 make the filesystem,
# 5 mount it from /dev/mapper/* ,
# 6 populate the filesystem with a "persistence.conf" file,
# 7 un mount it,
# 8 close it with luks

# DEFINE SOME VARIABLES
# NOOBS TIP: the = sign assigns a value to a variable
devicel="/dev/sdb" # CHOOSE A DEVICE
usb_size="4gb" # TELL PARTED ITS SIZE

mirror="http://ftp.debian.org/debian/"
security_mirror="http://ftp.debian.org/debian-security/"



# MAKE THE IMAGE
# remember sid doesnt always build easily.
# going to need to learn a bit about live-build
# or make a live image with remastersys that includes `cryptsetup'
# LOOK TO: http://live.debian.net/manual/3.x/html/ ... al.en.html --/
# FOR ANSWERS ON GETTING YOUR BUILD WORKING CORRECTLY <---------------/
# NOOBS TIP: pin some stuff to wheezy for sid

sudo apt-get install live-build live-config live-config-doc \
cryptsetup --force-yes

mkdir -p ~/crypto_buildl

cd ~/crypto_buildl

lb config \
-a amd64 \
-b hdd \
-d sid \
--linux-flavours amd64 \
--bootstrap cdebootstrap \
--cdebootstrap-options "--flavour=minimal" \
--bootappend-live "\
boot=live \
config \
persistent=cryptsetup \
persistence-encryption=luks \
username=joe \
hostname=shmoe \
persistence" \
--apt-indices false \
--apt-recommends false \
--linux-packages linux-image-3.7-trunk \
--mirror-bootstrap "${mirror}" \
--mirror-binary "${mirror}" \
--mirror-chroot-security "${security_mirror}" \
--mirror-binary-security "${security_mirror}" &&


# THIS ADDS SOME USUAL DEPENDENCIES + CRYPTSETUP
printf "\
user-setup
sudo
cryptsetup
apt-utils
" > config/package-lists/package.list.chroot
# NOTICE 'cryptsetup"
# NOOBS TIP: add cryptsetup package

# PIN SOME PACKAGES TO WHEEZY
# NOOBS TIP: pin packages to other debian releases to get what you need
cat >> config/archives/sid.pref.chroot << EOF
# USER-SETUP
Package: user-setup
Pin: release n=wheezy
Pin-Priority: 600

# SYSLINU*
Package: syslinu*
Pin: release n=wheezy
Pin-Priority: 600

Package: *
Pin: release n=experimental
Pin-Priority: 1

EOF

# ADD WHEEZY REPOS FOR ABOVE PIN LIST

echo "deb "${mirror}" wheezy main" > \
config/archives/wheezy.list.chroot

# ADD WHEEZY REPOS FOR ABOVE PIN LIST
echo "deb "${mirror}" experimental main" > \
config/archives/experimental.list.chroot

# BUILD THE IMAGE
sudo lb build


# CHECK FOR BAD BLOCKS ON THE DEVICE
sudo badblocks -c 10240 -s -w -t random -v "${devicel}"

# OVERWRITE THE DEVICE WITH RANDOM DATA
sudo dd if=/dev/urandom of="${devicel}"


# DD THE binary.img to a usb
sudo dd if=binary.img of="${devicel}"

# THIS JUST TELLS PARTED WHERE TO START
# TO MAKE THE SDB2 PARTITION
read bytes _ < <(du -bcm binary.img |tail -1); echo $bytes

# MAKE THE PARTITION,
sudo parted "${devicel}" mkpart primary "${bytes}" "${usb_size}"


# ENCRYPT THE PARTITION
sudo cryptsetup --verbose --verify-passphrase luksFormat "${devicel}2"

# OPEN THE ENCRYPTED PARTITION
sudo cryptsetup luksOpen "${devicel}2" my_usb

# MAKE A FILESYSTEM ON IT AND LABEL IT "persistence"
sudo mkfs.ext3 -L persistence /dev/mapper/my_usb

# MAKE A MOUNT POINT
sudo mkdir -p /mnt/my_usb

# MOUNT THE OPENED ENCYRYPTION PARTITION
sudo mount /dev/mapper/my_usb /mnt/my_usb/

# MAKE THE PERSISTENCE.CONF FILE
echo "/ union" > ~/persistence.conf && sudo mv ~/persistence.conf \
/persistence.conf && sudo mv /persistence.conf /mnt/my_usb

# UMOUNT IT
sudo umount /dev/mapper/my_usb

# CLOSE THE LUKS PARTITION
sudo cryptsetup luksClose /dev/mapper/my_usb

# TEST IT
sudo apt-get install qemu --force-yes

sudo kvm "${devicel}"

dzz
Posts: 257
Joined: 2007-02-05 20:39
Location: Devon, England

Re: Debian Live encrypted persistence

#17 Post by dzz »

Good to hear luks persistence works with official packages only, even if only for sid. The hook script mentioned earlier was only ever done, for wheezy, because the official stuff did not.

However in wheezy we are already advised to use live-* packages from sid.. What is the difference that it does not work for wheezy and how can it be made to?

Looking changelogs for live-boot, live-config and live-build in experimental I see no mention of luks persistence in any recent stuff.

fsmithred
Posts: 1873
Joined: 2008-01-02 14:52

Re: Debian Live encrypted persistence

#18 Post by fsmithred »

Code: Select all

apt-cache policy live-boot
live-boot:
  Installed: 3.0~a35-1
  Candidate: 3.0.0-1
  Version table:
     4.0~a5-1 0
          1 http://debian.lcs.mit.edu/debian/ experimental/main amd64 Packages
     3.0.0-1 0
        500 http://debian.lcs.mit.edu/debian/ sid/main amd64 Packages
 *** 3.0~a35-1 0
        500 http://debian.lcs.mit.edu/debian/ wheezy/main amd64 Packages
        100 /var/lib/dpkg/status
That might explain why it didn't work for me. I'm using live-* packages from wheezy. I already had a usb stick with an encrypted partition, so I added persistence.conf, labeled the filesystem and added the relevant boot options to my syslinux boot menu. Thinking there might be a typo in the instructions, I tried both "persistent=cryptsetup" and "persistence=cryptsetup". Also tried "persistence-path=/dev/mapper/persistence. I can try this later in a sid install.

dzz
Posts: 257
Joined: 2007-02-05 20:39
Location: Devon, England

Re: Debian Live encrypted persistence

#19 Post by dzz »

Some details of a live usb set up here:

part1: FAT32 with syslinux and a custom Wheezy live image. All live-* packages are very recent sid. Tested and works normally (including with non-luks persistence). Image includes cryptsetup.

Code: Select all

:~$ dpkg -l|grep live-
ii  live-boot                               3.0~b11-1                          all          Live System Boot Scripts
ii  live-boot-doc                           3.0~b11-1                          all          Live System Boot Scripts (documentation)
ii  live-boot-initramfs-tools               3.0~b11-1                          all          Live System Boot Scripts (initramfs-tools backend)
ii  live-config                             3.0.18-1                           all          Live System Configuration Scripts
ii  live-config-doc                         3.0.18-1                           all          Live System Configuration Scripts (documentation)
ii  live-config-sysvinit                    3.0.18-1                           all          Live System Configuration Scripts (sysvinit backend)
ii  live-tools                              3.0.17-1                           all          Live System Support Scripts
part2: LUKS volume containing a EXT2 filesystem labelled "persistence". Contains a file "persistence.conf" with text "/ union,sources=."

cmdline includes: " persistence persistence-encryption=luks " Very early in the boot process the luks key was prompted for.

In the live session: It is already mapped but not mounted according to <blkid> and <cat /proc/mounts> It can be mounted manually without needing luks key again. It is certainly is not being used for persistence.

If "debug" is added to the cmdline everything freezes after the first 3 screens. Power button is the only way out.. Not even busybox. No clue why and no log to save.

The actual live* packages still don't work for luks persistence in wheezy (custom hook script does) I don't know what is different about a sid build (except that every day, it is different, yesterday's is already outdated)

EDIT <man live-boot>
persistence-encryption=TYPE1,TYPE2 ... TYPEn
This option determines which types of encryption that we allow to be used
when probing devices for persistence media. If "none" is in the list, we
allow unencrypted media; if "luks" is in the list, we allow LUKS-encrypted
media. Whenever a device containing encrypted media is probed the user will
be prompted for the passphrase. The default value is "none"
.

EDIT2:
# WORK IN PROGRESS ... CHANGING until perfected
Looking forward to that so "hacks" are no longer necessary.. thanks Debian-Live devs for your good work.

/dev/null
Posts: 62
Joined: 2013-01-30 17:31

Re: Debian Live encrypted persistence

#20 Post by /dev/null »

LUKS volume containing a EXT2 filesystem labelled "persistence". Contains a file "persistence.conf" with text "/ union,sources=."

cmdline includes: " persistence persistence-encryption=luks " Very early in the boot process the luks key was prompted for.

In the live session: It is already mapped but not mounted according to <blkid> and <cat /proc/mounts> It can be mounted manually without needing luks key again. It is certainly is not being used for persistence.
I've got exactly the same results for custom sid

Right after password check there's an error:

Code: Select all

mount: mounting Loading on live/persistence/Loading failed : no such device
so it basically looks similar to this "Loading" thing I mentioned before:

Code: Select all

+ probe_for_fs_label live-rw home-rw persistence Loading /etc/boottime.kmap.gz /dev/mapper/sdb2
+ local overlays dev
+ overlays=live-rw home-rw persistence
+ dev=Loading
+ /sbin/blkid -s LABEL -o value Loading
+ [ = live-rw ]
+ /sbin/blkid -s LABEL -o value Loading
+ [ = home-rw ]
+ /sbin/blkid -s LABEL -o value Loading
+ [ = persistence ]
+ result=
+ [ -n ]
If "debug" is added to the cmdline everything freezes after the first 3 screens. Power button is the only way out.. Not even busybox. No clue why and no log to save.
again, same here
last displayed line is

Code: Select all

uhci_hcd: USB Universal Host Controller Interface driver
on my other laptop it is :

Code: Select all

ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
Btw what is this boot parameter: persistent=cryptsetup ?
I can't see it in man live-boot.

Tomorrow I'll try debian live irc channel :D

dzz
Posts: 257
Joined: 2007-02-05 20:39
Location: Devon, England

Re: Debian Live encrypted persistence

#21 Post by dzz »

no need for any "hacks"
it works, just takes a lot of research
Those of us who did plenty research, still failed and only then resorted to "hacks" remain mystified.

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=700902

The good news is: live-boot_4.0~a7-1_all.deb, which made experimental only today, actually does work. The fix is also noted in the changelog.

I just tested it using an existing custom wheezy image, booted from a rebuilt initrd containing the new stuff in /lib/live/

You can replace just the initrd in /live of a syslinux-type usb setup, maybe not if you use the less flexible dd method (IMO totally inflexible because the device is then useless for normal data storage)

There is still a problem: if you use "debug" on the cmdline (to get a full boot log) with luks the system will hang early on (and the log can't be saved)

pcalvert
Posts: 1939
Joined: 2006-04-21 11:19
Location: Sol Sector
Has thanked: 1 time
Been thanked: 2 times

Re: Debian Live encrypted persistence

#22 Post by pcalvert »

As an alternative, what about creating a "Data" or "Documents" directory inside one's home directory and then encrypting that with eCryptfs?

Phil
Freespoke is a new search engine that respects user privacy and does not engage in censorship.

dzz
Posts: 257
Joined: 2007-02-05 20:39
Location: Devon, England

Re: Debian Live encrypted persistence

#23 Post by dzz »

As an alternative, what about creating a "Data" or "Documents" directory inside one's home directory and then encrypting that with eCryptfs?
A good option for actual data files but dot files also containing personal data would still be unencrypted. (voip account, mail client, browser ,,,)

Pendrives are easily lost or stolen; proper luks persistence is important for "travelling OS" security. We're close to getting it sorted.

/dev/null
Posts: 62
Joined: 2013-01-30 17:31

Re: Debian Live encrypted persistence

#24 Post by /dev/null »

great news, thank you
I just tested it using an existing custom wheezy image, booted from a rebuilt initrd containing the new stuff in /lib/live/
how to rebuild initrd with new live-boot?
or how make wheezy or sid build with live-boot from experimental?

dzz
Posts: 257
Joined: 2007-02-05 20:39
Location: Devon, England

Re: Debian Live encrypted persistence

#25 Post by dzz »

how to rebuild initrd with new live-boot?
Initrd hacking, sid/experimental and live-image building are not really "Beginners Questions" material. The bottom line is, LUKS persistence doesn't work (yet) in the "mainstream". However you did ask.

Extract an initrd:

Code: Select all

# make a directory, copy into it the original initrd
# open a terminal, cd to that directory

INITRD=name_of_original_initrd
mkdir ./uz
cd uz
fakeroot zcat ../$INITRD | cpio -i
Now it's extracted you can change things.

/lib/live/ contains the live-boot scripts. what I did was replace everything in there with the new stuff from the previously extracted .deb

Rebuild it:

Code: Select all

cd uz
fakeroot find . -print0 | cpio -0 -H newc -o | gzip -c > ../initrd.rebuilt
Use initrd.rebuilt to boot with (rename it or edit the menu)

I'm not saying this is the "right" way to do things, nor is this a "recommendation" Just trying to find my own way through that which is about as clear as mud. And I'm working with wheezy not sid.

I'm sure this stuff will be sorted officially in due course (getting a bit late to make wheezy in time though)

/dev/null
Posts: 62
Joined: 2013-01-30 17:31

Re: Debian Live encrypted persistence

#26 Post by /dev/null »

It's working 8)
thank you dzz

/dev/null
Posts: 62
Joined: 2013-01-30 17:31

Re: Debian Live encrypted persistence

#27 Post by /dev/null »

It's working
Well - not anymore :x
They must messed something up with the new versions of boot scripts. It seem that scripts from experimental doesn't fit wheezy/sid builds anymore.

User avatar
llivv
Posts: 5340
Joined: 2007-02-14 18:10
Location: cold storage

Re: Debian Live encrypted persistence

#28 Post by llivv »

/dev/null wrote:They must messed something up with the new versions of boot scripts. It seem that scripts from experimental
is there a new feature in the version from experimental you wanted to try?

posting which versions worked for you
and which version don't work for you
helps a lot more than telling us that they messed up in experimental.
In memory of Ian Ashley Murdock (1973 - 2015) founder of the Debian project.

/dev/null
Posts: 62
Joined: 2013-01-30 17:31

Re: Debian Live encrypted persistence

#29 Post by /dev/null »

I'm talking about luks-encrypted persistence partition in Debian live.
There was a bug in live-boot that causes the luks-encrypted partitions aren't properly detected and cannot be used for persistence. They fixed it recently in live-boot 4.0, but it is still in experimental.
Since building an experimental live image isn't the best idea, "dzz" suggested to just put newest boot scripts from live-boot 4.0.deb package directly into previously built wheezy's/sid's initrd.img.
This method worked great a month ago, but yesterday when I tried it again, the system stuck somewhere during the boot procedure.
It seems that developers changed the newest (4.0) boot scripts in such way that they aren't compatible with wheezy/sid builds anymore.
I tried to install whole experimental live-boot 4.0 package (to overwrite default buggy 3.0) using "--interactive shell" in lb config. I changed repos to experimental but apt-get update doesn't seem to work as it should.
apt-get install live-boot - says that the package is already in the newest version, but it isn't - apt-cache policy live-boot clearly states that the installed version is 3.0.

So if anyone could tell me how to install live-boot from experimental during wheezy/sid build procedure I would be very grateful

dzz
Posts: 257
Joined: 2007-02-05 20:39
Location: Devon, England

Re: Debian Live encrypted persistence

#30 Post by dzz »

Experimental is just that, "experimental". It can change fast. Sometimes stuff is broken, or there might be undocumented changes. You need to be *very* selective what you use from there and without expectations. You might break something else unexpectedly.

That's why I preferred just a custom initrd to actually installing experimental packages. (revised) Just replace the entire /lib/live directory

live-boot 4.0~a9-1 is current. I got a kernel panic using an initrd rebuilt with that. I don't know if it's actually broken or something I did wrong.

You can find 4.0~a7-1 (we know that works) here:

http://snapshot.debian.org/binary/live-boot/
http://snapshot.debian.org/package/live-boot/4.0~a7-1/

BTW luks loopback files still don't work (there is a patch for that)

I use custom build scripts in preference to official live-build so can't help much with that. Rather than use the repo for one package (I think it's <apt-get -t experimental install>) I would dpkg -i the deb.. or just do the custom initrd after build

/dev/null
Posts: 62
Joined: 2013-01-30 17:31

Re: Debian Live encrypted persistence

#31 Post by /dev/null »

Thank you again dzz. I tried -dpkg -i yesterday with 4.0 a9, and luks-persistence didn't work. Thanks for the link to older version.
I think I found another bug: "--interactive shell" parameter doesn't work with "-d sid", lb build just skips it.

BTW maybe you can help me with this :oops: :
http://forums.debian.net/viewtopic.php?f=10&t=101246

IxoT0b
Posts: 1
Joined: 2014-03-14 18:17

Re: Debian Live encrypted persistence

#32 Post by IxoT0b »

Hey,

this thread made it possible for me to create an encrypted persistent debian live system - thanks to all!

I am using live boot 4.0~a7-1 in a rebuilt initrd, debian wheezy with fluxbox on a sandisc cruzer xtreme 64 gb - it works perfectly fine.

If anything new comes up pls share!

fsmithred
Posts: 1873
Joined: 2008-01-02 14:52

Re: Debian Live encrypted persistence

#33 Post by fsmithred »

IxoT0b wrote: If anything new comes up pls share!
Yeah, some new things have come up.

refracta2usb has been updated, and it uses live-boot 4.0~a7-1to rebuild the initrd so you can use encrypted loopback files for persistence or have a read/write media partition.
refracta2usb-0.9.5.deb
http://distro.ibiblio.org/refracta/files/Testing/

dzz has updated his patch-initrd script to use live-boot-4.0~alpha18. That hasn't been added to refracta2usb yet.
http://refracta.freeforums.org/alternat ... html#p3591

AreYouLoco?
Posts: 1
Joined: 2014-04-16 08:45

Re: Debian Live encrypted persistence

#34 Post by AreYouLoco? »

fsmithred wrote:
IxoT0b wrote: If anything new comes up pls share!
Yeah, some new things have come up.
Off Topic
I don't want to be rude but #debian-live channel didn't helped me at all. They don't have to I know. But why for they are sitting on IRC? Nerds...
Yeah some brand new info. I am sure that some of you will be really happy of that URLs which I have received like I am. now

This is what you are looking for for a long time! It's distro called Freepto.

Main Download Page
Torrents: Torrent en_GB Torrent es_ES Torrent it_IT

Here is also their GIT repo: https://github.com/AvANa-BBS/freepto-lb
And for those who don't want download whole ISO just configs to make encrypted persistance:
git -c http.sslVerify=false clone https://v.licheni.net/stack/code/debianlive.git

I suggest you using that distro as a base of your encrypted system with persistence. It's seems reliable and secure I am testing it now.
No one can help you better than Italian hackers:D And now I am ready to contribute to this project.

P.S. The site is in Italian too. I used google translate for you. Works fine to get the needed informations. Enjoy!

SeventhClaw
Posts: 1
Joined: 2014-05-31 10:27

Re: Debian Live encrypted persistence

#35 Post by SeventhClaw »

Hey there, thanks to this thread and the live-build manual I finally got encrypted persistence to work with wheezy live; figured I might as well post the steps I followed. All the official 7.5.0 live images miss in the end is cryptsetup and live-boot 4.* (jessie/testing now has 4.0~alpha21-1).

So first I installed a full Debian 7.5 amd64 DVD1 XFCE copy in VirtualBox + sudo + a "user" user to build this image with; should be fully reproducible.

The goal was just to to produce a slightly more liberal version of the XFCE live CD ISO with encryption. This is pretty sloppy but the end result worked. There was only one conflict with a "plymouth" package following this procedure, seems to be a current issue. Also note I actually ran this only with "--source true", but don't see how that would change anything.

Code: Select all

# Prep
sudo apt-get install cryptsetup debtags git cdebootstrap live-build live-boot live-config --yes
# Restart just in case

sudo -s

# Get stock config
cd /home/user
rm -rf live-images && mkdir live-images && cd live-images
lb config --config git://live.debian.net/git/live-images.git
cd images/xfce-desktop
lb clean

echo -e 'cryptsetup\n' > config/package-lists/custom.list.chroot
mkdir config/archives
echo -e 'deb http://ftp.ca.debian.org/debian/ jessie main contrib non-free' > config/archives/custom.list.chroot
echo -e 'Package: *\nPin: release n=wheezy\nPin-Priority: 990' > config/archives/custom.pref.chroot
# Note: jessie * pin must be -10; if set to 10 fails horribly
echo -e '\nPackage: *\nPin: release n=jessie\nPin-Priority: -10' >> config/archives/custom.pref.chroot
echo -e '\nPackage: live-boot\nPin: release n=jessie\nPin-Priority: 995' >> config/archives/custom.pref.chroot
echo -e '\nPackage: live-boot-initramfs-tools\nPin: release n=jessie\nPin-Priority: 995' >> config/archives/custom.pref.chroot
# Not sure really need these below, but fails without live-boot-initramfs-tools above
echo -e '\nPackage: live-boot-backend\nPin: release n=jessie\nPin-Priority: 995' >> config/archives/custom.pref.chroot
echo -e '\nPackage: live-tools\nPin: release n=jessie\nPin-Priority: 995' >> config/archives/custom.pref.chroot

# Debian bug fix (for ~7.5.0 images):
# IF GET A CONFLICT ERROR with "plymouth" package upon 'lb build':
pico config/package-lists/desktop.list.chroot
# -> Remove all "plymouth*" packages and save

# Change stock config (wheezy & amd64 implied)
lb config --source false --bootstrap cdebootstrap --bootappend-live 'boot=live config quiet splash persistence persistence-encryption=none,luks' --mirror-bootstrap "http://ftp.ca.debian.org/debian/" --mirror-binary "http://ftp.ca.debian.org/debian/" --archive-areas "main contrib non-free"

# Build
lb build

# Transfer to USB (I didn't actually do this, I used the ISO as CD, but for reference...)
dd if=binary.hybrid.iso of=/dev/sdX bs=4M

# Then in a different VM with a blank disk (/dev/sda), 
# needs something along the lines of (warning: from memory)...
fdisk /dev/sda
# o, n, w
cryptsetup luksFormat /dev/sda1
cryptsetup luksOpen /dev/sda1 test
mkfs.ext2 -L persistence /dev/mapper/test
mkdir /mnt/test
mount /dev/mapper/test /mnt/test
echo '/ union,source=.' > persistence.conf
# Not sure this is necessary but did it anyway
chown -R user /mnt/test
chmod -R 755 /mnt/test
umount /mnt/test
cryptsetup luksClose test

# Restart with ISO or USB plugged in, same machine as disk above, enter password when prompted... 
... Persistence!

Post Reply