Debian Live encrypted persistence

New to Debian (Or Linux in general)? Ask your questions here!

Re: Debian Live encrypted persistence

Postby haylo » 2013-02-12 16:04

here i just posted the code. That way there is some online documentation of this.


### LUKS PERSISTENCE ###
## for debian live ##

# Copyright (c) 2012-2013 cgraff , haylo
# LIVE BUILD SYNTAX AND PROGRAM FROM: Baumann, Armstrong and Lamb
# THIS IS FREE CODE AS LONG AS THESE COMMENTS ARE INCLUDED :-)

# WORK IN PROGRESS ... CHANGING until perfected


# BASIC LOW DOWN IS:
# 1 format the partition,
# 2 encrypt it with luks,
# 3 open it with luks,
# 4 make the filesystem,
# 5 mount it from /dev/mapper/* ,
# 6 populate the filesystem with a "persistence.conf" file,
# 7 un mount it,
# 8 close it with luks

# DEFINE SOME VARIABLES
# NOOBS TIP: the = sign assigns a value to a variable
devicel="/dev/sdb" # CHOOSE A DEVICE
usb_size="4gb" # TELL PARTED ITS SIZE

mirror="http://ftp.debian.org/debian/"
security_mirror="http://ftp.debian.org/debian-security/"



# MAKE THE IMAGE
# remember sid doesnt always build easily.
# going to need to learn a bit about live-build
# or make a live image with remastersys that includes `cryptsetup'
# LOOK TO: http://live.debian.net/manual/3.x/html/ ... al.en.html --/
# FOR ANSWERS ON GETTING YOUR BUILD WORKING CORRECTLY <---------------/
# NOOBS TIP: pin some stuff to wheezy for sid

sudo apt-get install live-build live-config live-config-doc \
cryptsetup --force-yes

mkdir -p ~/crypto_buildl

cd ~/crypto_buildl

lb config \
-a amd64 \
-b hdd \
-d sid \
--linux-flavours amd64 \
--bootstrap cdebootstrap \
--cdebootstrap-options "--flavour=minimal" \
--bootappend-live "\
boot=live \
config \
persistent=cryptsetup \
persistence-encryption=luks \
username=joe \
hostname=shmoe \
persistence" \
--apt-indices false \
--apt-recommends false \
--linux-packages linux-image-3.7-trunk \
--mirror-bootstrap "${mirror}" \
--mirror-binary "${mirror}" \
--mirror-chroot-security "${security_mirror}" \
--mirror-binary-security "${security_mirror}" &&


# THIS ADDS SOME USUAL DEPENDENCIES + CRYPTSETUP
printf "\
user-setup
sudo
cryptsetup
apt-utils
" > config/package-lists/package.list.chroot
# NOTICE 'cryptsetup"
# NOOBS TIP: add cryptsetup package

# PIN SOME PACKAGES TO WHEEZY
# NOOBS TIP: pin packages to other debian releases to get what you need
cat >> config/archives/sid.pref.chroot << EOF
# USER-SETUP
Package: user-setup
Pin: release n=wheezy
Pin-Priority: 600

# SYSLINU*
Package: syslinu*
Pin: release n=wheezy
Pin-Priority: 600

Package: *
Pin: release n=experimental
Pin-Priority: 1

EOF

# ADD WHEEZY REPOS FOR ABOVE PIN LIST

echo "deb "${mirror}" wheezy main" > \
config/archives/wheezy.list.chroot

# ADD WHEEZY REPOS FOR ABOVE PIN LIST
echo "deb "${mirror}" experimental main" > \
config/archives/experimental.list.chroot

# BUILD THE IMAGE
sudo lb build


# CHECK FOR BAD BLOCKS ON THE DEVICE
sudo badblocks -c 10240 -s -w -t random -v "${devicel}"

# OVERWRITE THE DEVICE WITH RANDOM DATA
sudo dd if=/dev/urandom of="${devicel}"


# DD THE binary.img to a usb
sudo dd if=binary.img of="${devicel}"

# THIS JUST TELLS PARTED WHERE TO START
# TO MAKE THE SDB2 PARTITION
read bytes _ < <(du -bcm binary.img |tail -1); echo $bytes

# MAKE THE PARTITION,
sudo parted "${devicel}" mkpart primary "${bytes}" "${usb_size}"


# ENCRYPT THE PARTITION
sudo cryptsetup --verbose --verify-passphrase luksFormat "${devicel}2"

# OPEN THE ENCRYPTED PARTITION
sudo cryptsetup luksOpen "${devicel}2" my_usb

# MAKE A FILESYSTEM ON IT AND LABEL IT "persistence"
sudo mkfs.ext3 -L persistence /dev/mapper/my_usb

# MAKE A MOUNT POINT
sudo mkdir -p /mnt/my_usb

# MOUNT THE OPENED ENCYRYPTION PARTITION
sudo mount /dev/mapper/my_usb /mnt/my_usb/

# MAKE THE PERSISTENCE.CONF FILE
echo "/ union" > ~/persistence.conf && sudo mv ~/persistence.conf \
/persistence.conf && sudo mv /persistence.conf /mnt/my_usb

# UMOUNT IT
sudo umount /dev/mapper/my_usb

# CLOSE THE LUKS PARTITION
sudo cryptsetup luksClose /dev/mapper/my_usb

# TEST IT
sudo apt-get install qemu --force-yes

sudo kvm "${devicel}"
haylo
 
Posts: 3
Joined: 2013-02-12 14:49

Re: Debian Live encrypted persistence

Postby dzz » 2013-02-12 21:14

Good to hear luks persistence works with official packages only, even if only for sid. The hook script mentioned earlier was only ever done, for wheezy, because the official stuff did not.

However in wheezy we are already advised to use live-* packages from sid.. What is the difference that it does not work for wheezy and how can it be made to?

Looking changelogs for live-boot, live-config and live-build in experimental I see no mention of luks persistence in any recent stuff.
dzz
 
Posts: 225
Joined: 2007-02-05 20:39
Location: Devon, England

Re: Debian Live encrypted persistence

Postby fsmithred » 2013-02-13 10:28

Code: Select all
apt-cache policy live-boot
live-boot:
  Installed: 3.0~a35-1
  Candidate: 3.0.0-1
  Version table:
     4.0~a5-1 0
          1 http://debian.lcs.mit.edu/debian/ experimental/main amd64 Packages
     3.0.0-1 0
        500 http://debian.lcs.mit.edu/debian/ sid/main amd64 Packages
 *** 3.0~a35-1 0
        500 http://debian.lcs.mit.edu/debian/ wheezy/main amd64 Packages
        100 /var/lib/dpkg/status


That might explain why it didn't work for me. I'm using live-* packages from wheezy. I already had a usb stick with an encrypted partition, so I added persistence.conf, labeled the filesystem and added the relevant boot options to my syslinux boot menu. Thinking there might be a typo in the instructions, I tried both "persistent=cryptsetup" and "persistence=cryptsetup". Also tried "persistence-path=/dev/mapper/persistence. I can try this later in a sid install.
fsmithred
 
Posts: 1673
Joined: 2008-01-02 14:52

Re: Debian Live encrypted persistence

Postby dzz » 2013-02-14 02:46

Some details of a live usb set up here:

part1: FAT32 with syslinux and a custom Wheezy live image. All live-* packages are very recent sid. Tested and works normally (including with non-luks persistence). Image includes cryptsetup.

Code: Select all
:~$ dpkg -l|grep live-
ii  live-boot                               3.0~b11-1                          all          Live System Boot Scripts
ii  live-boot-doc                           3.0~b11-1                          all          Live System Boot Scripts (documentation)
ii  live-boot-initramfs-tools               3.0~b11-1                          all          Live System Boot Scripts (initramfs-tools backend)
ii  live-config                             3.0.18-1                           all          Live System Configuration Scripts
ii  live-config-doc                         3.0.18-1                           all          Live System Configuration Scripts (documentation)
ii  live-config-sysvinit                    3.0.18-1                           all          Live System Configuration Scripts (sysvinit backend)
ii  live-tools                              3.0.17-1                           all          Live System Support Scripts


part2: LUKS volume containing a EXT2 filesystem labelled "persistence". Contains a file "persistence.conf" with text "/ union,sources=."

cmdline includes: " persistence persistence-encryption=luks " Very early in the boot process the luks key was prompted for.

In the live session: It is already mapped but not mounted according to <blkid> and <cat /proc/mounts> It can be mounted manually without needing luks key again. It is certainly is not being used for persistence.

If "debug" is added to the cmdline everything freezes after the first 3 screens. Power button is the only way out.. Not even busybox. No clue why and no log to save.

The actual live* packages still don't work for luks persistence in wheezy (custom hook script does) I don't know what is different about a sid build (except that every day, it is different, yesterday's is already outdated)

EDIT <man live-boot>
persistence-encryption=TYPE1,TYPE2 ... TYPEn
This option determines which types of encryption that we allow to be used
when probing devices for persistence media. If "none" is in the list, we
allow unencrypted media; if "luks" is in the list, we allow LUKS-encrypted
media. Whenever a device containing encrypted media is probed the user will
be prompted for the passphrase. The default value is "none"
.

EDIT2:
# WORK IN PROGRESS ... CHANGING until perfected

Looking forward to that so "hacks" are no longer necessary.. thanks Debian-Live devs for your good work.
dzz
 
Posts: 225
Joined: 2007-02-05 20:39
Location: Devon, England

Re: Debian Live encrypted persistence

Postby /dev/null » 2013-02-14 21:20

LUKS volume containing a EXT2 filesystem labelled "persistence". Contains a file "persistence.conf" with text "/ union,sources=."

cmdline includes: " persistence persistence-encryption=luks " Very early in the boot process the luks key was prompted for.

In the live session: It is already mapped but not mounted according to <blkid> and <cat /proc/mounts> It can be mounted manually without needing luks key again. It is certainly is not being used for persistence.

I've got exactly the same results for custom sid

Right after password check there's an error:
Code: Select all
mount: mounting Loading on live/persistence/Loading failed : no such device

so it basically looks similar to this "Loading" thing I mentioned before:
Code: Select all
+ probe_for_fs_label live-rw home-rw persistence Loading /etc/boottime.kmap.gz /dev/mapper/sdb2
+ local overlays dev
+ overlays=live-rw home-rw persistence
+ dev=Loading
+ /sbin/blkid -s LABEL -o value Loading
+ [ = live-rw ]
+ /sbin/blkid -s LABEL -o value Loading
+ [ = home-rw ]
+ /sbin/blkid -s LABEL -o value Loading
+ [ = persistence ]
+ result=
+ [ -n ]

If "debug" is added to the cmdline everything freezes after the first 3 screens. Power button is the only way out.. Not even busybox. No clue why and no log to save.

again, same here
last displayed line is
Code: Select all
uhci_hcd: USB Universal Host Controller Interface driver

on my other laptop it is :
Code: Select all
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver


Btw what is this boot parameter: persistent=cryptsetup ?
I can't see it in man live-boot.

Tomorrow I'll try debian live irc channel :D
/dev/null
 
Posts: 62
Joined: 2013-01-30 17:31

Re: Debian Live encrypted persistence

Postby dzz » 2013-02-19 20:09

no need for any "hacks"
it works, just takes a lot of research

Those of us who did plenty research, still failed and only then resorted to "hacks" remain mystified.

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=700902

The good news is: live-boot_4.0~a7-1_all.deb, which made experimental only today, actually does work. The fix is also noted in the changelog.

I just tested it using an existing custom wheezy image, booted from a rebuilt initrd containing the new stuff in /lib/live/

You can replace just the initrd in /live of a syslinux-type usb setup, maybe not if you use the less flexible dd method (IMO totally inflexible because the device is then useless for normal data storage)

There is still a problem: if you use "debug" on the cmdline (to get a full boot log) with luks the system will hang early on (and the log can't be saved)
dzz
 
Posts: 225
Joined: 2007-02-05 20:39
Location: Devon, England

Re: Debian Live encrypted persistence

Postby pcalvert » 2013-02-20 05:29

As an alternative, what about creating a "Data" or "Documents" directory inside one's home directory and then encrypting that with eCryptfs?

Phil
"I never had an interest in being a mayor 'cause that's a real job. You have to produce.
That's why I was able to be a senator for 36 years." - U.S. Vice President Joe Biden
pcalvert
 
Posts: 1558
Joined: 2006-04-21 11:19
Location: Sol Sector

Re: Debian Live encrypted persistence

Postby dzz » 2013-02-20 14:37

As an alternative, what about creating a "Data" or "Documents" directory inside one's home directory and then encrypting that with eCryptfs?


A good option for actual data files but dot files also containing personal data would still be unencrypted. (voip account, mail client, browser ,,,)

Pendrives are easily lost or stolen; proper luks persistence is important for "travelling OS" security. We're close to getting it sorted.
dzz
 
Posts: 225
Joined: 2007-02-05 20:39
Location: Devon, England

Re: Debian Live encrypted persistence

Postby /dev/null » 2013-02-20 22:40

great news, thank you

I just tested it using an existing custom wheezy image, booted from a rebuilt initrd containing the new stuff in /lib/live/

how to rebuild initrd with new live-boot?
or how make wheezy or sid build with live-boot from experimental?
/dev/null
 
Posts: 62
Joined: 2013-01-30 17:31

Re: Debian Live encrypted persistence

Postby dzz » 2013-02-21 01:47

how to rebuild initrd with new live-boot?


Initrd hacking, sid/experimental and live-image building are not really "Beginners Questions" material. The bottom line is, LUKS persistence doesn't work (yet) in the "mainstream". However you did ask.

Extract an initrd:

Code: Select all
# make a directory, copy into it the original initrd
# open a terminal, cd to that directory

INITRD=name_of_original_initrd
mkdir ./uz
cd uz
fakeroot zcat ../$INITRD | cpio -i


Now it's extracted you can change things.

/lib/live/ contains the live-boot scripts. what I did was replace everything in there with the new stuff from the previously extracted .deb

Rebuild it:

Code: Select all
cd uz
fakeroot find . -print0 | cpio -0 -H newc -o | gzip -c > ../initrd.rebuilt


Use initrd.rebuilt to boot with (rename it or edit the menu)

I'm not saying this is the "right" way to do things, nor is this a "recommendation" Just trying to find my own way through that which is about as clear as mud. And I'm working with wheezy not sid.

I'm sure this stuff will be sorted officially in due course (getting a bit late to make wheezy in time though)
dzz
 
Posts: 225
Joined: 2007-02-05 20:39
Location: Devon, England

Re: Debian Live encrypted persistence

Postby /dev/null » 2013-02-22 21:56

It's working 8)
thank you dzz
/dev/null
 
Posts: 62
Joined: 2013-01-30 17:31

Re: Debian Live encrypted persistence

Postby /dev/null » 2013-03-16 13:17

It's working

Well - not anymore :x
They must messed something up with the new versions of boot scripts. It seem that scripts from experimental doesn't fit wheezy/sid builds anymore.
/dev/null
 
Posts: 62
Joined: 2013-01-30 17:31

Re: Debian Live encrypted persistence

Postby llivv » 2013-03-16 13:41

/dev/null wrote:They must messed something up with the new versions of boot scripts. It seem that scripts from experimental

is there a new feature in the version from experimental you wanted to try?

posting which versions worked for you
and which version[s] don't work for you
helps a lot more than telling us that they messed up in experimental.
in the kitchen with Julia ....
[...]
Get on the D bus to B Can ....
[...]
User avatar
llivv
 
Posts: 5539
Joined: 2007-02-14 18:10
Location: will O' the tree hug

Re: Debian Live encrypted persistence

Postby /dev/null » 2013-03-16 15:39

I'm talking about luks-encrypted persistence partition in Debian live.
There was a bug in live-boot that causes the luks-encrypted partitions aren't properly detected and cannot be used for persistence. They fixed it recently in live-boot 4.0, but it is still in experimental.
Since building an experimental live image isn't the best idea, "dzz" suggested to just put newest boot scripts from live-boot 4.0.deb package directly into previously built wheezy's/sid's initrd.img.
This method worked great a month ago, but yesterday when I tried it again, the system stuck somewhere during the boot procedure.
It seems that developers changed the newest (4.0) boot scripts in such way that they aren't compatible with wheezy/sid builds anymore.
I tried to install whole experimental live-boot 4.0 package (to overwrite default buggy 3.0) using "--interactive shell" in lb config. I changed repos to experimental but apt-get update doesn't seem to work as it should.
apt-get install live-boot - says that the package is already in the newest version, but it isn't - apt-cache policy live-boot clearly states that the installed version is 3.0.

So if anyone could tell me how to install live-boot from experimental during wheezy/sid build procedure I would be very grateful
/dev/null
 
Posts: 62
Joined: 2013-01-30 17:31

Re: Debian Live encrypted persistence

Postby dzz » 2013-03-16 23:43

Experimental is just that, "experimental". It can change fast. Sometimes stuff is broken, or there might be undocumented changes. You need to be *very* selective what you use from there and without expectations. You might break something else unexpectedly.

That's why I preferred just a custom initrd to actually installing experimental packages. (revised) Just replace the entire /lib/live directory

live-boot 4.0~a9-1 is current. I got a kernel panic using an initrd rebuilt with that. I don't know if it's actually broken or something I did wrong.

You can find 4.0~a7-1 (we know that works) here:

http://snapshot.debian.org/binary/live-boot/
http://snapshot.debian.org/package/live-boot/4.0~a7-1/

BTW luks loopback files still don't work (there is a patch for that)

I use custom build scripts in preference to official live-build so can't help much with that. Rather than use the repo for one package (I think it's <apt-get -t experimental install>) I would dpkg -i the deb.. or just do the custom initrd after build
dzz
 
Posts: 225
Joined: 2007-02-05 20:39
Location: Devon, England

PreviousNext

Return to Beginners Questions

Who is online

Users browsing this forum: No registered users and 10 guests

fashionable