How to create a Debian live USB with persistence?

debian live-usb persistence

34,901

Solution 1

Debian live with persistence.

First try with official image from www.debian.org/CD/live/

From SE site (standard live):
```
wget https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/debian-live-10.0.0-amd64-standard.iso
```
From automatic mirror selection (mate live):
```
wget http://debian-cd.debian.net/debian-cd/10.0.0-live/amd64/iso-hybrid/debian-live-10.0.0-amd64-mate.iso
```
Then checksum you download file with https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/SHA256SUMS...

Ok ISO filesystem is read-only, but there is a little workaround: we could replace non vital bootparam by persistence in this way.
- Once file validated !
- you could alter them by using sed for replacing strings in binary.
```
LANG=C sed 's/splash quiet/persistence /;s/quiet splash/persistence /' \
    </tmp/debian-live-10.0.0-amd64-mate.iso \
    >/tmp/debian-live-10.0.0-amd64-mate-persist.iso
```
This will create a modified copy of your live binary file, by strictly replacing splash quiet or quiet splash by persistence, everywhere. Ok this will work only while grub boot command do contain this two words together.

But care to not miss the space after persistence:
```
"splash quiet" -> 12 characters
"persistence " -> 12 characters
```
Or your binary will be broken.
Install on USB key
```
dd if=debian-live-10.0.0-amd64-standard-persist.iso of=/dev/sdX
```
Then add your third partition for persistence:
```
fdisk /dev/sdX
n           # new partition
p           # primary
<Return>    # default: 3
<Return>    # default: next free sector
<Return>    # default: last addressable sector
w           # write and quit
```
This could be run without interaction:
```
fdisk /dev/sdX <<<$'n\np\n\n\n\nw'
```
Format and prepare persistence with union:
```
mkfs.ext4 -L persistence /dev/sdX3
mount /dev/sdX3 /mnt
echo '/ union' >/mnt/persistence.conf
sync
umount /mnt
```
Then eject and try!

If you use official, unmodified image, for using persistence, you have to interrupt boot selection:

Once menu screen is displayed, choose your boot option, then instead of Return, hit Tab.

The kernel commandline will be displayed, then add persistence with a space, after last word (quiet), then hit Return.

Unfortunately, as 1st partition is bundled with UEFI and is ISO, you can't modify the boot command.

Customized Debian live with persistence

You just have to add persistence to boot command line, but nothing else!? There is a way, using FAT and syslinux, but you have a lot of data manipulations. It's long and I find this not so well. I prefer:

Build your own Debian live

More regular, but a little longer (at least for 1st image),

Note: All this stuff was done under root user (this must work by using fakeroot, but this is not tested there and today).

apt install live-build

... and all recommendations.

I wrote a little XARGS function for dropping commented lines:

XARGS() { sed -ne '/#/d;s/ \t//g;H;${x;s/\n/ /g;s/^ //;p}'; }

First setting bootparams, with localisation and arguments for persistence:

ExtraBootParams=$(XARGS <<eobp
    boot=live
    config
    locales=ed_WT
    keyboard-layouts=ed
    keyboard-variant=wt
    persistence
eobp
)

Now your package list:

PackageList=$(XARGS <<-eopl
    gnome
    gnome-core
    # gnome-full
    # debian-forensics
    debian-installer-launcher
eopl
)

Very first step of lb: create initial tree:

lb config --architectures amd64 -d buster --debian-installer-gui \
    true --archive-areas 'main contrib non-free' \
    --bootappend-live "$ExtraBootParams"

Now, you have a small tree, you could:

printf "%s\n" > config/package-lists/standard.list.chroot \
    $Packages $PackageList

Ok, next command will take a loooong time! (Approx 1 hour on my host)

lb build

If everything's ok, you may find your own Debian live:

ls -l *.iso
-rw-r--r--  1 root root 1511817216 sep  7 15:32 live-image-amd64.hybrid.iso

Install on USB key (same operation than for downloaded binaries)

The 'iso-hybrid' image contains two partitions for UEFI and live mixed in a way both EFI and bios could boot on.

file live-image-amd64.hybrid.iso
live-image-amd64.hybrid.iso: DOS/MBR boot sector; partition 2 : ID
=0xef, start-CHS (0x3ff,254,63), end-CHS (0x3ff,254,63), startsect
or 708, 5696 sectors

You could simply put to your USB Key: (Note: ensure first your USB Key is not mounted!)

dd if=live-image-amd64.hybrid.iso of=/dev/sdX

Then add your third partition for persistence:

fdisk /dev/sdX <<<$'n\np\n\n\n\nw'

Format and prepare persistence with union:

mkfs.ext4 -L persistence /dev/sdX3
mount /dev/sdX3 /mnt
echo '/ union' >/mnt/persistence.conf
sync
umount /mnt

Eject and try...

Debian live with encrypted persistence

Build your own Debian live, but with encrypted persistence.

In order to boot with rootfs / on encrypted persistence, you have to add dm-crypt module and related binaries to initrd (initial ram disk) by adding setting CRYPTSETUP=y into an /etc/initramfs-tools/hooks/...

I wrote a little XARGS function for dropping commented lines:

XARGS() { sed -ne '/#/d;s/ \t//g;H;${x;s/\n/ /g;s/^ //;p}'; }

First setting bootparams, with localisation and arguments for persistence and cryptsetup:

ExtraBootParams=$(XARGS <<eobp
    boot=live
    config
    locales=ed_WT
    keyboard-layouts=ed
    keyboard-variant=wt
    persistent=cryptsetup
    persistence-encryption=luks
    persistence
eobp
)

Now your package list:

PackageList=$(XARGS <<-eopl
    gnome
    gnome-core
    # gnome-full
    # debian-forensics
    debian-installer-launcher
eopl
)

And your package selection:

Packages=$(XARGS <<-eopk
    cryptsetup
    cryptsetup-initramfs
    debian-installer-launcher
    firmware-linux-nonfree
    firmware-linux-free
    less
    ssh
#   openvpn
#   xtightvncviewer
    gsmartcontrol
    smartmontools
    partclone
    ntfs-3g
    task-gnome-desktop
    user-setup
    sudo
    apt-utils
eopk
)

Of course cryptsetup is required! ;-)

Very first step of lb: create initial tree:

lb config --architectures amd64 -d buster --debian-installer-gui \
    true --archive-areas 'main contrib non-free' \
    --bootappend-live "$ExtraBootParams"

Now, you have a small tree, you could:

printf "%s\n" > config/package-lists/standard.list.chroot \
    $Packages $PackageList

Ok, next two command will take a loooong time! (Approx 40' on my host)

lb bootstrap ; lb chroot

Now you could add your module and binaries:

echo dm-crypt >> chroot/etc/initramfs-tools/modules

sed '/CRYPTSETUP=/s/^#//;s/=.*/=y/' -i \
    chroot/etc/cryptsetup-initramfs/conf-hook 
ln -s ../../cryptsetup-initramfs/conf-hook \
    chroot/etc/initramfs-tools/hooks/cryptsetup
chroot chroot live-update-initramfs -u

Then run final stage (will take some more lot of time ~25'):

lb installer ; lb binary

Note: If you read cryptsetup: WARNING: Couldn't determine root device, it's fine! This mean that cryptsetup is installed on your initrd.

If everything's ok, you may find your own Debian live:

ls -l *.iso
-rw-r--r--  1 root root 1511817216 sep  7 15:32 live-image-amd64.hybrid.iso

Install on USB key

The 'iso-hybrid' umage do contain already two partitions for UEFI and live mixed in a way both EFI and BIOS could boot on.

file live-image-amd64.hybrid.iso
live-image-amd64.hybrid.iso: DOS/MBR boot sector; partition 2 : ID
=0xef, start-CHS (0x3ff,254,63), end-CHS (0x3ff,254,63), startsect
or 708, 5696 sectors

You could simply put to your USB Key: (Note: ensure first your USB Key is not mounted!)

dd if=live-image-amd64.hybrid.iso of=/dev/sdX

Add new Linux partion by using free space.

fdisk /dev/sdX <<<$'n\np\n\n\n\nw'

This will create partition 3 using free space on your USB Key.

Then prepare your crypted partition

cryptsetup -q luksFormat /dev/sdX3

Enter passphrase

cryptsetup -q luksOpen /dev/sdX3 persist

Enter passphrase again

mkfs.ext4 -L persistence /dev/mapper/persist
mount /dev/mapper/persist /mnt
echo '/ union' >/mnt/persistence.conf
sync
umount /mnt
cryptsetup luksClose persist

That's all.

eject /dev/sdX

Solution 2

Cloning and Debian persistent live

Cloning (copying each byte as it is, creating a one-to-one copy) is a very simple and reliable method to create a boot drive (live drive or install drive) from a linux hybrid iso file.

There is no conventional way to make a cloned Debian iso file persistent, because it has a read-only file system, ISO9660, (and read-only 'partition table').

Using mkusb (BIOS/UEFI)

It is possible (and I would say rather simple) to use mkusb to create a persistent live drive from a Debian live iso file. mkusb supports Debian 8 or newer, and you should use the current version of mkusb (version 12.3.3 or newer).

See the details at this link to install mkusb in Debian.
There are tips about persistent live drives made with mkusb.
See also this link (where Debian is one of the distros listed).

mkusb creates a partition table and 5 partitions:

NTFS partition for exchange of data with Windows computers (optional size)
Extended partition or bios_grub partition
FAT32 boot partition for booting with grub (both in UEFI and BIOS mode)
ISO9660 partition containing clone of the iso file
ext partition for persistence, where your modifications (and files) are stored

mkusb does the following tweaks automatically:

The boot option persistence (not persistent as in Ubuntu)
The label persistence on partition #5 (not casper-rw as in Ubuntu)
The file persistence.conf with the content / union at the top level of partition #5.

This structure is created by the bash shellscript dus-persistent, when you use mkusb version 12, alias mkusb-dus. If you want all the details, install mkusb and read the content of dus-persistent, or read it directly via phillw.net/isos/linux-tools/mkusb/dus-persistent.

The default settings of mkusb can be used for Debian 8, 9 and 10, and the persistent live drive will work when booted in BIOS mode and UEFI mode (but not with secure boot).

(In the previous version mkusb 12.3.2 you must set 'usb-pack-efi' manually in the 'settings' menu for Debian 10 in UEFI mode.)

Using manual setup (UEFI only)

If you want an UEFI only boot flash drive you do not need an installer at all.

You just need to format flash drive with FAT32 and set boot flag on. Then use whatever is your favorite extraction tool like 7zip to extract & copy ISO to FAT32 partition. See Do it yourself for more details.

I verified this method with the standard live iso file (small, no graphic desktop environment),

debian-live-10.0.0-amd64-standard.iso

So this way you can make a USB boot drive with Debian 10 (64-bit). It will be live-only and boot in UEFI mode.

Create a FAT32 partition big enough to store the files from the iso file (add approximately 5 % to the size of the iso file)
Mount the FAT32 partition
Extract the content (directory tree with all the files) of the iso file to the FAT32 partition

Now you have a live-only USB boot drive

Edit the word persistence to the end of the line(s) starting with linux in the file
```
/path/to/mountpoint/boot/grub/grub.cfg
```
Create an ext2 partition in the unallocated space (behind the FAT32 partition)
Put the label persistence on the ext2 partition
Mount the ext2 partition
Write / union to the file persistence.conf in the ext2 file system
Unmount all partitions on the flash drive before you unplug it, or reboot

Now you have a persistent live USB drive with Debian 10.

Command output seen from within the persistent live drive:

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            7.8G     0  7.8G   0% /dev
tmpfs           1.6G  9.0M  1.6G   1% /run
/dev/sdb1       4.0G  826M  3.2G  21% /run/live/persistence/sdb1
/dev/loop0      610M  610M     0 100% /run/live/rootfs/filesystem.squashfs
tmpfs           7.8G     0  7.8G   0% /run/live/overlay
/dev/sdb2        11G   38M   11G   1% /run/live/persistence/sdb2
overlay          11G   38M   11G   1% /
tmpfs           7.8G     0  7.8G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           7.8G     0  7.8G   0% /sys/fs/cgroup
tmpfs           7.8G     0  7.8G   0% /tmp
tmpfs           1.6G     0  1.6G   0% /run/user/1000

$ lsb_release -a
Distributor ID: Debian
Description:    Debian GNU/Linux 10 (buster)
Release:    10
Codename:   buster

$ lsblk -fm /dev/sdb
NAME   FSTYPE LABEL       UUID                                 FSAVAIL FSUSE% MOUNTPOINT                            SIZE OWNER GROUP MODE
sdb                                                                                                                14.8G             brw-rw----
├─sdb1 vfat   USBBOOT     7176-C538                               3.2G    20% /usr/lib/live/mount/persistence/sdb1    4G             brw-rw----
└─sdb2 ext2   persistence 2b324439-d63e-4a19-bf57-d49ecb881828     10G     0% /usr/lib/live/mount/persistence/sdb2 10.8G             brw-rw----

You can see that the size of overlay matches that of /dev/sdb2 which indicates that the persistence is working.

Using `mkusb-minp` (BIOS/UEFI)

It is possible (and I would say rather simple) to use mkusb-minp to create a persistent live drive from a Debian live iso file. mkusb-minp supports Debian 10 or newer. This simple shellscript is developed from mkusb-min. Both of them 'wrap a safety belt' around the cloning process to help avoid writing to the wrong device.

This is a good option, if you do not want to add software via a PPA or in general want to use only tools that you can understand.

I made this simple shellscript based on the simple cloning method by F. Hauri (described in another answer in this thread).
See the general description at help.ubuntu.com/community/mkusb/minp.
There are more details at help.ubuntu.com/community/mkusb/minp/details.

Solution 3

Why don't you make a new installation of Debian choosing the usb device instead your HDD as the destination? What capacity does your usb have?

I have a Debian LXDE installed and runing in a 4GB usb memory stick, updating the system whenever I want and installing packages when I need them.

Solution 4

First of all - I am no Linux guy - just familiar with it more than the average Joe.

You could use yummi installer to create a Debian 10 usb (I used a 16GB stick I had laying around) boot stick, then boot from it.

The once in, insert a large USB stick into a different port (I am doing this as we speak with a 128GB Samsung FIT flash-drive) and then double-click on the Install Debian icon on the desktop.

Run through the normal stuff until you get to the partitioning, then select manual..in the manual partitioning screen ensure your 128GB USB stick is selected (or whichever one you use - make sure it is not the local drive for whatever PC you are using to do this and make sure it is not the 16GB USB stick that the live Debian is currently running on)

Once done delete all existing partitions frm the 128GB usb stick and create the following 3 partitions.

500 MB /boot/efi 1942MB linuxswap the rest ext2 (mine is 119GB)

---- I wouldn't use ext3 or 4 for a USB stick - not sure why, some Linux guy told me that a few years ago and I stuck with it. If anyone knows why please tell us?

and finally ensure you're boot loader will be installed to the proper USB stick - this was on the same page as the partitioning when I did it and it is easy to miss - for me, it was at the very bottom and I had to change it from the local SDD of the PC I was using to do this to the Samsung 128GB FIT flash-drive. After all that, next...wait about an hour or so (in my case).

If Debian live locks the password is "live" without quotes of course.

It finished while I was typing this so I reboot and log onto my full blown Debian 10 system running from my Samsung 128GB FIT USB flash-drive. With the approx. 10 minutes for usage I've had on it I was able to confirm disk space is as it should be (LOTS) and it seems to be performing a bit slower than my NvMe local drive but faster than a reguler 7200RPM HDD.

There yah go.

View more solutions

34,901

Pierre

Updated on September 18, 2022

Comments

Pierre over 1 year
I would like to use Debian wherever I go by installing it on a USB flash drive, but I'm not sure how to make it save the settings and be able to install/update packages without loosing them on reboots.

So I downloaded a Hybrid ISO image from this link, and I followed the instructions in the FAQ:
```
dd if=image.iso of=/dev/sdb bs=4M; sync
```
This command copied the image to the flash drive by creating a partition which size is the same as the ISO image (1.3 GB out of 8GB), and the rest of the flash drive is unallocated.

I searched on how to do this, but every tutorial uses a different approach and some of them are outdated and talk about the old usb-hdd image.

So how should I install this hybrid image on the flash drive ?

How should I partition my flash drive to be able to install packages and save settings ? and how can I install this image without using dd ?
Pierre about 10 years

It's an 8GB flash drive, I want to use GNOME, and thought that it would take a lot of space, and the GNOME live image is 1.3 GB only, but my HDD install is larger but I don't know exactly how much, which installation image did you use ?
slm about 10 years

@Peter - 8GB should be fine for a default install.
YoMismo about 10 years

I used cdimage.debian.org/debian-cd/current-live/i386/iso-hybrid/…, since I wanted speed I picked lxde. You have a gnome image there too. Gnome from the usb may be a little bit slow, but if you have plenty of RAM, it will only be slow opening the apps the first time, next time they will open faster. Regarding space, obviously you will have reduce the number of apps you install in the usb to the ones you really use/need, and leave the rest for your home computer.
YoMismo about 10 years

By the way, you may want to create your /tmp on your ram instead on your disk, you will only need to add the line tmpfs /tmp tmpfs nodev,nosuid,size=2G 0 0 to your fstab (change 2G for the size you want it to be and have in mind that the amount you assign to tmp won't be available for the rest of the apps)
F. Hauri over 4 years

Using installed system is not suitable to USB key who have to boot on many different hardware. A real live is build in a way to permit multiple hardwares
sudodus over 4 years

This is interesting: 1. Are there any bin files available for the current versions of Debian? 2. Please let us know, if and how your method works (with the current versions of Debian).
Admin over 4 years

@sudodus Since Debian 7 (Wheezy), the USB-HDD images were no longer built for release. I can at least confirm that the USB-HDD images are available for Debian 5 (Lenny) and 6 (Squeeze). The USB-HDD images are .img and not .bin, but I guess these are essentially the same as per Debian Wiki.
F. Hauri over 4 years

Ok, I will rewrite all this, but not today... Sorry.
Admin over 4 years

General information for users: As of today (5 Sep 2019), mkusb 12.3.2-1ubuntu4 is available via PPA for Ubuntu 12.04 and newer. SparkyLinux, a Debian-based distro, has the same version in their repository as announced in March 2019.
F. Hauri over 4 years

@sudodus Rewite finished, and no, sorry: bin image published on debian.org don't boot with crypted persistence.
Admin over 4 years

Neither OP or the bounty provider myself had asked for encrypted persistence, just the normal persistence. Regardless, this answer still does not explain how is this different from the hybrid ISO image offered by Debian--why need to build everything from the beginning...?
F. Hauri over 4 years

@clearkimura ... but encryption on private usb key is cool!!
F. Hauri over 4 years

@sudodus Sorry for previous misinfo: binary file available to download is capable of using persistence, but you have to ask for at each boot.
F. Hauri over 4 years

@clearkimura Another edit (hope the last). Added one section concerning available binaries to download and another section using live build, but simplier and without encryption.
sudodus over 4 years

+1; @F.Hauri, I have tried to create the partition for persistence after the cloned part of a USB drive using parted and gparted without much luck. You show that fdisk works (does not destroy the booting of the cloned drive). And yes, I see that it will be necessary to add the boot option persistence at every boot because the file system iso9660 is read-only by design. And I agree, it can be cool (even very valuable) to have encrypted persistence for example while travelling.
F. Hauri over 4 years

@clearkimura I've found a nice solution based on downloaded images, using `sed 's/splash quiet/persistence /'. See edited 1st paragraph!
sudodus over 4 years

@F.Hauri, Thanks for this tip :-)
Admin over 4 years

@F.Hauri Never mind me; I am rather concerned if end-users will ever understand when this answer will work or will fail to work (yes, that "nice solution" can fail). Myself at present would understand why, but myself from many years ago, like any average end-user, might have disagreed with this answer instead.
Admin over 4 years

Post-bounty comment: The sed command is just a workaround, and yet nobody seems to comment or explain the limitations. I think copying the solution from other answer is doing more harm than good, unless you can clarify when that will fail to work.
sudodus over 4 years

@clearkimura, I have tested 'all the way' F.Hauri's to 'binary edit' persistence into a Debian 10 iso file, create a partition behind it with fdisk and tweak the partition to become a container for persistent data. I can confirm that it works with Debian 10 both in UEFI and BIOS mode, but I cannot say anything yet about previous versions (Debian 8 and 9). I remove the text from the answer and make 'only' this comment about it.
sudodus over 4 years

@clearkimura, Now I have tested Debian 8 and 9. Persistence works with Debian 8.8.0-standard (amd64 and i386) in BIOS mode, but not in UEFI mode, because sed does not see the pattern suggested in grub. It would need another pattern (than suggested by F. Hauri). The partition for persistence works, and it is possible to add persistence at the grub menu. Persistence does not work with Debian 9.6.0-cinnamon because sed does not see the pattern suggested. It would need other patterns.The partition for persistence works, and it is possible to add persistence at the grub and syslinux menus.
sudodus over 4 years

A corresponding method using a binary edited and cloned drive with an 'fdisked' partition behind works with Lubuntu Eoan (to become 19.10) but not with Lubuntu 18.04.1 LTS. We can expect it to work with the next Lubuntu LTS (and with the other flavours of Ubuntu) 20.04 LTS.
sudodus over 4 years

@F.Hauri, I am sorry, but 'this will work only while grub boot command do contain this two words together, but this is alway the case, at least from Debian 6' is not true for the live iso files of Debian 8 and 9 that I have tested. It works well for me with the Debian 10 standard amd64 live iso file.
Admin over 4 years

Not sure if my comment was misinterpreted, but I just wanted to hint that the sed command will not work for everyone. Given that is only a workaround, it is not worth the time to do further testing.
F. Hauri over 4 years

@sudodus which one did you test? Standard, gnome, kde?
sudodus over 4 years

@F.Hauri, I tested with the following four iso files downloaded from Debian recently or some time ago: debian-live-10.0.0-amd64-standard.iso debian-live-8.8.0-i386-standard.iso debian-live-8.8.0-amd64-standard.iso debian-live-9.6.0-amd64-cinnamon.iso (I found no standard live iso file for Debian 9).
GAD3R over 4 years

I have tested mkusb 2 years ago, it work perfectly on/for creating a persistent debian 8 and 9. here is my answer on U&L
Jon over 4 years

oh yeah, I used Debian 10 KDE non-free but i suppsoe anything would work this way..am going to pull the stick from my laptop which I used to create it and try it out on my desktop and my web sver to make sure it boots fine across multiple computer manufacturers and is not limited to my laptop we'll see...let you know in 10 minutes or so...
Jon over 4 years

Worked fine on all systems - I had to disable secure boot on those that had it in their BIOS and enabled (x2 - tested on five systems = 1 older model HP Inspiron, one REAL older model Dell XPS, one Alienware Aurora R4 - secure boot not enabled, one Alienware Aurora R8 - secure boot enabled-had to disable it to get boot to work and one MSI Infinite X also with secure boot enabled and I disabled it). Other than that np. This is for me to use for diagnosis purposes only - otherwise I'd just install it to the NvME and secure boot it from that drive and be done with it.
sudodus over 4 years

You describe in a good way how to create an installed system in a USB drive (installed like into an internal drive). This is a good alternative, but it is not answering the original question of this thread. Anyway, I think your effort is worth upvoting (+1).
Jonathan Holvey almost 4 years

Since the Debian 10.0.0 live image is no longer available, can anyone confirm if the sed technique works with debian-live-10.5.0-amd64-standard.iso? I've successfully replaced the boot parameter, but I'm still prompted to choose a boot option.
F. Hauri almost 4 years

@JonathanHolvey On prompt, hit tab key and look command line, if your update is present.
Jonathan Holvey almost 4 years

@F.Hauri yes, I can see the update when I tab at the prompt. However, I need the OS to boot without user interaction. Is this not possible?
F. Hauri almost 4 years

@JonathanHolvey This is not same question! Please ask a new one!
Thomas Guyot-Sionnest over 3 years

This is brilliant! Quite useful on smaller devices that doesn't have enough ram to support all changes in tmpfs. From what I've seen the startup scripts look for just any device with the proper FS, so it looks like this could even work with an actual CD or network boot, + properly formatted usb stick (you may still have to edit the cmdline...).
acme64 about 3 years

Manual install confirmed working with deb 10.8