Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

dd persistence and usb size limits

If none of the specific sub-forums seem right for your thread, ask here.
Post Reply
Message
Author
kbd
Posts: 6
Joined: 2015-02-26 00:48

dd persistence and usb size limits

#1 Post by kbd »

Im still basically a noob, but i have spent a short time with debian persistent systems (ubuntu, bt5 and kali). i have noticed dd 16gb usb images work fine on different usbs of 16gb size; but corrupt going to a bigger drive. I am imaging a 16gb debian gnome os persistent usb. I have tried imaging both the main fat32 drive(sdc1) and the persistent ext3 drive(sdc2) separately + mbr separate too. Also the entire usb together (sdc). I tried to re-burn the indivual drives(sdc1+sdc2) + mbr separely and the entire drive (sdc) separately (since dd just leaves unused space alone imaging a 16gb to a 32gb should just leave ~16gb unused at the end of the drive.) It seems there may be a 16gb limit for persistence and this would be debian related since it appears in both ubuntu, bt5 and kali.

The dd process to save the image works without error. The dd process to burn saved image to usb works without error. The image on 32gb usb boots 70-80% of the time. Forensic/Live modes work better than persistent mode - 99% of the time if the usb boots then forensic mode works perfectly, install packages, customize, work, everything flawless. Persistent mode boots slower on 32gb than 16gb or 8gb. Sometimes the user login screen appears and i must enter user and pass. About 40% of the time once it boots it works normally, but incredibly slow - packages install, files open, drives mount, customization are error free just everything is unbelievably slow. again on 8&16gb usbs everything works perfectly. Ubuntu tends to successfully boot less often since it defaults to persistence if you dont install, ubuntu does not have forensic or true live mode that i know of YET. The type of usb does not seem to matter - i can use any make and model of usb from any vendor, but the size is the issue -

What are the pro/cons to full install on usb vs. persistence. I like ultimate portability and computer flexability and wish to create a customizable system that runs on usb dependably. Full install tends to make the system computer specific, while persistence allows for movement of the system between computers - and i like that. I understand this may be a side question that some may want to answer separately, if i should move it to a different section pls let me know, i ask it here because i think it may relate directly to a solution to my 8-16-32gb usb clone issue.

note: as i have spent more time with debian my persistent usbs have become more complex, could persistent systems have a limit of packages, updates, installs, and customizations?

note: i started with 8gb persistent usbs and they seemed to dd without problem being imaged to 16gb usbs. But both 8gb and 16gb images fail on 32gb usbs or larger. Does debian not permit persistent systems to be built on drives larger than 16gb?

kbd
Posts: 6
Joined: 2015-02-26 00:48

Re: dd persistence and usb size limits

#2 Post by kbd »

i suspect mib vs. cylinder during partitioning....stay tuned

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: dd persistence and usb size limits

#3 Post by Head_on_a_Stick »

I have had success running distributions from a high quality 64GiB USB3.0 stick by just treating it as an attached hard drive and installing normally.

The speed was acceptable even via a USB2 port and very good via a USB3 port.

I currently have Kali running with persistence off a slow, cheap 8GiB stick and it is very slow when loading and saving data.
deadbang

kbd
Posts: 6
Joined: 2015-02-26 00:48

Re: dd persistence and usb size limits

#4 Post by kbd »

Yeah i was finally able to get a 32gb Lexar usb working with a 16gb usb image - the cylinders vs. mib selection during initial formatting of the individual 32gb harddrives was likely a large part of the problem. (After using cylinders the 32gb usb booted first try !) I burned separte images for each of the drives, one for the live system(sdc1) and one for the persisence file(sdc2). I am inclined to agree with you about usb2.0 vs usb3.0 causing persistence to respond more slowly, but i also suspect some other problem that may make the system behave a bit more sluggishly, or cause momentary hangs (3-10seconds). Head_on_a_Stick - sounds like you have tried persistence AND install methods, have you ever tried both methods using usb3.0 to see if there is a perfomance difference between persistence or treating it as an attached hard drive and installing normally. I will now try doing a full install on the same 32gb usb inorder to determine if installs inherently perform better than persistence.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: dd persistence and usb size limits

#5 Post by Head_on_a_Stick »

kbd wrote:Head_on_a_Stick - sounds like you have tried persistence AND install methods, have you ever tried both methods using usb3.0 to see if there is a perfomance difference between persistence or treating it as an attached hard drive and installing normally.
I've never bothered -- the Sandisk Extreme USB3.0 stick I use offers better performance via a USB3 port than the (slow) internal spinning rust hard drive on my crappy laptop.
deadbang

kbd
Posts: 6
Joined: 2015-02-26 00:48

Re: dd persistence and usb size limits

#6 Post by kbd »

dont really want to bump this thread again but @Head_on_a_Stick i have been using usb persistent kali drives for awhile now - curious if you are still aswell. and if so - i noticed that logs tend to fill up quickly, so much so i had to write a bash to sed delete the messages, kern, and a few other logs. Also after 75-100hrs of use i need to sync more often to get the system to respond.until finally i must reburn a fresh backup image 200-300 hrs of active use. doesnt seem alot of ppl use linux nor kali this way on a consistent basis - so thought i would ask your observations. take care

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: dd persistence and usb size limits

#7 Post by Head_on_a_Stick »

I have been playing around with my own custom live image [1] recently and that also has a persistence option and I have to say that it works very poorly indeed with delays during the booting and shutdown sequences and also during any extended sync's to the persistence medium.

I really would recommend purchasing a quality USB3 stick instead and treating it as a removable hard drive (ie, perform a conventional installation to the device), the results can be workable even on machines which only have USB2 ports thanks to the high quality controller chips in USB3 sticks.

EDIT: another option (for Debian stretch) is debootstick:
debootstick is used to generate a bootable image from a Debian or Ubuntu chroot environment (such as one generated with debootstrap, docker export, etc.). This image should then be copied to a USB stick or disk and used to boot any amd64 machine (BIOS- or UEFI-based). The embedded system is ready to be started live (no installation procedure needed), and is fully upgradeable (kernel and bootloader included).
https://github.com/drakkar-lig/debootst ... sign-Notes

:cool:
deadbang

Post Reply