How to format a USB stick efficiently.

Share your own howto's etc. Not for support questions!

How to format a USB stick efficiently.

Postby edbarx » 2014-07-13 05:28

If you have a small USB stick and you want to economize on space, you may save some space by avoiding to create a partition table. On the other hand, if your USB stick is huge, consider whether it is more suitable for you to have multiple partitions in which case you will need to create a partitition table first.

I would like to thank the two posters following the first post for drawing my attention that 'my' method may not appeal to those brandishing very large USB sticks. Excuse my reaction, but ridicule is not the best vehicle for correction. :)

The Method:
Using gparted to format a USB stick has its downside of creating a partition table and thus wasting several megabytes of space. Since a USB stick usually contains just one partition it is normally pointless to create a partition table like an MBR and GPT

mkdosfs can be instructed to create a partition without requiring a partition table as follows:
Code: Select all
# mkdosfs -I -F 32 /dev/sdx

Where:
  1. -I
    USB sticks usually don't hold more than one partition. This
    switch tells mkdosfs to create a file system without checking
    for the presence of a partition table, thereby utilizing all the
    available space without constraints.
    This switch will force mkdosfs to work properly.
  2. -F FAT-size
    Specifies the type of file allocation tables used (12, 16 or 32
    bit). If nothing is specified, mkdosfs will automatically
    select between 12, 16 and 32 bit, whatever fits better for the
    file system size.
  3. /dev/sdx
    Is the device node representing the USB stick.

Reference: man mkdosfs
Last edited by edbarx on 2014-07-14 13:12, edited 5 times in total.
Debian == { > 30, 000 packages }; Debian != systemd
The worst infection of all, is a false sense of security!
It is hard to get away from CLI tools.
User avatar
edbarx
 
Posts: 5388
Joined: 2007-07-18 06:19
Location: 35° 50 N, 14 º 35 E

Re: How to format a USB stick efficiently.

Postby reinob » 2014-07-14 08:01

But you're surely aware that a DOS (MBR) partition table requires all of, wait for it.. 512 BYTES. Are you willing to turn a huge USB disk into a mere floppy disk just for half-a-kilobyte?

Think of all the possibilites a modern (64, 128GB) USB disk has.
reinob
 
Posts: 520
Joined: 2014-06-30 11:42

Re: How to format a USB stick efficiently.

Postby wizard10000 » 2014-07-14 09:32

Good on you for thinking outside the box; but as reinob mentioned the boot record and partition table take up one 512-byte hardware sector. You'll waste more space than that adding just *one* file to the drive (think file size vs. block size) :)

But - discoveries are made by not following instructions. My own discovery was some years ago when I put a Windows swapfile on a ramdrive :)

My logic was unassailable - I was going to speed up pagefile access by putting the pagefile in RAM; sadly, that was wrong on a couple of levels but I thought it was a great idea. Fortunately the people who knew more than me kinda rolled their eyes, patted me on the head and told me to keep learning.

Keep thinking outisde the box; sure it'll result in a bad idea once in awhile, but who knows - one day you might come up with something really great.

cheers -
we see things not as they are, but as we are.
-- anais nin
User avatar
wizard10000
 
Posts: 1204
Joined: 2011-05-09 20:02
Location: everywhere i go, there i am!

Re: How to format a USB stick efficiently.

Postby reinob » 2014-07-14 10:29

wizard10000 wrote:But - discoveries are made by not following instructions. My own discovery was some years ago when I put a Windows swapfile on a ramdrive :)

My logic was unassailable - I was going to speed up pagefile access by putting the pagefile in RAM; sadly, that was wrong on a couple of levels but I thought it was a great idea. Fortunately the people who knew more than me kinda rolled their eyes, patted me on the head and told me to keep learning.


You were just missing one ingredient: compression! :)
Having compressed-swap on RAM is actually a very useful thing in devices with little RAM but good CPU (think Nokia N900).

It's called ZRAM, and it's in the Linux kernel :)
reinob
 
Posts: 520
Joined: 2014-06-30 11:42

Re: How to format a USB stick efficiently.

Postby edbarx » 2014-07-14 12:34

wizard10000 wrote:Good on you for thinking outside the box; but as reinob mentioned the boot record and partition table take up one 512-byte hardware sector. You'll waste more space than that adding just *one* file to the drive (think file size vs. block size) :)
.....
Keep thinking outisde the box; sure it'll result in a bad idea once in awhile, but who knows - one day you might come up with something really great.


Reading the tone of your post reminds me of a disgruntled politician who finds his policies crumbling in the face of better and more effective ones. The purpose of a partition table is to hold partitions: that is why it is called a partition table and since a USB stick is usually formatted as one whole continguous partition, it shouldn't make any difference.

Before I wrote my howto, I used gparted to format the USB stick. First I created an MBR partition table and then proceeded to create a partition spanning the whole unpartitioned space. The result was a loss of around 4.7% of the stick's entire capacity. 'My' workaround, resolved that with a gain of 2%.

Now regarding the illogical comparison of using swap in RAM. First and foremost, a swap file/partition, is used to supplement RAM, therefore it is illogical to use a swap that resides in RAM as that goes directly counter to the scope of implementing a swap file.

There was once a great author who said: ridicule is the lowest form of wit. When you have valid arguments, I will be all ears to listen to them, but jeering an opinion or a method you deem usuitable for your likes and believes, is nothing more than empty vessles making irrational sounds.

By the way, I used the method on an ancient 1GB USB stick.
Debian == { > 30, 000 packages }; Debian != systemd
The worst infection of all, is a false sense of security!
It is hard to get away from CLI tools.
User avatar
edbarx
 
Posts: 5388
Joined: 2007-07-18 06:19
Location: 35° 50 N, 14 º 35 E

Re: How to format a USB stick efficiently.

Postby reinob » 2014-07-14 15:50

@edbarx,

Although not directly addressed, I apologize for my tone in my response above to your post.
I still think that not having a MBR is asking for trouble. But this depends on the OS(s) and software you use.

As for swap on RAM, it is by no means illogical (as long as it's compressed). It's a perfectly valid tradeoff between memory and CPU. Even devices with 128MB (the Nokia N800) benefit(ted) from dedicating as much as half of the RAM to compressed swap space.
reinob
 
Posts: 520
Joined: 2014-06-30 11:42

Re: How to format a USB stick efficiently.

Postby edbarx » 2014-07-14 15:55

reinob wrote:As for swap on RAM, it is by no means illogical (as long as it's compressed). It's a perfectly valid tradeoff between memory and CPU. Even devices with 128MB (the Nokia N800) benefit(ted) from dedicating as much as half of the RAM to compressed swap space.

That is not 'swap' but intelligently using the existing swap implementation to create compressed memory management. Reimplimentation in completely different contexts is one advantage of the many of having open sourced software.
Debian == { > 30, 000 packages }; Debian != systemd
The worst infection of all, is a false sense of security!
It is hard to get away from CLI tools.
User avatar
edbarx
 
Posts: 5388
Joined: 2007-07-18 06:19
Location: 35° 50 N, 14 º 35 E

Re: How to format a USB stick efficiently.

Postby wizard10000 » 2014-07-14 16:43

edbarx wrote:Reading the tone of your post reminds me of a disgruntled politician who finds his policies crumbling in the face of better and more effective ones....


Then you missed the point completely. Rather than ridicule, my objective was to be encouraging and somewhat supportive but I won't make that mistake in this context again.
we see things not as they are, but as we are.
-- anais nin
User avatar
wizard10000
 
Posts: 1204
Joined: 2011-05-09 20:02
Location: everywhere i go, there i am!

Re: How to format a USB stick efficiently.

Postby Head_on_a_Stick » 2014-11-25 07:47

I use this:
Code: Select all
# dd if=/dev/zero of=/dev/sdb bs=512 count=1;sync
# mkfs.fat -F32 /dev/sdb

Just to make sure the MBR is blasted off first ;)
"Only the mediocre are always at their best." — Jean Giraudoux
User avatar
Head_on_a_Stick
 
Posts: 6666
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: How to format a USB stick efficiently.

Postby chickennugget » 2017-05-25 12:26

Gotbletu made a useful set of functions for formatting usb sticks in with various file systems
https://www.youtube.com/watch?v=7txO1cdNJsQ
see the show notes link under the video for the functions themselves
chickennugget
 
Posts: 3
Joined: 2012-06-17 14:46

Re: How to format a USB stick efficiently.

Postby sunrat » 2017-05-27 01:02

I use and recommend Head_on_a_Stick's method above. Except for this minor difference which makes FAT32:
Code: Select all
mkfs.vfat -I /dev/sdx


I think most flash keys come formatted in "superfloppy" mode when you buy them.
“ computer users can be divided into 2 categories:
Those who have lost data
...and those who have not lost data YET ”
Remember to BACKUP!
User avatar
sunrat
 
Posts: 2035
Joined: 2006-08-29 09:12
Location: Melbourne, Australia

Re: How to format a USB stick efficiently.

Postby phenest » 2017-05-27 08:25

Head_on_a_Stick wrote:
Code: Select all
# mkfs.fat -F32 /dev/sdb

sunrat wrote:
Code: Select all
mkfs.vfat -I /dev/sdx

I don't think mkfs.vfat chooses fat32 necessarily. From the man page:
-F FAT-SIZE
Specifies the type of file allocation tables used (12, 16 or 32 bit). If nothing is
specified, mkfs.fat will automatically select between 12, 16 and 32 bit, whatever fits
better for the filesystem size.

Given a small enough USB drive size, mkfs.vfat may not use FAT32.
NEC Spirit 550 P4 3.8GHz HT - 2GB RAM - nVidia 7600GT - Pioneer BDR-209DBK
ASUS Sabertooth P67 i7 3770K - 32GB RAM - 2x nVidia 660GTX SLI'd
User avatar
phenest
 
Posts: 1571
Joined: 2010-03-09 09:38
Location: The Matrix

Re: How to format a USB stick efficiently.

Postby sunrat » 2017-05-27 10:00

You're right, phenest. I just looked and mkfs.vfat is a symlink to mkfs.fat. :roll: :D

So my curiosity wanted to know what makes mkfs.fat determine FAT size if not specified. I found a MS spec paper - https://staff.washington.edu/dittrich/m ... gen103.pdf

If(CountofClusters < 4085) {
/* Volume is FAT12 */
} else if(CountofClusters < 65525) {
/* Volume is FAT16 */
} else {
/* Volume is FAT32 */
}

FAT16 was limited by the
maximum size of the FAT and the maximum valid cluster size to no more than a 2 GB volume if the
disk had 512-byte sectors. FAT32 addressed this limitation on the amount of disk space that one FAT
volume could occupy so that disks larger than 2 GB only had to have one partition defined.

So I take from that if a filesystem is over 2GB, it must be FAT32.
“ computer users can be divided into 2 categories:
Those who have lost data
...and those who have not lost data YET ”
Remember to BACKUP!
User avatar
sunrat
 
Posts: 2035
Joined: 2006-08-29 09:12
Location: Melbourne, Australia

Re: How to format a USB stick efficiently.

Postby pcalvert » 2017-05-27 12:46

edbarx wrote:Since a USB stick usually contains just one partition it is normally pointless to create a partition table like an MBR and GPT

Yes, it is a good idea, and some USB sticks are formatted that way by the manufacturer. By formatting the entire device instead of creating one or more partitions, it eliminates the need to make sure the partitions are properly aligned to ensure good read/write performance.

Phil
pcalvert
 
Posts: 1734
Joined: 2006-04-21 11:19
Location: Sol Sector

Re: How to format a USB stick efficiently.

Postby Head_on_a_Stick » 2017-05-28 19:49

Head_on_a_Stick wrote:I use this:
Code: Select all
# dd if=/dev/zero of=/dev/sdb bs=512 count=1;sync
# mkfs.fat -F32 /dev/sdb

Just to make sure the MBR is blasted off first ;)

This fool forgot that a USB stick pre-formatted with a GUID partition table will retain the backup table after this procedure and that may make the stick appear corrupted.

To solve this problem, use this command to eradicate all traces of any partition tables before running `mkfs`:
Code: Select all
# sgdisk --zap-all /dev/sdX

https://packages.debian.org/search?keyw ... ection=all
"Only the mediocre are always at their best." — Jean Giraudoux
User avatar
Head_on_a_Stick
 
Posts: 6666
Joined: 2014-06-01 17:46
Location: /dev/chair

Next

Return to Docs, Howtos, Tips & Tricks

Who is online

Users browsing this forum: No registered users and 1 guest

fashionable