Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

How to format a USB stick efficiently.

Share your HowTo, Documentation, Tips and Tricks. Not for support questions!.
Post Reply
Message
Author
User avatar
edbarx
Posts: 5401
Joined: 2007-07-18 06:19
Location: 35° 50 N, 14 º 35 E
Been thanked: 2 times

How to format a USB stick efficiently.

#1 Post by edbarx »

If you have a small USB stick and you want to economize on space, you may save some space by avoiding to create a partition table. On the other hand, if your USB stick is huge, consider whether it is more suitable for you to have multiple partitions in which case you will need to create a partitition table first.

I would like to thank the two posters following the first post for drawing my attention that 'my' method may not appeal to those brandishing very large USB sticks. Excuse my reaction, but ridicule is not the best vehicle for correction. :)

The Method:
Using gparted to format a USB stick has its downside of creating a partition table and thus wasting several megabytes of space. Since a USB stick usually contains just one partition it is normally pointless to create a partition table like an MBR and GPT

mkdosfs can be instructed to create a partition without requiring a partition table as follows:

Code: Select all

# mkdosfs -I -F 32 /dev/sdx
Where:
  1. -I
    USB sticks usually don't hold more than one partition. This
    switch tells mkdosfs to create a file system without checking
    for the presence of a partition table, thereby utilizing all the
    available space without constraints.
    This switch will force mkdosfs to work properly.
  2. -F FAT-size
    Specifies the type of file allocation tables used (12, 16 or 32
    bit). If nothing is specified, mkdosfs will automatically
    select between 12, 16 and 32 bit, whatever fits better for the
    file system size.
  3. /dev/sdx
    Is the device node representing the USB stick.
Reference: man mkdosfs
Last edited by edbarx on 2014-07-14 13:12, edited 5 times in total.
Debian == { > 30, 000 packages }; Debian != systemd
The worst infection of all, is a false sense of security!
It is hard to get away from CLI tools.

reinob
Posts: 1189
Joined: 2014-06-30 11:42
Has thanked: 97 times
Been thanked: 47 times

Re: How to format a USB stick efficiently.

#2 Post by reinob »

But you're surely aware that a DOS (MBR) partition table requires all of, wait for it.. 512 BYTES. Are you willing to turn a huge USB disk into a mere floppy disk just for half-a-kilobyte?

Think of all the possibilites a modern (64, 128GB) USB disk has.

reinob
Posts: 1189
Joined: 2014-06-30 11:42
Has thanked: 97 times
Been thanked: 47 times

Re: How to format a USB stick efficiently.

#3 Post by reinob »

wizard10000 wrote: But - discoveries are made by not following instructions. My own discovery was some years ago when I put a Windows swapfile on a ramdrive :)

My logic was unassailable - I was going to speed up pagefile access by putting the pagefile in RAM; sadly, that was wrong on a couple of levels but I thought it was a great idea. Fortunately the people who knew more than me kinda rolled their eyes, patted me on the head and told me to keep learning.
You were just missing one ingredient: compression! :)
Having compressed-swap on RAM is actually a very useful thing in devices with little RAM but good CPU (think Nokia N900).

It's called ZRAM, and it's in the Linux kernel :)

User avatar
edbarx
Posts: 5401
Joined: 2007-07-18 06:19
Location: 35° 50 N, 14 º 35 E
Been thanked: 2 times

Re: How to format a USB stick efficiently.

#4 Post by edbarx »

wizard10000 wrote:Good on you for thinking outside the box; but as reinob mentioned the boot record and partition table take up one 512-byte hardware sector. You'll waste more space than that adding just *one* file to the drive (think file size vs. block size) :)
.....
Keep thinking outisde the box; sure it'll result in a bad idea once in awhile, but who knows - one day you might come up with something really great.
Reading the tone of your post reminds me of a disgruntled politician who finds his policies crumbling in the face of better and more effective ones. The purpose of a partition table is to hold partitions: that is why it is called a partition table and since a USB stick is usually formatted as one whole continguous partition, it shouldn't make any difference.

Before I wrote my howto, I used gparted to format the USB stick. First I created an MBR partition table and then proceeded to create a partition spanning the whole unpartitioned space. The result was a loss of around 4.7% of the stick's entire capacity. 'My' workaround, resolved that with a gain of 2%.

Now regarding the illogical comparison of using swap in RAM. First and foremost, a swap file/partition, is used to supplement RAM, therefore it is illogical to use a swap that resides in RAM as that goes directly counter to the scope of implementing a swap file.

There was once a great author who said: ridicule is the lowest form of wit. When you have valid arguments, I will be all ears to listen to them, but jeering an opinion or a method you deem usuitable for your likes and believes, is nothing more than empty vessles making irrational sounds.

By the way, I used the method on an ancient 1GB USB stick.
Debian == { > 30, 000 packages }; Debian != systemd
The worst infection of all, is a false sense of security!
It is hard to get away from CLI tools.

reinob
Posts: 1189
Joined: 2014-06-30 11:42
Has thanked: 97 times
Been thanked: 47 times

Re: How to format a USB stick efficiently.

#5 Post by reinob »

@edbarx,

Although not directly addressed, I apologize for my tone in my response above to your post.
I still think that not having a MBR is asking for trouble. But this depends on the OS(s) and software you use.

As for swap on RAM, it is by no means illogical (as long as it's compressed). It's a perfectly valid tradeoff between memory and CPU. Even devices with 128MB (the Nokia N800) benefit(ted) from dedicating as much as half of the RAM to compressed swap space.

User avatar
edbarx
Posts: 5401
Joined: 2007-07-18 06:19
Location: 35° 50 N, 14 º 35 E
Been thanked: 2 times

Re: How to format a USB stick efficiently.

#6 Post by edbarx »

reinob wrote:As for swap on RAM, it is by no means illogical (as long as it's compressed). It's a perfectly valid tradeoff between memory and CPU. Even devices with 128MB (the Nokia N800) benefit(ted) from dedicating as much as half of the RAM to compressed swap space.
That is not 'swap' but intelligently using the existing swap implementation to create compressed memory management. Reimplimentation in completely different contexts is one advantage of the many of having open sourced software.
Debian == { > 30, 000 packages }; Debian != systemd
The worst infection of all, is a false sense of security!
It is hard to get away from CLI tools.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: How to format a USB stick efficiently.

#7 Post by Head_on_a_Stick »

I use this:

Code: Select all

# dd if=/dev/zero of=/dev/sdb bs=512 count=1;sync
# mkfs.fat -F32 /dev/sdb
Just to make sure the MBR is blasted off first ;)
deadbang

chickennugget
Posts: 3
Joined: 2012-06-17 14:46

Re: How to format a USB stick efficiently.

#8 Post by chickennugget »

Gotbletu made a useful set of functions for formatting usb sticks in with various file systems
https://www.youtube.com/watch?v=7txO1cdNJsQ
see the show notes link under the video for the functions themselves

User avatar
sunrat
Administrator
Administrator
Posts: 6412
Joined: 2006-08-29 09:12
Location: Melbourne, Australia
Has thanked: 116 times
Been thanked: 461 times

Re: How to format a USB stick efficiently.

#9 Post by sunrat »

I use and recommend Head_on_a_Stick's method above. Except for this minor difference which makes FAT32:

Code: Select all

mkfs.vfat -I /dev/sdx
I think most flash keys come formatted in "superfloppy" mode when you buy them.
“ computer users can be divided into 2 categories:
Those who have lost data
...and those who have not lost data YET ”
Remember to BACKUP!

User avatar
phenest
Posts: 1702
Joined: 2010-03-09 09:38
Location: The Matrix

Re: How to format a USB stick efficiently.

#10 Post by phenest »

Head_on_a_Stick wrote:

Code: Select all

# mkfs.fat -F32 /dev/sdb
sunrat wrote:

Code: Select all

mkfs.vfat -I /dev/sdx
I don't think mkfs.vfat chooses fat32 necessarily. From the man page:
-F FAT-SIZE
Specifies the type of file allocation tables used (12, 16 or 32 bit). If nothing is
specified, mkfs.fat will automatically select between 12, 16 and 32 bit, whatever fits
better for the filesystem size.
Given a small enough USB drive size, mkfs.vfat may not use FAT32.
ASRock H77 Pro4-M i7 3770K - 32GB RAM - Pioneer BDR-209D

User avatar
sunrat
Administrator
Administrator
Posts: 6412
Joined: 2006-08-29 09:12
Location: Melbourne, Australia
Has thanked: 116 times
Been thanked: 461 times

Re: How to format a USB stick efficiently.

#11 Post by sunrat »

You're right, phenest. I just looked and mkfs.vfat is a symlink to mkfs.fat. :roll: :D

So my curiosity wanted to know what makes mkfs.fat determine FAT size if not specified. I found a MS spec paper - https://staff.washington.edu/dittrich/m ... gen103.pdf
If(CountofClusters < 4085) {
/* Volume is FAT12 */
} else if(CountofClusters < 65525) {
/* Volume is FAT16 */
} else {
/* Volume is FAT32 */
}
FAT16 was limited by the
maximum size of the FAT and the maximum valid cluster size to no more than a 2 GB volume if the
disk had 512-byte sectors. FAT32 addressed this limitation on the amount of disk space that one FAT
volume could occupy so that disks larger than 2 GB only had to have one partition defined.
So I take from that if a filesystem is over 2GB, it must be FAT32.
“ computer users can be divided into 2 categories:
Those who have lost data
...and those who have not lost data YET ”
Remember to BACKUP!

pcalvert
Posts: 1939
Joined: 2006-04-21 11:19
Location: Sol Sector
Has thanked: 1 time
Been thanked: 2 times

Re: How to format a USB stick efficiently.

#12 Post by pcalvert »

edbarx wrote:Since a USB stick usually contains just one partition it is normally pointless to create a partition table like an MBR and GPT
Yes, it is a good idea, and some USB sticks are formatted that way by the manufacturer. By formatting the entire device instead of creating one or more partitions, it eliminates the need to make sure the partitions are properly aligned to ensure good read/write performance.

Phil
Freespoke is a new search engine that respects user privacy and does not engage in censorship.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: How to format a USB stick efficiently.

#13 Post by Head_on_a_Stick »

Head_on_a_Stick wrote:I use this:

Code: Select all

# dd if=/dev/zero of=/dev/sdb bs=512 count=1;sync
# mkfs.fat -F32 /dev/sdb
Just to make sure the MBR is blasted off first ;)
This fool forgot that a USB stick pre-formatted with a GUID partition table will retain the backup table after this procedure and that may make the stick appear corrupted.

To solve this problem, use this command to eradicate all traces of any partition tables before running `mkfs`:

Code: Select all

# sgdisk --zap-all /dev/sdX
https://packages.debian.org/search?keyw ... ection=all
deadbang

User avatar
sunrat
Administrator
Administrator
Posts: 6412
Joined: 2006-08-29 09:12
Location: Melbourne, Australia
Has thanked: 116 times
Been thanked: 461 times

Re: How to format a USB stick efficiently.

#14 Post by sunrat »

The same question came up at MX forum yesterday and revealed issues.
https://forum.mxlinux.org/viewtopic.php?f=97&t=42311
To format the drive as "superfloppy" mode, the "-I" option needs to be specified. Otherwise I guess a partition would need to be present so the command may need to be (not tested by me) -

Code: Select all

mkfs.fat -F32 /dev/sdb1
I checked a couple of flash drives I have which have not been altered since purchase and both have a partition so my previous belief that they come in superfloppy mode may be incorrect.
“ computer users can be divided into 2 categories:
Those who have lost data
...and those who have not lost data YET ”
Remember to BACKUP!

Post Reply