Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

USB Flash OS drive longevity.

New to Debian (Or Linux in general)? Ask your questions here!
Post Reply
Message
Author
ixaff
Posts: 4
Joined: 2018-01-25 03:21

USB Flash OS drive longevity.

#1 Post by ixaff »

I have decided to try building my own home server running off a full install Debian flash drive. However, I have read that this would cause a significant decrease in the longevity of the flash drive unless I do something to minimize writes.

Does that still apply to modern flash drives? If so, is there a step-by-step guide out there on the process needed to minimize writes to flash after Debian install. Ideally this guide should be as idiot proof as possible so a complete novice like myself can easily follow.

deborah-and-ian
Posts: 182
Joined: 2016-07-13 08:40

Re: USB Flash OS drive longevity.

#2 Post by deborah-and-ian »

SSD drives don't have this problem anymore. USB flashs drives are of much lower quality -- I wouldn't use those for a server that constantly writes and deletes stuff.
Debian GNU/Linux 9 Stretch w/Openbox

Acer Aspire E5-521G
AMD A8-6410 APU
4 GB RAM
integrated AMD Mullins
dedicated AMD Hainan Radeon R5 M240 2 GB
240 GB Toshiba Q300 SSD
Realtek RTL8111/8168/8411 ethernet
Qualcomm Atheros QCA9565 wireless

ixaff
Posts: 4
Joined: 2018-01-25 03:21

Re: USB Flash OS drive longevity.

#3 Post by ixaff »

deborah-and-ian wrote:SSD drives don't have this problem anymore. USB flashs drives are of much lower quality -- I wouldn't use those for a server that constantly writes and deletes stuff.
That's consistent with what I have read. That said, I did come across a discussion suggesting that writes to the USB boot drive can be minimized with some tweaks. (http://forums.debian.net/viewtopic.php? ... es#p540332)

However, my limited understanding as a novice allows me only to understand the gist that it CAN be done, and not comprehend enough of what is being discussed to know HOW I can achieve that outcome.

I've done lots of Googling since and cannot seem to find any step-by-step guides on the process for Debian. Hopefully someone can link a guide to me should it exist. Otherwise IF it's not too much trouble, perhaps help guide me through the steps directly.

dcihon
Posts: 279
Joined: 2012-09-23 16:18
Been thanked: 3 times

Re: USB Flash OS drive longevity.

#4 Post by dcihon »

This is pretty close to step by step as I have found. I have not went through this process so I cannot say whether any of this works.
Also there are many articles out there on this subject so if I was you I would read as much as I can.
https://askubuntu.com/questions/97153/i ... ermanently

User avatar
GarryRicketson
Posts: 5644
Joined: 2015-01-20 22:16
Location: Durango, Mexico

Re: USB Flash OS drive longevity.

#5 Post by GarryRicketson »

The various questions are rather broad, and not really related to Debian at all.
Does that still apply to modern flash drives?
Not all flash drives are created equal, modern or older,...some brands are more reliable then others, it is beyond the scope of this forum to go into which brands are better, but some research should help you decide.
I have decided to try building my own home server running off a full install Debian flash drive.
That is nice, but a couple of things here,.."home server" ?? there are many types of servers that can be used at home or at work,
What kinds of servers are there
Take your pick of the results, maybe this one:
https://www.webopedia.com/quick_ref/servers.asp
Any of these could be a "home server", or used at work as well,... With out knowing if this "home server" is going to be a WebServer, Proxy Server, or maybe a Print server, etc.
In any event, using a USB flash drive, to explore, experiment and learn more about setting up various types of servers would be ok, but for anything long term, or permanent, I wouldn't count on it.
Another thing :
running off a full install Debian flash drive.
For most servers, regardless of the type, including a WebServer, there is no reason or need for a full install of Debian, a full Debian install includes DE's or WM's, Xorg, and all sorts of things that are not needed or even used on a server,...even WebServers do not use most of the packages that come with a
full Debian install.
Again, I do think using a USB stick to experiment with is a reasonable option for learning more, as for a simple "step by step" guide,... first you need to be able to specify, exactly what kind of "home server", and then using the specifics , you will get better results using a search engine, or even if you don't like to use a search engine, you will be able to ask, and someone else might know of a good step by step instruction, or be able to find one for you.
Keep in mind "google" is not the only search engine, and far from the best, unless you enjoy spam and advertisements not related to the information you are looking for, but that is a whole other topic.
Asking multiple questions, in 1 post, makes it almost impossible to give any specific answer, and leads to more of a multi topic, off topic type of discussion.
Trying to get back to the topic/questions,...
If this so called "home server" is going to be WebServer, and you put it online, ( I don't recommend that at all), there is a whole lot of things more important then how long it will keep working, security issues .
When a server, any kind is put online it becomes a target for 1000's of other machines, bots, especially webservers. The automated bots are constantly trying to access the server, all though with proper fire walls, you can prevent them form actually accessing, you can not stop them from trying, and it is a 24 hour thing, 1000s of bots constantly trying to connect, even though they don't access, it still cause a continuous stream of activity on the server , logs get written, new blocks get created, etc,.... USB sticks are simply not intended for this ,... any way this is to long and I have other things to do.

User avatar
stevepusser
Posts: 12930
Joined: 2009-10-06 05:53
Has thanked: 41 times
Been thanked: 71 times

Re: USB Flash OS drive longevity.

#6 Post by stevepusser »

Is this going to be a full install onto the USB stick, or a Live session with persistence?

Maybe you can copy the OS to RAM and run from that in order to reduce wear.
MX Linux packager and developer

ixaff
Posts: 4
Joined: 2018-01-25 03:21

Re: USB Flash OS drive longevity.

#7 Post by ixaff »

@stevepusser
Full install on flash. Not live.

If that is the best solution I would try it, if I had something to follow.

@GarryRicketson
I gather from your signature that you have a thing about questions being asked in "the smart way". I apologize for not having done so in your opinion and will take note of the points you have raised.

@dcihon
I went through the link you provided. The process described seemed to center largely on shrinking and migrating root fs and the rest of it I just plain did not understand at all. It did still turn out very helpful because it gave me the right terms to search for. After having done so I have ended up with a bunch of reading that would take me awhile to process.

That being said, I also stumbled across something that might be a short cut of sorts. Apparently, Openmediavault created a flashmemory plugin Debian package that manages temporary file systems across reboots, to decrease writes on permanent storage allowing for the installation of OMV on flash thumb drives and SD cards. The Github page says that this can be implemented on different distros by copying it into /sbin.

Argus
Posts: 20
Joined: 2017-12-03 01:19

Re: USB Flash OS drive longevity.

#8 Post by Argus »

In general, people are correct in saying an SSD or HDD will last longer, but there are just two important points I want to make:

1) I, and a lot of people, run Linux from Micro-SD cards on things like the Raspberry Pi, which makes for an excellent little server. SD cards are far less reliable and shorter lived than USB drives, usually, so you should have no problem running your server off of a USB stick, at least for a good while.

2) If you are running a server that you care about, you should have backups anyway. It's easy to just make a copy of a microSD or USB stick, so you can make regular backups and give it a try. If your server is just hosting a web-page or something and the data on it doesn't change all that often, who cares if your USB stick fails at some point? Just keep a backup. Most likely, you won't even need it: and in the worst case scenario, you just have to plugin the backup USB stick and buy a new one to replace the old one for < $20.

p.H
Global Moderator
Global Moderator
Posts: 3049
Joined: 2017-09-17 07:12
Has thanked: 5 times
Been thanked: 132 times

Re: USB Flash OS drive longevity.

#9 Post by p.H »

@ixaff :
Is what you call "USB flash drive" this : https://en.wikipedia.org/wiki/USB_flash_drive
or this : https://en.wikipedia.org/wiki/Solid-state_drive ?

They have not the same endurance.
Argus wrote:I, and a lot of people, run Linux from Micro-SD cards on things like the Raspberry Pi, which makes for an excellent little server. SD cards are far less reliable and shorter lived than USB drives, usually, so you should have no problem running your server off of a USB stick, at least for a good while.
I have read a number of people complaining that Raspberry Pi killed their SD cards. I suspect that the cause is improper filesystem and usage pattern, not the Raspberry Pi itself.

Also I found the following filesystems :
- NILFS2 and F2FS, designed to work on flash-based block devices ;
- JFFS2 and UBIFS, with the block2mtd emulation layer because they are primarily designed to work on raw flash chips (MTD).

Does anyone have any experience or opinion about these solutions ? Do they provide real benefit on "dumb" flash drives (not SSD) over traditional filesystems ?

User avatar
Thorny
Posts: 542
Joined: 2011-02-27 13:40

Re: USB Flash OS drive longevity.

#10 Post by Thorny »

ixaff wrote:@GarryRicketson
I gather from your signature that you have a thing about questions being asked in "the smart way". I apologize for not having done so in your opinion and will take note of the points you have raised.
Just so you understand, a lot of us here feel that way, it's just that this time Garry's sig pointed it out to you first.

It might be worthwhile for you to read this post by a forum admin.
http://forums.debian.net/viewtopic.php?f=30&t=47078

ixaff
Posts: 4
Joined: 2018-01-25 03:21

Re: USB Flash OS drive longevity.

#11 Post by ixaff »

@p.H
dcihon posted a link that specifically mentions the possibility of implementing a file system more suited for usb flash drives (not ssd). However, the final solution was something completely different and something I do not quite understand. Maybe you might find it helpful.

https://askubuntu.com/questions/97153/i ... ermanently

User avatar
pylkko
Posts: 1802
Joined: 2014-11-06 19:02

Re: USB Flash OS drive longevity.

#12 Post by pylkko »

Well, I have used JFFS2 and F2FS, but also have run servers of EXT4 on flash.

AFAICS the quality of flash media vary quite much. This does not just pertain to the electronics and their QA but also to what kind of operations the device does on it's own to protect itself. Flash media have their own microcontrollers, RAM, wear leveling algorithms and so on. Low cost articles likely don't have the high quality versions of these. For example, I have run a music server, where daily changing music content is stored on flash and several servers run 24/7 (dlna, web, NFS, and what more) for almost 5 years now. This is every day and night, with but a few days of offline time. The entire disk is EXT4, and all the logs are written to disk. No problems yet. At the same time, you can read online anecdotes of people who have killed flash media - in as short time periods as some months - simply by using high write applications on them.

I have also used OpenWRT on routers for years, and it used (or always uses, perhaps?) JFFS2. From what I have understood, JFFS was first invented in order to effect wear leveling on simple and small size NOR flash used as ROM's. Therefore, not so much used on USB sticks and SD-cards, as they now mostly have firmware level wear leveling and are large, and NAND.

F2FS is used in phones, I believe it was desgined by Samsung for phones and other mobile devices that have embedded flash memories. It was built from the start to be optimal for NAND and is claimed (by many) to be much more performant, so that for example applications can launch faster,

User avatar
dotlj
Posts: 646
Joined: 2009-12-25 17:21

Re: USB Flash OS drive longevity.

#13 Post by dotlj »

pylkko wrote:AFAICS the quality of flash media vary quite much. This does not just pertain to the electronics and their QA but also to what kind of operations the device does on it's own to protect itself. Flash media have their own microcontrollers, RAM, wear leveling algorithms and so on. Low cost articles likely don't have the high quality versions of these. For example, I have run a music server, where daily changing music content is stored on flash and several servers run 24/7 (dlna, web, NFS, and what more) for almost 5 years now. This is every day and night, with but a few days of offline time. The entire disk is EXT4, and all the logs are written to disk. No problems yet. At the same time, you can read online anecdotes of people who have killed flash media - in as short time periods as some months - simply by using high write applications on them.
Agreed. I have a Sandisk Extreme Pro flashdisk, write speeds at about 200 MB/s (faster than my HDD) over USB 3.0 and many other flashdisks that average about 4 MB/s.
My recent Samsung 960 EVO (internal) has an up to date controller, so that I haven't needed to add discard to /etc/fstab.
dstat show writes are almost non existent during no activity.
There is a lot of documentation about flash memory but much of it applies to older types and hasn't been updated.

Post Reply