Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

[SOLVED] Password hashes under Debian?

New to Debian (Or Linux in general)? Ask your questions here!
Message
Author
Ahtiga Saraz
Posts: 1014
Joined: 2009-06-15 01:19

[SOLVED] Password hashes under Debian?

#1 Post by Ahtiga Saraz »

By default, as far as I know, Debian hashes passwords using a single application of md5 and stores them in /etc/shadow.

[EDIT: apparently true for at least some Debian 5.0.x Lenny installations. By default, Debian 6.0.x Squeeze apparently uses SHA-512 with a salt which is a publically known 8 char string. MD-5 is generally agreed to be much too weak to use for cryptographic purposes; SHA-512 is thought to be much stronger. Even better would be to iterate the SHA-512 "hashing" many times. See the last page for how to do this, and how to check that it is working.]

But the md5 digest/hash is said to be insecure; several years ago a double application of sha256 digest/hash was said to be much better. Is there a way to implement that in Debian Squeeze?
Last edited by Ahtiga Saraz on 2011-03-28 01:09, edited 2 times in total.
Ahtiga Saraz

Le peuple debout contre les tyrans! De l'audace, encore de l'audace, toujours l'audace!

User avatar
llivv
Posts: 5340
Joined: 2007-02-14 18:10
Location: cold storage

Re: Password hashes under Debian?

#2 Post by llivv »

Ahtiga Saraz wrote:Is there a way to implement that in Debian Squeeze?
if there is a way why wouldn't you choose sha512 or sha1024 instead?
You want to secure the box, no?
In memory of Ian Ashley Murdock (1973 - 2015) founder of the Debian project.

Ahtiga Saraz
Posts: 1014
Joined: 2009-06-15 01:19

Re: Password hashes under Debian?

#3 Post by Ahtiga Saraz »

llivv wrote:
Ahtiga Saraz wrote:Is there a way to implement that in Debian Squeeze?
if there is a way why wouldn't you choose sha512 or sha1024 instead?
You want to secure the box, no?
Obviously. There is a tradeoff between latest hash and best-tested hash. I try to follow the most recent "best practice" advice from experts I trust which I can find on-line.

In this case, advice several years old says to double hash passwords with sha256.

Does anyone know how to implement the double hashing? Or why Debian doesn't do that by default?
Ahtiga Saraz

Le peuple debout contre les tyrans! De l'audace, encore de l'audace, toujours l'audace!

User avatar
bugsbunny
Posts: 5354
Joined: 2008-07-06 17:04
Been thanked: 1 time

Re: Password hashes under Debian?

#4 Post by bugsbunny »

What are the contents of /etc/pam.d/common-password? Specifically the line that starts with "password". I'm not sure what the default is, but mine uses sha512, with a salt and 1000 iterations

Code: Select all

password	[success=1 default=ignore]	pam_unix.so obscure sha512 rounds=1000
Comments state that the default is the Unix crypt command rather than md5

See man pam_unix for more

Ahtiga Saraz
Posts: 1014
Joined: 2009-06-15 01:19

Re: Password hashes under Debian?

#5 Post by Ahtiga Saraz »

rounds=1000

Bugs, you read my mind!

I don't know right now what my Debian oldstable /et/pam.d/common-password says, but another linux says

Code: Select all

password   required   pam_unix.so nullok obscure md5
which is terrible. I believe that "crypt" is even more outdated than md5.

Can I ask some stupid questions without breaking any rules here?
  • do you know and if so can you explain what the bit in brackets does? And where in the line you quoted do you specify a salt? And how is the salt constructed?
  • if I install new Squeeze system from CD#1 on another machine, and have already created root and ordinary user by the time I can alter the file in question (if neccessary), obviously the new procedure will apply to a second ordinary user I then create, but will it be made to apply retroactively to root and first ordinary user?
  • I believe we can think of a digest algorithm as a pseudorandom many-to-one function, and we are iterating it one thousand times; does it really look like a new random function...but with different range? Does this improve or worsen cryptographic properties?
Ahtiga Saraz

Le peuple debout contre les tyrans! De l'audace, encore de l'audace, toujours l'audace!

User avatar
bugsbunny
Posts: 5354
Joined: 2008-07-06 17:04
Been thanked: 1 time

Re: Password hashes under Debian?

#6 Post by bugsbunny »

These are the comments at the top of the file:
# Explanation of pam_unix options:
#
# The "sha512" option enables salted SHA512 passwords. Without this option,
# the default is Unix crypt. Prior releases used the option "md5".
#
# The "obscure" option replaces the old `OBSCURE_CHECKS_ENAB' option in
# login.defs.
#
# See the pam_unix manpage for other options.
For the bit in brackets see

Code: Select all

man pam.d
Changes will not apply retroactively. They'll be applied at next password change.

Not sure on the last one. One thing iterations do is slow down an attack. They also (along with salts) make rainbow tables practically impossible.

You should also look at

Code: Select all

man crypt
, especially the Notes section

Ahtiga Saraz
Posts: 1014
Joined: 2009-06-15 01:19

Re: Password hashes under Debian?

#7 Post by Ahtiga Saraz »

Bugs, thanks so much, this is very helpful!
  • You're right, my question about value1=action1 in answered right there in the pam man page (blush),
  • So to make the changes take effect, I should edit the file as we discussed and then change root and ordinary-user passwords in the usual way, e.g. kuser,
  • On my Debian oldstable, the file is just the bad default I quoted above from looking a live CD system, so I definitely should have changed this long ago,
  • Yes, I take the point about why interating digest a thousand times prevents crackers from using rainbow tables
  • I guess that for now we will have to leave the question about the probability theory of sha512 iterated one thousand times as "we haven't a clue".
  • On my Debian oldstable, the crypt man page previously described outdated "crypt" but when I recently installed mcrypt it now describes mcrypt, which is much stronger.
Ahtiga Saraz

Le peuple debout contre les tyrans! De l'audace, encore de l'audace, toujours l'audace!

Ahtiga Saraz
Posts: 1014
Joined: 2009-06-15 01:19

Re: Password hashes under Debian?

#8 Post by Ahtiga Saraz »

Still trying to figure this out. I have some elementary followup questions.
  • In the /etc/pam.d/common-password file (default commented out and suggested replacement uncommented)

    Code: Select all

    password  [success=1 default=ignore] pam_unix.so obscure sha512 rounds=1000
    # password   required   pam_unix.so nullok obscure md5
    
    do I correctly understand the syntax?
    • type: password
    • control: required or more specifically [success=1 default=ignore]
    • module-path: pam_unix.so
    • module-arguments: obscure md5 or obscure sha512 rounds=1000
  • In the age of Wissenschaft <- Wikischaft, I may have been misled by the Wikipedia article on rainbow tables, which seems to imply (without giving details or citations?) that the default in all modern linux distributions (for logins? for any PAM-aware password authentication?) is salted and iterated md5 hash. If that is true, I have been unable to find any mention in the documentation I have been struggling to understand (mostly man pages, comments in the /etc/pam.d files). Does anyone know? Because of my limited background in this area, I doubt that looking at source code will help me very much, so I am dependent upon documentation plus this forum.
  • I tried to test the suggested line by making the change, creating a test user, and (as root user) looking at /etc/shadow. Unfortunately, that hash looks to be 128 bits long, not 512 bits, so it seems that I defaulted back to md5sum. The man page suggests as debug option, but I haven't figured out how to use it. How can I test that I really am using a large salt, that I really am using sha512, that I really am getting 1000 rounds?
Last edited by Ahtiga Saraz on 2011-03-04 16:42, edited 3 times in total.
Ahtiga Saraz

Le peuple debout contre les tyrans! De l'audace, encore de l'audace, toujours l'audace!

User avatar
bugsbunny
Posts: 5354
Joined: 2008-07-06 17:04
Been thanked: 1 time

Re: Password hashes under Debian?

#9 Post by bugsbunny »

Looks correct, although you missed nullok in your default module arguments.

That default line must be ancient, it's certainly not the default on current installs (although it's possibly/probably kept on upgrades from older systems)

Ahtiga Saraz
Posts: 1014
Joined: 2009-06-15 01:19

Re: Password hashes under Debian?

#10 Post by Ahtiga Saraz »

I was wondering about nullok; from the man pages it seemed that it might be a good idea, but I am confused about whether it increases or decreases security. So I guess the edited line should read

Code: Select all

password  [success=1 default=ignore] pam_unix.so nullok obscure sha512 rounds=1000
The ancient default line: this is what I found when I looked at my Debian oldstable (Lenny) system, which I am currently preparing to replace. That system is about 1.5 years old.

I installed mcrypt about one month ago. That seems to have replaced the man page for "crypt" with the man page for "mcrypt", so I can't read the notes mentioned above.

I have been wondering whether by installing some extra crypto stuff (from Lenny repos only) over past year, such as mcrypt, I have somehow caused my system to become confused about what crypto it knows, thus causing default to md5, or even worse, to unsalted uniterated md5, or even worse, to the oldest default, unix crypt. And I can't figure out how to check.

Another point which worries me: the number 1000 seems to have been pulled out of a hat, and I don't understand why it has been chosen. It has small prime factors, which just might be a problem. Crudely, I understand a one-way hash function to be a mapping from keyspace to a small subset of keyspace, which should behave like a "generic" such function, so that it is hard for attackers to systematically reverse engineer hash values. I want results describing a "typical" function from a large keyspace to a subspace, possibly smaller.

For comparison: A reference book quotes some results for a randomly chosen function from a message space to itself, with no restriction on the size of the output (hashes):
  • We can picture it as a directed graph consisting of cycles with trees attached to various points on each cycle.
  • For a message space of size 2^n, there should be about n/2 log 2 cycles,
  • About sqrt(pi) 2^(n/2) output words should belong to some cycle,
  • A fraction of about 1/e output words should not be the "hash" of any input word; iterating our "hash function" should reduce the variety of the words we obtain by a fraction something like this:
    • 1 round: 0.63212
    • 10 rounds: 0.15024
    • 100 rounds: 0.01935
    • 1000 rounds: 0.00199
(Oddly enough, another Wikipedia article offers proofs of these facts, but I have lost the link.)

If we pretend that our unhashed passwords are always 128 bits long, and that our hash function outputs 128 bit hashes, but takes some inputs to the same output, then as I understand it, the idea is that a hash function should offer no particular "special structure" an attacker can exploit, so it should behave like a "randomly chosen" function, whose "expected" behavior should be roughly as just described.

One of my concerns: if our hash function behaves like a "generic function", it seems possible that if we iterate 1000 times, we might often wind up in a small cycle, which would imply that further iteration is not having the expected effect, and if the length of the cycle is also divisible by 2,5, that might further assist an attacker.

Another thought: as I understand it, the general idea of a hash algorithm like md5 is that we want to verify that a long file has not been corrupted. So the hash value should have fixed length, typically shorter than the input file; we assume that because only a tiny fraction of the huge space from which actual files are "randomly selected" by complex human/cpu processes will occur in practice, "collisions" are sufficiently rare that we can ignore them. But it seems that disguising user input passwords for storage in /etc/shadow is a different problem, which might be better solved by using the user input password as a "seed" (perhaps with some publically known "salt") for some algorithm blending features of a random sequence generator with a hash function, which outputs a longer word than the input word, and which can be iterated.
Last edited by Ahtiga Saraz on 2011-03-04 17:36, edited 2 times in total.
Ahtiga Saraz

Le peuple debout contre les tyrans! De l'audace, encore de l'audace, toujours l'audace!

Ahtiga Saraz
Posts: 1014
Joined: 2009-06-15 01:19

What is a n00bie to do?

#11 Post by Ahtiga Saraz »

From another thread:
Telemachus wrote: By the way, salting and iterating md5s is utterly worthless. See these two articles for why:
I tried to explain why I think md5 and other hash functions are solving a quite different problem from the password storage problem. These two articles explain that point MUCH more clearly than I did.

Now I have some more pointed followup questions:
  • the first article suggests that all post 1970s password systems defeat rainbow tables, which appears to contradict information I have seen elsewhere
  • the second article (updated as recently as a week ago) suggests that bcrypt is the way to store passwords. But bcrypt is NOT a Debian default. Apparently it is however available as debs called "bcrypt" and "libcrypt-eksblowfish-perl". The second apparently implements the "slowpoke blowfish" needed to solve the password storage problem with bcrypt.
  • I installed them and read the manual pages. The man page for pam.d and the comments for /etc/pam.d/password-common do not mention how to make Debian use bcrypt to store passwords. The man page for bcrypt does not mention password storage at all.
See why I am confused?
Ahtiga Saraz

Le peuple debout contre les tyrans! De l'audace, encore de l'audace, toujours l'audace!

User avatar
bugsbunny
Posts: 5354
Joined: 2008-07-06 17:04
Been thanked: 1 time

Re: Password hashes under Debian?

#12 Post by bugsbunny »

I installed mcrypt about one month ago. That seems to have replaced the man page for "crypt" with the man page for "mcrypt", so I can't read the notes mentioned above.
Try

Code: Select all

man 3 crypt

Ahtiga Saraz
Posts: 1014
Joined: 2009-06-15 01:19

Re: Password hashes under Debian?

#13 Post by Ahtiga Saraz »

On my system, man crypt gives me the man page for mcrypt, a deb which I installed a few weeks ago. I presume the developers of mcrypt intended it that way. Anyway, it prevents me from seeing whatever you think I should read.

Let me try again: the default line in /etc/pam.d/common-password is

Code: Select all

password   required   pam_unix.so nullok obscure md5
We have now clearly established that md5 (or sha256, or sha512) are just what we do not want (iteration and salts make no difference to this point), so that default line (which is surely more recent than 1970s?) is just what Debian users very certainly do not want to have in their /etc/pam.d/common-password! So what line to I need to replace this line with?
Last edited by Ahtiga Saraz on 2011-03-06 22:21, edited 2 times in total.
Ahtiga Saraz

Le peuple debout contre les tyrans! De l'audace, encore de l'audace, toujours l'audace!

User avatar
bugsbunny
Posts: 5354
Joined: 2008-07-06 17:04
Been thanked: 1 time

Re: Password hashes under Debian?

#14 Post by bugsbunny »

If you want to use blowfish encryption see Debian -- Package Search Results -- libpam-unix2
Note that using this eliminates all the sha functions and just gives you blowfish, des, and md5 as options. Make sure you read the man pages.

Ahtiga Saraz
Posts: 1014
Joined: 2009-06-15 01:19

The needed debs, at last!

#15 Post by Ahtiga Saraz »

Bugs, I think you meant to say that I need to install the debs libpam-unix2, libxcrypt1. Done.

Quick clarification: md4, md5, sha1, sha256, sha512... all solve a different problem from the one we have here. So PAM aware applications should not even offer these as options. Looks like bcrypt with slowpoke blowfish should be the default option, and further, this is so important for the linux security model that doing something else should not be easy to do by mistake/ignorance. In any case, it should not be true that installing libpam-unix2 removes sha256sum and sha512sum, and it looks like it does not. Whew!

OK, now I can't find man pages telling me what I need to do now that I have libpam-unix2. Suggestions?
Ahtiga Saraz

Le peuple debout contre les tyrans! De l'audace, encore de l'audace, toujours l'audace!

User avatar
bugsbunny
Posts: 5354
Joined: 2008-07-06 17:04
Been thanked: 1 time

Re: Password hashes under Debian?

#16 Post by bugsbunny »

Ahtiga Saraz wrote:On my system, man crypt gives me the man page for mcrypt, a deb which I installed a few weeks ago. I presume the developers of mcrypt intended it that way. Anyway, it prevents me from seeing whatever you think I should read.

Let me try again: the default line in /etc/pam.d/common-password is

Code: Select all

password   required   pam_unix.so nullok obscure md5
We have now clearly established that md5 (or sha256, or sha512) are just what we do not want (iteration and salts make no difference to this point), so that default line (which is surely more recent than 1970s?) is just what Debian users very certainly do not want to have in their /etc/pam.d/common-password! So what line to I need to replace this line with?
Yes "man crypt" gives you the mcrypt page, in section 1 of the man pages. Note the 3 in what I gave you above. crypt is still found in section 3 of the man pages - it's a programming interface, not a cli program.

iteration does make a difference, see article 1. iteration slows down any potential attack, especially if the attacker doesn't know2 how many iterations are needed (although it slows them down even if they do)

hash(password+salt)=hash1
hash(hash1)=hash2
hash(hash2)=hash3

That will do to illustrate. That's 3 iterations. Rainbow tables are obviously useless here (salt was enough to take care of that), so all that's left is brute force. To brute force this you would have to take each possible combination, run it through the hash, try the result, if it doesn't work run it through the hash again, try again, run it through the hash again, try again. Then move on to the next combination. That assumes that you know the max hash attempts is 3.

If you don't know how many iterations it complicates things immensely. Using a slower function makes things ven worse fromk an attackers standpoint. But the bottom line is that iterations slow things down. blowfish is slower than any of the hash functions. Blowfish + iterations will defintely put a crimp in any possible attack.

That said sha512 with lots of iterations should be more than secure (I don't use 1000 iterations, I use significantly more). Even md5 with lots of iterations would probably be safe enough (I would not recommend that though, I'm just using it to demonstrate a point). I could (and have in the past) gone with Debian's unix2 (blowfish) implementation, but I don't like the fallback options (being only md5 and des), so I'm sticking (for the moment) with sha512 and lots of iterations.

Ahtiga Saraz
Posts: 1014
Joined: 2009-06-15 01:19

Re: Password hashes under Debian?

#17 Post by Ahtiga Saraz »

Do you agree that the author of the second article cited by Telemachus seems to say very clearly that iterating sha256 thousands of times,, even with a salt, is useless? So you seem to be directly contradicting that view.

[EDIT: my mistake. It was Thomas Ptacek, not Coda Hale, who mentioned "stretching", aka iterating the digest algorithm. And he seems to agree that when done properly, even using a conventional digest algorithm such as sha-256, this is adequately secure.]

Telemachus?

And I STILL don't know what I need to put in my /etc/pam.d/common-password to do either of the following
  • sha-256 or sha-512 hash iterated thousands of times, with an appropriate salt
  • use bcrypt with slowpoke blowfish to store passwords
I know that I am very confused, but I also know that this is durn well not my fault.
Last edited by Ahtiga Saraz on 2011-03-07 19:49, edited 1 time in total.
Ahtiga Saraz

Le peuple debout contre les tyrans! De l'audace, encore de l'audace, toujours l'audace!

User avatar
bugsbunny
Posts: 5354
Joined: 2008-07-06 17:04
Been thanked: 1 time

Re: Password hashes under Debian?

#18 Post by bugsbunny »

See also Unix crypt with SHA-256/512 as to why blowfish isn't there (in the default implementation) and the sha functions are

Ahtiga Saraz
Posts: 1014
Joined: 2009-06-15 01:19

Re: Password hashes under Debian?

#19 Post by Ahtiga Saraz »

I see that the author of that page claims to address the points raised in the second paper cited by Telemachus, but I don't think he has in fact attempted to address them. It seems to me that the point stands: md5, sha256, sha512 all solve an important but completely different problem from the one we have here.
Ahtiga Saraz

Le peuple debout contre les tyrans! De l'audace, encore de l'audace, toujours l'audace!

User avatar
bugsbunny
Posts: 5354
Joined: 2008-07-06 17:04
Been thanked: 1 time

Re: Password hashes under Debian?

#20 Post by bugsbunny »

Ahtiga Saraz wrote:Do you agree that the author of the second article cited by Telemachus seems to say very clearly that iterating sha256 thousands of times,, even with a salt, is useless? So you seem to be directly contradicting that view.

Telemachus?

And I STILL don't know what I need to put in my /etc/pam.d/common-password to do either of the following
  • sha-256 or sha-512 hash iterated thousands of times, with an appropriate salt
  • use bcrypt with slowpoke blowfish to store passwords
I know that I am very confused, but I also know that this is durn well not my fault.
No, I don't agree that that the author states that iterating is uselss. He never mention siterations at all. He states that a salt with a very fast hash function is basically useless (although I do disagree somewhat with even that conclusion, it depends on the size of the salt. Take a random salt that's a random size between 10 and 20 characters, meaning the string is between 16 and 20+passphrase length long (assuming a minimum password length ogf 6) and even fast systems would have some problems. Again, a long enough passphrase slows down an attacker since it radically increases the number of combinations that need to be checked. And making the salt a variable length adds to the complexity.

In any case the "work factor" that he mentions is what iterations do.

libpam_unix2 does remove the sha functions when it comes to storing the password, not for general purpose use.
see

Code: Select all

man pam_unix2

Post Reply