[SOLVED] Password hashes under Debian?

[bugsbunny]

The author of that paper was the coder. He also mentions NIST, and peer review by a number of people. "The code has been reviewed from security teams at HP, IBM, Red Hat, and Sun." I'm pretty sure that those people know which end is up.

[Ahtiga Saraz]

Telemachus linked to two blog posts:

• Thomas Pracek (Sept 2007)
• Coda Hale (updated Feb 2011)

Both recommend adaptive hashing rather than PHK's MD-5 iterated with salt. The first adaptive hashing algorithm is bcrypt, which was proposed in a Usenix paper by Provos and Mazieres which is available on-line (and which I have just read), and which is apparently implemented in the debs mentioned above (whose documentation I can't find). Pracek also describes SRP, which was introduced in another paper.

Bugs cited a blog by Ulrich Drepper (Sept 2007), which cites some more papers. The Drepper blog is what Bugs means by "this paper" in this just above. When Drepper says "the code has been reviewed" he appears to refer to an implementation of PHK's scheme using sha256 rather than md5. But the pam_unix man page and the /etc/pam.d/common-password comments seem to say very clearly that md5 is the Debian default, at least in oldstable (Lenny), which would not make sense for the time frame when Lenny was introduced.

If I understand this correctly, Drepper is the author of the software from which Debian's pam_unix is derived. In his 2007 defense of his choice to go with salted and iterated sha256 (not md5? contrary to what the man pages say?) rather than bcrypt, he seems to assume that the attacker will not learn the salt. I don't see why that is a safe assumption.

Pracek's point is that for storing passwords, a sloooow algorithm is just what you want to make off-line attacks infeasible. That seems very clear and reasonable to me.

In 2007, Pracek and Drepper came to different conclusions:

• Pracek felt that there were three workable solutions
  • PHK's iterated md5 (or sha256) with salt
  • adaptive hashing: bcrypt
  • SRP
  but of these, bcrypt was clearly the "correct choice".
• Drepper felt that PHK's scheme was the best choice, for...regulatory reasons? Because he trusted the sha256 code more than the bcrypt code? No, he clearly says that he considers bcrypt to be secure.

Since most home users probably don't have to satisfy regulators, it seems that bcrypt is the correct choice for home users.

I seem to have installed bcrypt, libpam-unix2, and any libraries I might need. Now all I need, I think, is the correct line to put in my /etc/pam.d/common-password.

[bugsbunny]

Digital Consumption - Blowfish shadow files on Debian

Notes: If you're currently using one of the sha's you need to convert to md5 before switching (and make sure that the passwords have all converted as well).
the "obscure" option won't work, neither will "nullok_secure"

Options that will work:
nullok
use_first_pass
use_authtok

valid encryption algorithms: md5, bigcrypt, blowfish, bf (alternate for blowfish)

for iteration support: rounds=xxxxx or crypt_rounds=xxxxx where xxxxx is an integer representing number of iterations

That's about it.

[Ahtiga Saraz]

Thanks for the link, but... wow, this keeps getting more complicated!

If you're currently using one of the sha's you need to convert to md5 before switching (and make sure that the passwords have all converted as well).

Sorry, I don't understand. I thought the idea was to modify at least one file in /etc/pam.d and to change all the user passwords using (for example) kuser?

Before I (as root user) modify every file in my /etc/pam.d I'd like to know a bit more:

• the original paper on "adaptive hashing" with bcrypt uses a modified "slowpoke" blowfish; is the "blowfish" invoked by making the changes suggested by Charles Darke this slowpoke blowfish, and if so, shouldn't it be called by its real name to avoid confusion? (On my system, a package like mcrypt and probably others claims to implement ordinary blowfish.)
• is there any way I can verify what password storage procedure my system is currently using? As I mentioned earlier, as root I did look at /etc/shadow, from which it seems likely that my system is using md5. If so that should be because it fell back on the default because (it seems) the line I tried in my /etc/pam.d/common-password is incorrect. But how can I tell whether it is using a salt (how big, approximately?) and iterations (how many, at least approximately?).

[EDIT: just noticed that Charles Darke says that if the "hash" in /etc/shadow begins with $1 it is "hashed" using md5, which is just what I had guessed. If it begins with $2 is is "hashed" with blowfish.]

[Ahtiga Saraz]

Well, I installed the deb pam-unix2. I then tried editing

• /etc/common-account
• /etc/common-auth
• /etc/common-password
• /etc/common-session

to replace pam_unix.so with pam_unix2.so and to take out the options which are illegal under pam_unix2. I tried changing md5 to blowfish and other options. I tried editing the conf file for pam_unix2. I tried rebooting. Each time I then changed the password of a test user and then peeked at /etc/shadow, I could see the hash change. And etc/auth.log didn't indicate that my system didn't recognize pam_unix2.so.

But no matter what I try, it is always an md5 hash, to judge by length and leading character.

Should I see pam_unix and or pam_unix2 as installed modules using lsmod?

In etc/shadow, if the last field is a 10 digit number, does that mean that I have at least used a salt in "hashing" that user's password?

Did installing libpam_unix2 mean that libpam_unix is now broken on my system? That would be a disaster since I changed the /etc/pam.d files back to the default lines for now. Both pam_unix and pam_unix2 man pages are visible and distinct, so probably this is OK.

[bugsbunny]

[EDIT: just noticed that Charles Darke says that if the "hash" in /etc/shadow begins with $1 it is "hashed" using md5, which is just what I had guessed. If it begins with $2 is is "hashed" with blowfish.]

Right, which is what you would have read if you ever issued the

Code: Select all

man 3 crypt

command blowfish can be either 2 or 2a

blowfish, by it's very nature, is "slowpoke" compared to any of the hashing functions. Iterating it will slow it even more.

The reason I say you have to make sure you're not using one of the sha functions before switching is that the unix2 module doesn't support those, which means you won't be able to login if you switch but the stored passwords are still using sha hashes. md5 is common to both (as is des, but that would be really foolish)

The salt is also in the shadow file.. Again I refer you to the Notes section of "man 3 crypt" (the earlier parts talk about des)

pam_unix is still available. Look for errors in the auth log file. (You can ignore any unix2 errors that say "this option is obsolete use /etc/default/passwd". First the file is /etc/security/pam_unix2.default, second the options are still applied from within pam.d (I looked at the source code)

Basically the shadow (or passwd) file entry looks like:
$id$salt$encrypted

If iteration is called you should see that reflected in there as well. Looking at my shadow file i get (a somewhat modified):
bbunny:$6$rounds=9999$96AXd2p8$0LxBZfnuDue7.n5<rest of string>

That gives
<login name>:$6<means sha512>$rounds=9999<self explanatory>$<salt>$<actual resulting hash value>

[Ahtiga Saraz]

Bugs, it is difficult for me to get to this forum and almost impossible for me to post. My government (don't ask) hates the internet. (I am NOT asking for help on my connection to this forum, because there is absolutely nothing which can be done about it.)

Which is why I was not able to previously tell you that when I try

Code: Select all

man 3 crypt

this does not work on my system. That is why I cannot view the crypt man page. I know this is ridiculous, but could you possibly just paste into your reply the contents of the crypt man page? Thanks.

blowfish, by it's very nature, is "slowpoke" compared to any of the hashing functions. Iterating it will slow it even more.

Good points, but I already knew all this. Do you see why I am confused by the fact that the slowpoke blowfish is in a deb which I am evidently supposed to download with bcrypt, yet from what you say, it appears that this slowpoke version is not actually used by Debian, no matter what one tries?

Looking at my shadow file i get (a somewhat modified):

Code: Select all

bbunny:$6$rounds=9999$96AXd2p8$0LxBZfnuDue7.n5<rest of string>

That gives

Code: Select all

<login name>:$6<means sha512>$rounds=9999<self explanatory>$<salt>$<actual resulting hash value>

Thank you indeed! That is very useful. No matter what options, added lines, I try, I can only get hashes which look like this (slightly modified):

Code: Select all

crackmeifucan:$1$r7DABCDE$XXC0BmRsABC/IIwFGs4Cz1:...

Which seems to mean that no matter what I do, I am using md5 with no salt and no iteration. Incredible.

Could you paste in the line from your /etc/pam.d/common-password, please? Do you use pam_unix or pam_unix2? Can you think of any line in your /etc/pam.d files which I might need?

If it helps, when I started this thread, the line in my /etc/pam.d/common-password was

Code: Select all

password   required   pam_unix.so nullok obscure md5

and I am now completely confused about whether that is even supposed to work, much less why such an allegedly archaic line is apparently the oldstable (Lenny) default.

I have been stuck for more than a week on a very small point in my quest to install Debian stable (Squeeze) which might resolve these problems on the new system. Since I am stuck, this hope is of no help here.

I understand, of course, that passwords are merely a first line of defense. Nonetheless, a very important line. So given the effort I have put into security, I am aghast to learn that apparently my system is doing everything it can to ensure that I am using a password storage method which has been well known to be insecure for... what, two decades?!

Could there be some kernel module which I don't know about which is somehow over-riding all my attempts to change to a better method of password storage?

[Ahtiga Saraz]

As I said, I changed all my /etc/pam.d files back to what they were when I started, so I should be using pam_unix again even though pam_unix2 is also installed. Just checked changing password of my test user again, and from /var/log/auth.log it seems that I am using pam_unix and that it is working when I su to my test user.

[bugsbunny]

Ok - man 3 crypt, to get the file (choice of languages):

Code: Select all

apt-file find man3/crypt.3.gz
manpages-de-dev: /usr/share/man/de/man3/crypt.3.gz
manpages-dev: /usr/share/man/man3/crypt.3.gz
manpages-es: /usr/share/man/es/man3/crypt.3.gz
manpages-fr-dev: /usr/share/man/fr/man3/crypt.3.gz
manpages-ja-dev: /usr/share/man/ja/man3/crypt.3.gz
manpages-pl-dev: /usr/share/man/pl/man3/crypt.3.gz
manpages-pt-dev: /usr/share/man/pt/man3/crypt.3.gz
manpages-tr: /usr/share/man/tr/man3/crypt.3.gz

Full contents (although we've pretty much covered this):

Code: Select all

crypt(3) - Man pages

dwww Home | Manual pages | Find package

CRYPT(3)                   Linux Programmer's Manual                  CRYPT(3)

NAME
       crypt, crypt_r - password and data encryption

SYNOPSIS
       #define _XOPEN_SOURCE
       #include <unistd.h>

       char *crypt(const char *key, const char *salt);

       char *crypt_r(const char *key, const char *salt,
                     struct crypt_data *data);

       Link with -lcrypt.

DESCRIPTION
       crypt()  is  the password encryption function.  It is based on the Data
       Encryption Standard algorithm with  variations  intended  (among  other
       things) to discourage use of hardware implementations of a key search.

       key is a user's typed password.

       salt is a two-character string chosen from the set [a–zA–Z0–9./].  This
       string is used to perturb the algorithm in one of 4096 different ways.

       By taking the lowest 7 bits of each of the first  eight  characters  of
       the  key, a 56-bit key is obtained.  This 56-bit key is used to encrypt
       repeatedly a constant  string  (usually  a  string  consisting  of  all
       zeros).   The returned value points to the encrypted password, a series
       of 13 printable ASCII characters (the first  two  characters  represent
       the salt itself).  The return value points to static data whose content
       is overwritten by each call.

       Warning: The key space consists of 2**56 equal 7.2e16 possible  values.
       Exhaustive searches of this key space are possible using massively par-
       allel computers.  Software, such as crack(1), is available  which  will
       search  the  portion of this key space that is generally used by humans
       for passwords.  Hence, password selection  should,  at  minimum,  avoid
       common words and names.  The use of a passwd(1) program that checks for
       crackable passwords during the selection process is recommended.

       The DES algorithm itself has a few quirks which make  the  use  of  the
       crypt()  interface  a very poor choice for anything other than password
       authentication.  If you are planning on using the crypt() interface for
       a  cryptography project, don't do it: get a good book on encryption and
       one of the widely available DES libraries.

       crypt_r() is a reentrant version of crypt().  The structure pointed  to
       by  data  is  used  to  store  result data and bookkeeping information.
       Other than allocating it, the only thing that the caller should do with
       this  structure  is  to  set data->initialized to zero before the first
       call to crypt_r().

RETURN VALUE
       On success, a pointer to the encrypted password is returned.  On error,
       NULL is returned.

ERRORS
       ENOSYS The  crypt()  function  was not implemented, probably because of
              U.S.A. export restrictions.

CONFORMING TO
       crypt(): SVr4, 4.3BSD, POSIX.1-2001.  crypt_r() is a GNU extension.

NOTES
   Glibc Notes
       The glibc2 version of  this  function  supports  additional  encryption
       algorithms.

       If  salt is a character string starting with the characters "$id$" fol-
       lowed by a string terminated by "$":

              $id$salt$encrypted

       then instead of using the DES machine,  id  identifies  the  encryption
       method  used  and  this  then  determines  how the rest of the password
       string is interpreted.  The following values of id are supported:

              ID  | Method
              ─────────────────────────────────────────────────────────
              1   | MD5
              2a  | Blowfish (not in mainline glibc; added in some
                  | Linux distributions)
              5   | SHA-256 (since glibc 2.7)
              6   | SHA-512 (since glibc 2.7)

       So   $5$salt$encrypted   is   an   SHA-256   encoded    password    and
       $6$salt$encrypted is an SHA-512 encoded one.

       "salt" stands for the up to 16 characters following "$id$" in the salt.
       The encrypted part of the password string is the actual computed  pass-
       word.  The size of this string is fixed:

       MD5     | 22 characters
       SHA-256 | 43 characters
       SHA-512 | 86 characters

       The  characters  in  "salt"  and  "encrypted"  are  drawn  from the set
       [a–zA–Z0–9./].  In the MD5 and SHA implementations the  entire  key  is
       significant (instead of only the first 8 bytes in DES).

SEE ALSO
       login(1),    passwd(1),   encrypt(3),   getpass(3),   passwd(5),   fea-
       ture_test_macros(7)

COLOPHON
       This page is part of release 3.27 of the Linux  man-pages  project.   A
       description  of  the project, and information about reporting bugs, can
       be found at http://www.kernel.org/doc/man-pages/.

                                  2010-06-20                          CRYPT(3)

Line from my common-password file:

Code: Select all

password	[success=1 default=ignore]	pam_unix.so obscure sha512 rounds=9999

Now if you really want you can try the following:

Code: Select all

password	[success=1 default=ignore]	pam_unix2.so  blowfish rounds=9999

that should work, but is untested by me at the moment. I have tried using this in the past, but ultimately decided it wasn't worth it. Whether you belive me or not I'm convinced that using what I have above is, for all practical purposes (including governments with massive computer arrays available) just as safe as using blowfish would be. The advantage of sticking with the standard pam_unix rather than switching to unix2 (and why I ultimately did) is that in the future, if (and when) they change/upgrade the available methods it will be much easier to implement by staying with the norm, since I'm certain it will be backwards compatible (meaning I won't have a problem logging in before I change the encryption method). Right now if I wanted to switch to blowfish I would have to change my passwords to md5 encryption, then change the relevant lines to unix2, then change to blowfish. Doable since I don't have too many accounts on here, but still a pain.

Ultimately though if your biggest worry is gov scrutiny then you need to do 2 things:
make sure they can't get in from the outside while the system is running. Relatively simple to do (I use relatively deliberately here, since it's not totally simple, but not that difficult. Don't allow root ssh access. Use keyfiles rather than passwords for ssh access. Depending on what else you have running run something like fail2ban, or elsr set up a more comprehensive firewall.

Number 2 - full disk encryption. Although this can lead to other problems if they decide to search. (eg if the fact that it's encrypted is enough to throw you into the fire). If that's the case then there's still things you can do, you just have to get sneakier about it. What can be done though depends to at least a certain extent to what material, and how much of it, is involved. In any case blowfish vs sha512 for login passwords would be the least of your worries. Either of them would provide more than enough security even against a very determined and competent adversary.

[Ahtiga Saraz]

I don't want to hijack my own thread, but on the man page mystery, what you just posted gave me this idea (should have thought it, I know...)

Code: Select all

# locate man3 | grep crypt
/usr/share/man/man3/Crypt::Eksblowfish::Bcrypt.3pm.gz
/usr/share/man/man3/libmcrypt.3.gz
/usr/share/man/man3/mcrypt.3.gz

And

Code: Select all

man Crypt::Eksblowfish

does give me a man page for the slowpoke blowfish, proper name eksblowfish. Also

Code: Select all

man Crypt::PasswdMD5

does give me a man page for how to use the MD5 method under the old default, crypt.

And now I have another problem: I don't know enough about Perl to understand those man pages.

[EDIT: never mind, neither of these Crypt:: man pages seems to have anything to do with shadow. I am so confused...]

Back to my immediate problem, of getting what you do to work for me: to use

Code: Select all

password   [success=1 default=ignore]   pam_unix.so obscure sha512 rounds=9999

do I need to uninstall pam_unix2.so before I change my current line to that line? Afterwards, I know that I need to change passwords for the line to take effect.

Right now, when I try your line (with all known options for the "hash algorithm"), I still get md5 and I DONT get any rounds, but now I am at least getting a ten character (randomly chosen?) salt.

Same when I try

Code: Select all

password   [success=1 default=ignore]   pam_unix2.so  blowfish rounds=9999

Is it possible that I somehow have inconsistent definitions in the sum total of the relevant config files, which is causing the system to refuse to do rounds or to use anything but md5? Is it possible that I lack the right version of glibc? I tried to check but failed to find anything.

Now I am really getting curious... would it be inappropriate for me to try to make my first poll? Question: what is in your /etc/pam.d/common-password? I would really like to know if the braindead line

Code: Select all

password   required   pam_unix.so nullok obscure md5

was ever the default line for Debian Lenny. Again, my Debian oldstable (Lenny) is about 1.5 years old, not 25 years old.

[Ahtiga Saraz]

• /etc/pam.d/common-account

  Code: Select all

  account required        pam_unix.so
  # account       required        pam_unix2.so
  

• /etc/pam.d/common-auth

  Code: Select all

  auth    required        pam_unix.so nullok_secure
  # auth  required        pam_unix2.so
  

• /etc/pam.d/common-password

  Code: Select all

  # password   required   pam_unix.so nullok obscure md5
  # password   [success=1 default=ignore]   pam_unix2.so blowfish rounds=9999
  # password   [success=1 default=ignore]   pam_unix.so sha256 rounds=9999
  password   [success=1 default=ignore]   pam_unix.so md5 rounds=9999
  

• /etc/pam.d/common-session - session-related modules common to all services

  Code: Select all

  session required        pam_unix.so
  # session       required        pam_unix2.so
  

The commented out lines give some idea of the changes I have tried without success. No matter what I try, I cannot get even md5 to go through multiple rounds (no matter how few I try), and I cannot get any "hash" algorithm" but md5.

[Ahtiga Saraz]

OK if I bump this?

I think the question of password hashes is obviously very important for Debian users. So it is very frustrating that I am still unable to coax my oldstable system into using anything but MD-5 with only one round. (I have at least been able to coax it into using a 10 character salt which is stored in the shadow file along with the hash.)

[Ahtiga Saraz]

OK if I bump this again?

Despite much effort, I am still stuck with salted but uniterated MD-5 hashes.

Passwords are only the first line of defence, but the first line is very important when you do not really have a strategy of "defense in depth".

Please help me figure this out!

[Ahtiga Saraz]

I think I finally have an unofficial answer for Squeeze users. This procedure doesn't work on my Lenny system for some reason, but I have tested it on a Debian 6.0.1a (Squeeze) installation.

In a shell, as root, examine /etc/shadow. From the form of the lines corresponding to your ordinary user, you should be able to see from the first field (the fields are separated by colons) how passwords are stored in "obscured form" in /etc/shadow. Even if an intruder is able to retrieve your /etc/shadow, our goal is to ensure that he will have a hard time recovering user passwords from what is stored there.

According to my best current understanding:

• in each line corresponding to a user who is allowed to access a login shell, the first field probably has form $1$string$string, where the one digit code names the algorithm used to store a "hashed" password, the first string is a salt (probably 8 characters long), and the second string is the "hashed" password,
• the algorithm codes are:
  • $1 means MD-5
  • $2 means the original blowfish block cipher algorithm, used here for storing passwords
  • $2a means eksblowfish, the version specially modified for storing passwords (runs very slowly by design)
  • $3 ???
  • $4 ???
  • $5 means SHA-256
  • $6 means SHA-512

This understanding is correct according to the source of all questionable wisdom, if that means anything.

It seems that in Debian 6.0.1a (Squeeze), by default /etc/pam.d/common-password has the line

Code: Select all

password  [ success=1 default=ignore  ]  pam_unix.so obscure sha512

So by default, passwords are stored with salted SHA-512"hash", where the salt is some 8 char publically known string, but no iterations. Furthermore, it seems that even if you install from CD-1 offline, your new Squeeze system knows about SHA-512, which is very good. If you want more rounds change this to

Code: Select all

password  [ success=1 default=ignore  ]  pam_unix.so obscure sha512 rounds=1000

You can use any positive integer, I think, and 1000 might not be the best choice. This method of iterating a hash function many times is called key stretching.

Next, exit from su, and check with whoami that you are once again functioning as your ordinary user. Now change your password by calling passwd and giving your old password then your new one.

Next, su back to root and look at /etc/shadow. You should now see that the line for your ordinary user begins with

Code: Select all

$6$rounds=1000$8_char_salt$long_string_which_is_your_hashed_password

This means: use SHA-512 algorithm with the given 8 character string as salt, and iterate 1000 times. That is thought to be adequate to defeat rainbow table attacks.

Next, change the root login password and the login passwords of any other ordinary users on your system.

Iterating 1000 times shouldn't cause any usability problems since this doesn't take long by human standards, and you probably don't type in your user login password very often. I think it is true to say that iterating SHA-512 is an inelegant brute force solution to the problem that "integrity check" hash algorithms (which should be fast) like MD-5, SHA-256, SHA-512 are not well suited to serve as "cryptographic" hash algorithms (which should be slow, and have other characteristics conducive to cryptographic strength).

You can now try logging in as your ordinary user. Here is what I think happens: as per the pam_unix.so library, you are prompted for your login password. The 8 char salt found (in cleartext) in /etc/shadow is appended to the string you type, then SHA-512 is applied 1000 times. The result is then compared to the long string found in /etc/shadow in the entry for your ordinary user. And if the strings agree, you are logged in.

There is an open source implemention, discussed above, of ek2sblowfish, which takes more work to achieve in Squeeze, but may not be much stronger than simply iterating SHA-512 approximately 1000 times. To use this method for storing passwords securely, I think you need to do something like this:

• download the appropriate package implementing eksblowfish
• change pam_unix.so to pam_unix2.so in the files in /etc/pam.d
• indicate change sha512 to blowfish in your /etc/pam.d/common-password

I have not tested this, though, so this probably needs some further tweaking.

[scruffyeagle]

I won't pretend to have understood very much of this thread, but I did read all the posts before commenting. I just had one thing to contribute: If you don't like the fact that 1000 breaks down into small number factors, use a prime number as the iterations setting.

[Ahtiga Saraz]

@Caisyy:

Sounds like you replaced a line in a configuration file, but which file?

@scruffyeagle:

Yes, we had the same thought.