[HowTo] managing logfiles

[wizard10000]

I spent some time yesterday cleaning up logfiles and configuring logrotate and journald and figured I'd write a howto on managing logfiles,

Two things we'll get to play with here - logrotate and journald.conf. Note that both logrotate and journald.conf have man pages available in case you get stuck. Let's get to it

logrotate

logrotate looks for its configuration in /etc/logrotate.conf and /etc/logrotate.conf.d and the format is pretty simple. Below is an unmolested copy of logrotate.conf - check it out -

Code: Select all

# see "man logrotate" for details.

# global options do not affect preceding include directives

# rotate log files weekly
#weekly

# keep 4 weeks worth of backlogs
#rotate 4

# create new (empty) log files after rotating old ones
#create

# use date as a suffix of the rotated file
#dateext

# uncomment this if you want your log files compressed
#compress

# packages drop log rotation information into this directory
include /etc/logrotate.d

# system-specific logs may also be configured here.

Pretty much self-explanatory, right? Now let's look at mine -

Code: Select all

# see "man logrotate" for details

# global options do not affect preceding include directives

# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 2

# create new (empty) log files after rotating old ones
create

# use date as a suffix of the rotated file
dateext

# uncomment this if you want your log files compressed
#compress

# packages drop log rotation information into this directory
include /etc/logrotate.d

I've made a couple of changes - I don't need four weeks worth of logs so I've cut that back to two weeks by changing the default "rotate 4" to "rotate 2". I've also told logrotate to create an empty logfile for any log it archives and to add a date to the archive's filename. I don't compress logs but you might want to take advantage of that option.

You can see the last line in the file says "include /etc/logrotate.d" and if you take a look in that directory you'll see configurations for a few other logs - check it out - this is /etc/lograte.d/apt

Code: Select all

/var/log/apt/term.log {
  rotate 2
  weekly
  compress
  missingok
  notifempty
}

/var/log/apt/history.log {
  rotate 2
  weekly
  compress
  missingok
  notifempty
}

Now apt by default rotates logs monthly and keeps a year's worth of logs so as you can see I've made a couple of changes so apt only maintains two weeks of logs. As mentioned logrotate's man page has a lot more information and is worth a look because there's a heck of a lot more stuff you can do with logrotate like run scripts, move archives to another directory and so on. You can create files in /etc/logrotate.d if you want to customize other logfiles.

journald.conf

journald maintains binary logs that can be accessed with journalctl. /etc/systemd/journald.conf is a little more complex than logrotate configs so I've linked to https://www.freedesktop.org/software/sy ... .conf.html so folks can get a better understanding of what goes on under the hood. Check this out -

Code: Select all

[Journal]
#Storage=auto
#Compress=yes
#Seal=yes
#SplitMode=uid
#SyncIntervalSec=5m
#RateLimitIntervalSec=30s
#RateLimitBurst=10000
SystemMaxUse=1G
#SystemKeepFree=
#SystemMaxFileSize=
#SystemMaxFiles=100
#RuntimeMaxUse=
#RuntimeKeepFree=
#RuntimeMaxFileSize=
#RuntimeMaxFiles=100
#MaxRetentionSec=0
#MaxFileSec=1month
#ForwardToSyslog=no
#ForwardToKMsg=no
#ForwardToConsole=no
#ForwardToWall=yes
#TTYPath=/dev/console
#MaxLevelStore=debug
#MaxLevelSyslog=debug
#MaxLevelKMsg=notice
#MaxLevelConsole=info
#MaxLevelWall=emerg
#MaxLevelSocket=debug
#LineMax=48K
#ReadKMsg=yes
#Audit=yes

If you look closely you'll see the only change I made was restricting the journal to 1GB. By default SystemMaxUse limits the journal to 10% of the filesystem or 4GB, whichever is smaller. I don't need 4GB of journal so I restrict journald to 1GB but I don't restrict it during runtime. If someone is concerned about a runaway log filling up a root partition you could also adjust RuntimeMaxUse to help protect your system.

The changes I made above reduced the size of my logs from a little more than 3GB to 1.2GB. As mentioned logrotate and journald.conf are well-documented but I would recommend backing up the default files before hacking away at them.

Enjoy!

[sunrat]

I restrict journal logs to 200M as I've never found the need for journal logs past the previous boot. YMMV.

Code: Select all

SystemMaxUse=200M

I do however leave apt logs at the default 12 which are archived monthly. On a few occasions I've wanted to find something that was installed long ago. My /var/log/apt/ directory currently amasses 394KiB so that's not much burden.

[lindi]

With systemd-journal-remote you can configure your systems to sync the journal to a remote system.

[wizard10000]

I restrict journal logs to 200M as I've never found the need for journal logs past the previous boot. YMMV.

Code: Select all

SystemMaxUse=200M

Me neither - I may shrink this further.

I do however leave apt logs at the default 12 which are archived monthly. On a few occasions I've wanted to find something that was installed long ago. My /var/log/apt/ directory currently amasses 394KiB so that's not much burden.

Now that I think about it I'm beginning to agree with you here. I think I may put this back where it was and just shove the 11 archives it'll create into their own directory. I don't care much for clutter and am thinking of creating /var/log/archives and telling journald and logrotate to move archives there.

Thanks for the input!

edit: here's where I ended up -

Code: Select all

/var/log/apt/term.log {
  rotate 12
  monthly
  olddir /var/log/archives/apt
  dateext
  missingok
  notifempty
}

/var/log/apt/history.log {
  rotate 12
  monthly
  olddir /var/log/archives/apt
  dateext
  missingok
  notifempty
}

[CwF]

For users to manipulate and fully utilize the journal remember to add those users to group ‘adm’ and/or ‘systemd-journal’. This also gives the user the ability to use the journalctl --vacuum.

It's possible on some system to remove rsyslog and logrotate packages since they are redundant. I have not completed a full audit of legacy programs that may still only use the old ways, but the journal is now capturing most things. apt/history would still be one reason to keep logrotate and on a few systems and I have that history years long and not really an issue.

So far I use only the file number limit and/or use active user management during the session.

Code: Select all

 
 $  cat /etc/systemd/journald.conf.d/MaxFiles.conf
SystemMaxFiles=11
 $  journalctl --disk-usage
Archived and active journals take up 62.3M in the file system.
 $  journalctl --vacuum-files=10
Vacuuming done, freed 0B of archived journals from /run/log/journal.
Vacuuming done, freed 0B of archived journals from /var/log/journal.
Vacuuming done, freed 0B of archived journals from /var/log/journal/7f00b65437ca41578997408626d105f8
 

--vacuum-size=BYTES
--vacuum-files=INT
--vacuum-time=TIME

If there is cruft existing before any declarations are made then we should manually vacuum them down to size to see immediate results. Old log files need manually deleted I think.

On Xfce I use genmon with some of this built in to report, and with a click, vacuum. I wish I could click something like that for my house...Where is that Robin...
Bleachbit also has a journal vacuum built in.

[wizard10000]

...I have not completed a full audit of legacy programs that may still only use the old ways

The list is pretty short. On this machine it's alternatives, apt, aptitude, cups, dpkg, fontconfig and popcon.