Code: Select all
# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 11 (bullseye)
Release: 11
Codename: bullseye
I administer around 40 Debian based virtual machines. About 70% are buster, the balance are bullseye. We're in the process of upgrading those Busters to Bulleye since I noticed earlier this month that Buster was EOL as of Sept 2022.
We monitor our systems for amongst over things disk space and have noticed several have at least 4GB of logs in /var/log/journal. We tend to be quite prudent when it comes to allocating resources to VMs to make the most out of a resource, and also consider things like back ups. We also operate a SEIM which all our servers send their logs to, so storing 4GB or more of logs locally is a bit wasteful.
I've set SystemMaxUse=500M in /etc/systemd/journald.conf and restarted the service by executing systemctl status systemd-journald.service, which immediately vaccumed the logs to 500mb.. however a few days later the logs are back up to 4GB.
Example...
root@SERVERNAME:~# systemctl status systemd-journald.service
● systemd-journald.service - Journal Service
Loaded: loaded (/lib/systemd/system/systemd-journald.service; static)
Active: active (running) since Mon 2022-12-05 16:39:12 UTC; 20s ago
TriggeredBy: ● systemd-journald-dev-log.socket
● systemd-journald-audit.socket
● systemd-journald.socket
Docs: man:systemd-journald.service(8)
man:journald.conf(5)
Main PID: 572900 (systemd-journal)
Status: "Processing requests..."
Tasks: 1 (limit: 9529)
Memory: 1.4M
CPU: 477ms
CGroup: /system.slice/systemd-journald.service
└─572900 /lib/systemd/systemd-journald
Dec 05 16:39:12 SERVERNAME systemd-journald[572900]: Journal started
Dec 05 16:39:12 SERVERNAME systemd-journald[572900]: System Journal (/var/log/journal/71b8a1315af275875e623e775d680544) is 488.0M, max 500.0M, 11.9M free.
Dec 05 16:39:12 SERVERNAME systemd-journald[572900]: System Journal (/var/log/journal/71b8a1315af275875e623e775d680544) is 488.0M, max 500.0M, 11.9M free.
Obviously journald is recognising I've set a hard limit of 500mb because it is vacuuming at start up.. so I can only conclude journald isn't performing it's auto vacuuming when running.
I've reviewed https://manpages.debian.org/testing/sys ... .5.en.html but have been unable to discover why SystemMaxUse=500M isn't being honoured.
I have only noticed this on the Bulleye machines but I don't have enough statistics to confirm it's specifically a Bullseye problem.
Any help gratefully received
Cheers
Steve