Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Apt (or something) eating up all HD space every day

If none of the specific sub-forums seem right for your thread, ask here.
Post Reply
Message
Author
kurros
Posts: 5
Joined: 2016-04-27 12:18

Apt (or something) eating up all HD space every day

#1 Post by kurros »

I am witnessing some annoying behaviour, I think by apt (on debian stretch). When I first start my computer each day some process runs for a while and just totally fills up my disk with the apt cache. I then have to run "apt-get clean" to get it all back. This is no big problem since I just go through this process every day, but it is kind of irritating behaviour and it would be nice to stop it if possible. It seems to have started after some recent update, though I can't be sure. I also sometimes see packagekitd running at this time, so it might be involved. Any ideas what is happening?

User avatar
debiman
Posts: 3063
Joined: 2013-03-12 07:18

Re: Apt (or something) eating up all HD space every day

#2 Post by debiman »

do you have unattended upgrades running or something?
otherwise apt shouldn't be running automatically.

Code: Select all

apt search packagekitd
Sorting... Done
Full Text Search... Done
hmmmm... :scratchbeard:

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: Apt (or something) eating up all HD space every day

#3 Post by Head_on_a_Stick »

Can we see

Code: Select all

systemctl list-unit-files --state=enabled --no-pager
Also, please let me know if the terminal text stays green after that command, I want to know if my bug has been fixed in stretch yet.
deadbang

kurros
Posts: 5
Joined: 2016-04-27 12:18

Re: Apt (or something) eating up all HD space every day

#4 Post by kurros »

All the "enabled"s are green, I guess you mean does it revert to the default color afterwards? Yes, the text looks like normal after the command finishes. Here is the output:

Code: Select all

$ systemctl list-unit-files --state=enabled --no-pager
UNIT FILE                                  STATE  
acpid.path                                 enabled
cups.path                                  enabled
accounts-daemon.service                    enabled
anacron.service                            enabled
atd.service                                enabled
autovt@.service                            enabled
avahi-daemon.service                       enabled
binfmt-support.service                     enabled
bluetooth.service                          enabled
console-setup.service                      enabled
cron.service                               enabled
cups-browsed.service                       enabled
cups.service                               enabled
dbus-org.bluez.service                     enabled
dbus-org.freedesktop.Avahi.service         enabled
dbus-org.freedesktop.ModemManager1.service enabled
dbus-org.freedesktop.nm-dispatcher.service enabled
getty@.service                             enabled
irqbalance.service                         enabled
keyboard-setup.service                     enabled
minissdpd.service                          enabled
ModemManager.service                       enabled
network-manager.service                    enabled
networking.service                         enabled
NetworkManager-dispatcher.service          enabled
NetworkManager.service                     enabled
portmap.service                            enabled
pppd-dns.service                           enabled
rpcbind.service                            enabled
rsync.service                              enabled
rsyslog.service                            enabled
rtkit-daemon.service                       enabled
syslog.service                             enabled
systemd-timesyncd.service                  enabled
acpid.socket                               enabled
avahi-daemon.socket                        enabled
cups.socket                                enabled
rpcbind.socket                             enabled
uuidd.socket                               enabled
remote-fs.target                           enabled
anacron.timer                              enabled
apt-daily-upgrade.timer                    enabled
apt-daily.timer                            enabled

43 unit files listed.
Last edited by kurros on 2017-11-03 09:07, edited 1 time in total.

kurros
Posts: 5
Joined: 2016-04-27 12:18

Re: Apt (or something) eating up all HD space every day

#5 Post by kurros »

debiman wrote:do you have unattended upgrades running or something?
otherwise apt shouldn't be running automatically.
Hmm well I didn't set up any such thing, so I think no?

User avatar
pylkko
Posts: 1802
Joined: 2014-11-06 19:02

Re: Apt (or something) eating up all HD space every day

#6 Post by pylkko »

kurros wrote:
debiman wrote:do you have unattended upgrades running or something?
otherwise apt shouldn't be running automatically.
Hmm well I didn't set up any such thing, so I think no?
Some desktop environments also do this I believe, automatic update, that is

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: Apt (or something) eating up all HD space every day

#7 Post by Head_on_a_Stick »

kurros wrote:

Code: Select all

apt-daily-upgrade.timer                    enabled
apt-daily.timer                            enabled
^ This.

EDIT: can we see what they do please:

Code: Select all

systemctl cat apt-daily{,-upgrade}.timer
deadbang

kurros
Posts: 5
Joined: 2016-04-27 12:18

Re: Apt (or something) eating up all HD space every day

#8 Post by kurros »

Hmm, interesting:

Code: Select all

$ systemctl cat apt-daily{,-upgrade}.timer
# /lib/systemd/system/apt-daily.timer
[Unit]
Description=Daily apt download activities

[Timer]
OnCalendar=*-*-* 6,18:00
RandomizedDelaySec=12h
Persistent=true

[Install]
WantedBy=timers.target

# /lib/systemd/system/apt-daily-upgrade.timer
[Unit]
Description=Daily apt upgrade and clean activities
After=apt-daily.timer

[Timer]
OnCalendar=*-*-* 6:00
RandomizedDelaySec=60m
Persistent=true

[Install]
WantedBy=timers.target

User avatar
debiman
Posts: 3063
Joined: 2013-03-12 07:18

Re: Apt (or something) eating up all HD space every day

#9 Post by debiman »

you need to look at the corresponding service file to see what is executed - that is the next step to determine what is filling up your hd.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: Apt (or something) eating up all HD space every day

#10 Post by Head_on_a_Stick »

^ Yes, I suppose I should have suggested

Code: Select all

systemctl cat apt-daily{,-upgrade}.{timer,service}
But anyway I'm sure we can use our imagination here :)

@OP: this will stop the madness:

Code: Select all

# systemctl disable --now apt-daily{,-upgrade}.timer
deadbang

kurros
Posts: 5
Joined: 2016-04-27 12:18

Re: Apt (or something) eating up all HD space every day

#11 Post by kurros »

Ahh, excellent, thanks. I'll get back to you if this doesn't work, but it seems pretty likely to be the problem.

Just for completeness, here is the output of that last command:

Code: Select all

$ systemctl cat apt-daily{,-upgrade}.{timer,service}

# /lib/systemd/system/apt-daily.timer
[Unit]
Description=Daily apt download activities

[Timer]
OnCalendar=*-*-* 6,18:00
RandomizedDelaySec=12h
Persistent=true

[Install]


# /lib/systemd/system/apt-daily.service
[Unit]
Description=Daily apt download activities
Documentation=man:apt(8)
ConditionACPower=true
After=network-online.target
Wants=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/lib/apt/apt.systemd.daily update


# /lib/systemd/system/apt-daily-upgrade.timer
[Unit]
Description=Daily apt upgrade and clean activities
After=apt-daily.timer

[Timer]
OnCalendar=*-*-* 6:00
RandomizedDelaySec=60m
Persistent=true

[Install]
WantedBy=timers.target

# /lib/systemd/system/apt-daily-upgrade.service
[Unit]
Description=Daily apt upgrade and clean activities
Documentation=man:apt(8)
ConditionACPower=true
After=apt-daily.service

[Service]
Type=oneshot
ExecStart=/usr/lib/apt/apt.systemd.daily install
KillMode=process
TimeoutStopSec=900

User avatar
debiman
Posts: 3063
Joined: 2013-03-12 07:18

Re: Apt (or something) eating up all HD space every day

#12 Post by debiman »

Head_on_a_Stick wrote:@OP: this will stop the madness:

Code: Select all

# systemctl disable --now apt-daily{,-upgrade}.timer
just keep in mind:
this will probably prevent apt from eating your hd, but it will also disable automatic updates!

actually i'm inclined to think that these services shouldn't eat your hd, and if they do, chances are a manual upgrade will do the same. but at this point it's really just a hunch, feel free to ignore me.

kopper
Posts: 137
Joined: 2016-09-30 14:30

Re: Apt (or something) eating up all HD space every day

#13 Post by kopper »

If you don't want to disable automated updates, you could create a cron task to clean everything up for you. E.g. for daily clean up you could do something like this.

Code: Select all

sudo crontab -e
#Add this line to the file 
0 11 * * * apt autoclean
# Save and close
Above cleans your downloaded apt packages every day at 11. You could also do that on reboot by adding following line to file instead.

Code: Select all

@reboot apt autoclean
There might be some more systemd way to do it but I can't give any advice on that. More about cron and crontab here: https://debian-handbook.info/browse/sta ... n-atd.html
Debian 10.2 Stable with i3
Secure your stuff: Securing Debian Manual
Don't break your stuff: Source List Management DontBreakDebian

User avatar
pylkko
Posts: 1802
Joined: 2014-11-06 19:02

Re: Apt (or something) eating up all HD space every day

#14 Post by pylkko »

Actually, the service that people told this guy to turn off runs a shell script located at /usr/lib/apt/ apt.systemd.daily. It has an option CleanInterval where you can set apt clean to occur in intervals of certain days. There is also an equivalen apt autoclean interval setting

So an even easier solution would be to turn it back on and configure it properly.

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: Apt (or something) eating up all HD space every day

#15 Post by Dai_trying »

pylkko wrote:So an even easier solution would be to turn it back on and configure it properly.
+1

User avatar
debiman
Posts: 3063
Joined: 2013-03-12 07:18

Re: Apt (or something) eating up all HD space every day

#16 Post by debiman »

^ of course.
but, assuming it performs familiar apt tasks, i wonder how it can "fill up all hd space"?
simply checking for available updates, that shouldn't take any hd space at all?

i just checked.

Code: Select all

cd /var/cache/apt
watch du -h &
watch ls -al
around 400MB altogether, most of it in archives subfolder.

now in another terminal

Code: Select all

apt update
apt upgrade
during every update, the /var/cache/apt/*.bin files get recreated.
during upgrade, there are some changes in archives - the value first goes down significantly, then goes up again, and because there were some updates this time, settles on a slightly higher value (first 412MB, now 424MB).

kurros, you might want to repeat this troubleshooting.
how full is the partition?

PS:
my wording was a little unfortunate; i am NOT the OP, i just tried to get a better view on the situation by performing similar tasks on my own machine.
in the end we still need to know what the apt-daily scripts actually do.
Last edited by debiman on 2017-11-09 18:31, edited 1 time in total.

User avatar
pylkko
Posts: 1802
Joined: 2014-11-06 19:02

Re: Apt (or something) eating up all HD space every day

#17 Post by pylkko »

I don't think that it can easily fill up an entire empty disk. However, if you are running a system with a partition too full to begin with then it is different. Also, it much depends on whether or not you are running a standard Debian stable install or not, as the packages do not change much (only, often very small security patches). However, as we well know, more than a few users out there add all kinds of third party repositories and try to use testing as a rolling release etc. The downloaded packages cache size will also vary depending on how many packages you are using. So I can imagine other situations where it might be more severe.

Post Reply