Reducing Footprint: One Command to Greatly Abet (Fixed)

If it doesn't relate to Debian, but you still want to share it, please do it here

Reducing Footprint: One Command to Greatly Abet (Fixed)

Postby thmtrxhsu » 2020-04-11 05:02

The end goal is reduce the footprint that apps and their dependencies use. To that end, we need to know the sizes of packages and their dependencies. Unfortunately as any experienced Linux user will tell you, there is no safe way to automatically remove dependencies i.e., through a script. Inevitably if you do this you will break something. So you have to go about manually. Henceforth the command to inform the user of what is taking up disk space because mere package size is insufficient. I had scoured the web, and such a command did not exist. So I made it myself. You require the aptitude package. I should add, these commands are for X64 systems, you need to modify if you use a 32 bit system.

Forward Dependencies (Use to identify large libraries you don't need)
Code: Select all
for z in $(dpkg -l | awk '/^[hi]i/{print $2}' | grep -E '^lib'); do \
printf "\n$z:" && \
aptitude show $z | grep -E 'Uncompressed Size' && \
printf "\n" && \
apt show 2>/dev/null $(aptitude search '!~i?reverse-depends("^'$z'$")' -F "%p" | \
sed 's/:i386$//') | grep -E 'Package|Installed-Size' | sed '/APT/d;s/^.*Package:/\t&/;N;s/\n/ /' &>> DistroAnalysis.txt; done


Reverse Dependencies (use this to reduce footprint):
Code: Select all
for z in $(dpkg -l | awk '/^[hi]i/{print $2}' | grep -v '^lib'); do \
pkg_size=$(apt show 2>/dev/null $z| grep -E 'Installed-Size:' |  sed "s/Installed-Size://;s/ kB/K/g;s/ mB/M/g")
let desc_resize=207-${#z}
description=$(apt show 2>/dev/null $z | grep -A3 -E 'Description' | sed "s/Description://;:a;N;$!ba;s/\n/ /g" | tr -d '\n' | colrm $desc_resize )
package=$(printf "\n$z$pkg_size D: $description\n")
rdependslist=$(apt-rdepends 2>/dev/null $z | grep -v "Depends" | sed -e "s/$z//g")
rdepends=$(apt show 2>/dev/null $rdependslist | grep -E "Package:|Installed-Size" | sed -e "s/Package: //g;/APT/d;s/Installed-Size: //;s/ kB/K/g;s/ MB/M/g")
echo $package &>> DistroAnalysis.txt
if [[ (${#rdepends} -le 700) ]]; then echo $rdepends &>> DistroAnalysis.txt; else :; fi
printf "\n" &>> DistroAnalysis.txt; done


Requires: apt-rdepends packages.
That last script took a lot of work. Let me interpret for you starting from the for loop. Input list of all packages, exclude some useless text, exclude libraries. Print a new line and the package name. Count the characters in the package name. To fit the description on two lines in geany with monospace regular 8 font, the maximum characters over two lines is 206, but the program colrm you need to add + 1 for some reason. Apt show without error, grep the package size and format it nicely and store into variable package_size. Now grab the description without error and put into one continuous stream of characters, then cut to resize with colrm; store into variable description. N.B. the -A3 means the first 3 lines of description, which is why the description is in a separate function then the former; because you can't grep -e 'x' -A3 -e 'x' that is illegal and if you A3 both that opens up another can of worms. Now print variables package_size and description. Get rdepends list and store it into result. Look the name and size of each rdepends stored in result, and store this in final. If rdpends is not ridiculously long i.e., whole OS, then and only then output it; name and size, otherwise just output main package, size and description. &>> Output without fail; freaking work.
Last edited by thmtrxhsu on 2020-04-23 12:40, edited 9 times in total.
thmtrxhsu
 
Posts: 18
Joined: 2020-04-09 13:46

Re: Reducing Footprint: One Command to Greatly Abet

Postby stevepusser » 2020-04-11 07:55

Since Debian installs recommended packages by default, and a program may or may not work well without those extra packages for users, how are you going to account for those?
The MX Linux repositories: Backports galore! If we don't have something, just ask and we'll try--we like challenges. New packages: Kodi 18.7, Strawberry 0.6.12, Pale Moon 28.10.0, Xscreensaver 5.44, Krita 4.3.0, Foliate 2.3.0
User avatar
stevepusser
 
Posts: 11772
Joined: 2009-10-06 05:53

Re: Reducing Footprint: One Command to Greatly Abet

Postby Head_on_a_Stick » 2020-04-11 08:13

thmtrxhsu wrote:the command to inform the user of what is taking up disk space

Yes, that's a nice script but:
Code: Select all
empty@E485 ~ % dpkg -l|grep -c "^ii"
2691
empty@E485 ~ %

It gives me several thousand lines of (alphabetically listed) output to scan through... :roll:

thmtrxhsu wrote:compare the services that start

This command is useless for that:
Code: Select all
sudo service --status-all

Many services no longer provide sysvinit scripts and not all that do will provide useful output for the --status-all option.

And what about distributions that don't use APT or sysvinit at all?

EDIT: off-topic for this thread but I'm curious:
stevepusser wrote:Since Debian installs recommended packages by default, and a program may or may not work well without those extra packages for users

Then why do MX disable recommended packages by default? Seems like a strange decision.
User avatar
Head_on_a_Stick
 
Posts: 12132
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: Reducing Footprint: One Command to Greatly Abet

Postby stevepusser » 2020-04-12 00:45

It is a carryover from antiX's goal of making the distro as sleek as possible, and many of the recommended packages are just bloat--we have to look at each package individually to see if it suffers from "recommends abuse". I do switch many recommended packages to dependencies for my MX ports if they seem to be really useful.
The MX Linux repositories: Backports galore! If we don't have something, just ask and we'll try--we like challenges. New packages: Kodi 18.7, Strawberry 0.6.12, Pale Moon 28.10.0, Xscreensaver 5.44, Krita 4.3.0, Foliate 2.3.0
User avatar
stevepusser
 
Posts: 11772
Joined: 2009-10-06 05:53

Re: Reducing Footprint: One Command to Greatly Abet

Postby thmtrxhsu » 2020-04-13 19:09

stevepusser wrote:Since Debian installs recommended packages by default, and a program may or may not work well without those extra packages for users, how are you going to account for those?


Those extra packages Steve are runtime dependencies aka reverse dependencies aka rdepends.
My original post was flawed in that I erroneously created a script to figure out the forward dependencies aka depends, but this is useless vis-a-vis reducing footprint; in other words the "troublemakers" are those exclusive dependencies which is code for for rdepends.

I have updated the rdepends script and post.
thmtrxhsu
 
Posts: 18
Joined: 2020-04-09 13:46

Re: Reducing Footprint: One Command to Greatly Abet

Postby thmtrxhsu » 2020-04-13 19:32

@stevepusser

What do you suggest then?
Last edited by thmtrxhsu on 2020-04-14 19:10, edited 1 time in total.
thmtrxhsu
 
Posts: 18
Joined: 2020-04-09 13:46

Re: Reducing Footprint: One Command to Greatly Abet

Postby Head_on_a_Stick » 2020-04-14 11:43

For systemd:
Code: Select all
systemctl list-unit-files --state=enabled --no-pager

For OpenRC:
Code: Select all
rc-status --servicelist

For s6:
Code: Select all
s6-svstat

For runit:
Code: Select all
sv status /path/to/service/files

Not sure about the rest though.

EDIT: and please don't full-quote unnecessarily, it degrades the readability of the thread.
User avatar
Head_on_a_Stick
 
Posts: 12132
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: Reducing Footprint: One Command to Greatly Abet (Fixed)

Postby thmtrxhsu » 2020-04-14 19:11

Thanks and I complete agree.
thmtrxhsu
 
Posts: 18
Joined: 2020-04-09 13:46

Re: Reducing Footprint: One Command to Greatly Abet (Fixed)

Postby stevepusser » 2020-04-15 04:45

There is a difference between runtime dependencies and Recommends.

Suggest you run

Code: Select all
apt show inkscape


to see some. Inkscape will still run without the recommends, but some features will not work. Inkscape will not run at all without the runtime dependencies, most of which are generated automatically by dh_shlibdeps during the package build.
The MX Linux repositories: Backports galore! If we don't have something, just ask and we'll try--we like challenges. New packages: Kodi 18.7, Strawberry 0.6.12, Pale Moon 28.10.0, Xscreensaver 5.44, Krita 4.3.0, Foliate 2.3.0
User avatar
stevepusser
 
Posts: 11772
Joined: 2009-10-06 05:53

Re: Reducing Footprint: One Command to Greatly Abet (Fixed)

Postby thmtrxhsu » 2020-04-20 21:26

@ stevepusser

You can interchange these words: "runtime dependencies" "rdepends" "reverse dependencies". Originally, I created the wrong script (for forward dependencies aka depends), but you should see I updated the post to include the rdpends version which works flawlessly. Debian has a flexible point of view hence the terminology. We would probably prefer treating dependency as meaning reverse dependency but that is simply because our perspective and understanding of dependency is "dependent of"(reverse dependency) vs "a dependent for"(forward dependency).

In any event, I have always finished brewing "MX NoFrills" the ISO which does not exceed 900MB. More details in a separate post.
thmtrxhsu
 
Posts: 18
Joined: 2020-04-09 13:46


Return to Offtopic

Who is online

Users browsing this forum: No registered users and 10 guests

fashionable