Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Reducing Footprint: One Command to Greatly Abet (Fixed)

Off-Topic discussions about science, technology, and non Debian specific topics.
Post Reply
Message
Author
thmtrxhsu
Posts: 18
Joined: 2020-04-09 13:46

Reducing Footprint: One Command to Greatly Abet (Fixed)

#1 Post by thmtrxhsu »

The end goal is reduce the footprint that apps and their dependencies use. To that end, we need to know the sizes of packages and their dependencies. Unfortunately as any experienced Linux user will tell you, there is no safe way to automatically remove dependencies i.e., through a script. Inevitably if you do this you will break something. So you have to go about manually. Henceforth the command to inform the user of what is taking up disk space because mere package size is insufficient. I had scoured the web, and such a command did not exist. So I made it myself. You require the aptitude package. I should add, these commands are for X64 systems, you need to modify if you use a 32 bit system.

Forward Dependencies (Use to identify large libraries you don't need)

Code: Select all

for z in $(dpkg -l | awk '/^[hi]i/{print $2}' | grep -E '^lib'); do \
printf "\n$z:" && \
aptitude show $z | grep -E 'Uncompressed Size' && \
printf "\n" && \
apt show 2>/dev/null $(aptitude search '!~i?reverse-depends("^'$z'$")' -F "%p" | \
sed 's/:i386$//') | grep -E 'Package|Installed-Size' | sed '/APT/d;s/^.*Package:/\t&/;N;s/\n/ /' &>> DistroAnalysis.txt; done
Reverse Dependencies (use this to reduce footprint):

Code: Select all

for z in $(dpkg -l | awk '/^[hi]i/{print $2}' | grep -v '^lib'); do \
pkg_size=$(apt show 2>/dev/null $z| grep -E 'Installed-Size:' |  sed "s/Installed-Size://;s/ kB/K/g;s/ mB/M/g")
let desc_resize=207-${#z}
description=$(apt show 2>/dev/null $z | grep -A3 -E 'Description' | sed "s/Description://;:a;N;$!ba;s/\n/ /g" | tr -d '\n' | colrm $desc_resize )
package=$(printf "\n$z$pkg_size D: $description\n")
rdependslist=$(apt-rdepends 2>/dev/null $z | grep -v "Depends" | sed -e "s/$z//g")
rdepends=$(apt show 2>/dev/null $rdependslist | grep -E "Package:|Installed-Size" | sed -e "s/Package: //g;/APT/d;s/Installed-Size: //;s/ kB/K/g;s/ MB/M/g")
echo $package &>> DistroAnalysis.txt
if [[ (${#rdepends} -le 700) ]]; then echo $rdepends &>> DistroAnalysis.txt; else :; fi 
printf "\n" &>> DistroAnalysis.txt; done
Requires: apt-rdepends packages.
That last script took a lot of work. Let me interpret for you starting from the for loop. Input list of all packages, exclude some useless text, exclude libraries. Print a new line and the package name. Count the characters in the package name. To fit the description on two lines in geany with monospace regular 8 font, the maximum characters over two lines is 206, but the program colrm you need to add + 1 for some reason. Apt show without error, grep the package size and format it nicely and store into variable package_size. Now grab the description without error and put into one continuous stream of characters, then cut to resize with colrm; store into variable description. N.B. the -A3 means the first 3 lines of description, which is why the description is in a separate function then the former; because you can't grep -e 'x' -A3 -e 'x' that is illegal and if you A3 both that opens up another can of worms. Now print variables package_size and description. Get rdepends list and store it into result. Look the name and size of each rdepends stored in result, and store this in final. If rdpends is not ridiculously long i.e., whole OS, then and only then output it; name and size, otherwise just output main package, size and description. &>> Output without fail; freaking work.
Last edited by thmtrxhsu on 2020-04-23 12:40, edited 9 times in total.

User avatar
stevepusser
Posts: 12930
Joined: 2009-10-06 05:53
Has thanked: 41 times
Been thanked: 71 times

Re: Reducing Footprint: One Command to Greatly Abet

#2 Post by stevepusser »

Since Debian installs recommended packages by default, and a program may or may not work well without those extra packages for users, how are you going to account for those?
MX Linux packager and developer

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 133 times

Re: Reducing Footprint: One Command to Greatly Abet

#3 Post by Head_on_a_Stick »

thmtrxhsu wrote:the command to inform the user of what is taking up disk space
Yes, that's a nice script but:

Code: Select all

empty@E485 ~ % dpkg -l|grep -c "^ii"
2691
empty@E485 ~ %
It gives me several thousand lines of (alphabetically listed) output to scan through... :roll:
thmtrxhsu wrote:compare the services that start
This command is useless for that:

Code: Select all

sudo service --status-all
Many services no longer provide sysvinit scripts and not all that do will provide useful output for the --status-all option.

And what about distributions that don't use APT or sysvinit at all?

EDIT: off-topic for this thread but I'm curious:
stevepusser wrote:Since Debian installs recommended packages by default, and a program may or may not work well without those extra packages for users
Then why do MX disable recommended packages by default? Seems like a strange decision.
deadbang

User avatar
stevepusser
Posts: 12930
Joined: 2009-10-06 05:53
Has thanked: 41 times
Been thanked: 71 times

Re: Reducing Footprint: One Command to Greatly Abet

#4 Post by stevepusser »

It is a carryover from antiX's goal of making the distro as sleek as possible, and many of the recommended packages are just bloat--we have to look at each package individually to see if it suffers from "recommends abuse". I do switch many recommended packages to dependencies for my MX ports if they seem to be really useful.
MX Linux packager and developer

thmtrxhsu
Posts: 18
Joined: 2020-04-09 13:46

Re: Reducing Footprint: One Command to Greatly Abet

#5 Post by thmtrxhsu »

stevepusser wrote:Since Debian installs recommended packages by default, and a program may or may not work well without those extra packages for users, how are you going to account for those?
Those extra packages Steve are runtime dependencies aka reverse dependencies aka rdepends.
My original post was flawed in that I erroneously created a script to figure out the forward dependencies aka depends, but this is useless vis-a-vis reducing footprint; in other words the "troublemakers" are those exclusive dependencies which is code for for rdepends.

I have updated the rdepends script and post.

thmtrxhsu
Posts: 18
Joined: 2020-04-09 13:46

Re: Reducing Footprint: One Command to Greatly Abet

#6 Post by thmtrxhsu »

@stevepusser

What do you suggest then?
Last edited by thmtrxhsu on 2020-04-14 19:10, edited 1 time in total.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 133 times

Re: Reducing Footprint: One Command to Greatly Abet

#7 Post by Head_on_a_Stick »

For systemd:

Code: Select all

systemctl list-unit-files --state=enabled --no-pager
For OpenRC:

Code: Select all

rc-status --servicelist
For s6:

Code: Select all

s6-svstat
For runit:

Code: Select all

sv status /path/to/service/files
Not sure about the rest though.

EDIT: and please don't full-quote unnecessarily, it degrades the readability of the thread.
deadbang

thmtrxhsu
Posts: 18
Joined: 2020-04-09 13:46

Re: Reducing Footprint: One Command to Greatly Abet (Fixed)

#8 Post by thmtrxhsu »

Thanks and I complete agree.

User avatar
stevepusser
Posts: 12930
Joined: 2009-10-06 05:53
Has thanked: 41 times
Been thanked: 71 times

Re: Reducing Footprint: One Command to Greatly Abet (Fixed)

#9 Post by stevepusser »

There is a difference between runtime dependencies and Recommends.

Suggest you run

Code: Select all

apt show inkscape
to see some. Inkscape will still run without the recommends, but some features will not work. Inkscape will not run at all without the runtime dependencies, most of which are generated automatically by dh_shlibdeps during the package build.
MX Linux packager and developer

thmtrxhsu
Posts: 18
Joined: 2020-04-09 13:46

Re: Reducing Footprint: One Command to Greatly Abet (Fixed)

#10 Post by thmtrxhsu »

@ stevepusser

You can interchange these words: "runtime dependencies" "rdepends" "reverse dependencies". Originally, I created the wrong script (for forward dependencies aka depends), but you should see I updated the post to include the rdpends version which works flawlessly. Debian has a flexible point of view hence the terminology. We would probably prefer treating dependency as meaning reverse dependency but that is simply because our perspective and understanding of dependency is "dependent of"(reverse dependency) vs "a dependent for"(forward dependency).

In any event, I have always finished brewing "MX NoFrills" the ISO which does not exceed 900MB. More details in a separate post.

Post Reply