Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Is there a program/script that could replace my build scripts

Programming languages, Coding, Executables, Package Creation, and Scripting.
Message
Author
larienna
Posts: 106
Joined: 2014-09-27 20:54

Is there a program/script that could replace my build scripts

#1 Post by larienna »

I am currently using Codeblocks and Cmake for plain C programming. Code blocks is used to build the debugging application for common development, and Cmake is used for more official builds, like multiplatform and installation for libraries mostly ran during CI.

My problem is that I end up in every project multiple simple script that does various tasks and as more time passes, the more script gets added. Just to give you a rough idea, this is an inventory of the scripts I have in almost every project:
  • build the software
    build for win 32 (used by CI)
    builld for win 64 (Used by CI)
    install the software/library (normally requires sudo, this is why it's a separate script)
    Build on a tmpfs (if I am using an SSD)
    Debug with GDB
    Run the program for memory check with valgrind
    Run unit tests
    Run examples
    Run demonstration program
In order to make things more convenient, I though of making 1 script to rule them all. But they say you should keep your bash script simple and short. So I was wondering, if there is another alternative to bash script. Maybe there is a tool, script or program somewhere that actually did the same job I intended to do before launching my self in the creation of the bash script.

Any recommendations?

LE_746F6D617A7A69
Posts: 932
Joined: 2020-05-03 14:16
Has thanked: 7 times
Been thanked: 65 times

Re: Is there a program/script that could replace my build scripts

#2 Post by LE_746F6D617A7A69 »

larienna wrote: 2022-09-18 20:05 (..) Maybe there is a tool, script or program somewhere that actually did the same job I intended to do before launching my self in the creation of the bash script.

Any recommendations?
GNU autotools.

Documentation:
https://www.gnu.org/software/autoconf/
https://www.gnu.org/software/automake/

Basic tutorial:
https://opensource.com/article/19/7/int ... -autotools

GNU Autotools are mainly a framework for creating build scripts, but it contains all the components needed to implement unit testing, debugging, OS-dependent source code modifications, etc - in fact, that enviroment can be easily extended in any way, so there is practically no limits on usage.
Therefore, millions projects are using autotools, including CodeBlocks itself.

However, there are 2 potential problems:
1. All the POSIX compatible platforms are supported directly - but in case of Winblows, autotools are working only trough MinGW/MSYS compatibiliy layer (TBH, I don'n know the current situation, because I've dropped the Winblows platform many years ago)
2. Autotools are hard to learn (because this is a complex framework) - but as soon as You create Your 1st working project, You'll never look back - autotools are limitless ;)

Regards.
Bill Gates: "(...) In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating system."
The_full_story and Nothing_have_changed

larienna
Posts: 106
Joined: 2014-09-27 20:54

Re: Is there a program/script that could replace my build scripts

#3 Post by larienna »

I thought CMAKE replaced Autotools?

Many of the stuff can be made with CMAKE, but one problem with using cmake directly is where the files get generated, not everything is at the root. So doing CMAKE and MAKE commands by hand is annoying because there are various folders I need to specify.

If I ignore the various directories to specify, Some task requires to run:
cmake + make
make install + run
tool + run

And various other combinations. So I can do it via command line, but it's long and annoying. Maybe a bash script is really the best option since I am looking for a shortcut to do something that can already be done via command line.

LE_746F6D617A7A69
Posts: 932
Joined: 2020-05-03 14:16
Has thanked: 7 times
Been thanked: 65 times

Re: Is there a program/script that could replace my build scripts

#4 Post by LE_746F6D617A7A69 »

larienna wrote: 2022-09-21 01:51 I thought CMAKE replaced Autotools?
Who said that? ;)
larienna wrote: 2022-09-21 01:51 Many of the stuff can be made with CMAKE, but one problem with using cmake directly is where the files get generated, not everything is at the root (..)
That's one of the reasons why I've mentioned Autotools.
But foremost, I've recommended Autotools, because You've asked about a "script to rule them all" -> the most fundamental difference between CMAKE and Autotools is that to build CMAKE project You need CMAKE, while to build Autotools project, You need just the shell.

The autoconf utility "compiles" configure.ac into a *portable shell script* that is, the "configure" script, which is exactly the script to rule them all ;)

Regards.
Bill Gates: "(...) In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating system."
The_full_story and Nothing_have_changed

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: Is there a program/script that could replace my build scripts

#5 Post by Head_on_a_Stick »

Alternative view of Autotools: http://www.cs.cmu.edu/afs/club.cc.cmu.e ... mful2.html

Disclaimer: I'm no programmer.
larienna wrote:if there is another alternative to bash script
Debian's /bin/sh (dash) is lighter, faster and less buggy than bash. All Debian system scripts have a /bin/sh shebang for those reasons.
deadbang

LE_746F6D617A7A69
Posts: 932
Joined: 2020-05-03 14:16
Has thanked: 7 times
Been thanked: 65 times

Re: Is there a program/script that could replace my build scripts

#6 Post by LE_746F6D617A7A69 »

Head_on_a_Stick wrote: 2022-09-21 21:03 Alternative view of Autotools: http://www.cs.cmu.edu/afs/club.cc.cmu.e ... mful2.html
This brings me to my first criticism of autotools: it is too complex. To understand and debug a project that uses autotools, you must understand Bourne shell scripting
Right, but the "problem" is that autoconf is not generating bash scripts (i.e. no bashism), but a POSIX shell scripts :lol: - don't believe every idiot on the internet ;)
Bill Gates: "(...) In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating system."
The_full_story and Nothing_have_changed

jmgibson1981
Posts: 295
Joined: 2015-06-07 14:38
Has thanked: 11 times
Been thanked: 32 times

Re: Is there a program/script that could replace my build scripts

#7 Post by jmgibson1981 »

But they say you should keep your bash script simple and short.
Yes they do say this. But no one tells you what to do and not to do with your own machine. If you want to build a 5000 line script go nuts. Probably a better way but it can be done. You could also split it in to functions and have each in it's own file that is sourced by the main script when it runs. All kinds of ways to skin this cat.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: Is there a program/script that could replace my build scripts

#8 Post by Head_on_a_Stick »

LE_746F6D617A7A69 wrote: 2022-09-21 21:28 but the "problem" is that autoconf is not generating bash scripts (i.e. no bashism), but a POSIX shell scripts
POSIX sh is a Bourne shell.
jmgibson1981 wrote: 2022-09-22 00:36 If you want to build a 5000 line script go nuts.
The advice against using bash is because it's memory-intensive, slow and very buggy with a metric sh*t-tonne of edge cases and special rules. There's even a site to check for problems because it's so difficult to code cleanly using bash.

If a shell script grows beyond a few hundred lines it should probably be re-written in a better language. Perl is the obvious choice for system scripts.
deadbang

larienna
Posts: 106
Joined: 2014-09-27 20:54

Re: Is there a program/script that could replace my build scripts

#9 Post by larienna »

Thank for the bash script checker.

I think the main difficulty in finding a replacement for bash is:

1. Dependencies: If I make a script in let say Python, I need to install Python.
2. Program launching: The language must have the capacity to easily launch programs with arguments.

Sure, I could do it in C, but I`Il need to do some forking and calling execve(). Not really convenient.

If perl can launch programs easily and is installed by default on linux systems, then I guess I could give it a try.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: Is there a program/script that could replace my build scripts

#10 Post by Head_on_a_Stick »

If the script is mostly just calling other utilities then use POSIX sh (/bin/sh in Debian), the script will be lighter, faster and less buggy than if you had used bash.

If the script has lots of loops and other bash features then re-writing in Perl will bring a speed improvement and the handling of arrays and suchlike is not completely brain-dead, unlike with bash. Perl can launch other programs: https://perlmaven.com/running-external- ... -from-perl
deadbang

larienna
Posts: 106
Joined: 2014-09-27 20:54

Re: Is there a program/script that could replace my build scripts

#11 Post by larienna »

I was wondering, can I run make inside a make file?

Can I have in a make file, with a bin file as target, where the steps to generate the bin are made by running: cmake and make.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: Is there a program/script that could replace my build scripts

#12 Post by Head_on_a_Stick »

larienna wrote: 2022-09-22 13:36 can I run make inside a make file?
Apparently: https://stackoverflow.com/questions/220 ... r-makefile
deadbang

larienna
Posts: 106
Joined: 2014-09-27 20:54

Re: Is there a program/script that could replace my build scripts

#13 Post by larienna »

I am trying to find differences between bash and sh at the scripting level, because there is a lot of additional features when using the terminal. I could not find that much info.

Right now, speed is not much an issue so I don't think I have any reason to downgrade to sh unless there is way more risk of making mistakes with bash. I still like the idea of using perl, I'Il consider that option.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: Is there a program/script that could replace my build scripts

#14 Post by Head_on_a_Stick »

larienna wrote: 2022-09-22 13:47 I am trying to find differences between bash and sh at the scripting level
https://mywiki.wooledge.org/Bashism

Debian has the checkbashisms package and shellcheck.net will also pick them up.

Resources:

https://drewdevault.com/2018/02/05/Intr ... shell.html

https://pubs.opengroup.org/onlinepubs/9 ... hap02.html

https://github.com/dylanaraps/pure-sh-bible

https://www.grymoire.com/Unix/Sh.html

https://www.etalabs.net/sh_tricks.html
deadbang

LE_746F6D617A7A69
Posts: 932
Joined: 2020-05-03 14:16
Has thanked: 7 times
Been thanked: 65 times

Re: Is there a program/script that could replace my build scripts

#15 Post by LE_746F6D617A7A69 »

Head_on_a_Stick wrote: 2022-09-22 07:00
LE_746F6D617A7A69 wrote: 2022-09-21 21:28 but the "problem" is that autoconf is not generating bash scripts (i.e. no bashism), but a POSIX shell scripts
POSIX sh is a Bourne shell.
Nah, the author claims that the "main problem" with autotools is that autoconf uses Bourne shell scripts. The point was that the POSIX shell scripting language is the simplest language under the Sun - it was designed to be used by regular end users of the system. Now a "professional" programmer claims that autotools are hard to debug because it is based on that simple language (what if it would be based on a bash?).
He also claims that simple M4 macros are "an extremely low level of abstraction" (exact quoting). Using this logic, the C language must be at the microcode abstraction level :lol:

Anyway, the CMAKE project has been started as an "easy way to get a build system" -> the reality have quickly revised that assumption -> today cmake project is basically replicating the functionality of autotools, but without any confirmed benefits.
I dare to say that CMAKE is an effect of the SNS syndrome -> learning how to use existing but complex solutions is boring -> let's create a new one -> who cares that it is incomplete? -> it's new!

(of course CMAKE project is not new today, but still, it tries to implement functionalities that the autotools had 20+ years ago ;) )
Bill Gates: "(...) In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating system."
The_full_story and Nothing_have_changed

linuxCowboy
Posts: 15
Joined: 2022-09-22 09:02
Been thanked: 1 time

Re: Is there a program/script that could replace my build scripts

#16 Post by linuxCowboy »

Take a look at the Meson Build system.

It impressed me very much. It is simple and logical like perl and git was. But extremly extensible.

[rant about autotools-mess kicked]

And my meson.build has a few lines and thats it!

LE_746F6D617A7A69
Posts: 932
Joined: 2020-05-03 14:16
Has thanked: 7 times
Been thanked: 65 times

Re: Is there a program/script that could replace my build scripts

#17 Post by LE_746F6D617A7A69 »

linuxCowboy wrote: 2022-09-22 21:42 Take a look at the Meson Build system.

It impressed me very much. It is simple and logical like perl and git was. But extremly extensible.
(...)
And my meson.build has a few lines and thats it!
Simple build system == crippled build system.
Build systems *have* to be complex to be able to handle complex scenarios during build process.

Meson is good for simple projects with trivial dependencies. In cases when the project consists of sub-projects that have to be build with different compilers/flags or when the build process must be modified by the results of custom tests or when there are complex conditional dependencies to solve meson is *completely* useless. This also mostly applies to cmake, but there are workarounds.

The funny thing is, that autotools handles such scenarios (and many more non-typical cases) natively, for over 3 decades ;)
Bill Gates: "(...) In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating system."
The_full_story and Nothing_have_changed

larienna
Posts: 106
Joined: 2014-09-27 20:54

Re: Is there a program/script that could replace my build scripts

#18 Post by larienna »

I'll take a look at Messon, but right now, I do not dislike CMAKE besides maybe that it creates a makefile that does the job for you instead of doing it himself. So It it has an intermediary step. One thing for sure that I like to have is control, because if there is a problem, you can fix it. When programming in Java, I had to deal with Gradle. And even today, I consider that there is black magic involved somewhere in the build process. When it does not work, good luck finding a solution.

I quickly took a look at Perl, and it could be something I would consider. The last time I programmed in perl was to create cgi-bin 25+ years ago, before PHP got created (I think). I am not sure if I still have my perl book. It was not that very convenient for web development, but it could be easy to use for scripting. So I think I'll give it a try.

larienna
Posts: 106
Joined: 2014-09-27 20:54

Re: Is there a program/script that could replace my build scripts

#19 Post by larienna »

I took a quick look at Meson reviews. Most of the criticism of Meson, is the lack of flexibility therefore the lost of control. It probably follows a convention over configuration paradigm. But it seems easier to use unless you are doing something obscure and unconventional. The readability is apparently better.

As for CMAKE, there is apparently a new way (v.3+) and an old way (v.2) of doing things. I am currently using v.3 in my project. Another annoying thing in cmake is the need to specify each .c file manually. I also need a special script to find the allegro and sqlite library; a Script that I don't understand much. I also need special config for mingw32 and mingw64 cross compilation building.

Meson seems to be willing to make all those things easier to deal with. I guess it depends more on the complexity of the project, and your personal preferences.

Maybe I should indeed take a look at meson. I could try it on a small project of mine and see how I can make it work.

linuxCowboy
Posts: 15
Joined: 2022-09-22 09:02
Been thanked: 1 time

Re: Is there a program/script that could replace my build scripts

#20 Post by linuxCowboy »

Simple build system == crippled build system.
Build systems *have* to be complex to be able to handle complex scenarios during build process.
It should be "Make easy things easy and hard things possible." Often is neither, but Meson, like Perl, have it!
Meson is good for simple projects with trivial dependencies.
Wikipedia:
"GNOME has made it a goal to port its projects to Meson. As of late 2017, GNOME Shell itself exclusively requires Meson after abandoning Autotools, and central components like GTK+, Clutter-GTK, GLib and GStreamer can be built with Meson. Systemd relies on Meson since dropping Autotools in version 234. Also X.Org and Mesa were ported to Meson."

Yes, very simple projects. :wink:

Post Reply