Re: No unanswered question

On Jul 8, 2:48 pm, "Alf P. Steinbach /Usenet" <alf.p.steinbach
+use...@gmail.com> wrote:


At great risk of being offtopic (though I would argue it's quite
important to using the C++ language, and thus on topic), here's a fun
question for you. (Well, a series of questions.) Why is there no
incrementally correct build system, or easy to use incrementally
correct build framework, for C, C++, and Java? As far as I can tell,
all publicly available solutions fail basic incremental correctness
tests.

Let me further define the problem.

A build is the act of following a set of steps, a plan, a process, of
creating executable "files" from source "files".

Let me try to define incremental. A user has a codebase, a local view
of source control, a bunch of source files on disk. the source
includes the build script files as well, such as the makefiles. The
user does a full clean build. The user then makes some changes to the
source, such as adding, removing, or editing source files (which
include build script files). The user then does another build, called
an incremental build, which selectively skips (some) portions of the
full build which are unnecessary because they would produce equivalent
output as the already existing files. This partial build is called an
incremental build.

A correct incremental build is an incremental build which produces
output files equivalent to what a full clean build would produce. An
incremental build process, or incremental build system, is
(incrementally) correct if it can only produce correct incremental
builds, that is, if it will always produce output equivalent to a full
clean build.

An incremental build can be done by hand. A build system is just a
build process, a plan to do a build, a set of actionable items to do a
build. The dependency analysis can be done manually. However, such
analysis tends to take longer than just a full clean build, and
mistakes can be made the human doing the analysis, so it's not really
a correct build system either. Thus any correct incremental build must
automate the tracking of dependencies.

Under that definition, all the build systems and build frameworks
known to me are not incrementally correct, to varying degrees.

The common GNU Make solution described in Recursive Make Considered
Harmful for C and C++ is the closest, but still misses out on corner
cases, including:
1- Removing a new C++ source file when using wildcards will not result
in a relink of its library or executable.
2- Adding a new include file which "hides" another include file on an
include path will not result in a recompile of all corresponding
object files.
3- A change to a makefile itself won't always be caught, such as a
change to a command line preprocessor maco definition.
4- Other various changes to the makefiles which might inadvertently
break incremental correctness.

One might argue that 3, and to a larger extent 4, are outside the
scope of a build system. I disagree with that assessment. When I'm
working on my company's large codebase, and I do a sync to changelist,
this includes changing makefiles which I know nothing about. I want
the build system to correctly incrementally build affected files
without requiring a full clean build. However, with the common GNU
Make solution described in Recursive Make Considered Harmful, this
will not happen; the build can be incorrect.

I'll skip the in depth discussion of other various publicly available
build systems, but as far as I can tell, they are all incorrect under
this set of criteria.

So, my questions were, is there some build system for C, C++, Java,
and extensible to other sane programming languages, which is
incrementally correct, which I somehow glossed over?

Why is there no incrementally correct build system out there? I would
argue that incremental builds represent the most effective approach to
decreasing build times. If your build is taking too long, you can
throw hardware at it, parallelization at it, distributed systems on a
grid at it, throw better faster compilers at it, etc., but all of
these approaches are just taking off a coefficient of the build.
Incremental builds tend to result in build times which are
proportional to the size of the change, not the size of the code base,
which makes them asymptotically much faster than any other possible
change to the build. (pimp is an interesting exception. pimpl does
improve full clean build times by more than just some coefficient by
reducing the size of the effective source given the compiler. However,
pimpl plays with incremental by decreasing the number of dependencies
of the build, thereby improving an incremental build.)

Finally, if I manage to get my company's legal to let me open source
my own little build tool which I've been writing myself in my spare
time, what would I license it under? (I was leaning GNU GPL.) Where
would I put it up to get people to actually consider using it?

Perhaps more generally, who should I bug about the huge shortcoming in
all modern build technology?