nonrecursive makeﬁle for clastool - atlas.fis.utfsm.clatlas.fis.utfsm.cl/atlas/clastool.pdf ·...

UNIVERSIDAD TECNICA FEDERICO SANTA MARIADEPARTAMENTO DE F ISICA

UTFSM ATLAS GRID TEAM

Nonrecursive Makefile for ClasTool

Sebastian Mancilla [email protected]

Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 1 / 20

Makefiles and make command

make is a utility intended to automate and optimize the construction ofprograms.

A descriptor file named Makefile describes the relationship amongsource files and provides commands for updating each file.

make invokes Makefile to automatically rebuild a programwhenever one of the source files is modified.

It only recompiles the files that were affected by changes, thussaving compiling time.

This helps reduce the likelihood of human errors when makingentries from the command line.

Makefiles and make commandExample

The executable project1 is built from the following sources files:

project||_ main.c|_ data.h|_ data.c|_ io.h|_ io.c

The source main.c includes data.h and io.h.

The dependencies between files are shown in the following tree:

And this is a simple Makefile to build the executable:

project1: main.o data.o io.ogcc main.o data.o io.o -o project1

main.o: data.h io.h main.cgcc -c main.c

data.o: data.c data.hgcc -c data.c

io.o: io.h io.ccc -c io.c

make runs the rule for first target project1 and figures itsdependencies on data.o, main.o and io.o.

make next checks if any of the three object files are listed as targets. Ifyes, it runs the rule for first prerequisite, that is, main.o, to find itsdependencies.

make checks whether the prerequisites of main.o have furtherdependencies. If not, as in the example, it checks if main.o is up todate. If not, it runs the command for main.o by compiling main.c to getthe object file.

make looks at the targets data.o and io.o and compiles these objectfiles in a similar fashion.

make returns to first target project1. Since it now has all up-to-dateobject files for the rule, it executes the command to build project1.

Recursive and nonrecursive Makefiles

Traditionally the software development projects use a hierarchy of directoriescontaining source files for the modules which make up the project.

In the recursive make method each of the sub-directories contains aMakefile which describes the rules and instructions for the makeprogram.The complete project build is done by arranging for the top-levelMakefile to change directory into each of the sub-directories andrecursively invoke make.

A nonrecursive make uses one single top-level Makefile to build theproject and one make include file for each sub-directory containing filelists and module-specific rules.

Recursive and nonrecursive Makefiles

Traditionally the software development projects use a hierarchy of directoriescontaining source files for the modules which make up the project.

In the recursive make method each of the sub-directories contains aMakefile which describes the rules and instructions for the makeprogram.The complete project build is done by arranging for the top-levelMakefile to change directory into each of the sub-directories andrecursively invoke make.

A nonrecursive make uses one single top-level Makefile to build theproject and one make include file for each sub-directory containing filelists and module-specific rules.

Recursive Makefile issues

Recursive makefiles are bad primarily because they partition thedependency tree into several trees.

This prevents dependencies between make instances from beingexpressed correctly.

Also causes (parts of) the dependency tree to be recalculatedmultiple times, which is a performance issue in the end.

This affects both phases of the operation of make:

It causes make to construct an inaccurate dependency tree.

It forces make to traverse the tree in an inappropriate order.

Simple example of dependency problem

Recursive Makefile issues

Other problems include:

Recursive makes which take forever to work out that they need todo nothing.

Recursive makes which do too much, or too little.

Recursive makes which are overly sensitive to changes in thesource code and require constant Makefile intervention to keepthem working.

It is very hard to get the order of the recursion into thesub-directories correct.

It is often necessary to do more than one pass over thesub-directories to build the whole system.

Using nonrecursive Makefiles

The use of a whole project make is not as difficult to put intopractice as it may at first appear.

It solves the problems found in the recursive make approach.

It handles dependencies correctly. Treating the whole tree as asingle entity is really the right way.Specific targets can be built (say a particular program) correctly,since make always creates a full dependency graph.It is faster.

It requires a paradigm shift for developers used to recursive make.

The ROOT Makefile itself was structured as described in thepaper Recursive Make Considered Harmful.

The main philosophy is that it is better to have a single largeMakefile describing the entire project than many small Makefiles,one for each sub-project, that are recursively called from the mainMakefile. By cleverly using the include mechanism the singleMakefile solution is as modular as the recursive approach withoutthe problems of incomplete dependency graphs.

The single Makefile is fast.About 1 second to check if anything needs to be recompiled.

My approach for ClasTool

I just took the old ClasTool Makefile and turned it into anonrecursive Makefile.

My purpose was to eliminate the problems associated with using arecursive Makefile.

Because that is not the right way.

All the dictionary, objects and dependency files are created insideof their source directory, so in the new approach this needs aseries of specific rules for every directory.

The automatic dependency generation was inefficient, and with acouple of issues.

Implementation

Three main files in the project root directory:

Makefile system: it has all the system-dependent definitions(almost what Makefile top used to be).

Makefile: the project’s nonrecursive Makefile.

Makefile templates: it has all the templates to build thespecific rules for every directory, and the macros used by themodule.mk files.

A module.mk file in each directory for building the libraries.

All this files are included in the main Makefile.

All share the same structure.

Implementation

Each module.mk file has the following lines:local_class := $(wildcard $(module)/*.cc)local_files :=lib_name := $(module)lib_deps :=extra_libs :=extra_flags :=

$(eval $(call make-module,$(module),$(lib_name),$(lib_deps)))$(eval $(call make-libs,$(module),$(lib_name),$(lib_deps), \

$(local_class),$(local_files), \$(extra_libs)))

$(eval $(call make-objs,$(module),$(extra_flags)))

For every directory the only thing that needs to be changed is thevalues of the variables.

Implementation

Each module.mk file has the following lines:local_class := $(wildcard $(module)/*.cc)local_files :=lib_name := $(module)lib_deps :=extra_libs :=extra_flags :=

Implementation

Each module.mk file has the following lines:local_class := $(wildcard $(module)/*.cc)local_files :=lib_name := $(module)lib_deps := ClasBanks VirtualReaderextra_libs := -L$(ROOTLIBDIR) -lPhysics -lTreeextra_flags :=

Implementation

The macros make-module, make-libs and make-objs aredefined in Makefile templates. Example of a macro definition

By using the eval function, all the necessary rules are autogenerated by make itself according to the above macros.

If some rules must be changed, this is done in one place:Makefile templates.

Auto-dependency generation is significantly improved using theadvanced method invented by Tom Tromey1.

1http://make.paulandlesley.org/autodep.htmlSebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 16 / 20

Results

I hope all the work be useful.

As ClasTool, it is not finished, and it could be improved.

But right now it works very good.

Results

I hope all the work be useful.

As ClasTool, it is not finished, and it could be improved.

But right now it works very good.

References

Recursive Make Considered Harmful, Peter Miller, 1997http://miller.emu.id.au/pmiller/books/rmch/

Managing Projects with GNU make, 3rd Edition, RobertMecklenburg, 2004http://www.makelinux.net/make3/main.html

My Git repository (here you can find the sources)http://github.com/smancill/ClasTool_Makefile

Nonrecursive Makefile dependencies problem

Suppose make is called first for ModuleA and then for ModuleB.

What happens if we modified file C2 and we call make command?

make-libs macrodefine make-libsSOURCES_DEPS += $(call dep-list,$(1),$(4) $(5))SOURCES_DICTS += $(call dict-list,$(1),$(4))

$(SLIBDIR)/lib$(2).$(DllSuf): $(call obj-list,$(1),$(4) $(5)) \$(call dict-obj-list,$(1),$(4))

@echo CREATING SHARED LIBRARY $$@ifeq (,$(7))

$(LD) $(SOFLAGS) $(LDFLAGS) -o $$@ \$$ˆ $(6) $(if $3,-L$(SLIBDIR) $(addprefix -l,$3))

else$(7) $(SOFLAGS) $(LDFLAGS) -o $$@ \$$ˆ $(6) $(if $3,-L$(SLIBDIR) $(addprefix -l,$3))

endif$(POST_LINK_COMMAND)

$(LIBDIR)/lib$(2).a: $(call obj-list,$(1),$(4) $(5)) \$(call dict-obj-list,$(1),$(4))

@echo CREATING LIBRARY $$@ar r $$@ $$ˆ

nonrecursive makeﬁle for clastool - atlas.fis.utfsm.clatlas.fis.utfsm.cl/atlas/clastool.pdf ·...

Documents