The GNU build systemESA 2008/2009

Eelco Schatborneelco@os3.nl

18 september 2008

1/ 31

Agenda

Today

I A bit of history

I Make

I GNU Build Tools

2/ 31

Computers and programming

I first computers were function specific

I created to work on a specific problem with variables

I analog systems, ‘programmed’ by connecting tubes

I programming took a long time

I finding problems took even longer

I preparation of programs was much more important

3/ 31

The ‘old’ days

Colossus

I created during the2nd world war,finished in 1943

I used to crackgerman codes atBletchley Park inEngland

4/ 31

SNE visit to Bletchley park in 2006

5/ 31

The ‘old’ days

ENIAC

I also created duringthe 2nd world warby the americans,finished in 1946

I used to computetrajectories forgranades

I programming meant connecting the tubes with wires

6/ 31

Evolution in programming

I moved from hardware tosoftware, so no more messywires

I programs now stored in memory

I but each machine instructionhad to be entered by hand

I can take a long time, prone toerrors

I memory would be empty atstartup

I storing programs and dataoff-line would be nice

7/ 31

Storage: punch cards or tapes

I already invented for automatedweaving looms

I paper with holes in it,each hole a bit

I can store any data, includingprograms

I writing programs meant punching the correct sequence ofcodes

I cumbersome, time consuming

I errors meant re-punching and re-evaluating

8/ 31

Storage: bigger and faster

I storage capabilities grew, no longer a bottleneck

I creating and changing software became easier

I more intermediate steps were possible

I creation and testing faster, less time needed to program

I more and more languages start appearing

I larger programs became possible

9/ 31

Compilers

I higher level languages need to be translated to machine code

I this includes assembly languages

I compilers are needed

Bigger programs

I larger programs made programming more difficult

I maintainability an issue

I compilation time now also a factor

10/ 31

Splitting source code

I sources can be split over more files

I problem: How to compile them into one?

I issues: dependancy, multiple includes . . .

Solutions

1 concatenate the files and compile the whole

– can create very big sources, takes up a lot of memory– compilation can take a long time

2 compile each far as possible and then glue them togetherafterwards

– adds extra step into the compilation process+ less memory needed

11/ 31

Object files

I each source file is compiled into anobject file

I object files are independant (checkingis done at linking time)

I compiling the sources one by onesaves memory

src

��

src

��

obj obj

I contains the machine code, data, program symbols, debugginginformation. . .

I object files can later be linked together

I only have to re-compile the source file that were changed

12/ 31

Linking object files

I object files produced by acompiler can be linkedtogether to form anexecutable (.com, .a, .exe,. . . )

I object files only have to belinked once

I function references are linkedto the relevant code

lib

""DDDDDDDDobj

��

obj

||yyyyyyyy

?> =<89 :;linker

||yyyyyyyy

�� ""FFFF

FFFF

F

lib so exe

I the linker resolves the symbols in each object file while linking

13/ 31

Shared Objects (Libraries)

I object files can also be linked into libraries of symbols

I not executable, but can be used for linking yet again

I Libraries are reusable for all programs

I can be linked statically or dynamically

I static linking adds the library (or the used parts) to anexecutable

I dynamic linking means a library is loaded into memory whenthe executable requires it

I static linking makes for bigger programs

I dynamic linking can be slower at starting up

14/ 31

Recap: building an executable

I say we have 2 source files for our program

I we compile each:

#> compile main.SNEcompiling... main.obj created#> compile rest.SNEcompiling... rest.obj created

I we’ll now link them into an executable, or binary

#> link main.obj rest.obj myprog.alinking... myprog.a created

15/ 31

Compiling large programs

I compiling and linking by hand becomes tedious when creatinglarge programs

I automation was required

I early automation used simple make.sh and install.shscripts

I created by hand per project, the programmer must think ofeverything

I introduced inconsistancies, only works on local system,dependancy checking is boring. . .

I generic automation is required

16/ 31

Make (1)

I one of the first build utilities, build automation

I very wide spread due to inclusion into Unix

I created (originally) by Stuart Feldman in 1977 at Bell Labs

I Feldman received the ACM Software System Award

I make has undergone rewrites, many versions now

I BSD make, GNU make, Micrsoft nmake

Makefile layout

target: dependenciescommand 1command 2...

17/ 31

Make (2)

Provides

I automated building using a Makefile

I only files that have changed

I methods for maintaining dependencies

I not language specific (we will use c)

Problems

I no dependency checking, left to programmer

I tailoring to multiple platforms is problematic

18/ 31

Make (3)

Example Makefile

helloworld: helloworld.occ -o $@ $<

helloworld.o: helloworld.ccc -c -o $@ $<

clean:rm -f helloworld helloworld.o

19/ 31

Build systems

Tools for automating the creation of platform specific buildingtools (Makefiles and such)

make-based systems

I GNU automake, CMake, imake, qmake

othersI Apache Ant

I Debian Package Maker

I GNU Build Tools

I MSBuild

I CMake

I . . .

See http://en.wikipedia.org/wiki/List_of_build_automation_software

20/ 31

GNU Build Tools (a.k.a autotools) (1)

I suite of programming tools produced by the GNU project

I designed to assist in creating portable (Unix) source packages

I widely used in many free software and open source packages

I intended to be used with other GNU tools like the GNU CCompiler

I one of the biggest users of the GNU M4 macro processor

I the system requires (relatively) little input

I configure.in, Makefile.am, acconfig.h

21/ 31

GNU Build Tools (2)

I addresses portability by providing a way to check forconditions that may change from one system to another

I header files, libraries, platform versions, . . .

I a special header file (config.h) can be created for inclusioninto the sources

I it contains the gathered information on the system for usewithin your program

22/ 31

GNU Autoconf (1)

I Autoconf is used to generate a configure script

I the configure script will do the checking when used at buildtime

I it will also process other template files (.in) likeMakefile.in to create a specific Makefile

I Autoconf can work around quirks found in various Unix-likeoperating systems, including some bugs

I these fixes are included in the configure script

Usage

I create configure.in to specify what functionality is required

I then run autoconf to generate a configure script

23/ 31

GNU Autoconf (2)

Usage example

I not all systems support the gettimeofday function

I you want to use it when available, and some other functionwhen not

I put AC_CHECK_FUNCS(gettimeofday) in configure.in

I if the function is available the (generated) configure script willarrange to define the preprocessor macroHAVE_GETTIMEOFDAY

I otherwise it will not define the macro at all

I use #ifdef to test whether it is safe to call gettimeofdayfrom your code

24/ 31

GNU Autoconf (3)

Auxiliary programs part of Autoconf

autoheader used to help manage C header files, will warn you ifcertain header requirements are not met (need to beadded to acconfig.h)

autoscan can create an initial input file for Autoconf, it willscan your sources to find possible portability problems

ifnames can list C pre-processor identifiers already in use inyour program

25/ 31

GNU Automake

I helps to create portable Makefiles for use with the make utility

I input Makefile.am, output Makefile.in

I Makefile.in is then used by the configure script to generatea Makefile

Usage example

I your package builds a program sne, simply add the followingto the Makefile.am in the directory where the program isbuilt

bin_PROGRAMS = snesne_SOURCES = sne1 sne2 ...

26/ 31

GNU Libtool

I helps manage the creation of static and dynamic libraries onvarious Unix-like operating systems

I hides the complexity of using shared libraries on differentplatforms

I provides a generic interface for developers

clean Remove files from the build directorycompile Compile a source file into a libtool object.execute Automatically set the library path, then run a program.

finish Complete the installation of libtool libraries.install Install libraries or executables.

link Create a library or an executable.uninstall Remove libraries from an installed directory.

27/ 31

Developer: Created and generated files

28/ 31

Build time: Created and generated files

29/ 31

References

http://en.wikipedia.org/wiki/Apollo_Guidance_Computerhttp://www.gnu.org/software/autoconf/http://www.gnu.org/software/libtool/libtool.htmlhttp://www.gnu.org/software/automake/automake.htmlhttp://www.gnu.org/software/m4/m4.html. . .

30/ 31

AssignmentI Study the Debian packaging system

I how does it workI how does it deal with dependanciesI does it use the GNU build tools? How?I if not what does it use?I . . .

I Study the Open SUSE packaging system and answer theabove questions again

I now look for a small and simple game on the web

I download its source package for Ubuntu (debian package?)

I create a binary package from it for the ppc and install that totest

I now try and create a binary package from the source for thei386 architecture

I try and install that on your experimentation computer31/ 31