How to run configure and make
*****************************

A package constructed using Autoconf will come with a configure script.  A user who wants to build and install the package must run this script in order to prepare their source tree in order to build it on their particular system.  The actual build process is performed using the make program.

The configure script tests system features.  For example, it might test whether the C library defines the time_t data type for use by the time() C library function.  The configure script then makes the results of those tests available to the program while it is being built.

This chapter explains how to invoke a configure script from the perspective of a user---someone who just wants to take your package and compile it on their system with a minimum of fuss.  It is because Autoconf works as well as it does that it is usually possible to build a package on any kind of machine with a simple configure; make command line.  The topics covered in this chapter include how to invoke configure, the files that configure generates and the most useful Makefile targets--actions that you want make to perform--that will be available when compiling the package (see Introducing Makefiles).

Configuring
===========

A configure script takes a large number of command line options. The set of options can vary from one package to the next, although a number of basic options are always present.  The available options can be discovered by running configure with the --help option.  Although many of these options are esoteric, it's worthwhile knowing of their existence when configuring packages with special installation requirements.  Each option will be briefly described below:

--cache-file=FILE
     configure runs tests on your system to determine the availability of features (or bugs!).  The results of these tests can be stored in a cache file to speed up subsequent invocations of configure.  The presence of a well primed cache file makes a big improvement when configuring a complex tree which has configure scripts in each subtree.

--help
     Outputs a help message.  Even experienced users of configure need to use --help occasionally, as complex projects will include additional options for per-project configuration.  For example, configure in the GCC package allows you to control whether the GNU assembler will be built and used by GCC in preference to a vendor's assembler.

--no-create
     One of the primary functions of configure is to generate output files.  This option prevents configure from generating such output files.  You can think of this as a kind of dry run, although the cache will still be modified.

--quiet
--silent
     As configure runs its tests, it outputs brief messages telling the user what the script is doing.  This was done because configure can be slow.  If there was no such output, the user would be left wondering what is happening.  By using this option, you too can be left wondering!

--version
     Prints the version of Autoconf that was used to generate the configure script.

--prefix=PREFIX
     The --prefix option is one of the most frequently used. If generated Makefiles choose to observe the argument you pass with this option, it is possible to entirely relocate the architecture-independent portion of a package when it is installed. For example, when installing a package like Emacs, the following command line will cause the Emacs Lisp files to be installed in /opt/gnu/share:

          	$ ./configure --prefix=/opt/gnu

     It is important to stress that this behavior is dependent on the generated files making use of this information.  For developers writing these files, Automake simplifies this process a great deal.  Automake is introduced in Introducing GNU Automake.

--exec-prefix=EPREFIX
     Similar to --prefix, except that it sets the location of installed files which are architecture-dependent.  The compiled emacs binary is such a file.  If this option is not given, the default exec-prefix value inserted into generated files is set to the same values at the prefix.

--bindir=DIR
     Specifies the location of installed binary files.  While there may be other generated files which are binary in nature, binary files here are defined to be programs that are run directly by users.

--sbindir=DIR
     Specifies the location of installed superuser binary files.  These are programs which are usually only run by the superuser.

--libexecdir=DIR
     Specifies the location of installed executable support files. Contrasted with ``binary files'', these files are never run directly by users, but may be executed by the binary files mentioned above.

--datadir=DIR
     Specifies the location of generic data files.

--sysconfdir=DIR
     Specifies the location of read-only data used on a single machine.

--sharedstatedir=DIR
     Specifies the location of data which may be modified, and which may be shared across several machines.

--localstatedir=DIR
     Specifies the location of data which may be modified, but which is specific to a single machine.

--libdir=DIR
     Specifies where object code library should be installed.

--includedir=DIR
     Specifies where C header files should be installed.  Header files for other languages such as C++ may be installed here also.

--oldincludedir=DIR
     Specifies where C header files should be installed for compilers other than GCC.

--infodir=DIR
     Specifies where Info format documentation files should be installed. Info is the documentation format used by the GNU project.

--mandir=DIR
     Specifies where manual pages should be installed.

--srcdir=DIR
     This option does not affect installation.  Instead, it tells configure where the source files may be found.  It is normally not necessary to specify this, since the configure script is normally in the same directory as the source files.

--program-prefix=PREFIX
     Specifies a prefix which should be added to the name of a program when installing it.  For example, using --program-prefix=g when configuring a program normally named tar will cause the installed program to be named gtar instead.  As with the other installation options, this configure option only works if it is utilized by the Makefile.in file.

--program-suffix=SUFFIX
     Specifies a suffix which should be appended to the name of a program when installing it.

--program-transform-name=PROGRAM
     Here, PROGRAM is a sed script.  When a program is installed, its name will be run through sed -e SCRIPT to produce the installed name.

--build=BUILD
     Specifies the type of system on which the package will be built.  If not specified, the default will be the same configuration name as the host.

--host=HOST
     Specifies the type of system on which the package will run---or be hosted.  If not specified, the host triplet is determined by executing config.guess.

--target=TARGET
     Specifies the type of system which the package is to be targeted to. This makes the most sense in the context of programming language tools like compilers and assemblers.  If not specified, the default will be the same configuration name as the host.

--disable-FEATURE
     Some packages may choose to provide compile-time configurability for large-scale options such as using the Kerberos authentication system or an experimental compiler optimization pass.  If the default is to provide such features, they may be disabled with --disable-FEATURE, where FEATURE is the feature's designated name.  For example:

          $ ./configure --disable-gui

--enable-FEATURE[=ARG]
     Conversely, some packages may provide features which are disabled by default.  To enable them, use --enable-FEATURE, where FEATURE is the feature's designated name.  A feature may accept an optional argument.  For example:

          	$ ./configure --enable-buffers=128

     Using --enable-FEATURE=no is synonymous with --disable-FEATURE, described above.

--with-PACKAGE[=ARG]
     In the free software community, there is a healthy tendency to reuse existing packages and libraries where possible.  At the time when a source tree is configured by configure, it is possible to provide hints about other installed packages.  For example, the BLT widget toolkit relies on Tcl and Tk.  To configure BLT, it may be necessary to give configure some hints about where you have installed Tcl and Tk:

          	$ ./configure --with-tcl=/usr/local --with-tk=/usr/local

     Using --with-PACKAGE=no is synonymous with --without-PACKAGE which is described below.

--without-PACKAGE
     Sometimes you may not want your package to inter-operate with some pre-existing package installed on your system.  For example, you might not want your new compiler to use GNU ld.  You can prevent this by using an option such as:

          	$ ./configure --without-gnu-ld

--x-includes=DIR
     This option is really a specific instance of a --with-package option.  At the time when Autoconf was initially being developed, it was common to use configure to build programs to run on the X Window System as an alternative to Imake.  The --x-includes option provides a way to guide the configure script to the directory containing the X11 header files.

--x-libraries=DIR
     Similarly, the --x-libraries option provides a way to guide configure to the directory containing the X11 libraries.

It is unnecessary, and often undesirable, to run configure from within the source tree.  Instead, a well-written Makefile generated by configure will be able to build packages whose source files reside in another tree.  The advantages of building derived files in a separate tree to the source code are fairly obvious: the derived files, such as object files, would clutter the source tree. This would also make it impossible to build those same object files on a different system or with a different configuration.  Instead, it is recommended to use three trees: a source tree, a build tree and an install tree.  Here is a closing example of how to build the GNU malloc package in this way:

       $ gtar zxf mmalloc-1.0.tar.gz
       $ mkdir build && cd build
       $ ../mmalloc-1.0/configure
       creating cache ./config.cache
       checking for gcc... gcc
       checking whether the C compiler (gcc  ) works... yes
       checking whether the C compiler (gcc  ) is a cross-compiler... no
       checking whether we are using GNU C... yes
       checking whether gcc accepts -g... yes
       checking for a BSD compatible install... /usr/bin/install -c
       checking host system type... i586-pc-linux-gnu
       checking build system type... i586-pc-linux-gnu
       checking for ar... ar
       checking for ranlib... ranlib
       checking how to run the C preprocessor... gcc -E
       checking for unistd.h... yes
       checking for getpagesize... yes
       checking for working mmap... yes
       checking for limits.h... yes
       checking for stddef.h... yes
       updating cache ../config.cache
       creating ./config.status

Now that this build tree is configured, it is possible to go on and build the package and install it into the default location of /usr/local:

       $ make all && make install

Files generated by configure
============================

After you have invoked configure, you will discover a number of generated files in your build tree.  The build directory structure created by configure and the number of files will vary from package to package.  Each of the generated files are described below and their relationships are shown in Generated File Dependencies:

config.cache
     configure can cache the results of system tests that have been performed to speed up subsequent tests.  This file contains the cache data and is a plain text file that can be hand-modified or removed if desired.

config.log
     As configure runs, it outputs a message describing each test it performs and the result of each test.  There is substantially more output produced by the shell and utilities that configure invokes, but it is hidden from the user to keep the output understandable.  The output is instead redirected to config.log.  This file is the first place to look when configure goes hay-wire or a test produces a nonsense result.  A common scenario is that configure, when run on a Solaris system, will tell you that it was unable to find a working C compiler.  An examination of config.log will show that Solaris' default /usr/ucb/cc is a program that informs the user that the optional C compiler is not installed.

config.status
     configure generates a shell script called config.status that may be used to recreate the current configuration.  That is, all generated files will be regenerated.  This script can also be used to re-run configure if the --recheck option is given.

config.h
     Many packages that use configure are written in C or C++.  Some of the tests that configure runs involve examining variability in the C and C++ programming languages and implementations thereof.  So that source code can programmatically deal with these differences, #define preprocessor directives can be optionally placed in a config header, usually called config.h, as configure runs.  Source files may then include the config.h file and act accordingly:

          #if HAVE_CONFIG_H
          #  include <config.h>
          #endif /* HAVE_CONFIG_H */

          #if HAVE_UNISTD_H
          #  include <unistd.h>
          #endif /* HAVE_UNISTD_H */

     We recommend always using a config header.

Makefile
     One of the common functions of configure is to generate Makefiles and other files.  As it has been stressed, a Makefile is just a file often generated by configure from a corresponding input file (usually called Makefile.in).  The following section will describe how you can use make to process this Makefile.  There are other cases where generating files in this way can be helpful.  For instance, a Java developer might wish to make use of a defs.java file generated from defs.java.in.

The most useful Makefile targets
================================

By now configure has generated the output files such as a Makefile.  Most projects include a Makefile with a basic set of well-known targets (see Targets and dependencies).  A target is a name of a task that you want make to perform -- usually it is to build all of the programs belonging to your package (commonly known as the all target).  From your build directory, the following commands are likely to work for a configured package:

make all
     Builds all derived files sufficient to declare the package built.

make check
     Runs any self-tests that the package may have.

make install
     Installs the package in a predetermined location.

make clean
     Removes all derived files.

There are other less commonly used targets which are likely to be recognized, particularly if the package includes a Makefile which conforms to the GNU Makefile standard or is generated by automake.  You may wish to inspect the generated Makefile to see what other targets have been included.

Configuration Names
===================

The COLLECTIVE name all types of computer systems using a configuration name.  This is a name for the system in a standardized format.

Some example configuration names are sparc-sun-solaris2.7, i586-pc-linux-gnu, or i386-pc-cygwin.

All configuration names used to have three parts, and in some documentation they are still called configuration triplets.  A three part configuration name is CPU-MANUFACTURER-OPERATING_SYSTEM.  Currently configuration names are permitted to have four parts on systems which distinguish the kernel and the operating system, such as GNU/Linux.  In these cases, the configuration name is CPU-MANUFACTURER-KERNEL-OPERATING_SYSTEM.

When using a configuration name in an option to a tool such as configure, it is normally not necessary to specify an entire name.  In particular, the middle field (MANUFACTURER, described below) is often omitted, leading to strings such as i386-linux or sparc-sunos.  The shell script config.sub is used to translate these shortened strings into the canonical form.

On most Unix variants, the shell script config.guess will print the correct configuration name for the system it is run on.  It does this by running the standard uname program, and by examining other characteristics of the system.  On some systems, config.guess requires a working C compiler or an assembler.

Because config.guess can normally determine the configuration name for a machine, it is only necessary for a user or developer to specify a configuration name in unusual cases, such as when building a cross-compiler.

Here is a description of each field in a configuration name:

cpu
     The type of processor used on the system.  This is typically something like i386 or sparc.  More specific variants are used as well, such as mipsel to indicate a little endian MIPS processor.

manufacturer
     A somewhat freeform field which indicates the manufacturer of the system.  This is often simply unknown.  Other common strings are pc for an IBM PC compatible system, or the name of a workstation vendor, such as sun.

operating_system
     The name of the operating system which is run on the system.  This will be something like solaris2.5 or winnt4.0.  There is no particular restriction on the version number, and strings like aix4.1.4.0 are seen.

     Configuration names may be used to describe all sorts of systems, including embedded systems which do not run any operating system.  In this case, the field is normally used to indicate the object file format, such as elf or coff.

kernel
     This is used mainly for GNU/Linux systems.  A typical GNU/Linux configuration name is i586-pc-linux-gnulibc1.  In this case the kernel, linux, is separated from the operating system, gnulibc1.

configure allows fine control over the format of binary files. It is not necessary to build a package for a given kind of machine on that machine natively---instead, a cross-compiler can be used.  Moreover, if the package you are trying to build is itself capable of operating in a cross configuration, then the build system need not be the same kind of machine used to host the cross-configured package once the package is built! Consider some examples:

Compiling a simple package for a GNU/Linux system.
     HOST = BUILD = TARGET = i586-pc-linux-gnu

Cross-compiling a package on a GNU/Linux system that is intended to
     run on an IBM AIX machine: BUILD = i586-pc-linux-gnu, HOST = TARGET = rs6000-ibm-aix3.2

Building a Solaris-hosted MIPS-ECOFF cross-compiler on a GNU/Linux
     system. BUILD = i586-pc-linux-gnu, HOST = sparc-sun-solaris2.4, TARGET = mips-idt-ecoff

Introducing Makefiles
*********************

A Makefile is a specification of dependencies between files and how to resolve those dependencies such that an overall goal, known as a target, can be reached.  Makefiles are processed by the make utility.  Other references describe the syntax of Makefiles and the various implementations of make in detail.  This chapter provides an overview into Makefiles and gives just enough information to write custom rules in a Makefile.am (see Introducing GNU Automake) or Makefile.in.

Targets and dependencies
========================

The make program attempts to bring a target up to date by bring all of the target's dependencies up to date.  These dependencies may have further dependencies.  Thus, a potentially complex dependency graph forms when processing a typical Makefile.  From a simple Makefile that looks like this:

     all: foo

     foo: foo.o bar.o baz.o

     .c.o:
             $(CC) $(CFLAGS) -c $< -o $@

     .l.c:
             $(LEX) $< && mv lex.yy.c $@

We can draw a dependency graph that looks like this:

                    all
                     |
                    foo
                     |
             .-------+-------.
            /        |        \
         foo.o     bar.o     baz.o
           |         |         |
         foo.c     bar.c     baz.c
                               |
                             baz.l

Unless the Makefile contains a directive to make, all targets are assumed to be filename and rules must be written to create these files or somehow bring them up to date.

When leaf nodes are found in the dependency graph, the Makefile must include a set of shell commands to bring the dependent up to date with the dependency.  Much to the chagrin of many make users, up to date means the dependent has a more recent timestamp than the target.  Moreover, each of these shell commands are run in their own sub-shell and, unless the Makefile instructs make otherwise, each command must exit with an exit code of 0 to indicate success.

Target rules can be written which are executed unconditionally.  This is achieved by specifying that the target has no dependents.  A simple rule which should be familiar to most users is:

     clean:
     	-rm *.o core

Makefile syntax
===============

Makefiles have a rather particular syntax that can trouble new users.  There are many implementations of make, some of which provide non-portable extensions.  An abridged description of the syntax follows which, for portability, may be stricter than you may be used to.

Comments start with a # and continue until the end of line.  They may appear anywhere except in command sequences---if they do, they will be interpreted by the shell running the command.  The following Makefile shows three individual targets with dependencies on each:

     target1:  dep1 dep2 ... depN
     <tab>	  cmd1
     <tab>	  cmd2
     <tab>	  ...
     <tab>	  cmdN

     target2:  dep4 dep5
     <tab>	  cmd1
     <tab>	  cmd2

     dep4 dep5:
     <tab>	  cmd1

Target rules start at the beginning of a line and are followed by a colon.  Following the colon is a whitespace separated list of dependencies.  A series of lines follow which contain shell commands to be run by a sub-shell (the default is the Bourne shell).  Each of these lines must be prefixed by a horizontal tab character. This is the most common mistake made by new make users.

These commands may be prefixed by an @ character to prevent make from echoing the command line prior to executing it.  They may also optionally be prefixed by a - character to allow the rule to continue if the command returns a non-zero exit code.  The combination of both characters is permitted.

Macros
======

A number of useful macros exist which may be used anywhere throughout the Makefile.  Macros start with a dollar sign, like shell variables.  Our first Makefile used a few:

     $(CC) $(CFLAGS) -c $< -o $@

Here, syntactic forms of $(..) are make variable expansions.  It is possible to define a make variable using a VAR=VALUE syntax:

     CC = ec++

In a Makefile, $(CC) will then be literally replaced by ec++.  make has a number of built-in variables and default values.  The default value for $(CC) is cc.

Other built-in macros exist with fixed semantics.  The two most common macros are $@ and $<.  They represent the names of the target and the first dependency for the rule in which they appear. $@ is available in any rule, but for some versions of make $< is only available in suffix rules.  Here is a simple Makefile:

     all:    dummy
     	@echo "$@ depends on dummy"

     dummy:
     	touch $@

This is what make outputs when processing this Makefile:

     $ make
     touch dummy
     all depends on dummy

The GNU Make manual documents these macros in more detail.

Suffix rules
============

To simplify a Makefile, there is a special kind of rule syntax known as a suffix rule.  This is a wildcard pattern that can match targets.  Our first Makefile used some.  Here is one:

     .c.o:
     	$(CC) $(CFLAGS) -c $< -o $@

Unless a more specific rule matches the target being sought, this rule will match any target that ends in .o.  These files are said to always be dependent on .c.  With some background material now presented, let's take a look at these tools in use.

A Minimal COLLECTIVE Project
****************************

This chapter describes how to manage a minimal project using the COLLECTIVE.  A minimal project is defined to be the smallest possible project that can still illustrate a sufficient number of principles in using the tools.  By studying a smaller project, it becomes easier to understand the more complex interactions between these tools when larger projects require advanced features.

The example project used throughout this chapter is a fictitious command interpreter called foonly.  foonly is written in C, but like many interpreters, uses a lexical analyzer and a parser expressed using the lex and yacc tools.  The package will be developed to adhere to the GNU Makefile standard, which is the default behavior for Automake.

There are many features of the COLLECTIVE that this small project will not utilize.  The most noteworthy one is libraries; this package does not produce any libraries of its own, so Libtool will not feature in this chapter.  The more complex projects presented in A Small COLLECTIVE Project and A Large COLLECTIVE Project will illustrate how Libtool participates in the build system.  The purpose of this chapter will be to provide a high-level overview of the user-written files and how they interact.

User-Provided Input Files
=========================

The smallest project requires the user to provide only two files.  The remainder of the files needed to build the package are generated by the COLLECTIVE (see Generated Output Files).

  * Makefile.am is an input to automake.

  * configure.in is an input to autoconf.

I like to think of Makefile.am as a high-level, bare-bones specification of a project's build requirements: what needs to be built, and where does it go when it is installed?  This is probably Automake's greatest strength--the description is about as simple as it could possibly be, yet the final product is a Makefile with an array of convenient make targets.

The configure.in is a template of macro invocations and shell code fragments that are used by autoconf to produce a configure script (see Generated File Dependencies). autoconf copies the contents of configure.in to configure, expanding macros as they occur in the input.  Other text is copied verbatim.

Let's take a look at the contents of the user-provided input files that are relevant to this minimal project.  Here is the Makefile.am:

     m4_include(examples/foonly/Makefile.texi)

This Makefile.am specifies that we want a program called foonly to be built and installed in the bin directory when make install is run.  The source files that are used to build foonly are the C source files main.c, foo.c, nly.c and foo.h, the lex program in scanner.l and a yacc grammar in parser.y.  This points out a particularly nice aspect about Automake: because lex and yacc both generate intermediate C programs from their input files, Automake knows how to build such intermediate files and link them into the final executable.  Finally, we must remember to link a suitable lex library, if configure concludes that one is needed.

And here is the configure.in:

     m4_include(examples/foonly/configure.texi)

This configure.in invokes some mandatory Autoconf and Automake initialization macros, and then calls on some Autoconf macros from the AC_PROG family to find suitable C compiler, lex, and yacc programs.  Finally, the AC_OUTPUT macro is used to cause the generated configure script to output a Makefile---but from what?  It is processed from Makefile.in, which Automake produces for you based on your Makefile.am (see Generated File Dependencies).

Generated Output Files
======================

By studying the diagram in Generated File Dependencies, it should be possible to see which commands must be run to generate the required output files from the input files shown in the last section.

First, we generate configure:

     $ aclocal
     $ autoconf

Because configure.in contains macro invocations which are not known to autoconf itself--AM_INIT_AUTOMAKE being a case in point, it is necessary to collect all of the macro definitions for autoconf to use when generating configure.  This is done using the aclocal program, so called because it generates aclocal.m4 (see Generated File Dependencies).  If you were to examine the contents of aclocal.m4, you would find the definition of the AM_INIT_AUTOMAKE macro contained within.

After running autoconf, you will find a configure script in the current directory.  It is important to run aclocal first because automake relies on the contents of configure.in and aclocal.m4.  On to automake:

     $ automake --add-missing
     automake: configure.in: installing `./install-sh'
     automake: configure.in: installing `./mkinstalldirs'
     automake: configure.in: installing `./missing'
     automake: Makefile.am: installing `./INSTALL'
     automake: Makefile.am: required file `./NEWS' not found
     automake: Makefile.am: required file `./README' not found
     automake: Makefile.am: installing `./COPYING'
     automake: Makefile.am: required file `./AUTHORS' not found
     automake: Makefile.am: required file `./ChangeLog' not found

The --add-missing option copies some boilerplate files from your Automake installation into the current directory.  Files such as COPYING, which contain the GNU General Public License change infrequently, and so can be generated without user intervention.  A number of utility scripts are also installed--these are used by the generated Makefiles, particularly by the install target. Notice that some required files are still missing.  These are:

NEWS
     A record of user-visible changes to a package.  The format is not strict, but the changes to the most recent version should appear at the top of the file.

README
     The first place a user will look to get an overview for the purpose of a package, and perhaps special installation instructions.

AUTHORS
     Lists the names, and usually mail addresses, of individuals who worked on the package.

ChangeLog
     The ChangeLog is an important file--it records the changes that are made to a package.  The format of this file is quite strict (see Documentation and ChangeLogs).

For now, we'll do enough to placate Automake:

     $ touch NEWS README AUTHORS ChangeLog
     $ automake --add-missing

Automake has now produced a Makefile.in.  At this point, you may wish to take a snapshot of this directory before we really let loose with automatically generated files.

By now, the contents of the directory will be looking fairly complete and reminiscent of the top-level directory of a GNU package you may have installed in the past:

     AUTHORS	   INSTALL      NEWS        install-sh    mkinstalldirs
     COPYING    Makefile.am  README      configure     missing
     ChangeLog  Makefile.in  aclocal.m4  configure.in

It should now be possible to package up your tree in a tar file and give it to other users for them to install on their own systems. One of the make targets that Automake generates in Makefile.in makes it easy to generate distributions (see Rolling Distribution Tarballs).  A user would merely have to unpack the tar file, run configure (see Invoking configure) and finally type make all:

     $ ./configure
     creating cache ./config.cache
     checking for a BSD compatible install... /usr/bin/install -c
     checking whether build environment is sane... yes
     checking whether make sets ${MAKE}... yes
     checking for working aclocal... found
     checking for working autoconf... found
     checking for working automake... found
     checking for working autoheader... found
     checking for working makeinfo... found
     checking for gcc... gcc
     checking whether the C compiler (gcc  ) works... yes
     checking whether the C compiler (gcc  ) is a cross-compiler... no
     checking whether we are using GNU C... yes
     checking whether gcc accepts -g... yes
     checking how to run the C preprocessor... gcc -E
     checking for flex... flex
     checking for flex... (cached) flex
     checking for yywrap in -lfl... yes
     checking lex output file root... lex.yy
     checking whether yytext is a pointer... yes
     checking for bison... bison -y
     updating cache ./config.cache
     creating ./config.status
     creating Makefile

     $ make all
     gcc -DPACKAGE=\"foonly\" -DVERSION=\"1.0\" -DYYTEXT_POINTER=1  -I. -I. \
       -g -O2 -c main.c
     gcc -DPACKAGE=\"foonly\" -DVERSION=\"1.0\" -DYYTEXT_POINTER=1  -I. -I. \
       -g -O2 -c foo.c
     flex   scanner.l && mv lex.yy.c scanner.c
     gcc -DPACKAGE=\"foonly\" -DVERSION=\"1.0\" -DYYTEXT_POINTER=1  -I. -I. \
       -g -O2 -c scanner.c
     bison -y   parser.y && mv y.tab.c parser.c
     if test -f y.tab.h; then \
       if cmp -s y.tab.h parser.h; then rm -f y.tab.h; \
       else mv y.tab.h parser.h; fi; \
     else :; fi
     gcc -DPACKAGE=\"foonly\" -DVERSION=\"1.0\" -DYYTEXT_POINTER=1  -I. -I. \
       -g -O2 -c parser.c
     gcc  -g -O2  -o foonly  main.o foo.o scanner.o parser.o -lfl

Maintaining Input Files
=======================

If you edit any of the COLLECTIVE input files in your package, it is necessary to regenerate the machine generated files for these changes to take effect.  For instance, if you add a new source file to the foonly_SOURCES variable in Makefile.am.  It is necessary to re-generate the derived file Makefile.in.  If you are building your package, you need to re-run configure to re-generate the site-specific Makefile, and then re-run make to compile the new source file and link it into foonly.

It is possible to regenerate these files by running the required tools, one at a time.  However, as we can see above, it can be difficult to compute the dependencies---does a particular change require aclocal to be run?  Does a particular change require autoconf to be run?  There are two solutions to this problem.

The first solution is to use the autoreconf command.  This tool regenerates all derived files by re-running all of the necessary tools in the correct order.  It is somewhat of a brute force solution, but it works very well, particularly if you are not trying to accommodate other maintainers, or regular maintenance that would render this command bothersome.

The alternative is Automake's ``maintainer mode''.  By invoking the AM_MAINTAINER_MODE macro from configure.in, automake will activate an --enable-maintainer-mode option in configure.  This is explained at length in Bootstrapping.

Packaging Generated Files
=========================

The debate about what to do with generated files is one which is keenly contested on the relevant Internet mailing lists.  There are two points of view and I will present both of them to you so that you can try to decide what the best policy is for your project.

One argument is that generated files should not be included with a package, but rather only the ``preferred form'' of the source code should be included.  By this definition, configure is a derived file, just like an object file, and it should not be included in the package.  Thus, the user should use the COLLECTIVE to bootstrap themselves prior to building the package.  I believe there is some merit to this purist approach, as it discourages the practice of packaging derived files.

The other argument is that the advantages of providing these files can far outweigh the violation of good software engineering practice mentioned above.  By including the generated files, users have the convenience of not needing to be concerned with keeping up to date with all of the different versions of the tools in active use.  This is especially true for Autoconf, as configure scripts are often generated by maintainers using locally modified versions of autoconf and locally installed macros.  If configure were regenerated by the user, the result could be different to that intended.  Of course, this is poor practice, but it happens to reflect reality.

I believe the answer is to include generated files in the package when the package is going to be distributed to a wide user community (ie. the general public).  For in-house packages, the former argument might make more sense, since the tools may also be held under version control.

Documentation and ChangeLogs
============================

As with any software project, it is important to maintain documentation as the project evolves--the documentation must reflect the current state of the software, but it must also accurately record the changes that have been made in the past.  The GNU coding standard rigorously enforces the maintenance of documentation.  Automake, in fact, implements some of the standard by checking for the presence of a ChangeLog file when automake is run!

A number of files exist, with standardized filenames, for storing documentation in GNU packages.  The complete GNU coding standard, which offers some useful insights, can be found at http://www.gnu.org/prep/standards.html.

Other projects, including in-house projects, can use these same tried-and-true techniques.  The purpose of most of the standard documentation files was outlined earlier see Generated Output Files, but the ChangeLog deserves additional treatment.

When recording changes in a ChangeLog, one entry is made per person.  Logical changes are grouped together, while logically distinct changes (ie. ``change sets'') are separated by a single blank line. Here is an example from Automake's own ChangeLog:

     1999-11-21  Tom Tromey  <tromey@cygnus.com>

             * automake.in (finish_languages): Only generate suffix rule
             when not doing dependency tracking.

             * m4/init.m4 (AM_INIT_AUTOMAKE): Use AM_MISSING_INSTALL_SH.
             * m4/missing.m4 (AM_MISSING_INSTALL_SH): New macro.

             * depend2.am: Use @SOURCE@, @OBJ@, @LTOBJ@, @OBJOBJ@,
             and @BASE@.  Always use -o.

Another important point to make about ChangeLog entries is that they should be brief.  It is not necessary for an entry to explain in details why a change was made, but rather what the change was.  If a change is not straightforward then the explanation of why belongs in the source code itself.  The GNU coding standard offers the complete set of guidelines for keeping ChangeLogs.  Although any text editor can be used to create ChangeLog entries, Emacs provides a major mode to help you write them.

Writing configure.in
********************

Writing a portable configure.in is a tricky business.  Since you can put arbitrary shell code into configure.in, your options seem overwhelming.  There are many questions the first-time Autoconf user asks: What constructs are portable and what constructs aren't portable? How do I decide what to check for?  What shouldn't I check for?  How do I best use Autoconf's features?  What shouldn't I put in configure.in?  In what order should I run my checks?  When should I look at the name of the system instead of checking for specific features?

What is Portability?
====================

Before we talk about the mechanics of deciding what to check for and how to check for it, let's ask ourselves a simple question: what is portability?  Portability is a quality of the code that enables it to be built and run on a variety of platforms.  In the Autoconf context, portability usually refers to the ability to run on Unix-like systems---sometimes including Windows.

When I first started using Autoconf, I had a hard time deciding what to check for in my configure.in.  At the time, I was maintaining a proprietary program that ran only on SunOS 4.  However, I was interested in porting it to Solaris, OSF/1, and possibly Irix.

The approach I took, while workable, was relatively time-consuming and painful: I wrote a minimal configure.in and then proceeded to simply try to build my program on Solaris.  Each time I encountered a build problem, I updated configure.in and my source and started again.  Once it built correctly, I started testing to see if there were runtime problems related to portability.

Since I didn't start with a relatively portable base, and since I was unaware of the tools available to help with adding Autoconf support to a package (see Migrating Existing Packages), it was much more difficult than it had to be.  If at all possible, it is better to write portable code to begin with.

There are a large number of Unix-like systems in the world, including many systems which, while still running, can only be considered obsolete.  While it is probably possible to port some programs to all such systems, typically it isn't useful to even try.  Porting to everything is a difficult process, especially given that it usually isn't possible to test on all platforms, and that new operating systems, with their own bugs and idiosyncrasies are released every year.

We advocate a pragmatic approach to portability: we write our programs to target a fairly large, but also fairly modern, cross-section of Unix-like systems.  As deficiencies are discovered in our portability framework, we update configure.in and our sources, and move on. In practice, this is an effective approach.

Brief introduction to portable sh
=================================

If you read a number of configure.ins, you'll quickly notice that they tend to be written in an unusual style.  For instance, you'll notice you hardly ever see the [ program used; instead you'll see test invoked.  We won't go into all the details of writing a portable shell script here; instead we leave that for Writing Portable Bourne Shell.

Like other aspects of portability, the approach you take to writing shell scripts in configure.in and Makefile.am should depend on your goals.  Some platforms have notoriously broken sh implementations.  For instance, Ultrix sh doesn't implement unset.  Of course, the COLLECTIVE are written in the most portable style possible, so as not to limit your possibilities.

Also, it doesn't really make sense to talk about portable sh programming in the abstract.  sh by itself does very little; most actual work is done by separate programs, each with its own potential portability problems.  For instance, some options are not portable between systems, and some seemingly common programs don't exist on every system -- so not only do you have to know which sh constructs are not portable, but you also must know which programs you can (and cannot) use, and which options to those programs are portable.

This seems daunting, but in practice it doesn't seem to be too hard to write portable shell scripts -- once you've internalized the rules. Unfortunately, this process can take a long time.  Meanwhile, a pragmatic ``try and see'' approach, while noting other portable code you've seen elsewhere, works fairly well.  Once again, it pays to be aware of which architectures you'll probably care about -- you will make different choices if you are writing an extremely portable program like emacs or gcc than if you are writing something that will only run on various flavors of Linux.  Also, the cost of having unportable code in configure.in is relatively low -- in general it is fairly easy to rewrite pieces on demand as unportable constructs are found.

Ordering Tests
==============

In addition to the problem of writing portable sh code, another problem which confronts first-time configure.in writers is determining the order in which to run the various tests.  Autoconf indirectly (via the autoscan program, which we cover in Migrating Existing Packages) suggests a standard ordering, which is what we describe here.

The standard ordering is:

 1. Boilerplate.  This section should include standard boilerplate code, such as the call to AC_INIT (which must be first), AM_INIT_AUTOMAKE, AC_CONFIG_HEADER, and perhaps AC_REVISION.

 2. Options.  The next section should include macros which add command-line options to configure, such as AC_ARG_ENABLE.  It is typical to put support code for the option in this section as well, if it is short enough, like this example from libgcj:

          AC_ARG_ENABLE(getenv-properties,
          [  --disable-getenv-properties
                                    don't set system properties from GCJ_PROPERTIES])

          dnl Whether GCJ_PROPERTIES is used depends on the target.
          if test -n "$enable_getenv_properties"; then
             enable_getenv_properties=${enable_getenv_properties_default-yes}
          fi
          if test "$enable_getenv_properties" = no; then
             AC_DEFINE(DISABLE_GETENV_PROPERTIES)
          fi

 3. Programs.  Next it is traditional to check for programs that are either needed by the configure process, the build process, or by one of the programs being built.  This usually involves calls to macros like AC_CHECK_PROG and AC_PATH_TOOL.

 4. Libraries.  Checks for libraries come before checks for other objects visible to C (or C++, or anything else).  This is necessary because some other checks work by trying to link or run a program; by checking for libraries first you ensure that the resulting programs can be linked.

 5. Headers.  Next come checks for existence of headers.

 6. Typedefs and structures.  We do checks for typedefs after checking for headers for the simple reason that typedefs appear in headers, and we need to know which headers we can use before we look inside them.

 7. Functions.  Finally we check for functions.  These come last because functions have dependencies on the preceding items: when searching for functions, libraries are needed in order to correctly link, headers are needed in order to find prototypes (this is especially important for C++, which has stricter prototyping rules than C), and typedefs are needed for those functions which use or return types which are not built in.

 8. Output.  This is done by invoking AC_OUTPUT.

This ordering should be considered a rough guideline, and not a list of hard-and-fast rules.  Sometimes it is necessary to interleave tests, either to make configure.in easier to maintain, or because the tests themselves do need to be in a different order.  For instance, if your project uses both C and C++ you might choose to do all the C++ checks after all the C checks are done, in order to make configure.in a bit easier to read.

What to check for
=================

Deciding what to check for is really the central part of writing configure.in.  Once you've read the Autoconf reference manual, the "how"s of writing a particular test should be fairly clear.  The "when"s might remain a mystery -- and it's just as easy to check for too many things as it is to check for too few.

One notable area of divergence between various Unix-like systems is that the same programs don't exist on all systems, and, even when they do, they don't always work in the same way.  For these problems we recommend, when possible, following the advice of the GNU Coding Standards: use the most common options from a relatively limited set of programs.  Failing that, try to stick to programs and options specified by POSIX, perhaps augmenting this approach by doing checks for known problems on platforms you care about.

Checking for tools and their differences is usually a fairly small part of a configure script; more common are checks for functions, libraries, and the like.

Except for a few core libraries like libc and, usually, libm and libraries like libX11 which typically aren't considered system libraries, there isn't much agreement about library names or contents between Unix systems.  Still, libraries are easy to handle, because decisions about libraries almost always only affect the various Makefiles.  That means that checking for another library typically doesn't require major (or even, sometimes, any) changes to the source code.  Also, because adding a new library test has a small impact on the development cycle -- effectively just re-running configure and then a relink -- you can effectively adopt a lax approach to libraries.  For instance, you can just make things work on the few systems you immediately care about and then handle library changes on an as-needed basis.

Suppose you do end up with a link problem.  How do you handle it?  The first thing to do is use nm to look through the system libraries to see if the missing function exists.  If it does, and it is in a library you can use then the solution is easy -- just add another AC_CHECK_LIB.  Note that just finding the function in a library is not enough, because on some systems, some "standard" libraries are undesirable; libucb is the most common example of a library which you should avoid.

If you can't find the function in a system library then you have a somewhat more difficult problem: a non-portable function.  There are basically three approaches to a missing function.  Below we talk about functions, but really these same approaches apply, more or less, to typedefs, structures, and global variables.

The first approach is to write a replacement function and either conditionally compile it, or put it into an appropriately-named file and use AC_REPLACE_FUNCS.  For instance, Tcl uses AC_REPLACE_FUNCS(strstr) to handle systems that have no strstr function.

The second approach is used when there is a similar function with a different name.  The idea here is to check for all the alternatives and then modify your source to use whichever one might exist.  The idiom here is to use break in the second argument to AC_CHECK_FUNCS; this is used both to skip unnecessary tests and to indicate to the reader that these checks are related.  For instance, here is how libgcj checks for inet_aton or inet_addr; it only uses the first one found:

     AC_CHECK_FUNCS(inet_aton inet_addr, break)

Code to use the results of these checks looks something like:

     #if HAVE_INET_ATON
       ... use inet_aton here
     #else
     #if HAVE_INET_ADDR
       ... use inet_addr here
     #else
     #error Function missing!
     #endif
     #endif

Note how we've made it a compile-time error if the function does not exist.  In general it is best to make errors occur as early as possible in the build process.

The third approach to non-portable functions is to write code such that these functions are only optionally used.  For instance, if you are writing an editor you might decide to use mmap to map a file into the editor's memory.  However, since mmap is not portable, you would also write a function to use the more portable read.

Handling known non-portable functions is only part of the problem, however.  The pragmatic approach works fairly well, but it is somewhat inefficient if you are primarily developing on a more modern system, like GNU/Linux, which has few functions missing.  In this case the problem is that you might not notice non-portable constructs in your code until it has largely been finished.

Unfortunately, there's no high road to solving this problem.  In the end, you need to have a working knowledge of the range of existing Unix systems.  Knowledge of standards such as POSIX and XPG can be useful here, as a first cut -- if it isn't in POSIX, you should at least consider checking for it.  However, standards are not a panacea -- not all systems are POSIX compliant, and sometimes there are bugs in systems functions which you must work around.

One final class of problems you might encounter is that it is also easy to check for too much.  This is bad because it adds unnecessary maintenance burden to your program.  For instance, sometimes you'll see code that checks for <sys/types.h>.  However, there's no point in doing that -- using this header is mostly portable.  Again, this can only be addressed by having a practical knowledge, which is only really possible by examining your target systems.

Using Configuration Names
=========================

While feature tests are definitely the best approach, a configure script may occasionally have to make a decision based on a configuration name.  This may be necessary if certain code must be compiled differently based on something which can not be tested using a standard Autoconf feature test.  For instance, the expect package needs to find information about the system's tty implementation; this can't reliably be done when cross compiling without examining the particular configuration name.

It is normally better to test for particular features, rather than to test for a particular system type.  This is because as Unix and other operating systems evolve, different systems copy features from one another.

When there is no alternative to testing the configuration name in a configure script, it is best to define a macro which describes the feature, rather than defining a macro which describes the particular system.  This permits the same macro to be used on other systems which adopt the same feature (see Writing New Macros for Autoconf).

Testing for a particular system is normally done using a case statement in the autoconf configure.in file.  The case statement might look something like the following, assuming that host is a shell variable holding a canonical configuration system---which will be the case if configure.in uses the AC_CANONICAL_HOST or AC_CANONICAL_SYSTEM macros.

     case "${host}" in
     i[[3456]]86-*-linux-gnu*) do something ;;
     sparc*-sun-solaris2.[[56789]]*) do something ;;
     sparc*-sun-solaris*) do something ;;
     mips*-*-elf*) do something ;;
     esac

Note the doubled square brackets in this piece of code.  These are used to work around an ugly implementation detail of autoconf---it uses M4 under the hood.  Without these extra brackets, the square brackets in the case statement would be swallowed by M4, and would not appear in the resulting configure.  This nasty detail is discussed at more length in M4.

It is particularly important to use * after the operating system field, in order to match the version number which will be generated by config.guess.  In most cases you must be careful to match a range of processor types.  For most processor families, a trailing * suffices, as in mips* above.  For the i386 family, something along the lines of i[34567]86 suffices at present.  For the m68k family, you will need something like m68*.  Of course, if you do not need to match on the processor, it is simpler to just replace the entire field by a *, as in *-*-irix*.

Introducing GNU Automake
************************

The primary goal of Automake is to generate Makefile.ins compliant with the GNU Makefile Standards.  Along the way, it tries to remove boilerplate and drudgery.  It also helps the Makefile writer by implementing features (for instance automatic dependency tracking and parallel make support) that most maintainers don't have the patience to implement by hand.  It also implements some best practices as well as workarounds for vendor make bugs -- both of which require arcane knowledge not generally available.

A secondary goal for Automake is that it works well with other free software, and, specifically, GNU tools.  For example, Automake has support for Dejagnu-based test suites.

Chances are that you don't care about the GNU Coding Standards. That's okay.  You'll still appreciate the convenience that Automake provides, and you'll find that the GNU standards compliance feature, for the most part, assists rather than impedes.

Automake helps the maintainer with five large tasks, and countless minor ones.  The basic functional areas are:

 1. Build

 2. Check

 3. Clean

 4. Install and uninstall

 5. Distribution

We cover the first three items in this chapter, and the others in later chapters.  Before we get into the details, let's talk a bit about some general principles of Automake.

General Automake principles
===========================

Automake at its simplest turns a file called Makefile.am into a GNU-compliant Makefile.in for use with configure.  Each Makefile.am is written according to make syntax; Automake recognizes special macro and target names and generates code based on these.

There are a few Automake rules which differ slightly from make rules:

  * Ordinary make comments are passed through to the output, but comments beginning with ## are Automake comments and are not passed through.

  * Automake supports include directives.  These directives are not passed through to the Makefile.in, but instead are processed by automake -- files included this way are treated as if they were textually included in Makefile.am at that point.  This can be used to add boilerplate to each Makefile.am in a project via a centrally-maintained file.  The filename to include can start with $(top_srcdir) to indicate that it should be found relative to the top-most directory of the project; if it is a relative path or if it starts with $(srcdir) then it is relative to the current directory.  For example, here is how you would reference boilerplate code from the file config/Make-rules (where config is a top-level directory in the project):

          include $(top_srcdir)/config/Make-rules

  * Automake supports conditionals which are not passed directly through to Makefile.in.  This feature is discussed in Advanced GNU Automake Usage.

  * Automake supports macro assignment using +=; these assignments are translated by Automake into ordinary = assignments in Makefile.in.

All macros and targets, including those which Automake does not recognize, are passed through to the generated Makefile.in -- this is a powerful extension mechanism.  Sometimes Automake will define macros or targets internally.  If these are also defined in Makefile.am then the definition in Makefile.am takes precedence.  This feature provides an easy way to tailor specific parts of the output in small ways.

Note, however, that it is a mistake to override parts of the generated code that aren't documented (and thus ``exported'' by Automake). Overrides like this stand a good chance of not working with future Automake releases.

Automake also scans configure.in.  Sometimes it uses the information it discovers to generate extra code, and sometimes to provide extra error checking.  Automake also turns every AC_SUBST into a Makefile variable.  This is convenient in more ways than one: not only does it mean that you can refer to these macros in Makefile.am without extra work, but, since Automake scans configure.in before it reads any Makefile.am, it also means that special variables and overrides Automake recognizes can be defined once in configure.in.

Introduction to Primaries
=========================

Each type of object that Automake understands has a special root variable name associated with it.  This root is called a primary. Many actual variable names put into Makefile.am are constructed by adding various prefixes to a primary.

For instance, scripts---interpreted executable programs---are associated with the SCRIPTS primary.  Here is how you would list scripts to be installed in the user's bindir:

     bin_SCRIPTS = magic-script

(Note that the mysterious bin_ prefix will be discussed later.)

The contents of a primary-derived variable are treated as targets in the resulting Makefile.  For instance, in our example above, we could generate magic-script using sed by simply introducing it as a target:

     bin_SCRIPTS = magic-script

     magic-script: magic-script.in
     	sed -e 's/whatever//' < $(srcdir)/magic-script.in > magic-script
     	chmod +x magic-script

The easy primaries
==================

This section describes the common primaries that are relatively easy to understand; the more complicated ones are discussed in the next section.

DATA
     This is the easiest primary to understand.  A macro of this type lists a number of files which are installed verbatim.  These files can appear either in the source directory or the build directory.

HEADERS
     Macros of this type list header files.  These are separate from DATA macros because this allows for extra error checking in some cases.

SCRIPTS
     This is used for executable scripts (interpreted programs).  These are different from DATA because they are installed with different permissions and because they have the program name transform applied to them (e.g., the --program-transform-name argument to configure).  Scripts are also different from compiled programs because the latter can be stripped while scripts cannot.

MANS
     This lists man pages.  Installing man pages is more complicated than you might think due to the lack of a single common practice.  One developer might name a man page in the source tree foo.man and then rename to the real name (foo.1) at install time.  Another developer might instead use numeric suffixes in the source tree and install using the same name.  Sometimes an alphabetic code follows the numeric suffix (e.g., quux.3n); this code must be stripped before determining the correct install directory (this file must still be installed in $(man3dir)).  Automake supports all of these modes of operation:

       * man_MANS can be used when numeric suffixes are already in place:

               man_MANS = foo.1 bar.2 quux.3n

       * man1_MANS, man2_MANS, etc., can be used to force renaming at install time.  This renaming is skipped if the suffix already begins with the correct number.  For instance:

               man1_MANS = foo.man
               man3_MANS = quux.3n

          Here foo.man will be installed as foo.1 but quux.3n will keep its name at install time.

TEXINFOS
     GNU programs traditionally use the Texinfo documentation format, not man pages.  Automake has full support for Texinfo, including some additional features such as versioning and install-info support. We won't go into that here except to mention that it exists.  See the Automake reference manual for more information.

Automake supports a variety of lesser-used primaries such as JAVA and LISP (and, in the next major release, PYTHON).  See the reference manual for more information on these.

Programs and libraries
======================

The preceding primaries have all been relatively easy to use.  Now we'll discuss a more complicated set, namely those used to build programs and libraries.  These primaries are more complex because building a program is more complex than building a script (which often doesn't even need building at all).

Use the PROGRAMS primary for programs, LIBRARIES for libraries, and LTLIBRARIES for Libtool libraries (see Introducing GNU Libtool).  Here is a minimal example:

     bin_PROGRAMS = doit

This creates the program doit and arranges to install it in bindir.  First make will compile doit.c to produce doit.o.  Then it will link doit.o to create doit.

Of course, if you have more than one source file, and most programs do, then you will want to be able to list them somehow.  You will do this via the program's SOURCES variable.  Each program or library has a set of associated variables whose names are constructed by appending suffixes to the ``normalized'' name of the program.  The normalized name is the name of the object with non-alphanumeric characters changed to underscores.  For instance, the normalized name of quux is quux, but the normalized name of install-info is install_info.  Normalized names are used because they correspond to make syntax, and, like all macros, Automake propagates these definitions into the resulting Makefile.in.

So if doit is to be built from files main.c and doit.c, we would write:

     bin_PROGRAMS = doit
     doit_SOURCES = doit.c main.c

The same holds for libraries.  In the zlib package we might make a library called libzlib.a.  Then we would write:

     lib_LIBRARIES = libzlib.a
     libzlib_a_SOURCES = adler32.c compress.c crc32.c deflate.c deflate.h \
     gzio.c infblock.c infblock.h infcodes.c infcodes.h inffast.c inffast.h \
     inffixed.h inflate.c inftrees.c inftrees.h infutil.c infutil.h trees.c \
     trees.h uncompr.c zconf.h zlib.h zutil.c zutil.h

We can also do this with libtool libraries.  For instance, suppose we want to build libzlib.la instead:

     lib_LTLIBRARIES = libzlib.la
     libzlib_la_SOURCES = adler32.c compress.c crc32.c deflate.c deflate.h \
     gzio.c infblock.c infblock.h infcodes.c infcodes.h inffast.c inffast.h \
     inffixed.h inflate.c inftrees.c inftrees.h infutil.c infutil.h trees.c \
     trees.h uncompr.c zconf.h zlib.h zutil.c zutil.h

As you can see, making shared libraries with Automake and Libtool is just as easy as making static libraries.

In the above example, we listed header files in the SOURCES variable.  These are ignored (except by make dist (see Rolling Distribution Tarballs)) but can serve to make your Makefile.am a bit clearer (and sometimes shorter, if you aren't installing headers).

Note that you can't use configure substitutions in a SOURCES variable.  Automake needs to know the static list of files which can be compiled into your program.  There are still various ways to conditionally compile files, for instance Automake conditionals or the use of the LDADD variable.

The static list of files is also used in some versions of Automake's automatic dependency tracking.  The general rule is that each source file which might be compiled should be listed in some SOURCES variable.  If the source is conditionally compiled, it can be listed in an EXTRA variable.  For instance, suppose in this example @FOO_OBJ@ is conditionally set by configure to foo.o when foo.c should be compiled:

     bin_PROGRAMS = foo
     foo_SOURCES = main.c
     foo_LDADD = @FOO_OBJ@
     foo_DEPENDENCIES = @FOO_OBJ@
     EXTRA_foo_SOURCES = foo.c

In this case, EXTRA_foo_SOURCES is used to list sources which are conditionally compiled; this tells Automake that they exist even though it can't deduce their existence automatically.

In the above example, note the use of the foo_LDADD macro.  This macro is used to list other object files and libraries which should be linked into the foo program.  Each program or library has several such associated macros which can be used to customize the link step; here we list the most common ones:

_DEPENDENCIES
     Extra dependencies which are added to the program's dependency list.  If not specified, this is automatically computed based on the value of the program's _LDADD macro.

_LDADD
     Extra objects which are passed to the linker.  This is only used by programs and shared libraries.

_LDFLAGS
     Flags which are passed to the linker.  This is separate from _LDADD to allow _DEPENDENCIES to be auto-computed.

_LIBADD
     Like _LDADD, but used for static libraries and not programs.

You aren't required to define any of these macros.

Frequently Asked Questions
==========================

Experience has shown that there are several common questions that arise as people begin to use automake for their own projects.  It seemed prudent to mention these issues here.

Users often want to make a library (or program, but for some reason it comes up more frequently with libraries) whose sources live in subdirectories:

     lib_LIBRARIES = libsub.a
     libsub_a_SOURCES = subdir1/something.c ...

If you try this with Automake 1.4, you'll get an error:

     $ automake
     automake: Makefile.am: not supported: source file `subdir1/something.c' is in subdirectory

For libraries, this problem is mostly simply solve by using libtool convenience libraries.  For programs, there is no simple solution.  Many people elect to restructure their package in this case.

The next major release of Automake addresses this problem.

Another general problem that comes up is that of setting compilation flags.  Most rules have flags---for instance, compilation of C code automatically uses CFLAGS.  However, these variables are considered user variables.  Setting them in Makefile.am is unsafe, because the user will expect to be able to override them at will.

To handle this, for each flag variable, Automake introduce an AM_ version which can be set in Makefile.am.  For instance, we could set some flags for C and C++ compilation like so:

     AM_CFLAGS = -DFOR_C
     AM_CXXFLAGS = -DFOR_CXX

Finally, people often ask how to compile a single source file in two different ways.  For instance, the etags.c file which comes with Emacs can be compiled with different -D options to produce the etags and ctags programs.

With Automake 1.4 this can only be done by writing your own compilation rules, like this:

     bin_PROGRAMS = etags ctags
     etags_SOURCES = etags.c
     ctags_SOURCES =
     ctags_LDADD = ctags.o

     etags.o: etags.c
     	$(CC) $(CFLAGS) -DETAGS ...

     ctags.o: etags.c
     	$(CC) $(CFLAGS) -DCTAGS ...

This is tedious and hard to maintain for larger programs.  Automake 1.5 will support a much more natural approach:

     bin_PROGRAMS = etags ctags
     etags_SOURCES = etags.c
     etags_CFLAGS = -DETAGS
     ctags_SOURCES = etags.c
     ctags_CFLAGS = -DCTAGS

Multiple directories
====================

So far, we've only dealt with single-directory projects.  Automake can also handle projects with many directories.  The variable SUBDIRS is used to list the subdirectories which should be built.  Here is an example from Automake itself:

     SUBDIRS = . m4 tests

Automake does not need to know the list of subdirectories statically, so there is no EXTRA_SUBDIRS variable.  You might think that Automake would use SUBDIRS to see which Makefile.ams to scan, but it actually gets this information from configure.in. This means that, if you have a subdirectory which is optionally built, you should still list it unconditionally in your call to AC_OUTPUT and then arrange for it to be substituted (or not, as appropriate) at configure time.

Subdirectories are always built in the order they appear, but cleaning rules (e.g., maintainer-clean) are always run in the reverse order.  The reason for this odd reversal is that it is wrong to remove a file before removing all the files which depend on it.

You can put . into SUBDIRS to control when the objects in the current directory are built, relative to the objects in the subdirectories.  In the example above, targets in . will be built before subdirectories are built.  If . does not appear in SUBDIRS, it is built following all the subdirectories.

Testing
=======

Automake also includes simple support for testing your program.

The most simple form of this is the TESTS variable.  This variable holds a list of tests which are run when the user runs make check.  Each test is built (if necessary) and then executed. For each test, make prints a single line indicating whether the test has passed or failed.  Failure means exiting with a non-zero status, with the special exception that an exit status of 77 (A number chosen arbitrarily by the Automake developers.) means that the test should be ignored.  make check also prints a summary showing the number of passes and fails.

Automake also supports the notion of an xfail, which is a test which is expected to fail.  Sometimes this is useful when you want to track a known failure, but you aren't prepared to fix it right away. Tests which are expected to fail should be listed in both TESTS and XFAIL_TESTS.

The special prefix check can be used with primaries to indicate that the objects should only be built at make check time.  For example, here is how you can build a program that will only be used during the testing process:

     check_PROGRAMS = test-program
     test_program_SOURCES = ...

Automake also supports the use of DejaGNU, the GNU test framework. DejaGNU support can be enabled using the dejagnu option:

     AUTOMAKE_OPTIONS = dejagnu

The resulting Makefile.in will include code to invoke the runtest program appropriately.

Bootstrapping
*************

m4_include(chapters/bootstrap.texi)

A Small COLLECTIVE Project
**************************

m4_include(chapters/small-project.texi)

Introducing GNU Libtool
***********************

m4_include(chapters/intro-libtool.texi)

Using GNU Libtool with configure.in and Makefile.am
***************************************************

m4_include(chapters/using-libtool.texi)

A Large COLLECTIVE Project
**************************

m4_include(chapters/large-project.texi)

Rolling Distribution Tarballs
*****************************

There's something about the word ``tarballs'' that make you want to avoid them altogether, let alone get involved in the disgusting process of rolling one.  And, in the past, that was apparently the attitude of most developers, as witnessed by the strange ways distribution tar archives were created and unpacked.  Automake largely automates this tedious process, in a sense providing you with the obliviousness you crave.

Introduction to Distributions
=============================

The basic approach to creating a tar distribution is to run

     make
     make dist

The generated tar file is named PACKAGE-VERSION.tar.gz, and will unpack into a directory named PACKAGE-VERSION.  These two rules are mandated by the GNU Coding Standards, and are just good ideas in any case, because it is convenient for the end user to have the version information easily accessible while building a package. It removes any doubt when she goes back to an old tree after some time away from it.  Unpacking into a fresh directory is always a good idea -- in the old days some packages would unpack into the current directory, requiring an annoying clean-up job for the unwary system administrator.

The unpacked archive is completely portable, to the extent of Automake's ability to enforce this.  That is, all the generated files (e.g., configure) are newer than their inputs (e.g., configure.in), and the distributed Makefile.in files should work with any version of make. Of course, some of the responsibility for portability lies with you: you are free to introduce non-portable code into your Makefile.am, and Automake can't diagnose this.  No special tools beyond the minimal tool list (see Utilities in Makefiles (The GNU Coding Standards)) plus whatever your own Makefile and configure additions use, will be required for the end user to build the package.

By default Automake creates a .tar.gz file.  It notices if you are using GNU tar and arranges to create portable archives in this case. (By default, GNU tar can create non-portable archives in certain (rare) situations.  To be safe, Automake arranges to use the -o compatibility flag when GNU tar is used.)

People do sometimes want to make other sorts of distributions.  Automake allows this through the use of options.

dist-bzip2
     Add a dist-bzip2 target, which creates a .tar.bz2 file. These files are frequently smaller than the corresponding .tar.gz file.

dist-shar
     Add a dist-shar target, which creates a shar archive.

dist-zip
     Add a dist-zip target, which creates a zip file. These files are popular for Windows distributions.

dist-tarZ
     Add a dist-tarZ target, which creates a .tar.Z file. This exists mostly for die-hard old-time Unix hackers; the rest of the world has moved on to gzip or bzip2.

What goes in
============

Automake tries to make creating a distribution as easy as possible.  The rules are set up by default to distribute those things which Automake knows belong in a distribution.  For instance, Automake always distributes your configure script and your NEWS file.  All the files Automake automatically distributes are shown by automake --help:

     $ automake --help
     ...
     Files which are automatically distributed, if found:
       ABOUT-GNU         README           config.guess      ltconfig
       ABOUT-NLS         THANKS           config.h.bot      ltmain.sh
       AUTHORS           TODO             config.h.top      mdate-sh
       BACKLOG           acconfig.h       config.sub        missing
       COPYING           acinclude.m4     configure         mkinstalldirs
       COPYING.LIB       aclocal.m4       configure.in      stamp-h.in
       ChangeLog         ansi2knr.1       elisp-comp        stamp-vti
       INSTALL           ansi2knr.c       install-sh        texinfo.tex
       NEWS              compile          libversion.in     ylwrap
     ...

Automake also distributes some files about which it has no built-in knowledge, but about which it learns from your Makefile.am.  For instance, the source files listed in a _SOURCES variable go into the distribution.  This is why you ought to list uninstalled header files in the _SOURCES variable: otherwise you'll just have to introduce another variable to distribute them -- Automake will only know about them if you tell it.

Not all primaries are distributed by default.  The rule is arbitrary, but pretty simple: of all the primaries, only _TEXINFOS and _HEADERS are distributed by default.  (Sources that make up programs and libraries are also distributed by default, but, perhaps confusingly, _SOURCES is not considered a primary.)

While there is no rhyme, there is a reason: defaults were chosen based on feedback from users.  Typically, ``enough'' reports of the form ``I auto-generate my _SCRIPTS.  How do I prevent them from ending up in the distribution?'' would cause a change in the default.

Although the defaults are adequate in many situations, sometimes you have to distribute files which aren't covered automatically. It is easy to add additional files to a distribution; simply list them in the macro EXTRA_DIST.  You can list files in subdirectories here.  You can also list a directory's name here and the entire contents will be copied into the distribution by make dist. Use this last feature with care.  A typical failure is that you'll put a ``temporary'' file in the directory and then it will end up in the distribution when you forget to remove it.  Similarly, version control files, such as a CVS subdirectory, can easily end up in a distribution this way.

If a primary is not distributed by default, but in your case it ought to be, you can easily correct it with EXTRA_DIST:

     EXTRA_DIST = $(bin_SCRIPTS)

The next major Automake release  (Probably numbered 1.5.) will have a better method for controlling whether primaries do or do not go into the distribution.  In 1.5 you will be able to use the dist and nodist prefixes to control distribution on a per-variable basis.  You will even be able to simultaneously use both prefixes with a given primary to include some files and omit others:

     dist_bin_SCRIPTS = distribute-this
     nodist_bin_SCRIPTS = but-not-this

The distcheck rule
==================

The make dist documentation sounds nice, and make dist did do something, but how do you know it really works?  It is a terrible feeling when you realize your carefully crafted distribution is missing a file and won't compile on a user's machine.

I wouldn't write such an introduction unless Automake provided a solution.  The solution is a smoke test known as make distcheck. This rule performs a make dist as usual, but it doesn't stop there.  Instead, it then proceeds to untar the new archive into a fresh directory, build it in a fresh build directory separate from the source directory, install it into a third fresh directory, and finally run make check in the build tree.  If any step fails, distcheck aborts, leaving you to fix the problem before it will create a distribution.

While not a complete test -- it only tries one architecture, after all -- distcheck nevertheless catches most packaging errors (as opposed to portability bugs), and its use is highly recommended.

Some caveats
============

Earlier, if you were awake, you noticed that I recommended the use of make before make dist or make distcheck.  This practice ensures that all the generated files are newer than their inputs.  It also solves some problems related to dependency tracking (see Advanced GNU Automake Usage).

Note that currently Automake will allow you to make a distribution when maintainer mode is off, or when you do not have all the required maintainer tools.  That is, you can make a subtly broken distribution if you are motivated or unlucky.  This will be addressed in a future version of Automake.

Implementation
==============

In order to understand how to use the more advanced dist-related features, you must first understand how make dist is implemented.  For most packages, what we've already covered will suffice.  Few packages will need the more advanced features, though I note that many use them anyway.

The dist rules work by building a copy of the source tree and then archiving that copy.  This copy is made in stages: a Makefile in a particular directory updates the corresponding directory in the shadow tree.  In some cases, automake is run to create a new Makefile.in in the new distribution tree.

After each directory's Makefile has had a chance to update the distribution directory, the appropriate command is run to create the archive.  Finally, the temporary directory is removed.

If your Makefile.am defines a dist-hook rule, then Automake will arrange to run this rule when the copying work for this directory is finished. This rule can do literally anything to the distribution directory, so some care is required -- careless use will result in an unusable distribution.  For instance, Automake will create the shadow tree using links, if possible.  This means that it is inadvisable to modify the files in the dist tree in a dist hook. One common use for this rule is to remove files that erroneously end up in the distribution (in rare situations this can happen).  The variable distdir is defined during the dist process and refers to the corresponding directory in the distribution tree; top_distdir refers to the root of the distribution tree.

Here is an example of removing a file from a distribution:

     dist-hook:
             -rm $(distdir)/remove-this-file

Installing and Uninstalling Configured Packages
***********************************************

Have you ever seen a package where, once built, you were expected to keep the build tree around forever, and always cd there before running the tool?  You might have to cast your mind way, way back to the bad old days of 1988 to remember such a horrible thing.

The COLLECTIVE provides a canned solution to this problem.  While not without flaws, it does provide a reasonable and easy-to-use framework. In this chapter we discuss how the COLLECTIVE installation model, how to convince automake to install files where you want them, and finally we conclude with some information about uninstalling, including a brief discussion of its flaws.

Where files are installed
=========================

If you've ever run configure --help, you've probably been frightened by the huge number of options offered.  Although nobody ever uses more than two or three of these, they are still important to understand when writing your package; their proper use will help you figure out where each file should be installed.  For a background on these standard directories and their uses, refer to Invoking configure.

We do recommend using the standard directories as described.  While most package builders only use --prefix or perhaps --exec-prefix, some packages (eg. GNU/Linux distributions) require more control.  For instance, if your package quux puts a file into sysconfigdir, then in the default configuration it will end up in /usr/local/var.  However, for a GNU/Linux distribution it would make more sense to configure with --sysconfigdir=/var/quux.

Automake makes it very easy to use the standard directories.  Each directory, such as bindir, is mapped onto a Makefile variable of the same name.  Automake adds three useful variables to the standard list:

pkgincludedir
     This is a convenience variable whose value is $(includedir)/$(PACKAGE).

pkgdatadir
     A convenience variable whose value is $(datadir)/$(PACKAGE).

pkglibdir
     A variable whose value is $(libdir)/$(PACKAGE).

These cannot be set on the configure command line but are always defined as above.   (There has been some debate in the Autoconf community about extending Autoconf to allow new directories to be set on the configure command line.  Currently the consensus seems to be that there are too many arguments to configure already.)

In Automake, a directory variable's name, without the dir suffix, can be used as a prefix to a primary to indicate install location. Confused yet?  And example will help: items listed in bin_PROGRAMS are installed in bindir.

Automake's rules are actually a bit more precise than this: the directory and the primary must agree.  It doesn't make sense to install a library in datadir, so Automake won't let you.  Here is a complete list showing primaries and the directories which can be used with them:

PROGRAMS
     bindir, sbindir, libexecdir, pkglibdir.

LIBRARIES
     libdir, pkglibdir.

LTLIBRARIES
     libdir, pkglibdir.

SCRIPTS
     bindir, sbindir, libexecdir, pkgdatadir.

DATA
     datadir, sysconfdir, sharedstatedir, localstatedir, pkgdatadir.

HEADERS
     includedir, oldincludedir, pkgincludedir.

TEXINFOS
     infodir.

MANS
     man, man0, man1, man2, man3, man4, man5, man6, man7, man8, man9, mann, manl.

There are two other useful prefixes which, while not directory names, can be used in their place.  These prefixes are valid with any primary. The first of these is noinst.  This prefix tells Automake that the listed objects should not be installed, but should be built anyway. For instance, you can use noinst_PROGRAMS to list programs which will not be installed.

The second such non-directory prefix is check.  This prefix tells Automake that this object should not be installed, and furthermore that it should only be built when the user runs make check.

Early in Automake history we discovered that even Automake's extended built-in list of directories was not enough -- basically anyone who had written a Makefile.am sent in a bug report about this.  Now Automake lets you extend the list of directories.

First you must define your own directory variable.  This is a macro whose name ends in dir.  Define this variable however you like. We suggest that you define it relative to an autoconf directory variable; this gives the user some control over the value.  Don't hardcode it to something like /etc; absolute hardcoded paths are rarely portable.

Now you can attach the base part of the new variable to a primary just as you can with the built-in directories:

     foodir = $(datadir)/foo
     foo_DATA = foo.txt

Automake lets you attach such a variable to any primary, so you can do things you ordinarily wouldn't want to do or be allowed to do. For instance, Automake won't diagnose this piece of code that tries to install a program in an architecture-independent location:

     foodir = $(datadir)/foo
     foo_PROGRAMS = foo

Fine-grained control of install
===============================

The second most common way  (The most common way being to simply set prefix.) to configure a package is to set prefix and exec-prefix to different values.  This way, a system administrator on a heterogeneous network can arrange to have the architecture-independent files shared by all platforms.  Typically this doesn't save very much space, but it does make in-place bug fixing or platform-independent runtime configuration a lot easier.

To this end, Automake provides finer control to the user than a simple make install.  For instance, the user can strip all the package executables at install time by running make install-strip (though we recommend setting the various INSTALL environment variables instead; this is discussed later).  More importantly, Automake provides a way to install the architecture-dependent and architecture-independent parts of a package independently.

In the above scenario, installing the architecture-independent files more than once is just a waste of time.  Our hypothetical administrator can install those pieces exactly once, with make install-data, and then on each type of build machine install only the architecture-dependent files with make install-exec.

Nonstandard directories specified in Makefile.am are also separated along data and exec lines, giving the user complete control over installation.  If, and only if, the directory variable name contains the string exec, then items ending up in that directory will be installed by install-exec and not install-data.

At some sites, the paths referred to by software at runtime differ from those used to actually install the software.  For instance, suppose /usr/local is mounted read-only throughout the network.  On the server, where new packages are built, the file system is available read-write as /w/usr/local -- a directory which is not mounted anywhere else.  In this situation the sysadmin can configure and build using the runtime values, but use the DESTDIR trick to temporarily change the paths at install time:

     ./configure --prefix=/usr/local
     make
     make DESTDIR=/w install

Note that DESTDIR operates as a prefix only.  Sometimes this isn't enough.  In this situation you can explicitly override each directory variable:

     ./configure --prefix=/usr/local
     make
     make prefix=/w/usr/local datadir=/w/usr/share install

Here is a full example  (This example assumes the use of GNU tar when extracting; this is standard on Linux but does not come with Solaris.) showing how you can unpack, configure, and build a typical GNU program on multiple machines at the same time:

     sunos$ tar zxf foo-0.1.tar.gz
     sunos$ mkdir sunos linux

In one window:

     sunos$ cd sunos
     sunos$ ../foo-0.1/configure --prefix=/usr/local \
     > --exec-prefix=/usr/local/sunos
     sunos$ make
     sunos$ make install

And in another window:

     sunos$ rsh linux
     linux$ cd ~/linux
     linux$ ../foo-0.1/configure --prefix=/usr/local \
     > --exec-prefix=/usr/local/linux
     linux$ make
     linux$ make install-exec

In this example we install everything on the sunos machine, but we only install the platform-dependent files on the linux machine.  We use a different exec-prefix, so for example GNU/Linux executables will end up in /usr/local/linux/bin/.

Install hooks
=============

As with dist, the install process allows for generic targets which can be used when the existing install functionality is not enough. There are two types of targets which can be used: local rules and hooks.

A local rule is named either install-exec-local or install-data-local, and is run during the course of the normal install procedure.  This rule can be used to install things in ways that Automake usually does not support.

For instance, in libgcj we generate a number of header files, one per Java class.  We want to install them in pkgincludedir, but we want to preserve the hierarchical structure of the headers (e.g., we want java/lang/String.h to be installed as $(pkgincludedir)/java/lang/String.h, not $(pkgincludedir)/String.h), and Automake does not currently support this.  So we resort to a local rule, which is a bit more complicated than you might expect:

     install-data-local:
             @for f in $(nat_headers) $(extra_headers); do \
     ## Compute the install directory at runtime.
               d="`echo $$f | sed -e 's,/[^/]*$$,,'`"; \
     ## Make the install directory.
               $(mkinstalldirs) $(DESTDIR)$(includedir)/$$d; \
     ## Find the header file -- in our case it might be in srcdir or
     ## it might be in the build directory.  "p" is the variable that
     ## names the actual file we will install.
               if test -f $(srcdir)/$$f; then p=$(srcdir)/$$f; else p=$$f; fi; \
     ## Actually install the file.
               $(INSTALL_DATA) $$p $(DESTDIR)$(includedir)/$$f; \
             done

A hook is guaranteed to run after the install of objects in this directory has completed.  This can be used to modify files after they have been installed.  There are two install hooks, named install-data-hook and install-exec-hook.

For instance, suppose you have written a program which must be setuid root.  You can accomplish this by changing the permissions after the program has been installed:

     bin_PROGRAMS = su
     su_SOURCES = su.c

     install-exec-hook:
             chown root $(bindir)/su
             chmod u+s $(bindir)/su

Unlike an install hook, and install rule is not guaranteed to be after all other install rules are run.  This lets it be run in parallel with other install rules when a parallel make is used.  Ordinarily this is not very important, and in practice you almost always see local hooks and not local rules.

The biggest caveat to using a local rule or an install hook is to make sure that it will work when the source and build directories are not the same---many people forget to do this.  This means being sure to look in $(srcdir) when the file is a source file.

It is also very important to make sure that you do not use a local rule when install order is important -- in this case, your Makefile will succeed on some machines and fail on others.

Uninstall
=========

As if things aren't confusing enough, there is still one more major installation-related feature which we haven't mentioned: uninstall. Automake adds an uninstall target to your Makefile which does the reverse of install: it deletes the newly installed package.

Unlike install, there is no uninstall-data or uninstall-exec; while possible in theory we don't think this would be useful enough to actually use.  Like install, you can write uninstall-local or uninstall-hook rules.

In our experience, uninstall is not a very useful feature. Automake implements it because it is mandated by the GNU Standards, but it doesn't work reliably across packages.  Maintainers who write install hooks typically neglect to write uninstall hooks.  Also, since it can't reliably uninstall a previously installed version of a package, it isn't useful for what most people would want to use it for anyway.  We recommend using a real packaging system, several of which are freely available.  In particular, GNU Stow, RPM, and the Debian packaging system seem like good choices.

Writing Portable C with COLLECTIVE
**********************************

COLLECTIVE permits you to write highly portable programs.  However, using COLLECTIVE is not by itself enough to make your programs portable. You must also write them portably.

In this chapter we will give an introduction to writing portable programs in C.  We will start with some notes on portable use of the C language itself.  We will then discuss cross-Unix portability.  We will finish with some notes on portability between Unix and Windows.

Portability is a big topic, and we can not cover everything in this chapter.  The basic rule of portable code is to remember that every system is in some ways unique.  Do not assume that every other system is like yours.  It is very helpful to be familiar with relevant standards, such as the ISO C standard and the POSIX.1 standard.  Finally, there is no substitute for experience; if you have the opportunity to build and test your program on different systems, do so.

C Language Portability
======================

The C language makes it easy to write non-portable code.  In this section we discuss these portability issues, and how to avoid them.

We concentrate on differences that can arise on systems in common use today.  For example, all common systems today define char to be 8 bits, and define a pointer to hold the address of an 8-bit byte.  We do not discuss the more exotic possibilities found on historical machines or on certain supercomputers.  If your program needs to run in unusual settings, make sure you understand the characteristics of those systems; the system documentation should include a C portability guide describing the problems you are likely to encounter.

ISO C
-----

The ISO C standard first appeared in 1989 (the standard is often called ANSI C).  It added several new features to the C language, most notably function prototypes.  This led to many years of portability issues when deciding whether to use ISO C features.

We think that programs written today can assume the presence of an ISO C compiler.  Therefore, we will not discuss issues related to the differences between ISO C compilers and older compilers---often called K&R compilers, from the first book on C by Kernighan and Ritchie.  You may see these differences handled in older programs.

There is a newer C standard called ``C9X''.  Because compilers that support it are not widely available as of this writing, this discussion does not cover it.

C Data Type Sizes
-----------------

The C language defines data types in terms of a minimum size, rather than an exact size.  As of this writing, this mainly matters for the types int and long.  A variable of type int must be at least 16 bits, and is often 32 bits.  A variable of type long must be at least 32 bits, and is sometimes 64 bits.

The range of a 16 bit number is -32768 to 32767 for a signed number, or 0 to 65535 for an unsigned number.  If a variable may hold numbers larger than 16 bits, use long rather than int.  Never assume that int or long have a specific size, or that they will overflow at a particular point.  When appropriate, use variables of system defined types rather than int or long:

size_t
     Use this to hold the size of an object, as returned by sizeof.

ptrdiff_t
     Use this to hold the difference between two pointers into the same array.

time_t
     Use this to hold a time value as returned by the time function.

off_t
     On a Unix system, use this to hold a file position as returned by lseek.

ssize_t
     Use this to hold the result of the Unix read or write functions.

Some books on C recommend using typedefs to specify types of particular sizes, and then adjusting those typedefs on specific systems. COLLECTIVE supports this using the AC_CHECK_SIZEOF macro. However, while we agree with using typedefs for clarity, we do not recommend using them purely for portability.  It is safest to rely only on the minimum size assumptions made by the C language, rather than to assume that a type of a specific size will always be available.  Also, most C compilers will define int to be the most efficient type for the system, so it is normally best to simply use int when possible.

C Endianness
------------

When a number longer than a single byte is stored in memory, it must be stored in some particular format.  Modern systems do this by storing the number byte by byte such that the bytes can simply be concatenated into the final number.  However, the order of storage varies: some systems store the least significant byte at the lowest address in memory, while some store the most significant byte there.  These are referred to as little-endian and big-endian systems, respectively. (These names come from Gulliver's Travels.)

This difference means that portable code may not make any assumptions about the order of storage of a number.  For example, code like this will act differently on different systems:

       /* Example of non-portable code; don't do this */
       int i = 4;
       char c = *(char *) i;

Although that was a contrived example, real problems arise when writing numeric data in a file or across a network connection.  If the file or network connection may be read on a different type of system, numeric data must be written in a format which can be unambiguously recovered. It is not portable to simply do something like

       /* Example of non-portable code; don't do this */
       write (fd, &i, sizeof i);

This example is non-portable both because of endianness and because it assumes that the size of the type of i are the same on both systems.

Instead, do something like this:

       int j;
       char buf[4];
       for (j = 0; j < 4; ++j)
         buf[j] = (i >> (j * 8)) & 0xff;
       write (fd, buf, 4); /* In real code, check the return value */

This unambiguously writes out a little endian 4 byte value.  The code will work on any system, and the result can be read unambiguously on any system.

Another approach to handling endianness is to use the htonS and ntohS functions available on most systems.  These functions convert between network endianness and host endianness. Network endianness is big-endian; it has that name because the standard TCP/IP network protocols use big-endian ordering.

These functions come in two sizes: htonl and ntohl operate on 4-byte quantities, and htons and ntohs operate on 2-byte quantities.  The hton functions convert host endianness to network endianness.  The ntoh functions convert network endianness to host endianness.  On big-endian systems, these functions simply return their arguments; on little-endian systems, they return their arguments after swapping the bytes.

Although these functions are used in a lot of existing code, they can be difficult to use in highly portable code, because they require knowing the exact size of your data types.  If you know that the type int is exactly 4 bytes long, then it is possible to write code like the following:

       int j;
       j = htonl (i);
       write (fd, &j, 4);

However, if int is not exactly 4 bytes long, this example will not work correctly on all systems.

C Structure Layout
------------------

C compilers on different systems lay out structures differently.  In some cases there can even be layout differences between different C compilers on the same system.  Compilers add gaps between fields, and these gaps have different sizes and are at different locations.  You can normally assume that there are no gaps between fields of type char or array of char.  However, you can not make any assumptions about gaps between fields of any larger type.  You also can not make any assumptions about the layout of bitfield types.

These structure layout issues mean that it is difficult to portably use a C struct to define the format of data which may be read on another type of system, such as data in a file or sent over a network connection.  Portable code must read and write such data field by field, rather than trying to read an entire struct at once.

Here is an example of non-portable code when reading data which may have been written to a file or a network connection on another type of system.  Don't do this.

       /* Example of non-portable code; don't do this */
       struct {
         short i;
         int j;
       } s;
       read (fd, &s, sizeof s);

Instead, do something like this (the struct s is assumed to be the same as above):

       unsigned char buf[6];
       read (fd, buf, sizeof buf); /* Should check return value */
       s.i = buf[0] | (buf[1] << 8);
       s.j = buf[2] | (buf[3] << 8) | (buf[4] << 16) | (buf[5] << 24);

Naturally the code to write out the structure should be similar.

C Floating Point
----------------

Most modern systems handle floating point following the IEEE-695 standard.  However, there are still portability issues.

Most processors use 64 bits of precision when computing floating point values.  However, the widely used Intel x86 series of processors compute temporary values using 80 bits of precision, as do most instances of the Motorola 68k series.  Some other processors, such as the PowerPC, provide fused multiply-add instructions which perform a multiplication and an addition using high precision for the intermediate value. Optimizing compilers will generate such instructions based on sequences of C operations.

For almost all programs, these differences do not matter.  However, for programs which do intensive floating point operations, the differences can be significant.  It is possible to write floating point loops which terminate on one sort of processor but not on another.

Unfortunately, there is no rule of thumb that can be used to avoid these problems.  Most compilers provide an option to disable the use of extended precision (for GNU cc, the option is -ffloat-store).  However, on the one hand, this merely shifts the portability problem elsewhere, and, on the other, the extended precision is often good rather than bad.  Although these portability problems can not be easily avoided, you should at least be aware of them if you write programs which require very precise floating point operations.

The IEEE-695 standard specifies certain flags which the floating point processor should make available (e.g., overflow, underflow, inexact), and specifies that there should be some control over the floating point rounding mode.  Most processors make these flags and controls available; however, there is no portable way to access them.  A portable program should not assume that it will have this degree of control over floating point operations.

GNU cc Extensions
-----------------

The GNU cc compiler has several useful extensions, which are documented in the GNU cc manual.  A program which must be portable to other C compilers must naturally avoid these extensions; the -pedantic option may be used to warn about any accidental use of an extension.

However, the GNU cc compiler is itself highly portable, and it runs on all modern Unix platforms as well as on Windows.  Depending upon your portability requirements, you may be able to simply assume that GNU cc is available, in which case your program may use extensions when they are useful.  Note that some extensions are inherently non-portable, such as inline assembler code, or using attributes to specify a particular section for a function or a global variable.

Cross-Unix Portability
======================

In the previous section, we discussed issues related to the C language. Here we will discuss the portability of C programs across different Unix implementations.  All modern Unix systems conform to the POSIX.1 (1990 edition) and POSIX.2 (1992 edition) standards.  They also all support the sockets interface for networking code.  However, there are still significant differences between systems which can affect portability.

We will not discuss portability to older Unix systems which do not conform to the POSIX standards.  If you need this sort of portability, you can often find some valuable hints in the set of macros defined by autoconf, and in the configure.in files of older programs which use autoconf.

Cross-Unix Function Calls
-------------------------

Functions not mentioned in POSIX.1 may not be available on all systems. If you want to use one of these functions, you should normally check for its presence by using AC_CHECK_FUNCS in your configure.in script, and adapt to its absence if possible.  Here is a list of some popular functions which are available on many, but not all, modern Unix systems:

alloca
     There are several portability issues with alloca.  See the description of AC_FUNC_ALLOCA in the autoconf manual.  Although this function can be very convenient, it is normally best to avoid it in highly portable code.

dlopen
     GNU libtool provides a portable alternate interface to dlopen.  See Dynamic Loading.

getline
     In some cases fgets may be used as a fallback.  In others, you will need to provide your own version of this function.

getpagesize
     On some systems, the page size is available as the macro PAGE_SIZE in the header file sys/param.h.  On others, the page size is available via the sysconf function.  If none of those work, you must generally simply guess a value such as 4096.

gettimeofday
     When this is not available, fall back to a less precise function such as time or ftime (which itself is not available on all systems).

mmap
     In some cases you can use either mmap or ordinary file I/O.  In others, a program which uses mmap will simply not be portable to all Unix systems.  Note that mmap is an optional part of the 1996 version of POSIX.1, so it is likely to be added to all Unix systems over time.

ptrace
     Unix systems without ptrace generally provide some other mechanism for debugging subprocesses, such as /proc.  However, there is no widely portable method for controlling subprocesses, as evidenced by the source code to the GNU debugger, gdb.

setuid
     Different Unix systems handle this differently.  On some systems, any program can switch between the effective user ID of the executable and the real user ID.  On others, switching to the real user ID is final; some of those systems provide the setreuid function instead to switch the effective and real user ID.  The effect when a program run by the superuser calls setuid varies among systems.

snprintf
     If this is not available, then in some cases it will be reasonable to simply use sprintf, and in others you will need to write a little routine to estimate the required length and allocate an appropriate buffer before calling sprintf.

strcasecmp
strdup
strncasecmp
     You can normally provide your own version of these simple functions.

valloc
     When this is not available, just use malloc instead.

vfork
     When this is not available, just use fork instead.

Cross-Unix System Interfaces
----------------------------

There are several Unix system interfaces which have associated portability issues.  We do not have the space here to discuss all of these in detail across all Unix systems.  However, we mention them here to indicate issues where you may need to consider portability.

curses
termcap
terminfo
     Many Unix systems provide the curses interface for simple graphical terminal access, but the name of the library varies.  Typical names are -lcurses or -lncurses.  Some Unix systems do not provide curses, but do provide the -ltermcap or -lterminfo library.  The latter libraries only provide an interface to the termcap file or terminfo files.  These files contain information about specific terminals, the difference being mainly the manner in which they are stored.

proc file system
     The /proc file system is not available on all Unix systems, and when it is available the actual set of files and their format varies.

pseudo terminals
     All Unix systems provide pseudo terminals, but the interface to obtain them varies widely.  We recommend examining the configuration of an existing program which uses them, such as GNU emacs or Expect.

shared libraries
     Shared libraries differ across Unix systems.  The GNU libtool program was written to provide an interface to hide the differences. See Introducing GNU Libtool.

termios
termio
tty
     The termios interface to terminals is standard on modern Unix systems.  Avoid the older, non-portable, termio and tty interfaces (these interfaces are defined in termio.h and sgtty.h, respectively).

threads
     Many, but not all, Unix systems support multiple threads in a single process, but the interfaces differ.  One thread interface, pthreads, was standardized in the 1996 edition of POSIX.1, so Unix systems are likely to converge on that interface over time.

utmp
wtmp
     Most Unix systems maintain the utmp and wtmp files to record information about which users are logged onto the system. However, the format of the information in the files varies across Unix systems, as does the exact location of the files and the functions which some systems provide to access the information.  Programs which merely need to obtain login information will be more portable if they invoke a program such as w.  Programs which need to update the login information must be prepared to handle a range of portability issues.

X Window System
     Version 11 of the X Window System is widely available across Unix systems.  The actual release number varies somewhat, as does the set of available programs and window managers.  Extensions such as OpenGL are not available on all systems.

Unix/Windows Portability
========================

Unix and Windows are very different operating systems, with very different APIs and functionality.  However, it is possible to write programs which run on both Unix and Windows, with significant extra work and some sacrifice in functionality.  For more information on how COLLECTIVE can help you write programs which run on both Unix and Windows, see Integration with Cygnus Cygwin.

Unix/Windows Emulation
----------------------

The simplest way to write a program which runs on both Unix and Windows is to use an emulation layer.  This generally results in a program which runs, but does not really feel like other programs for the operating system in question.

For example, the Cygwin package, which is freely available from Cygnus Solutions (http://sourceware.cygnus.com/cygwin/), provides a Unix API which works on Windows.  This permits Unix programs to be compiled to run on Windows.  It is even possible to run an X server in the Cygwin environment, so graphical programs will work as well, although they will not have the Windows look and feel.  The Cygwin package is discussed in more detail in see Integration with Cygnus Cygwin.

There are also commercial packages available to compile Unix programs for Windows (e.g., Interix) and to compile Windows programs on Unix (e.g., Bristol Technology).

The main disadvantage with using an emulation layer is that the resulting programs have the wrong look and feel.  They do not behave as users expect, so they are awkward to use.  This is generally not acceptable for high quality programs.

Unix/Windows Portable Scripting Language
----------------------------------------

Another approach to Unix/Windows portability is to develop the program using a portable scripting language.  An example of such a scripting language is Tcl/Tk (http://www.scriptics.com/).  Programs written in Tcl/Tk will work on both Unix and Windows (and on the Apple Macintosh operating system as well, for that matter).  Graphical programs will more or less follow the look and feel for the platform upon which they are run.  Since Tcl/Tk was originally developed on Unix, graphical Tcl/Tk programs will typically not look quite right to experienced Windows users, but they will be usable and of reasonable quality.  Other portable scripting languages are Perl, Python, and Guile.

One disadvantage of this approach is that scripting languages tend to be less efficient than straight C code, but it is often possible to recode important routines in C.  Another disadvantage is the need to learn a new language, one which furthermore may not be well designed for large programming projects.

Unix/Windows User Interface Library
-----------------------------------

Some programs' main interaction with the operating system is drawing on the screen.  It is often possible to write such programs using a cross platform user interface library.

A cross-platform user interface library is a library providing basic windowing functions which has been implemented separately for Unix and Windows.  The program calls generic routines which are translated into the appropriate calls on each platform.  These libraries generally provide a good look and feel on each platform, so this can be a reasonable approach for programs which do not require additional services from the system.

The main disadvantage is the least common denominator effect: the libraries often only provide functionality which is available on both Unix and Windows.  Features specific to either Unix or Windows may be very useful for the program, but they may not be available via the library.

Unix/Windows Specific Code
--------------------------

When writing a program which should run on both Unix and Windows, it is possible to simply write different code for the two platforms.  This requires a careful separation of the operating system interface, including the graphical user interface, from the rest of the program. An API must be designed to provide the system needs, and that API must be implemented separately on Unix and Windows.  The API should be set at an appropriate level to avoid the least common denominator effect.

This approach can be useful for a program which has significant platform independent computation as well as significant user interface or other system needs.  It generally produces better results than the other approaches discussed above.  The disadvantage is that this approach requires much more work that the others discussed above.

Unix/Windows Issues
-------------------

Whatever approach is used to support the program on both Unix and Windows, there are certain issues which may affect the design of the program, or many specific areas of the program.

Text and Binary Files
.....................

Windows supports two different types of files: text files and binary files.  On Unix, there is no such distinction.  On Windows, any program which uses files must know whether each file is text or binary, and open and use them accordingly.

In a text file on Windows, each line is terminated with a carriage return character followed by a line feed character.  When the file is read by a C program in text mode, the C library converts each carriage return/line feed pair into a single line feed character.  If the file is read in binary mode, the program will see both the carriage return and the line feed.

You may have seen this distinction when transferring files between Unix and Window systems via FTP.  You need to set the FTP program into binary or text mode as appropriate for the file you want to transfer.

When transferring a binary file, the FTP program simply transfers the data unchanged.  When transferring a text file, the FTP program must convert each carriage return/line feed pair into a single line feed.

When using the C standard library, a binary file is indicated by adding b after the r, w, or a in the call to fopen.  When reading a text file, the program can not simply count characters and use that when computing arguments to fseek.

File system Issues
..................

There are several differences between the file systems used on Unix and Windows, mainly in the areas of what names can be used for files.  The program doschk, which can be found in the gcc distribution, may be used on Unix to check for filenames which are not permitted on DOS or Windows.

DOS Filename Restrictions
.........................

The older DOS FAT file systems have severe limitations on file names.  These limitations no longer apply to Windows, but they do apply to DOS based systems such as DJGPP.

A file name may consist of no more than 8 characters, followed by an optional extension of no more than 3 characters.  This is commonly referred to as an 8.3 file name.  Filenames are case insensitive.

There are a couple of filenames which are treated specially.  You can not name a file aux or prn.  In some cases, you can not even use an extension, such as aux.c.  These restrictions apply to DOS and also to at least some versions of Windows.

Windows File Name Case
......................

Windows normally folds case when referring to files, unlike Unix.  That is, on Windows, the file names file, File, and FiLe all refer to the same file.  You must be aware of this when porting Unix programs to Windows, as the Unix programs may expect that using different case is reflected in the file system.

For example, the procedure used to build the program perl from source relies on distinguishing between the files PERL and perl.  This fails on Windows.

As a matter of interest, the Windows file system stores files under the name with which they were created.  The DOS shell displays the names in all upper case.  The Explorer shell displays them with each word in the file name capitalized.

Whitespace in File Names
........................

Both Unix and Windows file systems permit whitespace in file names. However, Unix users rarely take advantage of this, while Windows users often do.  For example, many Windows systems use a directory named Program Files, whose name has an embedded space.  This is a clash of conventions.

Many programs developed on Unix unintentionally assume that there will be no spaces in file and directory names, and behave mysteriously if any are encountered.  On Unix these bugs will almost never be seen.  On Windows, they will pop up immediately.

When writing a program which must run on Windows, consider these issues. Don't forget to test it on directories and files with embedded spaces.

Windows Separators and Drive Letters
....................................

On Unix, directories in a file name are separated by a forward slash (/).  On Windows, directories are separated by a backward slash (\).  For example, the Unix file dir/file on Windows would be dir\file. (Windows does permit a program to use a forward slash to separate directories when calling routines such as fopen.  However, Windows users do not expect to type forward slashes when they enter file names, and they do not expect to see forward slashes when a file name is printed.)

On Unix, a list of directories is normally separated by a colon (:).  On Windows, a list of directories is normally separated by a semicolon (;).  For example, a simple Unix search path might look like this: /bin:/usr/bin.  The same search path on Windows would probably look like this: c:\bin;c:\usr\bin.

On Unix, the file system is a single tree rooted at the directory simply named /.  On Windows, there are multiple file system trees. Absolute file names often start with a drive letter followed by a colon. Windows maintains a default drive, and a default directory on each drive, which can make it difficult for a program to convert a relative file name into the absolute file name intended by the user.  Windows permits referring to files on other systems by using a file name which starts with two slashes followed by a system name.

Miscellaneous Issues
....................

Windows shared libraries (DLLs) are different from typical Unix shared libraries.  They require special declarations for global variables declared in a shared library.  Programs which use shared libraries must generally use special macros in their header files to define these appropriately.  GNU libtool can help with some shared library issues, but not all.

There are some Unix system features which are not supported under Windows: pseudo terminals, effective user ID, file modes with user/group/other permission, named FIFOs, an executable overriding functions called by shared libraries, select on anything other than sockets.

There are some Windows system features which are not supported under Unix: the Windows event loop, many graphical capabilities, some aspects of the rich set of interthread communication mechanisms, the WSAAsyncSelect function.  You should keep these issues in mind when designing and writing a program which should run on both Unix and Windows.

Writing Portable C++ with COLLECTIVE
************************************

My first task in industry was to port a large C++ application from one Unix platform to another.  My colleagues immediately offered their sympathies and I remember my initial reaction--``what's the big deal?''. After all, this application used the C++ standard library, a modest subset of common Unix system calls and C++ was approaching ISO standardization.  Little did I know what lay ahead---endless hurdles imposed by differences to C++ implementations in use on those platforms.

Being essentially a superset of the C programming language, C++ suffers from all of the machine-level portability issues described in Writing Portable C.  In addition to this, variability in the language and standard libraries present additional trouble when writing portable C++ programs.

There have been comprehensive guides written on C++ portability (see Further Reading).  This chapter will attempt to draw attention to the less portable areas of the C++ language and describe how the COLLECTIVE can help you overcome these (see How COLLECTIVE Can Help). In many instances, the best approach to multi-platform C++ portability is to simply re-express your programs using more widely supported language constructs.  Fortunately, this book has been written at a time when the C++ language standard has been ratified and C++ implementations are rapidly conforming.  Gladly, as time goes on the necessity for this chapter will diminish.

Brief History of C++
====================

C++ was developed in 1983 by Bjarne Stroustrup at AT&T. Stroustrup was seeking a new object-oriented language with which to write simulations.  C++ has now become a mainstream systems programming language and is increasingly being used to implement free software packages. C++ underwent a lengthy standardization process and was ratified as an ISO standard in 1998.

The first specification of C++ was available in a book titled ``The Annotated C++ Reference Manual'' by Stroustrup and Ellis, also known as the ``ARM''.  Since this initial specification, C++ has developed in some areas.  These developments will be discussed in Changeable C++.

The first C++ compiler, known as cfront, was produced by Stroustrup at AT&T.  Because of its strong ties to C and because C is such a general purpose systems language, cfront consisted of a translator from C++ to C.  After translation, an existing C compiler was used to compile the intermediate C code down to machine code for almost any machine you care to mention.  C++ permits overloaded functions---that is, functions with the same name but different argument lists, so cfront implemented a name mangling algorithm (see Name Mangling) to give each function a unique name in the linker's symbol table.

In 1989, the first true C++ compiler, G++, was written by Michael Tiemann of Cygnus Support.  G++ mostly consisted of a new front-end to the GCC portable compiler, so G++ was able to produce code for most of the targets that GCC already supported.

In the years following, a number of new C++ compilers were produced. Unfortunately many were unable to keep pace with the development of the language being undertaken by the standards committee.  This divergence of implementations is the fundamental cause of non-portable C++ programs.

Changeable C++
==============

The C++ standard encompasses the language and the interface to the standard library, including the Standard Template Library (see Standard Template Library).  The language has evolved somewhat since the ARM was published; mostly driven by the experience of early C++ users.

In this section, the newer features of C++ will be briefly explained. Alternatives to these features, where available, will be presented when compiler support is lacking.  The alternatives may be used if you need to make your code work with older C++ compilers or to avoid these features until the compilers you are concerned with are mature.  If you are releasing a free software package to the wider community, you may need to specify a minimum level of standards conformance for the end-user's C++ compiler, or use the unappealing alternative of using lowest-common denominator C++ features.

In covering these, we'll address the following language features:

  * Built-in bool type

  * Exceptions

  * Casts

  * Variable scoping in for loops

  * Namespaces

  * The explicit keyword

  * The mutable keyword

  * The typename keyword

  * Runtime Type Identification (RTTI)

  * Templates

  * Default template arguments

  * Standard library headers

  * Standard Template Library (STL)

Built-in bool type
------------------

C++ introduced a built-in boolean data type called bool.  The presence of this new type makes it unnecessary to use an int with the values 0 and 1 and improves type safety.  The two possible values of a bool are true and false--these are reserved words.  The compiler knows how to coerce a bool into an int and vice-versa.

If your compiler does not have the bool type and false and true keywords, an alternative is to produce such a type using a typedef of an enumeration representing the two possible values:

     enum boolvals { false, true };
     typedef enum boolvals bool;

What makes this simple alternative attractive is that it prevents having to adjust the prolific amount of code that might use bool objects once your compiler supports the built-in type.

Exceptions
----------

Exception handling is a language feature present in other modern programming languages.  Ada and Java both have exception handling mechanisms.  In essence, exception handling is a means of propagating a classified error by unwinding the procedure call stack until the error is caught by a higher procedure in the procedure call chain.  A procedure indicates its willingness to handle a kind of error by catching it:

     void foo ();

     void
     func ()
     {
       try {
         foo ();
       }
       catch (...) {
         cerr << "foo failed!" << endl;
       }
     }

Conversely, a procedure can throw an exception when something goes wrong:

     typedef int io_error;

     void
     init ()
     {
       int fd;
       fd = open ("/etc/passwd", O_RDONLY);
       if (fd < 0) {
         throw io_error(errno);
       }
     }

C++ compilers tend to implement exception handling in full, or not at all.  If any C++ compiler you may be concerned with does not implement exception handling, you may wish to take the lowest common denominator approach and eliminate such code from your project.

Casts
-----

C++ introduced a collection of named casting operators to replace the conventional C-style cast of the form (type) expr.  The new casting operators are static_cast, reinterpret_cast, dynamic_cast and const_cast.  They are reserved words.

These refined casting operators are vastly preferred over conventional C casts for C++ programming.  In fact, even Stroustrup recommends that the older style of C casts be banished from programming projects where at all possible The C++ Programming Language, 3rd edition.  Reasons for preferring the new named casting operators include:

  - They provide the programmer with a mechanism for more explicitly specifying the kind of type conversion.  This assists the compiler in identifying incorrect conversions.

  - They are easier to locate in source code, due to their unique syntax: X_cast<type>(expr).

If your compiler does not support the new casting operators, you may have to continue to use C-style casts---and carefully!  I have seen one project agree to use macros such as the one shown below to encourage those involved in the project to adopt the new operators.  While the syntax does not match that of the genuine operators, these macros make it easy to later locate and alter the casts where they appear in source code.

     #define static_cast(T,e) (T) e

Variable Scoping in For Loops
-----------------------------

C++ has always permitted the declaration of a control variable in the initializer section of for loops:

     for (int i = 0; i < 100; i++)
     {
       ...
     }

The original language specification allowed the control variable to remain live until the end of the scope of the loop itself:

     for (int i = 0; i < j; i++)
     {
       if (some condition)
         break;
     }

     if (i < j)
       // loop terminated early

In a later specification of the language, the control variable's scope only exists within the body of the for loop.  The simple resolution to this incompatible change is to not use the older style. If a control variable needs to be used outside of the loop body, then the variable should be defined before the loop:

     int i;

     for (i = 0; i < j; i++)
     {
       if (some condition)
         break;
     }

     if (i < j)
       // loop terminated early

Namespaces
----------

C++ namespaces are a facility for expressing a relationship between a set of related declarations such as a set of constants.  Namespaces also assist in constraining names so that they will not collide with other identical names in a program.  Namespaces were introduced to the language in 1993 and some early compilers were known to have incorrectly implemented namespaces.  Here's a small example of namespace usage:

     namespace Animals {
       class Bird {
       public:
         fly (); {} // fly, my fine feathered friend!
       };
     };

     // Instantiate a bird.
     Animals::Bird b;

For compilers which do not correctly support namespaces it is possible to achieve a similar effect by placing related declarations into an enveloping structure.  Note that this utilises the fact that C++ structure members have public protection by default:

     struct Animals {
       class Bird {
       public:
         fly (); {} // fly, my find feathered friend!
       };
     protected
       // Prohibit construction.
       Animals ();
     };

     // Instantiate a bird.
     Animals::Bird b;

The explicit Keyword
--------------------

C++ adopted a new explicit keyword to the language.  This keyword is a qualifier used when declaring constructors.  When a constructor is declared as explicit, the compiler will never call that constructor implicitly as part of a type conversion.  This allows the compiler to perform stricter type checking and to prevent simple programming errors.  If your compiler does not support the explicit keyword, you should avoid it and do without the benefits that it provides.

The mutable Keyword
-------------------

C++ classes can be designed so that they behave correctly when const objects of those types are declared.  Methods which do not alter internal object state can be qualified as const:

     class String
     {
     public:
       String (const char* s);
       ~String ();

       size_t Length () const { return strlen (buffer); }

     private:
       char* buffer;
     };

This simple, though incomplete, class provides a Length method which guarantees, by virtue of its const qualifier, to never modify the object state.  Thus, const objects of this class can be instantiated and the compiler will permit callers to use such objects' Length method.

The mutable keyword enables classes to be implemented where the concept of constant objects is sensible, but details of the implementation make it difficult to declare essential methods as const.  A common application of the mutable keyword is to implement classes that perform caching of internal object data.  A method may not modify the logical state of the object, but it may need to update a cache--an implementation detail.  The data members used to implement the cache storage need to be declared as mutable in order for const methods to alter them.

Let's alter our rather farfetched String class so that it implements a primitive cache that avoids needing to call the strlen library function on each invocation of Length ():

     class String
     {
     public:
       String (const char* s) :length(-1) { /* copy string, etc. */ }
       ~String ();

       size_t Length () const
       {
         if (length < 0)
           length = strlen(buffer);
         return length;
       }

     private:
       char* buffer;
       mutable size_t length;
     }

When the mutable keyword is not available, your alternatives are to avoid implementing classes that need to alter internal data, like our caching string class, or to use the const_cast casting operator (see Casts) to cast away the ``constness'' of the object.

The typename Keyword
--------------------

The typename keyword was added to C++ after the initial specification and is not recognized by all compilers.  It is a hint to the compiler that a name following the keyword is the name of a type. In the usual case, the compiler has sufficient context to know that a symbol is a defined type, as it must have been encountered earlier in the compilation:

     class Foo
     {
     public:
       typedef int map_t;
     };

     void
     func ()
     {
       Foo::map_t m;
     }

Here, map_t is a type defined in class Foo.  However, if func happened to be a function template, the class which contains the map_t type may be a template parameter.  In this case, the compiler simply needs to be guided by qualifying T::map_t as a type name:

     class Foo
     {
     public:
       typedef int map_t;
     };

     template <typename T>
     void func ()
     {
       typename T::map_t t;
     }

Runtime Type Identification (RTTI)
----------------------------------

Run-time Type Identification, or RTTI, is a mechanism for interrogating the type of an object at runtime.  Such a mechanism is useful for avoiding the dreaded switch-on-type technique used before RTTI was incorporated into the language.  Until recently, some C++ compilers did not support RTTI, so it is necessary to assume that it may not be widely available.

Switch-on-type involves giving all classes a method that returns a special type token that an object can use to discover its own type.  For example:

             class Shape
             {
             public:
               enum types { TYPE_CIRCLE, TYPE_SQUARE };
               virtual enum types type () = 0;
             };

             class Circle: public Shape
             {
             public:
              enum types type () { return TYPE_CIRCLE; }
             };

             class Square: public Shape
             {
             public:
               enum types type () { return TYPE_SQUARE; }
             };

Although switch-on-type is not elegant, RTTI isn't particularly object-oriented either.  Given the limited number of times you ought to be using RTTI, the switch-on-type technique may be reasonable.

Templates
---------

Templates---known in other languages as generic types---permit you to write C++ classes which represent parameterized data types.  A common application for class templates is container classes. That is, classes which implement data structures that can contain data of any type.  For instance, a well-implemented binary tree is not interested in the type of data in its nodes.  Templates have undergone a number of changes since their initial inclusion in the ARM.  They are a particularly troublesome C++ language element in that it is difficult to implement templates well in a C++ compiler.

Here is a fictitious and overly simplistic C++ class template that implements a fixed-sized stack.  It provides a pair of methods for setting (and getting) the element at the bottom of the stack.  It uses the modern C++ template syntax, including the new typename keyword (see The typename Keyword).

     template <typename T> class Stack
     {
     public:
       T first () { return stack[9]; }
       void set_first (T t) { stack[9] = t; }

     private:
       T stack[10];
     };

C++ permits this class to be instantiated for any type you like, using calling code that looks something like this:

     int
     main ()
     {
       Stack<int> s;
       s.set_first (7);
       cout << s.first () << endl;
       return 0;
     }

An old trick for fashioning class templates is to use the C preprocessor.  Here is our limited Stack class, rewritten to avoid C++ templates:

     #define Stack(T) \
       class Stack__##T##__LINE__ \
       { \
       public: \
         T first () { return stack[0]; } \
         void set_first (T t) { stack[0] = t; } \
       \
       private: \
         T stack[10]; \
       }

There is a couple of subtleties being used here that should be highlighted.  This generic class declaration uses the C preprocessor operator ## to generate a type name which is unique amongst stacks of any type.  The __LINE__ macro is defined by the preprocessor and is used here to maintain unique names when the template is instantiated multiple times.  The trailing semicolon that must follow a class declaration has been omitted from the macro.

     int
     main ()
     {
       Stack (int) s;
       s.set_first (7);
       cout << s.first () << endl;
       return 0;
     }

The syntax for instantiating a Stack is slightly different to modern C++, but it does work relatively well, since the C++ compiler still applies type checking after the preprocessor has expanded the macro.  The main problem is that unless you go to great lengths, the generated type name (such as Stack__int) could collide with other instances of the same type in the program.

Default template arguments
--------------------------

A later refinement to C++ templates was the concept of default template arguments.  Templates allow C++ types to be parameterized and as such, the parameter is in essence a variable that the programmer must specify when instantiating the template. This refinement allows defaults to be specified for the template parameters.

This feature is used extensively throughout the Standard Template Library (see Standard Template Library) to relieve the programmer from having to specify a comparison function for sorted container classes. In most circumstances, the default less-than operator for the type in question is sufficient.

If your compiler does not support default template arguments, you may have to suffer without them and require that users of your class and function templates provide the default parameters themselves. Depending on how inconvenient this is, you might begrudgingly seek some assistance from the C preprocessor and define some preprocessor macros.

Standard library headers
------------------------

Newer C++ implementations provide a new set of standard library header files.  These are distinguished from older incompatible header files by their filenames---the new headers omit the conventional .h extension.  Classes and other declarations in the new headers are placed in the std namespace.  Detecting the kind of header files present on any given system is an ideal application of Autoconf.  For instance, the header <vector> declares the class std::vector<T>. However, if it is not available, <vector.h> declares the class vector<T> in the global namespace.

Standard Template Library
-------------------------

The Standard Template Library (STL) is a library of containers, iterators and algorithms.  I tend to think of the STL in terms of the container classes it provides, with algorithms and iterators necessary to make these containers useful.  By segregating these roles, the STL becomes a powerful library---containers can store any kind of data and algorithms can use iterators to traverse the containers.

There are about half a dozen STL implementations.  Since the STL relies so heavily on templates, these implementations tend to inline all of their method definitions.  Thus, there are no precompiled STL libraries, and as an added bonus, you're guaranteed to get the source code to your STL implementation.  Hewlett-Packard and SGI produce freely redistributable STL implementations.

It is widely known that the STL can be implemented with complex C++ constructs and is a certain workout for any C++ compiler.  The best policy for choosing an STL is to use a modern compiler such as GCC 2.95 or to use the STL that your vendor may have provided as part of their compiler.

Unfortunately, using the STL is pretty much an ``all or nothing'' proposition.  If it is not available on a particular system, there are no viable alternatives.  There is a macro in the Autoconf macro archive (see Autoconf macro archive) that can test for a working STL.

Compiler Quirks
===============

C++ compilers are complex pieces of software.  Sadly, sometimes the details of a compiler's implementations leak out and bother the application programmer.  The two aspects of C++ compiler implementation that have caused grief in the past are efficient template instantiation and name mangling.  Both of these aspects will be explained.

Template Instantiation
----------------------

The problem with template instantiation exists because of a number of complex constraints:

  - The compiler should only generate an instance of a template once, to speed the compilation process.

  - The linker needs to be smart about where to locate the object code for instantiations produced by the compiler.

This problem is exacerbated by separate compilation---that is, the method bodies for List<T> may be located in a header file or in a separate compilation unit.  These files may even be in a different directory than the current directory!

Life is easy for the compiler when the template definition appears in the same compilation unit as the site of the instantiation---everything that is needed is known:

     template <class T> class List
     {
     private:
       T* head;
       T* current;
     };

     List<int> li;

This becomes significantly more difficult when the site of a template instantiation and the template definition is split between two different compilation units.  In Linkers and Loaders, Levine describes in detail how the compiler driver deals with this by iteratively attempting to link a final executable and noting, from undefined symbol errors produced by the linker, which template instantiations must be performed to successfully link the program.

In large projects where templates may be instantiated in multiple locations, the compiler may generate instantiations multiple times for the same type.  Not only does this slow down compilation, but it can result in some difficult problems for linkers which refuse to link object files containing duplicate symbols.  Suppose there is the following directory layout:

     m4_changequote(,)m4_dnl
     src
     |
     `--- core
     |    `--- core.cxx
     `--- modules
     |    `--- http.cxx
     `--- lib
          `--- stack.h
     m4_changequote(`,')m4_dnl

If the compiler generates core.o in the core directory and libhttp.a in the http directory, the final link may fail because libhttp.a and the final executable may contain duplicate symbols---those symbols generated as a result of both http.cxx and core.cxx instantiating, say, a Stack<int>.  Linkers, such as that provided with AIX will allow duplicate symbols during a link, but many will not.

Some compilers have solved this problem by maintaining a template repository of template instantiations.  Usually, the entire template definition is expanded with the specified type parameters and compiled into the repository, leaving the linker to collect the required object files at link time.

The main concerns about non-portability with repositories center around getting your compiler to do the right thing about maintaining a single repository across your entire project.  This often requires a vendor-specific command line option to the compiler, which can detract from portability.  It is conceivable that Libtool could come to the rescue here in the future.

Name Mangling
-------------

Early C++ compilers mangled the names of C++ symbols so that existing linkers could be used without modification.  The cfront C++ translator also mangled names so that information from the original C++ program would not be lost in the translation to C.  Today, name mangling remains important for enabling overloaded function names and link-time type checking.  Here is an example C++ source file which illustrates name mangling in action:

     class Foo
     {
     public:
       Foo ();

       void go ();
       void go (int where);

     private:
       int pos;
     };

     Foo::Foo ()
     {
       pos = 0;
     }

     void
     Foo::go ()
     {
       go (0);
     }

     void
     Foo::go (int where)
     {
       pos = where;
     }

     int
     main ()
     {
       Foo f;
       f.go (10);
     }

     $ g++ -Wall example.cxx -o example.o

     $ nm --defined-only example.o
     00000000 T __3Foo
     00000000 ? __FRAME_BEGIN__
     00000000 t gcc2_compiled.
     0000000c T go__3Foo
     0000002c T go__3Fooi
     00000038 T main

Even though Foo contains two methods with the same name, their argument lists (one taking an int, one taking no arguments) help to differentiate them once their names are mangled.  The go__3Fooi is the version which takes an int argument.  The __3Foo symbol is the constructor for Foo.  The GNU binutils package includes a utility called c++filt that can demangle names.  Other proprietary tools sometimes include a similar utility, although with a bit of imagination, you can often demangle names in your head.

     $ nm --defined-only example.o | c++filt
     00000000 T Foo::Foo(void)
     00000000 ? __FRAME_BEGIN__
     00000000 t gcc2_compiled.
     0000000c T Foo::go(void)
     0000002c T Foo::go(int)
     00000038 T main

Name mangling algorithms differ between C++ implementations so that object files assembled by one tool chain may not be linked by another if there are legitimate reasons to prohibit linking.  This is a deliberate move, as other aspects of the object file may make them incompatible---such as the calling convention used for making function calls.

This implies that C++ libraries and packages cannot be practically distributed in binary form.  Of course, you were intending to distribute the source code to your package anyway, weren't you?

How COLLECTIVE Can Help
=======================

Each of the COLLECTIVE contribute to C++ portability.  Now that you are familiar with the issues, the following subsections will outline precisely how each tool contributes to achieving C++ portability.

Testing C++ Implementations with Autoconf
-----------------------------------------

Of the COLLECTIVE, perhaps the most valuable contribution to the portability of your C++ programs will come from Autoconf.  All of the portability issues raised in Changeable C++ can be detected using Autoconf macros.

Luc Maisonobe has written a large suite of macros for this purpose and they can be found in the Autoconf macro archive (see Autoconf macro archive).  If any of these macros become important enough, they may become incorporated into the core Autoconf release.  These macros perform their tests by compiling small fragments of C++ code to ensure that the compiler accepts them.  As a side effect, these macros typically use AC_DEFINE to define preprocessor macros of the form HAVE_feature, which may then be exploited through conditional compilation.

Automake C++ support
--------------------

Automake provides support for compiling C++ programs.  In fact, it makes it practically trivial: files listed in a SOURCES primary may include .c++, .cc, .cpp, .cxx or .C extensions and Automake will know to use the C++ compiler to build them.

For a project containing C++ source code, it is necessary to invoke the AC_PROG_CXX macro in configure.in so that Automake knows how to run the most suitable compiler.  Fortunately, when little details like this happen to escape you, automake will produce a warning:

     $ automake
     automake: Makefile.am: C++ source seen but `CXX' not defined in
     automake: Makefile.am: ``configure.in''

Libtool C++ support
-------------------

At the moment, Libtool is the weak link in the chain when it comes to working with C++.  It is very easy to naively build a shared library from C++ source using libtool:

     $ libtool -mode=link g++ -o libfoo.la -rpath /usr/local/lib foo.c++

This works admirably for trivial examples, but with real code, there are several things that can go wrong:

  - On many architectures, for a variety of reasons, libtool needs to perform object linking using ld.  Unfortunately, the C++ compiler often links in standard libraries at this stage, and using ld causes them to be dropped.

     This can be worked around (at the expense of portability) by explicitly adding these missing libraries to the link line in your Makefile. You could even write an Autoconf macro to probe the host machine to discover likely candidates.

  - The C++ compiler likes to instantiate static constructors in the library objects, which C++ programmers often rely on.  Linking with ld will cause this to fail.

     The only reliable way to work around this currently is to not write C++ that relies on static constructors in libraries.   You might be lucky enough to be able to link with LD=$CXX in your environment with some projects, but it would be prone to stop working as your project develops.

  - Libtool's inter-library dependency analysis can fail when it can't find the special runtime library dependencies added to a shared library by the C++ compiler at link time.

     The best way around this problem is to explicitly add these dependencies to libtool's link line:

          $ libtool -mode=link g++ -o libfoo.la -rpath /usr/local/lib foo.cxx \
          -lstdc++ -lg++

Now that C++ compilers on Unix are beginning to see widespread acceptance and are converging on the ISO standard, it is becoming unacceptable for Libtool to impose such limits.  There is work afoot to provide generalized multi-language and multi-compiler support into Libtool----currently slated to arrive in Libtool 1.5.  Much of the work for supporting C++ is already finished at the time of writing, pending beta testing and packaging (Visit the Libtool home page at http://www.gnu.org/software/libtool for breaking news.).

Further Reading
===============

A number of books have been published which are devoted to the topic of C++ portability.  Unfortunately, the problem with printed publications that discuss the state of C++ is that they date quickly.  These publications may also fail to cover inadequacies of your particular compiler, since portability know-how is something that can only be acquired by collective experience.

Instead, online guides such as the Mozilla C++ Portability Guide (http://www.mozilla.org/hacking/portable-cpp.html) tend to be a more useful resource.  An online guide such as this can accumulate the knowledge of a wider developer community and can be readily updated as new facts are discovered.  Interestingly, the Mozilla guide is aggressive in its recommendations for achieving true C++ portability: item 3, for instance, states ``Don't use exceptions''. While you may not choose to follow each recommendation, there is certainly a lot of useful experience captured in this document.

Dynamic Loading
***************

m4_include(chapters/dynamic-load.texi)

Using GNU libltdl
*****************

m4_include(chapters/libltdl.texi)

Advanced GNU Automake Usage
***************************

This chapter covers a few seemingly unrelated Automake features which are commonly considered ``advanced'': conditionals, user-added language support, and automatic dependency tracking.

Conditionals
============

Automake conditionals are a way to omit or include different parts of the Makefile depending on what configure discovers.  A conditional is introduced in configure.in using the AM_CONDITIONAL macro.  This macro takes two arguments: the first is the name of the condition, and the second is a shell expression which returns true when the condition is true.

For instance, here is how to make a condition named TRUE which is always true:

     AM_CONDITIONAL(TRUE, true)

As another example, here is how to make a condition named DEBUG which is true when the user has given the --enable-debug option to configure:

     AM_CONDITIONAL(DEBUG, test "$enable_debug" = yes)

Once you've defined a condition in configure.in, you can refer to it in your Makefile.am using the if statement.  Here is a part of a sample Makefile.am that uses the conditions defined above:

     if TRUE
     ## This is always used.
     bin_PROGRAMS = foo
     endif

     if DEBUG
     AM_CFLAGS = -g -DDEBUG
     endif

It's important to remember that Automake conditionals are configure-time conditionals.  They don't rely on any special feature of make, and there is no way for the user to affect the conditionals from the make command line.  Automake conditionals work by rewriting the Makefile -- make is unaware that these conditionals even exist.

Traditionally, Automake conditionals have been considered an advanced feature.  However, practice has shown that they are often easier to use and understand than other approaches to solving the same problem.  I now recommend the use of conditionals to everyone.

For instance, consider this example:

     bin_PROGRAMS = echo
     if FULL_ECHO
     echo_SOURCES = echo.c extras.c getopt.c
     else
     echo_SOURCES = echo.c
     endif

In this case, the equivalent code without conditionals is more confusing and correspondingly more difficult for the new Automake user to figure out:

     bin_PROGRAMS = echo
     echo_SOURCES = echo.c
     echo_LDADD   = @echo_extras@
     EXTRA_echo_SOURCES = extras.c getopt.c

Automake conditionals have some limitations.  One known problem is that conditionals don't interact properly with += assignment.  For instance, consider this code:

     bin_PROGRAMS = z
     z_SOURCES = z.c
     if SOME_CONDITION
     z_SOURCES += cond.c
     endif

This code appears to have an unambiguous meaning, but Automake 1.4 doesn't implement this and will give an error.  This bug will be fixed in the next major Automake release.

Language support
================

Automake comes with built-in knowledge of the most common compiled languages: C, C++, Objective C, Yacc, Lex, assembly, and Fortran. However, programs are sometimes written in an unusual language, or in a custom language that is translated into something more common.  Automake lets you handle these cases in a natural way.

Automake's notion of a ``language'' is tied to the suffix appended to each source file written in that language.  You must inform Automake of each new suffix you introduce.  This is done by listing them in the SUFFIXES macro.  For instance, suppose you are writing part of your program in the language M, which is compiled to object code by a program named mc.  The typical suffix for an M source file is .m.  In your Makefile.am you would write:

     SUFFIXES = .m

This differs from ordinary make usage, where you would use the special .SUFFIX target to list suffixes.

Now you need to tell Automake (and make) how to compile a .m file to a .o file.  You do this by writing an ordinary make suffix rule:

     MC = mc
     .m.o:
             $(MC) $(MCFLAGS) $(AM_MCFLAGS) -c $<

Note that we introduced the MC, MCFLAGS, and AM_MCFLAGS variables.  While not required, this is good style in case you want to override any of these later (for instance from the command line).

Automake understands enough about suffix rules to recognize that .m files can be treated just like any file it already understands, so now you can write:

     bin_PROGRAMS = myprogram
     myprogram_SOURCES = foo.c something.m

Note that Automake does not really understand chained suffix rules; however, frequently the right thing will happen anyway.  For instance, if you have a .m.c rule, Automake will naively assume that .m files should be turned into .o files -- and then it will proceed to rely on make to do the real work.  In this example, if the translation takes three steps---from .m to .x, then from .x to .c, and finally to .o---then Automake's simplistic approach will break. Fortunately, these cases are very rare.

Automatic dependency tracking
=============================

Keeping track of dependencies for a large program is tedious and error-prone.  Many edits require the programmer to update dependencies, but for some changes, such as adding a #include to an existing header, the change is large enough that he simply refuses (or does it incorrectly).  To fix this problem, Automake supports automatic dependency tracking.

The implementation of automatic dependency tracking in Automake 1.4 requires gcc and GNU make.  These programs are only required for maintainers; the Makefiles generated by make dist are completely portable.  If you can't use gcc or GNU make for your project, then you are simply out of luck; you have to disable dependency tracking.

Automake 1.5 will include a completely new dependency tracking implementation.  This new implementation will work with any compiler and any version of make.

Another limitation of the current scheme is that the dependencies included into the portable Makefiles by make dist are derived from the current build environment.  First, this means that you must use make all before you can meaningfully run make dist (otherwise the dependencies won't have been created).  Second, this means that any files not built in your current tree will not have dependencies in the distributed Makefiles.  The new implementation will avoid both of these shortcomings as well.

Automatic dependency tracking is on by default; you don't have to do anything special to get it.  To turn it off, either run automake -i instead of plain automake, or put no-dependencies into the AUTOMAKE_OPTIONS macro in each Makefile.am.

A Complex COLLECTIVE Project
****************************

m4_include(chapters/complex-project.texi)