Chapter 26. Design of the pkgsrc infrastructure
Prev | Part III. The pkgsrc infrastructure internals | Next

## Chapter 26. Design of the pkgsrc infrastructure

The pkgsrc infrastructure consists of many small Makefile fragments. Each such fragment needs a properly specified interface. This chapter explains how such an interface looks like.

## 26.1. The meaning of variable definitions

Whenever a variable is defined in the pkgsrc infrastructure, the location and the way of definition provide much information about the intended use of that variable. Additionally, more documentation may be found in a header comment or in this pkgsrc guide.

A special file is `mk/defaults/mk.conf`, which lists all variables that are intended to be user-defined. They are either defined using the `?=` operator or they are left undefined because defining them to anything would effectively mean “yes”. All these variables may be overridden by the pkgsrc user in the `MAKECONF` file.

Outside this file, the following conventions apply: Variables that are defined using the `?=` operator may be overridden by a package.

Variables that are defined using the `=` operator may be used read-only at run-time.

Variables whose name starts with an underscore must not be accessed outside the pkgsrc infrastructure at all. They may change without further notice.

### Note

These conventions are currently not applied consistently to the complete pkgsrc infrastructure.

## 26.2. Avoiding problems before they arise

All variables that contain lists of things should default to being empty. Two examples that do not follow this rule are `USE_LANGUAGES` and `DISTFILES`. These variables cannot simply be modified using the `+=` operator in package `Makefile`s (or other files included by them), since there is no guarantee whether the variable is already set or not, and what its value is. In the case of `DISTFILES`, the packages “know” the default value and just define it as in the following example.

    DISTFILES=      ${DISTNAME}${EXTRACT_SUFX} additional-files.tar.gz

Because of the selection of this default value, the same value appears in many package Makefiles. Similarly for `USE_LANGUAGES`, but in this case the default value (“`c`”) is so short that it doesn't stand out. Nevertheless it is mentioned in many files.

## 26.3. Variable evaluation

### 26.3.1. At load time

Variable evaluation takes place either at load time or at runtime, depending on the context in which they occur. The contexts where variables are evaluated at load time are:

- The right hand side of the `:=` and `!=` operators,

- Make directives like `.if` or `.for`,

- Dependency lines.

A special exception are references to the iteration variables of `.for` loops, which are expanded inline, no matter in which context they appear.

As the values of variables may change during load time, care must be taken not to evaluate them by accident. Typical examples for variables that should not be evaluated at load time are `DEPENDS` and `CONFIGURE_ARGS`. To make the effect more clear, here is an example:

    CONFIGURE_ARGS=         # none
    CFLAGS=                 -O
    CONFIGURE_ARGS+=        CFLAGS=${CFLAGS:Q}

    CONFIGURE_ARGS:=        ${CONFIGURE_ARGS}

    CFLAGS+=                -Wall

This code shows how the use of the `:=` operator can quickly lead to unexpected results. The first paragraph is fairly common code. The second paragraph evaluates the `CONFIGURE_ARGS` variable, which results in `CFLAGS=-O`. In the third paragraph, the `-Wall` is appended to the `CFLAGS`, but this addition will not appear in `CONFIGURE_ARGS`. In actual code, the three paragraphs from above typically occur in completely unrelated files.

### 26.3.2. At runtime

After all the files have been loaded, the values of the variables cannot be changed anymore. Variables that are used in the shell commands are expanded at this point.

## 26.4. How can variables be specified?

There are many ways in which the definition and use of a variable can be restricted in order to detect bugs and violations of the (mostly unwritten) policies. A package can be checked with `pkglint -Wall` to see whether it meets these rules.

## 26.5. Designing interfaces for Makefile fragments

Most of the `.mk` files fall into one of the following classes. Cases where a file falls into more than one class should be avoided as it often leads to subtle bugs.

### 26.5.1. Procedures with parameters

In a traditional imperative programming language some of the `.mk` files could be described as procedures. They take some input parameters and—after inclusion—provide a result in output parameters. Since all variables in `Makefile`s have global scope care must be taken not to use parameter names that have already another meaning. For example, `PKGNAME` is a bad choice for a parameter name.

Procedures are completely evaluated at preprocessing time. That is, when calling a procedure all input parameters must be completely resolvable. For example, `CONFIGURE_ARGS` should never be an input parameter since it is very likely that further text will be added after calling the procedure, which would effectively apply the procedure to only a part of the variable. Also, references to other variables will be modified after calling the procedure.

A procedure can declare its output parameters either as suitable for use in preprocessing directives or as only available at runtime. The latter alternative is for variables that contain references to other runtime variables.

Procedures shall be written such that it is possible to call the procedure more than once. That is, the file must not contain multiple-inclusion guards.

Examples for procedures are `mk/bsd.options.mk` and `mk/buildlink3/bsd.builtin.mk`. To express that the parameters are evaluated at load time, they should be assigned using the `:=` operator, which should be used only for this purpose.

### 26.5.2. Actions taken on behalf of parameters

Action files take some input parameters and may define runtime variables. They shall not define loadtime variables. There are action files that are included implicitly by the pkgsrc infrastructure, while other must be included explicitly.

An example for action files is `mk/subst.mk`.

## 26.6. The order in which files are loaded

Package `Makefile`s usually consist of a set of variable definitions, and include the file `../../mk/bsd.pkg.mk` in the very last line. Before that, they may also include various other `*.mk` files if they need to query the availability of certain features like the type of compiler or the X11 implementation. Due to the heavy use of preprocessor directives like `.if` and `.for`, the order in which the files are loaded matters.

This section describes at which point the various files are loaded and gives reasons for that order.

### 26.6.1. The order in `bsd.prefs.mk`

The very first action in `bsd.prefs.mk` is to define some essential variables like `OPSYS`, `OS_VERSION` and `MACHINE_ARCH`.

Then, the user settings are loaded from the file specified in `MAKECONF`, which is usually `mk.conf`. After that, those variables that have not been overridden by the user are loaded from `mk/defaults/mk.conf`.

After the user settings, the system settings and platform settings are loaded, which may override the user settings.

Then, the tool definitions are loaded. The tool wrappers are not yet in effect. This only happens when building a package, so the proper variables must be used instead of the direct tool names.

As the last steps, some essential variables from the wrapper and the package system flavor are loaded, as well as the variables that have been cached in earlier phases of a package build.

### 26.6.2. The order in `bsd.pkg.mk`

First, `bsd.prefs.mk` is loaded.

Then, the various `*-vars.mk` files are loaded, which fill default values for those variables that have not been defined by the package. These variables may later be used even in unrelated files.

Then, the file `bsd.pkg.error.mk` provides the target `error-check` that is added as a special dependency to all other targets that use `DELAYED_ERROR_MSG` or `DELAYED_WARNING_MSG`.

Then, the package-specific hacks from `hacks.mk` are included.

Then, various other files follow. Most of them don't have any dependencies on what they need to have included before or after them, though some do.

The code to check `PKG_FAIL_REASON` and `PKG_SKIP_REASON` is then executed, which restricts the use of these variables to all the files that have been included before. Appearances in later files will be silently ignored.

Then, the files for the main targets are included, in the order of later execution, though the actual order should not matter.

At last, some more files are included that don't set any interesting variables but rather just define make targets to be executed.

Prev | Up | Next
Part III. The pkgsrc infrastructure internals | Home | Chapter 27. Regression tests