Chapter 21. Making your package work
Prev | Part II. The pkgsrc developer's guide | Next

## Chapter 21. Making your package work

## 21.1. General operation

One appealing feature of pkgsrc is that it runs on many different platforms. As a result, it is important to ensure, where possible, that packages in pkgsrc are portable. This chapter mentions some particular details you should pay attention to while working on pkgsrc.

### 21.1.1. How to pull in user-settable variables from `mk.conf`

The pkgsrc user can configure pkgsrc by overriding several variables in the file pointed to by `MAKECONF`, which is `mk.conf` by default. When you want to use those variables in the preprocessor directives of make(1) (for example `.if` or `.for`), you need to include the file `../../mk/bsd.prefs.mk` before, which in turn loads the user preferences.

But note that some variables may not be completely defined after `../../mk/bsd.prefs.mk` has been included, as they may contain references to variables that are not yet defined. In shell commands (the lines in `Makefile` that are indented with a tab) this is no problem, since variables are only expanded when they are used. But in the preprocessor directives mentioned above and in dependency lines (of the form `target: dependencies`) the variables are expanded at load time.

### Note

To check whether a variable can be used at load time, run pkglint -Wall on your package.

### 21.1.2. User interaction

Occasionally, packages require interaction from the user, and this can be in a number of ways:

- When fetching the distfiles, some packages require user interaction such as entering username/password or accepting a license on a web page.

- When extracting the distfiles, some packages may ask for passwords.

- help to configure the package before it is built

- help during the build process

- help during the installation of a package

A package can set the `INTERACTIVE_STAGE` variable to define which stages need interaction. This should be done in the package's `Makefile`, e.g.:

    INTERACTIVE_STAGE=      configure install

The user can then decide to skip this package by setting the `BATCH` variable. Packages that require interaction are also excluded from bulk builds.

### 21.1.3. Handling licenses

Authors of software can choose the licence under which software can be copied. The Free Software Foundation has declared some licenses "Free", and the Open Source Initiative has a definition of "Open Source".

By default, pkgsrc allows packages with Free or Open Source licenses to be built. To allow packages with other licenses to be built as well, the pkgsrc user needs to add these licenses to the `ACCEPTABLE_LICENSES` variable in `mk.conf`. Note that this variable only affects which packages may be built, while the license terms often also restrict the actual use of the package and its redistribution.

One might want to only install packages with a BSD license, or the GPL, and not the other. The free licenses are added to the default `ACCEPTABLE_LICENSES` variable. The pkgsrc user can override the default by setting the `ACCEPTABLE_LICENSES` variable with "=" instead of "+=". The licenses accepted by default are defined in the `DEFAULT_ACCEPTABLE_LICENSES` variable in the file `pkgsrc/mk/license.mk`.

The license tag mechanism is intended to address copyright-related issues surrounding building, installing and using a package, and not to address redistribution issues (see `RESTRICTED` and `NO_SRC_ON_FTP`, etc.). Packages with redistribution restrictions should set these tags.

Denoting that a package may be copied according to a particular license is done by placing the license in `pkgsrc/licenses` and setting the `LICENSE` variable to a string identifying the license, e.g. in `graphics/xv`:

    LICENSE=        xv-license

When trying to build, the user will get a notice that the package is covered by a license which has not been placed in the `ACCEPTABLE_LICENSES` variable:

    % make
    ===> xv-3.10anb9 has an unacceptable license: xv-license.
    ===>     To view the license, enter "/usr/bin/make show-license".
    ===>     To indicate acceptance, add this line to your /etc/mk.conf:
    ===>     ACCEPTABLE_LICENSES+=xv-license
    *** Error code 1

In case a package requires multiple licenses, you can add all of them to the LICENSE variable, connected with the upper-case keyword `AND`. If the user has a choice between multiple licenses, you can add them connected with the upper-case keyword `OR`, for example:

    LICENSE=        isc AND apache-2.0
    LICENSE=        2-clause-bsd OR ruby-license

The license can be viewed with make show-license, and if the user so chooses, the line printed above can be added to `mk.conf` to convey to pkgsrc that it should not in the future fail because of that license:

    ACCEPTABLE_LICENSES+=xv-license

The use of `LICENSE=shareware`, `LICENSE=no-commercial-use`, and similar language is deprecated because it does not crisply refer to a particular license text. Another problem with such usage is that it does not enable a user to tell pkgsrc to proceed for a single package without also telling pkgsrc to proceed for all packages with that tag.

#### 21.1.3.1. Adding a package with a new license

When adding a package with a new license, the following steps are required:

1. Check whether the license qualifies as Free or Open Source by referencing Various Licenses and Comments about Them and Licenses by Name | Open Source Initiative. If this is the case, the filename in `pkgsrc/licenses/` does not need the `-license` suffix, and the license name should be added to:

   - DEFAULT_ACCEPTABLE_LICENSES in `pkgsrc/mk/license.mk`

   - default_acceptable_licenses in `pkgsrc/pkgtools/pkg_install/files/lib/license.c`

2. The license text should be added to `pkgsrc/licenses` for displaying. A list of known licenses can be seen in this directory.

#### 21.1.3.2. Change to the license

When the license changes (in a way other than formatting), make sure that the new license has a different name (e.g., append the version number if it exists, or the date). Just because a user told pkgsrc to build programs under a previous version of a license does not mean that pkgsrc should build programs under the new licenses. The higher-level point is that pkgsrc does not evaluate licenses for reasonableness; the only test is a mechanistic test of whether a particular text has been approved by either of two bodies (FSF or OSI).

### 21.1.4. Restricted packages

Some licenses restrict how software may be re-distributed. By declaring the restrictions, package tools can automatically refrain from e.g. placing binary packages on FTP sites.

There are four possible restrictions, which are the cross product of sources (distfiles) and binaries not being placed on FTP sites and CD-ROMs. Because this is rarely the exact language in any license, and because non-Free licenses tend to be different from each other, pkgsrc adopts a definition of FTP and CD-ROM. "FTP" means making the source or binary file available over the Internet at no charge. "CD-ROM" means making the source or binary available on some kind of media, together with other source and binary packages, which is sold for a distribution charge.

In order to encode these restrictions, the package system defines five make variables that can be set to note these restrictions:

- `RESTRICTED`

  This variable should be set whenever a restriction exists (regardless of its kind). Set this variable to a string containing the reason for the restriction. It should be understood that those wanting to understand the restriction will have to read the license, and perhaps seek advice of counsel.

- `NO_BIN_ON_CDROM`

  Binaries may not be placed on CD-ROM containing other binary packages, for which a distribution charge may be made. In this case, set this variable to `${RESTRICTED}`.

- `NO_BIN_ON_FTP`

  Binaries may not made available on the Internet without charge. In this case, set this variable to `${RESTRICTED}`. If this variable is set, binary packages will not be included on ftp.NetBSD.org.

- `NO_SRC_ON_CDROM`

  Distfiles may not be placed on CD-ROM, together with other distfiles, for which a fee may be charged. In this case, set this variable to `${RESTRICTED}`.

- `NO_SRC_ON_FTP`

  Distfiles may not made available via FTP at no charge. In this case, set this variable to `${RESTRICTED}`. If this variable is set, the distfile(s) will not be mirrored on ftp.NetBSD.org.

Please note that packages will be removed from pkgsrc when the distfiles are not distributable and cannot be obtained for a period of one full quarter branch. Packages with manual/interactive fetch must have a maintainer and it is his/her responsibility to ensure this.

### 21.1.5. Handling dependencies

Your package may depend on some other package being present, and there are various ways of expressing this dependency. pkgsrc supports the `DEPENDS`, `BUILD_DEPENDS`, `TOOL_DEPENDS`, and `TEST_DEPENDS` definitions, the `USE_TOOLS` definition, as well as dependencies via `buildlink3.mk`, which is the preferred way to handle dependencies, and which uses the variables named above. See Chapter 18, Buildlink methodology for more information.

The basic difference is that the `DEPENDS` definition registers that pre-requisite in the binary package so it will be pulled in when the binary package is later installed, whilst the `BUILD_DEPENDS`, `TOOL_DEPENDS`, and `TEST_DEPENDS` definitions do not, marking a dependency that is only needed for building or testing the resulting package. See also Chapter 14, Creating a new pkgsrc package from scratch for more information.

This means that if you only need a package present whilst you are building or testing, it should be noted as a `TOOL_DEPENDS`, `BUILD_DEPENDS`, or `TEST_DEPENDS`. When cross-compiling, `TOOL_DEPENDS` are native packages, i.e. packages for the architecture where the package is built; `BUILD_DEPENDS` are target packages, i.e., packages for the architecture for which the package is built.

The format for a `DEPENDS`, `BUILD_DEPENDS`, `TOOL_DEPENDS`, and `TEST_DEPENDS` definition is:

    <pre-req-package-name>:../../<category>/<pre-req-package>

Please note that the “pre-req-package-name” may include any of the wildcard version numbers recognized by pkg_info(1).

1. If your package needs another package's binaries or libraries to build and run, and if that package has a `buildlink3.mk` file available, use it:

       .include "../../graphics/jpeg/buildlink3.mk"

2. If your package needs another package's binaries or libraries only for building, and if that package has a `buildlink3.mk` file available, use it:

       .include "../../graphics/jpeg/buildlink3.mk"

   but set `BUILDLINK_DEPMETHOD.`jpeg`?=build` to make it a build dependency only. This case is rather rare.

3. If your package needs binaries from another package to build, use the `TOOL_DEPENDS` definition:

       TOOL_DEPENDS+= itstool-[0-9]*:../../textproc/itstool

4. If your package needs static libraries to link against, header files to include, etc. from another package to build, use the `BUILD_DEPENDS` definition.

5. If your package needs a library with which to link and there is no `buildlink3.mk` file available, create one. Using `DEPENDS` won't be sufficient because the include files and libraries will be hidden from the compiler.

6. If your package needs some executable to be able to run correctly and if there's no `buildlink3.mk` file, this is specified using the `DEPENDS` variable. The `print/lyx` package needs to be able to execute the latex binary from the tex-latex-bin package when it runs, and that is specified:

       DEPENDS+=        tex-latex-bin-[0-9]*:../../print/tex-latex-bin

7. If your package includes a test suite that has extra dependencies only required for this purpose (frequently this can be run as a “make test” target), use the `TEST_DEPENDS` variable.

8. You can use wildcards in package dependencies. Note that such wildcard dependencies are retained when creating binary packages. The dependency is checked when installing the binary package and any package which matches the pattern will be used. Wildcard dependencies should be used with care.

   The “-[0-9]*” should be used instead of “-*” to avoid potentially ambiguous matches such as “tk-postgresql” matching a “tk-*” `DEPENDS`.

   Wildcards can also be used to specify that a package will only build against a certain minimum version of a pre-requisite:

       DEPENDS+=       ImageMagick>=6.0:../../graphics/ImageMagick

   This means that the package will build using version 6.0 of ImageMagick or newer. Such a dependency may be warranted if, for example, the command line options of an executable have changed.

   If you need to depend on minimum versions of libraries, set `BUILDLINK_API_DEPENDS.`pkg`` to the appropriate pattern before including its `buildlink3.mk` file, e.g.

       BUILDLINK_API_DEPENDS.jpeg+=    jpeg>=9.0
       .include "../../graphics/jpeg/buildlink3.mk"

   For security fixes, please update the package vulnerabilities file. See Section 21.1.9, “Handling packages with security problems” for more information.

If your package needs files from another package to build, add the relevant distribution files to `DISTFILES`, so they will be extracted automatically. See the `print/ghostscript` package for an example. (It relies on the jpeg sources being present in source form during the build.)

### 21.1.6. Handling conflicts with other packages

Your package may conflict with other packages users might already have installed on their system, e.g., if your package installs the same set of files as another package in the pkgsrc tree.

For example, `x11/libXaw3d` and `x11/Xaw-Xpm` install the same shared library, thus you set in `pkgsrc/x11/libXaw3d/Makefile`:

    CONFLICTS=      Xaw-Xpm-[0-9]*

and in `pkgsrc/x11/Xaw-Xpm/Makefile`:

    CONFLICTS=      libXaw3d-[0-9]*

pkg_add(1) is able to detect attempts to install packages that conflict with existing packages and abort. However, in many situations this is too late in the process. Binary package managers will not know about the conflict until they attempt to install the package after already downloading it and all its dependencies. Users may also waste time building a package and its dependencies only to find out at the end that it conflicts with another package they have installed.

To avoid these issues `CONFLICTS` entries should be added in all cases where it is known that packages conflict with each other. These `CONFLICTS` entries are exported in pkg_summary(5) files and consumed by binary package managers to inform users that packages cannot be installed onto the target system.

### 21.1.7. Packages that cannot or should not be built

There are several reasons why a package might be instructed to not build under certain circumstances. If the package builds and runs on most platforms, the exceptions should be noted with `BROKEN_ON_PLATFORM`. If the package builds and runs on a small handful of platforms, set `BROKEN_EXCEPT_ON_PLATFORM` instead. Both `BROKEN_ON_PLATFORM` and `BROKEN_EXCEPT_ON_PLATFORM` are OS triples (OS-version-platform) that can use glob-style wildcards.

If a package is not appropriate for some platforms (as opposed to merely broken), a different set of variables should be used as this affects failure reporting and statistics. If the package is appropriate for most platforms, the exceptions should be noted with `NOT_FOR_PLATFORM`. If the package is appropriate for only a small handful of platforms (often exactly one), set `ONLY_FOR_PLATFORM` instead. Both `ONLY_FOR_PLATFORM` and `NOT_FOR_PLATFORM` are OS triples (OS-version-platform) that can use glob-style wildcards.

Some packages are tightly bound to a specific version of an operating system, e.g. LKMs or `sysutils/lsof`. Such binary packages are not backwards compatible with other versions of the OS, and should be uploaded to a version specific directory on the FTP server. Mark these packages by setting `OSVERSION_SPECIFIC` to “yes”. This variable is not currently used by any of the package system internals, but may be used in the future.

If the package should be skipped (for example, because it provides functionality already provided by the system), set `PKG_SKIP_REASON` to a descriptive message. If the package should fail because some preconditions are not met, set `PKG_FAIL_REASON` to a descriptive message.

### 21.1.8. Packages which should not be deleted, once installed

To ensure that a package may not be deleted, once it has been installed, the `PKG_PRESERVE` definition should be set in the package Makefile. This will be carried into any binary package that is made from this pkgsrc entry. A “preserved” package will not be deleted using pkg_delete(1) unless the “-f” option is used.

### 21.1.9. Handling packages with security problems

When a vulnerability is found, this should be noted in `pkgsrc/doc/pkg-vulnerabilities`. Entries in that file consist of three parts:

- package version pattern

- type of vulnerability (please cut'n'paste an existing one where possible)

- URL providing additional information about the issue

For the package version pattern please always use `<' to mark an upper bound (not `<='!). This will avoid possible problems due unrelated `PKGREVISION` bumps not related to security fixes. Lower bounds can be added too, using '>' or '>='. For example, “`foo>=1<1.2`” would mark versions 1.0 (included) to 1.2 (excluded) of “`foo`” as affected by the security issue.

Entries should always be added at the bottom of the file.

When fixing packages, please modify the upper bound of the corresponding entry. To continue the previous example, if a fix was backported to version 1.1nb2, change the previous pattern to “`foo>=1<1.1nb2`”.

To locally test a package version pattern against a `PKGNAME` you can use the pkg_admin pmatch command.

The URL should be as permanent as possible and provide as much information about the issue as possible. CVE entries are preferred.

After committing that file, ask pkgsrc-security@NetBSD.org to update the file on ftp.NetBSD.org.

After fixing the vulnerability by a patch, its `PKGREVISION` should be increased (this is of course not necessary if the problem is fixed by using a newer release of the software), and the pattern in the pkg-vulnerabilities file must be updated.

Also, if the fix should be applied to the stable pkgsrc branch, be sure to submit a pullup request!

Binary packages already on ftp.NetBSD.org will be handled semi-automatically by a weekly cron job.

In case a security issue is disputed, please contact pkgsrc-security@NetBSD.org.

### 21.1.10. How to handle incrementing versions when fixing an existing package

When making fixes to an existing package it can be useful to change the version number in `PKGNAME`. To avoid conflicting with future versions by the original author, a “nb1”, “nb2”, ... suffix can be used on package versions by setting `PKGREVISION=1` (2, ...). The “nb” is treated like a “.” by the package tools. e.g.

    DISTNAME=       foo-17.42
    PKGREVISION=    9

will result in a `PKGNAME` of “foo-17.42nb9”. If you want to use the original value of `PKGNAME` without the “nbX” suffix, e.g. for setting `DIST_SUBDIR`, use `PKGNAME_NOREV`.

When a new release of the package is released, the `PKGREVISION` should be removed, e.g. on a new minor release of the above package, things should be like:

    DISTNAME=       foo-17.43

`PKGREVISION` should be incremented for any non-trivial change in the resulting binary package. Without a `PKGREVISION` bump, someone with the previous version installed has no way of knowing that their package is out of date. Thus, changes without increasing `PKGREVISION` are essentially labeled "this is so trivial that no reasonable person would want to upgrade", and this is the rough test for when increasing `PKGREVISION` is appropriate. Examples of changes that do not merit increasing `PKGREVISION` are:

- Changing `HOMEPAGE`, `MAINTAINER`, `OWNER`, or comments in Makefile.

- Changing build variables if the resulting binary package is the same.

- Changing `DESCR`.

- Adding `PKG_OPTIONS` if the default options don't change.

Examples of changes that do merit an increase to `PKGREVISION` include:

- Security fixes

- Changes or additions to a patch file

- Changes to the `PLIST`

- A dependency is changed or renamed.

PKGREVISION must also be incremented when dependencies have ABI changes.

### 21.1.11. Substituting variable text in the package files (the SUBST framework)

When you want to replace the same text in multiple files, or multiple times in the same file, it is cumbersome to maintain a patch file for this. This is where the SUBST framework steps in. It provides an easy-to-use interface for replacing text in files. It just needs the following information:

- In which phase of the package build cycle should the replacement happen?

- In which files should the replacement happen?

- Which text should be replaced with what?

This information is encoded in a block of `SUBST` variables. A minimal example is:

    SUBST_CLASSES+=         paths
    SUBST_STAGE.paths=      pre-configure
    SUBST_FILES.paths=      src/*.c
    SUBST_SED.paths=        -e 's,/usr/local,${PREFIX},g'

Translated into English, it means: In the pre-configure stage (that is, after applying the patches from the patches/ directory and before running the configure script and the portability check), replace the text `/usr/local` with the content of the variable `PREFIX`.

Each SUBST block starts by appending an identifier to `SUBST_CLASSES` (note the `+=`). This identifier can be chosen freely by the package. If there should ever be duplicate identifiers, the pkgsrc infrastructure will catch this and fail early, so don't worry about name collisions.

Except for `SUBST_CLASSES`, all variables in a SUBST block are parameterized using this identifier. In the remainder of this section, these parameterized variables are written as `SUBST_STAGE.*`.

    SUBST_CLASSES+=         paths
    SUBST_STAGE.paths=      pre-configure
    SUBST_MESSAGE.paths=    Fixing absolute paths.
    SUBST_FILES.paths=      src/*.c
    SUBST_FILES.paths+=     scripts/*.sh
    SUBST_SED.paths=        -e 's,"/usr/local,"${PREFIX},g'
    SUBST_SED.paths+=       -e 's,"/var/log,"${VARBASE}/log,g'
    SUBST_VARS.paths=       LOCALBASE PREFIX PKGVERSION

To get a complete picture about the SUBST substitutions, run bmake show-all-subst. If something doesn't work as expected, run pkglint on the package, which detects several typical mistakes surrounding the SUBST blocks. For any questions that might remain after this, have a look at `mk/subst.mk`.

#### 21.1.11.1. Choosing the time where the substitutions happen

The `SUBST_STAGE.*` is one of {pre,do,post}-{extract,patch,configure,build,test,install}. Of these, `pre-configure` is used most often, by far. The most popular stages are, in chronological order:

`post-extract`

  The substitutions are applied immediately after the distfiles are extracted. Running bmake extract on the package will leave no traces of the original files.

  When the substitution applies to files for which there is also a patch in the `patches/` directory, this means that the patches will be computed based on the result of the substitution. When these patches are sent to the upstream maintainer later, to be fixed in the upstream package, these patches may no longer match what the upstream author is used to. Because of this, `pre-configure` is often a better choice.

`pre-configure`

  The substitutions are applied after the patches from the `patches/` directory. This makes it possible to run bmake patch on the package, after which the patches can be edited using the tools pkgvi and mkpatches from the `pkgtools/pkgdiff` package.

  When updating the patches, it is helpful to explicitly separate the bmake patch from the bmake configure, and to only edit the patches between these commands. Otherwise the substitutions from the SUBST block will end up in the patch file. When this happens in really obvious ways, pkglint will complain about patches that contain a hard-coded `/usr/pkg` instead of the correct and intended `@PREFIX@`, but it can only detect these really obvious cases.

`do-configure`

  This stage should only be used if the package defines a `pre-configure` action itself, and the substitution must happen after that. Typical examples are packages that use the `pre-configure` stage to regenerate the GNU configure script from `configure.ac`.

`post-configure`

  This stage is used to fix up any mistakes by the configure stage.

`pre-build`

  This stage should only be used for substitutions that are clearly related to building the package, not for fixing the configuration. Substitutions for pathnames (such as replacing `/usr/local` with `${PREFIX}`) or user names (such as replacing `@MY_USER@` with the actual username) belong in pre-configure or post-configure instead.

`post-build`

  Just as with pre-build, this stage should only be used for substitutions that are clearly related to building the package, not for fixing the configuration. Substitutions for pathnames (such as replacing `/usr/local` with `${PREFIX}`) or user names (such as replacing `@MY_USER@` with the actual username) belong in pre-configure or post-configure instead.

  A typical use is to update pkg-config files to include the rpath compiler options.

`pre-install`

  In general, the install phase should be as simple as possible. As with the pre-build and post-build stages, it should not be used to fix pathnames or user names, these belong in pre-configure instead. There are only few legitimate use cases for applying substitutions in this stage.

#### 21.1.11.2. Choosing the files where the substitutions happen

The `SUBST_FILES.*` variable contains a list of filename patterns. These patterns are relative to `WRKSRC` since that is where most substitutions happen. A typical example is:

    SUBST_FILES.path=       Makefile */Makefile */*/Makefile *.[ch]

The above patterns, especially the last, are quite broad. The SUBST implementation checks that each filename pattern that is mentioned here has an effect. For example, if none of the `*/*/Makefile` files contains the patterns to be found and substituted, that filename pattern is redundant and should be left out. By default, the SUBST framework will complain with an error message. If the text to be substituted occurs in some of the files from a single pattern, but not in all of them, that is totally ok, and the SUBST framework will only print an INFO message for those files.

If there is a good reason for having redundant filename patterns, set `SUBST_NOOP_OK.*` to `yes`.

Another popular way of choosing the files for the substitutions is via a shell command, like this:

    C_FILES_CMD=            cd ${WRKSRC} && ${FIND} . -name '*.c'
    SUBST_FILES.path=       ${C_FILES_CMD:sh}

The variable name `C_FILES_CMD` in this example is freely chosen and independent of the SUBST framework.

In this variant, the `SUBST_FILES.*` variable lists each file individually. Thereby chances are higher that there are filename patterns in which no substitution happens. Since the SUBST framework cannot know whether the filename patterns in `SUBST_FILES.*` have been explicitly listed in the Makefile (where any redundant filename pattern would be suspicious) or been generated by a shell command (in which redundant filename patterns are more likely and to be expected), it will complain about these redundant filename patterns. Therefore, SUBST blocks that use a shell command to generate the list of filename patterns often need to set `SUBST_NOOP_OK.*` to `yes`.

#### 21.1.11.3. Choosing what to substitute

In most cases, the substitutions are given using one or more sed(1) commands, like this:

    SUBST_SED.path=         -e 's|/usr/local|${PREFIX}|g'

Each of the sed commands needs to be preceded by the `-e` option and should be specified on a line of its own, to avoid hiding short sed commands at the end of a line.

Since the sed commands often contain shell metacharacters as the separator (the `|` in the above example), it is common to enclose them in single quotes.

A common substitution is to replace placeholders of the form `@VARNAME@` with their pkgsrc counterpart variable `${VARNAME}`. A typical example is:

    SUBST_VARS.path=        PREFIX

This type of substitutions is typically done by the GNU configure scripts during the do-configure stage, but in some cases these need to be overridden. The same pattern is also used when a package defines patches that replace previously hard-coded paths like `/usr/local` with a `@PREFIX@` placeholder first, which then gets substituted by the actual `${PREFIX}` in the pre-configure stage. In many of these cases, it works equally well to just use the SUBST framework to directly replace `/usr/local` with `${PREFIX}`, thereby omitting the intermediate patch file.

If the above is not flexible enough, it is possible to not use sed at all for the substitution but to specify an entirely different command, like this:

    SUBST_FILTER_CMD.path=  LC_ALL=C ${TR} -d '\r'

This is used for the few remaining packages in which the distributed files use Windows-style line endings that need to be converted to UNIX-style line endings.

#### 21.1.11.4. Other SUBST variables

When a SUBST block is applied during a package build, a message is logged. The default message is fine for most purposes but can be overridden by setting `SUBST_MESSAGE.*` to an individual message.

## 21.2. The fetch phase

### 21.2.1. Packages whose distfiles aren't available for plain downloading

If you need to download from a dynamic URL you can set `DYNAMIC_MASTER_SITES` and a make fetch will call `files/getsite.sh` with the name of each file to download as an argument, expecting it to output the URL of the directory from which to download it. `graphics/ns-cult3d` is an example of this usage.

If the download can't be automated, because the user must submit personal information to apply for a password, or must pay for the source, or whatever, you can set `FETCH_MESSAGE` to a list of lines that are displayed to the user before aborting the build. Example:

    FETCH_MESSAGE=  "Please download the files"
    FETCH_MESSAGE+= "    "${DISTFILES:Q}
    FETCH_MESSAGE+= "manually from "${MASTER_SITES:Q}"."

### 21.2.2. How to handle modified distfiles with the 'old' name

Sometimes authors of a software package make some modifications after the software was released, and they put up a new distfile without changing the package's version number. If a package is already in pkgsrc at that time, the checksum will no longer match. The contents of the new distfile should be compared against the old one before changing anything, to make sure the distfile was really updated on purpose, and that no trojan horse or so crept in. Please mention that the distfiles were compared and what was found in your commit message.

Then, the correct way to work around this is to set `DIST_SUBDIR` to a unique directory name, usually based on `PKGNAME_NOREV` (but take care with python or ruby packages, where `PKGNAME` includes a variable prefix). All `DISTFILES` and `PATCHFILES` for this package will be put in that subdirectory of the local distfiles directory. (See Section 21.1.10, “How to handle incrementing versions when fixing an existing package” for more details.) In case this happens more often, `PKGNAME` can be used (thus including the `nbX` suffix) or a date stamp can be appended, like `${PKGNAME_NOREV}-YYYYMMDD`.

`DIST_SUBDIR` is also used when a distfile's name does not contain a version and the distfile is apt to change. In cases where the likelihood of this is very small, `DIST_SUBDIR` might not be required. Additionally, `DIST_SUBDIR` must not be removed unless the distfile name changes, even if a package is being moved or renamed.

Do not forget regenerating the `distinfo` file after that, since it contains the `DIST_SUBDIR` path in the filenames. Also, increase the PKGREVISION if the installed package is different. Furthermore, a mail to the package's authors seems appropriate telling them that changing distfiles after releases without changing the file names is not good practice.

### 21.2.3. Packages hosted on github.com

Helper methods exist for packages hosted on github.com which will often have distfile names that clash with other packages, for example `1.0.tar.gz`. Use one of the three recipes from below:

#### 21.2.3.1. Fetch based on a tagged release

If your distfile URL looks similar to `https://github.com/username/example/archive/v1.0.zip`, then you are packaging a tagged release.

    DISTNAME=       example-1.0
    MASTER_SITES=   ${MASTER_SITE_GITHUB:=username/}
    #GITHUB_PROJECT= example    # can be omitted if same as DISTNAME
    GITHUB_TAG=     v${PKGVERSION_NOREV}
    EXTRACT_SUFX=   .zip

Here, DISTNAME combined with use of GITHUB_TAG leads the file fetching infrastructure to save the resulting file locally as `example-1.0.zip`.

#### 21.2.3.2. Fetch based on a specific commit before the first release

If your distfile looks similar to `https://github.com/username/example/archive/988881adc9fc3655077dc2d4d757d480b5ea0e11` and is from a commit before the first release, then set the package version to 0.0.0.N, where N is the number of commits to the repository, and set GITHUB_TAG to the commit hash. This will (almost) ensure that the first tagged release will have a version greater than this one so that package upgrades will function properly.

    DISTNAME=       example-0.0.0.347
    MASTER_SITES=   ${MASTER_SITE_GITHUB:=username/}
    #GITHUB_PROJECT= example    # can be omitted if same as DISTNAME
    GITHUB_TAG=     988881adc9fc3655077dc2d4d757d480b5ea0e11

#### 21.2.3.3. Fetch based on a specific commit after a release

If your distfile looks similar to `https://github.com/username/example/archive/988881adc9fc3655077dc2d4d757d480b5ea0e11` and is from a commit after a release, then include the last release version and the commit count since that release in the package version and set GITHUB_TAG to the commit hash. The latest release and commit count are shown in the output of "git describe --tags":

    # git clone https://github.com/username/example
    # cd example
    # git describe --tags
    1.2.3-5-g988881a

    DISTNAME=       example-1.2.3.5
    MASTER_SITES=   ${MASTER_SITE_GITHUB:=username/}
    #GITHUB_PROJECT= example    # can be omitted if same as DISTNAME
    GITHUB_TAG=     988881adc9fc3655077dc2d4d757d480b5ea0e11

#### 21.2.3.4. Fetch based on release

If your distfile URL looks similar to `https://github.com/username/example/releases/download/rel-1.6/offensive-1.6.zip`, then you are packaging a release.

    DISTNAME=       offensive-1.6
    PKGNAME=        ${DISTNAME:S/offensive/proper/}
    MASTER_SITES=   ${MASTER_SITE_GITHUB:=username/}
    GITHUB_PROJECT= example
    GITHUB_RELEASE= rel-${PKGVERSION_NOREV} # usually just set this to ${DISTNAME}
    EXTRACT_SUFX=   .zip

## 21.3. The configure phase

### 21.3.1. Shared libraries - libtool

pkgsrc supports many different machines, with different object formats like a.out and ELF, and varying abilities to do shared library and dynamic loading at all. To accompany this, varying commands and options have to be passed to the compiler, linker, etc. to get the Right Thing, which can be pretty annoying especially if you don't have all the machines at your hand to test things. The `devel/libtool` pkg can help here, as it just “knows” how to build both static and dynamic libraries from a set of source files, thus being platform-independent.

Here's how to use libtool in a package in seven simple steps:

1. Add `USE_LIBTOOL=yes` to the package Makefile.

2. For library objects, use “${LIBTOOL} --mode=compile ${CC}” in place of “${CC}”. You could even add it to the definition of `CC`, if only libraries are being built in a given Makefile. This one command will build both PIC and non-PIC library objects, so you need not have separate shared and non-shared library rules.

3. For the linking of the library, remove any “ar”, “ranlib”, and “ld -Bshareable” commands, and instead use:

       ${LIBTOOL} --mode=link \
           ${CC} -o ${.TARGET:.a=.la} \
               ${OBJS:.o=.lo} \
               -rpath ${PREFIX}/lib \
               -version-info major:minor

   Note that the library is changed to have a `.la` extension, and the objects are changed to have a `.lo` extension. Change `OBJS` as necessary. This automatically creates all of the `.a`, `.so.major.minor`, and ELF symlinks (if necessary) in the build directory. Be sure to include “-version-info”, especially when major and minor are zero, as libtool will otherwise strip off the shared library version.

   From the libtool manual:

       So, libtool library versions are described by three integers:

       CURRENT
       The most recent interface number that this library implements.

       REVISION
       The implementation number of the CURRENT interface.

       AGE
       The difference between the newest and oldest interfaces that
       this library implements.  In other words, the library implements
       all the interface numbers in the range from number `CURRENT -
       AGE' to `CURRENT'.

       If two libraries have identical CURRENT and AGE numbers, then the
       dynamic linker chooses the library with the greater REVISION number.

   The “-release” option will produce different results for a.out and ELF (excluding symlinks) in only one case. An ELF library of the form “libfoo-release.so.x.y” will have a symlink of “libfoo.so.x.y” on an a.out platform. This is handled automatically.

   The “-rpath argument” is the install directory of the library being built.

   In the `PLIST`, include only the `.la` file, the other files will be added automatically.

4. When linking shared object (`.so`) files, i.e. files that are loaded via dlopen(3), NOT shared libraries, use “-module -avoid-version” to prevent them getting version tacked on.

   The `PLIST` file gets the `foo.so` entry.

5. When linking programs that depend on these libraries before they are installed, preface the cc(1) or ld(1) line with “${LIBTOOL} --mode=link”, and it will find the correct libraries (static or shared), but please be aware that libtool will not allow you to specify a relative path in -L (such as “-L../somelib”), because it expects you to change that argument to be the `.la` file. e.g.

       ${LIBTOOL} --mode=link ${CC} -o someprog -L../somelib -lsomelib

   should be changed to:

       ${LIBTOOL} --mode=link ${CC} -o someprog ../somelib/somelib.la

   and it will do the right thing with the libraries.

6. When installing libraries, preface the install(1) or cp(1) command with “${LIBTOOL} --mode=install”, and change the library name to `.la`. e.g.

       ${LIBTOOL} --mode=install ${BSD_INSTALL_LIB} ${SOMELIB:.a=.la} ${PREFIX}/lib

   This will install the static `.a`, shared library, any needed symlinks, and run ldconfig(8).

7. In your `PLIST`, include only the `.la` file (this is a change from previous behaviour).

### 21.3.2. Using libtool on GNU packages that already support libtool

Add `USE_LIBTOOL=yes` to the package Makefile. This will override the package's own libtool in most cases. For older libtool using packages, libtool is made by ltconfig script during the do-configure step; you can check the libtool script location by doing make configure; find work*/ -name libtool.

`LIBTOOL_OVERRIDE` specifies which libtool scripts, relative to `WRKSRC`, to override. By default, it is set to “libtool */libtool */*/libtool”. If this does not match the location of the package's libtool script(s), set it as appropriate.

If you do not need `*.a` static libraries built and installed, then use `SHLIBTOOL_OVERRIDE` instead.

If your package makes use of the platform-independent library for loading dynamic shared objects, that comes with libtool (libltdl), you should include devel/libltdl/buildlink3.mk.

Some packages use libtool incorrectly so that the package may not work or build in some circumstances. Some of the more common errors are:

- The inclusion of a shared object (-module) as a dependent library in an executable or library. This in itself isn't a problem if one of two things has been done:

  1. The shared object is named correctly, i.e. `libfoo.la`, not `foo.la`

  2. The -dlopen option is used when linking an executable.

- The use of libltdl without the correct calls to initialisation routines. The function lt_dlinit() should be called and the macro `LTDL_SET_PRELOADED_SYMBOLS` included in executables.

### 21.3.3. GNU Autoconf/Automake

If a package needs GNU autoconf or automake to be executed to regenerate the `configure` script and `Makefile.in` makefile templates from `configure.ac` and `Makefile.am`, then they should be executed in a pre-configure target:

    USE_TOOLS+=	autoconf automake autoreconf
    GNU_CONFIGURE=	yes
    ...

    pre-configure:
            set -e; cd ${WRKSRC} && autoreconf -fi
    ...

Packages which use GNU Automake will sometimes require GNU Make (`gmake` in `USE_TOOLS`), but not always. Note that autoreconf only needs to be executed if `configure.ac` or Makefiles are modified, or `configure` is not present.

There are times when the configure process makes additional changes to the generated files, which then causes the build process to try to re-execute the automake sequence. This is prevented by touching various files in the configure stage. If this causes problems with your package you can set `AUTOMAKE_OVERRIDE=NO` in the package Makefile.

### 21.3.4. Meson / ninja

Packages using Meson to configure need to include:

    .include "../../devel/meson/build.mk"

In nearly all cases (any program with dependencies), pkg-config needs to be added to `USE_TOOLS`. If the package installs translation files for non-English languages, also add msgfmt and xgettext:

    USE_TOOLS+=	pkg-config msgfmt xgettext

If any options need to be passed to Meson, use `MESON_ARGS` instead of `CONFIGURE_ARGS`:

    MESON_ARGS+=	-Dx11=false

### 21.3.5. Determining the endianness

To determine the endianness of the target platform, have a look at `mk/endian.mk`.

## 21.4. Programming languages

### 21.4.1. C, C++, and Fortran

Compilers for the C and C++ languages come with the NetBSD base system. By default, pkgsrc assumes that a package is written in C and will hide all other compilers (via the wrapper framework, see Chapter 18, Buildlink methodology).

To declare which languages should be made available through pkgsrc's compiler wrappers, use the `USE_LANGUAGES` variable. Allowed values currently are:

    c, c++, fortran, fortran77, java, objc, obj-c++, and ada.

(and any combination). The default is “c”. Packages using GNU configure scripts, even if written in C++, usually need a C compiler for the configure phase.

To declare which features a package requires from the compiler, set either `USE_CC_FEATURES` or `USE_CXX_FEATURES`. Allowed values for `USE_CC_FEATURES` are currently:

    c11, c99, has_include

Allowed values for `USE_CXX_FEATURES` are currently:

    c++11, c++14, c++17, c++20, has_include, regex, filesystem,
    charconv, parallelism_ts, unique_ptr, put_time,
    is_trivially_copy_constructible

Note at present these variables only affect use of GCC, not other compilers.

Language variants like `gnu99` and `c++11` can be specified in `FORCE_C_STD` and `FORCE_CXX_STD` if the package does not explicitly set `-std=...` when compiling (i.e. the package assumes the compiler defaults to C++11 or some other standard). This is a common bug in upstream build systems.

Allowed values for `FORCE_C_STD` are currently:

    c90, c99, c11, gnu90, gnu99, gnu11

Allowed values for `FORCE_CXX_STD` are currently:

    c++03, c++11, c++14, c++17, c++20,
    gnu++03, gnu++11, gnu++14, gnu++17, gnu++20

Note at present these variables only affect use of GCC and Clang.

### 21.4.2. Java

If a program is written in Java, use the Java framework in pkgsrc. The package must include `../../mk/java-vm.mk`. This Makefile fragment provides the following variables:

- `USE_JAVA` defines if a build dependency on the JDK is added. If `USE_JAVA` is set to “run”, then there is only a runtime dependency on the JDK. The default is “yes”, which also adds a build dependency on the JDK.

- Set `USE_JAVA2` to declare that a package needs a Java2 implementation. The supported values are “yes”, “1.4”, and “1.5”. “yes” accepts any Java2 implementation, “1.4” insists on versions 1.4 or above, and “1.5” only accepts versions 1.5 or above. This variable is not set by default.

- `PKG_JAVA_HOME` is automatically set to the runtime location of the used Java implementation dependency. It may be used to set `JAVA_HOME` to a good value if the program needs this variable to be defined.

### 21.4.3. Go

If a program is written in Go and has any dependencies on other Go modules, have the package include `../../lang/go/go-module.mk`.

1. Generate a list of those dependencies with make clean && make patch && make show-go-modules > go-modules.mk.

2. Prepend `.include "go-modules.mk"` to any other `.include`s.

3. Incorporate these modules in `distinfo` with make makesum.

### 21.4.4. Rust

If a program is written in Rust and uses Cargo to build, have the package include `../../lang/rust/cargo.mk`.

1. Generate a list of those dependencies with make CARGO_ARGS="build --release" build && make print-cargo-depends > cargo-depends.mk.

2. Prepend `.include "cargo-depends.mk"` to any other `.include`s.

3. Incorporate these modules in `distinfo` with make makesum.

### 21.4.5. Packages containing Perl scripts

If your package contains interpreted Perl scripts, add “perl” to the `USE_TOOLS` variable and set `REPLACE_PERL` to ensure that the proper interpreter path is set. `REPLACE_PERL` should contain a list of scripts, relative to `WRKSRC`, that you want adjusted. Every occurrence of `*/bin/perl` in a she-bang line will be replaced with the full path to the Perl executable.

If a particular version of Perl is needed, set the `PERL5_REQD` variable to the version number. The default is “5.0”.

See Section 21.6.6, “Packages installing Perl modules” for information about handling Perl modules.

There is also the `REPLACE_PERL6` variable for the language now known as Raku.

### 21.4.6. Packages containing shell scripts

`REPLACE_SH`, `REPLACE_BASH`, `REPLACE_CSH`, and `REPLACE_KSH` can be used to replace shell she-bangs in files. Please use the appropriate one, preferring `REPLACE_SH` when this shell is sufficient. Each should contain a list of scripts, relative to `WRKSRC`, that you want adjusted. Every occurrence of the matching shell in a she-bang line will be replaced with the full path to the shell executable. When using `REPLACE_BASH`, don't forget to add `bash` to `USE_TOOLS`.

### 21.4.7. Other programming languages

There are further similar REPLACE variables available, e.g., `REPLACE_AWK` for packages containing awk scripts, and `REPLACE_R` for R. These two, like the others noted above, have their actions defined centrally in `mk/configure/replace-interpreter.mk`. Other languages define the actions of these variables within their own dedicated part of the tree, e.g., `REPLACE_PHP` is actioned in `lang/php/replace.mk`, and `REPLACE_PYTHON` is actioned in `lang/python/application.mk`. For other languages, consult the mk files found within their specific directories (the naming convention varies), or check the list found in Appendix E, Help topics.

Currently, special handling for other languages varies in pkgsrc. If a compiler package provides a `buildlink3.mk` file, include that, otherwise just add a (build) dependency on the appropriate compiler package.

## 21.5. The build phase

The most common failures when building a package are that some platforms do not provide certain header files, functions or libraries, or they provide the functions in a library that the original package author didn't know. To work around this, you can rewrite the source code in most cases so that it does not use the missing functions or provides a replacement function.

### 21.5.1. Compiling C and C++ code conditionally

If a package already comes with a GNU configure script, the preferred way to fix the build failure is to change the configure script, not the code. In the other cases, you can utilize the C preprocessor, which defines certain macros depending on the operating system and hardware architecture it compiles for. These macros can be queried using for example `#if defined(__i386)`. Almost every operating system, hardware architecture and compiler has its own macro. For example, if the macros `__GNUC__`, `__i386__` and `__NetBSD__` are all defined, you know that you are using NetBSD on an i386 compatible CPU, and your compiler is GCC.

The list of the following macros for hardware and operating system depends on the compiler that is used. For example, if you want to conditionally compile code on Solaris, don't use `__sun__`, as the SunPro compiler does not define it. Use `__sun` instead.

#### 21.5.1.1. C preprocessor macros to identify the operating system

To distinguish between specific NetBSD versions, you should use the following code.

    #ifdef __NetBSD__
    #include <sys/param.h>
    #if __NetBSD_Prereq__(9,99,17)
    /* use a newer feature */
    #else
    /* older code */
    #endif
    #endif

    #ifndef _WIN32
    /* Unix-like specific code */
    #endif

To distinguish between 4.4 BSD-derived systems and the rest of the world, you should use the following code.

    #include <sys/param.h>
    #if (defined(BSD) && BSD >= 199306)
    /* BSD-specific code goes here */
    #else
    /* non-BSD-specific code goes here */
    #endif

You can also test for the following macros:

    Cygwin      __CYGWIN__
    DragonFly   __DragonFly__
    FreeBSD     __FreeBSD__
    Haiku       __HAIKU__
    Interix     __INTERIX
    IRIX        __sgi (TODO: get a definite source for this)
    Linux       __linux
    Mac OS X    __APPLE__
    MirBSD      __MirBSD__ (__OpenBSD__ is also defined)
    Minix3      __minix
    NetBSD      __NetBSD__
    OpenBSD     __OpenBSD__
    Solaris     sun, __sun

#### 21.5.1.2. C preprocessor macros to identify the hardware architecture

    i386        i386, __i386, __i386__
    x86-64      __amd64__, __x86_64__
    ARM         __arm__
    MIPS        __mips
    SPARC       sparc, __sparc
    PowerPC     __powerpc

#### 21.5.1.3. C preprocessor macros to identify the compiler

    GCC         __GNUC__ (major version), __GNUC_MINOR__
    MIPSpro     _COMPILER_VERSION (0x741 for MIPSpro 7.41)
    SunPro      __SUNPRO_C (0x570 for Sun C 5.7)
    SunPro C++  __SUNPRO_CC (0x580 for Sun C++ 5.8)

### 21.5.2. How to handle compiler bugs

Some source files trigger bugs in the compiler, based on combinations of compiler version and architecture and almost always relation to optimisation being enabled. Common symptoms are gcc internal errors or never finishing compiling a file.

Typically, a workaround involves testing the `MACHINE_ARCH` and compiler version, disabling optimisation for that combination of file, `MACHINE_ARCH` and compiler.

This used to be a big problem in the past, but is rarely needed now as compiler technology has matured. If you still need to add a compiler specific workaround, please do so in the file `hacks.mk` and describe the symptom and compiler version as detailed as possible.

### 21.5.3. No such file or directory

Compilation sometimes fails with an error message like this:

    .../x11/gtk3/work/gtk+-3.24.12/gdk/gdktypes.h:35:10:
        fatal error: pango/pango.h: No such file or directory

The proper way to fix this problem depends on the type of the header, which is described in the following sections.

#### 21.5.3.1. Headers from other packages

If the header name looks like it comes from a different package, that other package should be included via the buildlink3 framework.

First, look whether the header is somewhere in the buildlink3 directory below `WRKDIR`. In the above case of the missing Pango header:

    $ find work/.buildlink/ -print | grep -F pango/pango.h

In the case of Pango, the output is:

    work/.buildlink/include/pango-1.0/pango/pango.h

If the `pango/pango.h` file were placed directly in the `.buildlink` directory, it would have been found automatically. There is an extra `pango-1.0` path component though, which means that the compiler command line must contain an option of the form `-I${BUILDLINK3_PREFIX.pango}/include/pango-1.0`. In most cases this option is generated by the configure script, which can be examined using:

    $ $ grep -o '[-]I[^[:space:]]*/pango[^[:space:]]*' work/*/Makefile
    -I/usr/pkg/include/pango-1.0
    -I/usr/pkg/include/pango-1.0
    -I/usr/pkg/include/pango-1.0
    -I/usr/pkg/include/pango-1.0
    -I/usr/pkg/include/pango-1.0

This looks good. These options are transformed by the buildlink wrapper to refer to the correct path inside `work/.buildlink`.

Since the compilation fails though, examine the compiler command lines in `work/.work.log` to see whether the `-I` option is included in the particular command line.

To further analyze the situation, run bmake build-env, which sets up an interactive, realistic environment including all the pkgsrc wrapper commands and environment variables. From there, try to compile some simple example programs that use the header.

#### 21.5.3.2. Headers generated during the build

If the name of the header seems to come from the package itself, and if the build is run with parallel jobs, the package may have some undeclared dependencies between the `.c` and the `.h` files, and a C file is compiled before its required header is generated.

To see whether the build runs with parallel jobs, run bmake show-all-build | grep JOBS. Its output looks like this:

      usr   MAKE_JOBS=              7
      pkg   MAKE_JOBS_SAFE          # undefined
      def   _MAKE_JOBS_N=           7

In this case the pkgsrc user has asked pkgsrc to build packages with 7 jobs in parallel (`MAKE_JOBS`). The package could have disabled parallel builds by setting `MAKE_JOBS_SAFE` to `no`, but in this case it hasn't.

To see whether the build failure is caused by parallel builds, first save the exact error message and a bit of context, maybe you need it later for reporting a bug. Next, run:

    MAKE_JOBS_SAFE=no bmake clean build

If that succeeds, file a bug report against the pkgsrc package, including the exact error message and the contents of your `mk.conf` file.

#### 21.5.3.3. Symlinks

Pkgsrc does not work reliably if any of `LOCALBASE`, `VARBASE` or `WRKDIR` contains a symlink. Since 2019Q2, the pkgsrc bootstrap program prevents installing pkgsrc in symlink-based directories. Existing pkgsrc installations are not checked for symlinks though.

The "No such file or directory" error messages are a typical symptom of symlinks, and it's quite difficult to find out that this is the actual cause.

#### 21.5.3.4. Stale working directories

When building a hierarchy of packages, it may happen that one package is built and then pkgsrc is updated. This situation can provoke various hard to diagnose build errors. To clean up the situation:

    $ (cd ../../ && cat mk/bsd.pkg.mk >/dev/null && rm -rf */*/work)

(The only purpose of the `bsd.pkg.mk` is to prevent running this command in the wrong directory.)

If you have set `WRKOBJDIR` in `mk.conf`, remove that directory as well.

#### 21.5.3.5. Other possible reasons

On platforms other than BSD, third-party packages are installed in `/usr/include`, together with the base system. This means that pkgsrc cannot distinguish between headers provided by the base system (which it needs) and headers from third-party packages (which are often included in pkgsrc as well). This can lead to subtle version mismatches.

In pkgsrc installations that have been active for several years, it may happen that some files are manually deleted. To exclude this unlikely reason, run pkg_admin check.

It may help to run pkg_admin rebuild-tree to check/fix dependencies.

If all of the above doesn't help, see Chapter 2, Getting help for contact information. Be prepared to describe what you have tried so far and what any error messages were.

### 21.5.4. Undefined reference to “...”

This error message often means that a package did not link to a shared library it needs. The following functions are known to cause this error message over and over.

Function | Library | Affected platforms
--- | --- | ---
accept, bind, connect | -lsocket | Solaris
crypt | -lcrypt | DragonFly, NetBSD
dlopen, dlsym | -ldl | Linux
gethost* | -lnsl | Solaris
inet_aton | -lresolv | Solaris
nanosleep, sem_*, timer_* | -lrt | Solaris
openpty | -lutil | Linux

To fix these linker errors, it is often sufficient to add `LIBS.`OperatingSystem`+= -l`foo`` to the package `Makefile` and then run bmake clean; bmake.

#### 21.5.4.1. The SunPro compiler and inline functions

When you are using the SunPro compiler, there is another possibility. That compiler cannot handle the following code:

    extern int extern_func(int);

    static inline int
    inline_func(int x)
    {
            return extern_func(x);
    }

    int main(void)
    {
            return 0;
    }

It generates the code for `inline_func` even if that function is never used. This code then refers to `extern_func`, which can usually not be resolved. To solve this problem you can try to tell the package to disable inlining of functions.

#### 21.5.4.2. Missing atomic functions

When building for older machine architectures (e.g., i386, PowerPC), builds may fail because the package expects modern 64-bit atomic functions which the underlying hardware either doesn't support, or will only support with specific compiler flags. This is generally handled via inclusion of mk/atomic64.mk.

### 21.5.5. Running out of memory

Sometimes packages fail to build because the compiler runs into an operating system specific soft limit. With the `UNLIMIT_RESOURCES` variable pkgsrc can be told to unlimit the resources. The allowed values are any combination of “cputime”, “datasize”, “memorysize”, “stacksize” and “virtualsize”. Setting this variable is similar to running the shell builtin ulimit command to raise the maximum data segment size or maximum stack size of a process, respectively, to their hard limits.

## 21.6. The install phase

### 21.6.1. Creating needed directories

The BSD-compatible install supplied with some operating systems cannot create more than one directory at a time. As such, you should call `${INSTALL_*_DIR}` like this:

    ${INSTALL_DATA_DIR} ${PREFIX}/dir1
    ${INSTALL_DATA_DIR} ${PREFIX}/dir2

Instead of running the install commands directly, you can also append “`dir1 dir2`” to the `INSTALLATION_DIRS` variable, which will automatically do the right thing.

### 21.6.2. Where to install documentation

In general, documentation should be installed into `${PREFIX}/share/doc/${PKGBASE}` or `${PREFIX}/share/doc/${PKGNAME_NOREV}` (the latter includes the version number of the package).

Many modern packages using GNU autoconf allow to set the directory where HTML documentation is installed with the “--with-html-dir” option. Sometimes using this flag is needed because otherwise the documentation ends up in `${PREFIX}/share/doc/html` or other places. In pkgsrc, the HTML documentation should go into the package-specific directory, just like any other documentation.

An exception to the above is that library API documentation generated with the `textproc/gtk-doc` tools, for use by special browsers (devhelp) should be left at their default location, which is `${PREFIX}/share/gtk-doc`. Such documentation can be recognized from files ending in `.devhelp` or `.devhelp2`. (It is also acceptable to install such files in `${PREFIX}/share/doc/${PKGBASE}` or `${PREFIX}/share/doc/${PKGNAME}`; the `.devhelp*` file must be directly in that directory then, no additional subdirectory level is allowed in this case. This is usually achieved by using “--with-html-dir=${PREFIX}/share/doc”. `${PREFIX}/share/gtk-doc` is preferred though.)

### 21.6.3. Installing highscore files

Certain packages, most of them in the games category, install a score file that allows all users on the system to record their highscores. In order for this to work, the binaries need to be installed setgid and the score files owned by the appropriate group and/or owner (traditionally the "games" user/group). Set `USE_GAMESGROUP` to yes to support this. The following variables, documented in more detail in `mk/defaults/mk.conf`, control this behaviour: `GAMEDATAMODE`, `GAMEDIRMODE`, `GAMES_GROUP`, `GAMEMODE`, `GAME_USER`. Other useful variables are: `GAMEDIR_PERMS`, `GAMEDATA_PERMS` and `SETGID_GAMES_PERMS`.

An example that illustrates some of the variables described above is `games/moon-buggy`. `OWN_DIRS_PERMS` is used to properly set directory permissions of the directory where the scorefile is saved, `REQD_FILES_PERMS` is used to create a dummy scorefile (`mbscore`) with the proper permissions and `SPECIAL_PERMS` is used to install setgid the game binary:

    USE_GAMESGROUP=         yes

    BUILD_DEFS+=            VARBASE

    OWN_DIRS_PERMS+=        ${VARBASE}/games/moon-buggy ${GAMEDIR_PERMS}
    REQD_FILES_PERMS+=      /dev/null ${VARBASE}/games/moon-buggy/mbscore ${GAMEDATA_PERMS}
    SPECIAL_PERMS+=         ${PREFIX}/bin/moon-buggy ${SETGID_GAMES_PERMS}

Various `INSTALL_*` variables are also available: `INSTALL_GAME` to install setgid game binaries, `INSTALL_GAME_DIR` to install game directories that are needed to be accessed by setgid games and `INSTALL_GAME_DATA` to install scorefiles.

A package should therefore never hard code file ownership or access permissions but rely on `*_PERMS` as described above or alternatively on `INSTALL_GAME`, `INSTALL_GAME_DATA` and `INSTALL_GAME_DIR` to set these correctly.

### 21.6.4. Adding DESTDIR support to packages

`DESTDIR` support means that a package installs into a staging directory, not the final location of the files. Then a binary package is created which can be used for installation as usual. There are two ways: Either the package must install as root (“destdir”) or the package can install as non-root user (“user-destdir”).

- `PKG_DESTDIR_SUPPORT` has to be set to “destdir” or “user-destdir”. By default `PKG_DESTDIR_SUPPORT` is set to “user-destdir” to help catching more potential packaging problems. If bsd.prefs.mk is included in the Makefile, `PKG_DESTDIR_SUPPORT` needs to be set before the inclusion.

- All installation operations have to be prefixed with `${DESTDIR}`.

- automake gets this DESTDIR mostly right automatically. Many manual rules and pre/post-install often are incorrect; fix them.

- If files are installed with special owner/group use `SPECIAL_PERMS`.

- In general, packages should support `UNPRIVILEGED` to be able to use DESTDIR.

### 21.6.5. Packages with hardcoded paths to other interpreters

Your package may also contain scripts with hardcoded paths to other interpreters besides (or as well as) Perl. To correct the full pathname to the script interpreter, you need to set the following definitions in your `Makefile` (we shall use tclsh in this example):

    REPLACE_INTERPRETER+=   tcl
    REPLACE.tcl.old=        .*/bin/tclsh
    REPLACE.tcl.new=        ${PREFIX}/bin/tclsh
    REPLACE_FILES.tcl=      # list of tcl scripts which need to be fixed,
    # relative to ${WRKSRC}, just as in REPLACE_PERL

### 21.6.6. Packages installing Perl modules

Makefiles of packages providing perl5 modules should include the Makefile fragment `../../lang/perl5/module.mk`. It provides a do-configure target for the standard perl configuration for such modules as well as various hooks to tune this configuration. See comments in this file for details.

Perl5 modules will install into different places depending on the version of perl used during the build process. To address this, pkgsrc will append lines to the `PLIST` corresponding to the files listed in the installed `.packlist` file generated by most perl5 modules. This is invoked by defining `PERL5_PACKLIST` to a space-separated list of packlist files relative to `PERL5_PACKLIST_DIR` (`PERL5_INSTALLVENDORARCH` by default), e.g.:

    PERL5_PACKLIST= auto/Pg/.packlist

The perl5 config variables `installarchlib`, `installscript`, `installvendorbin`, `installvendorscript`, `installvendorarch`, `installvendorlib`, `installvendorman1dir`, and `installvendorman3dir` represent those locations in which components of perl5 modules may be installed, provided as variable with uppercase and prefixed with `PERL5_`, e.g. `PERL5_INSTALLARCHLIB` and may be used by perl5 packages that don't have a packlist. These variables are also substituted for in the `PLIST` as uppercase prefixed with `PERL5_SUB_`.

### 21.6.7. Packages installing pkg-config files

Some packages, usually those providing libraries, install pkg-config files so that their headers and libraries can easily be found. The file names end with `.pc`.

Most of the time, these files only provide the linker flags for the library, but do not include the flags for setting the rpath so the libraries can also be found at runtime. Since this is so common, pkgsrc provides `PKGCONFIG_OVERRIDE` for this. Many packages generate the `.pc` from `.pc.in`, in that case add those files to the `PKGCONFIG_OVERRIDE` variable:

    PKGCONFIG_OVERRIDE+=   foo.pc.in

For packages using meson, the files are generated during build and you also need to change the phase in which the replacement is done. For example:

    PKGCONFIG_OVERRIDE+=            output/meson-private/foo.pc
    PKGCONFIG_OVERRIDE_STAGE=       post-build

### 21.6.8. Packages installing info files

Some packages install info files or use the “makeinfo” or “install-info” commands. `INFO_FILES` should be defined in the package Makefile so that `INSTALL` and `DEINSTALL` scripts will be generated to handle registration of the info files in the Info directory file. The “install-info” command used for the info files registration is either provided by the system, or by a special purpose package automatically added as dependency if needed.

`PKGINFODIR` is the directory under `${PREFIX}` where info files are primarily located. `PKGINFODIR` defaults to “info” and can be overridden by the user.

The info files for the package should be listed in the package `PLIST`; however any split info files need not be listed.

A package which needs the “makeinfo” command at build time must add “makeinfo” to `USE_TOOLS` in its Makefile. If a minimum version of the “makeinfo” command is needed it should be noted with the `TEXINFO_REQD` variable in the package `Makefile`. By default, a minimum version of 3.12 is required. If the system does not provide a makeinfo command or if it does not match the required minimum, a build dependency on the `devel/gtexinfo` package will be added automatically.

The build and installation process of the software provided by the package should not use the install-info command as the registration of info files is the task of the package `INSTALL` script, and it must use the appropriate makeinfo command.

To achieve this goal, the pkgsrc infrastructure creates overriding scripts for the install-info and makeinfo commands in a directory listed early in `PATH`.

The script overriding install-info has no effect except the logging of a message. The script overriding makeinfo logs a message and according to the value of `TEXINFO_REQD` either runs the appropriate makeinfo command or exit on error.

### 21.6.9. Packages installing man pages

All packages that install manual pages should install them into the same directory, so that there is one common place to look for them. In pkgsrc, this place is `${PREFIX}/${PKGMANDIR}`, and this expression should be used in packages. The default for `PKGMANDIR` is “`man`”. Another often-used value is “`share/man`”.

### Note

The support for a custom `PKGMANDIR` is far from complete.

The `PLIST` files can just use `man/` as the top level directory for the man page file entries, and the pkgsrc framework will convert as needed. In all other places, the correct `PKGMANDIR` must be used.

Packages that are configured with `GNU_CONFIGURE` set as “yes”, by default will use the `./configure` --mandir switch to set where the man pages should be installed. The path is `GNU_CONFIGURE_MANDIR` which defaults to `${PREFIX}/${PKGMANDIR}`.

Packages that use `GNU_CONFIGURE` but do not use --mandir, can set `CONFIGURE_HAS_MANDIR` to “no”. Or if the `./configure` script uses a non-standard use of --mandir, you can set `GNU_CONFIGURE_MANDIR` as needed.

See Section 19.5, “Man page compression” for information on installation of compressed manual pages.

### 21.6.10. Packages installing X11 fonts

If a package installs font files, you will need to rebuild the fonts database in the directory where they get installed at installation and deinstallation time. This can be automatically done by using the pkginstall framework.

You can list the directories where fonts are installed in the `FONTS_DIRS.`type`` variables, where `type` can be one of “ttf”, “type1” or “x11”. Also make sure that the database file `fonts.dir` is not listed in the PLIST.

Note that you should not create new directories for fonts; instead use the standard ones to avoid that the user needs to manually configure his X server to find them.

### 21.6.11. Packages installing SGML or XML data

If a package installs SGML or XML data files that need to be registered in system-wide catalogs (like DTDs, sub-catalogs, etc.), you need to take some extra steps:

1. Include `../../textproc/xmlcatmgr/catalogs.mk` in your `Makefile`, which takes care of registering those files in system-wide catalogs at installation and deinstallation time.

2. Set `SGML_CATALOGS` to the full path of any SGML catalogs installed by the package.

3. Set `XML_CATALOGS` to the full path of any XML catalogs installed by the package.

4. Set `SGML_ENTRIES` to individual entries to be added to the SGML catalog. These come in groups of three strings; see xmlcatmgr(1) for more information (specifically, arguments recognized by the 'add' action). Note that you will normally not use this variable.

5. Set `XML_ENTRIES` to individual entries to be added to the XML catalog. These come in groups of three strings; see xmlcatmgr(1) for more information (specifically, arguments recognized by the 'add' action). Note that you will normally not use this variable.

### 21.6.12. Packages installing extensions to the MIME database

If a package provides extensions to the MIME database by installing `.xml` files inside `${PREFIX}/share/mime/packages`, you need to take some extra steps to ensure that the database is kept consistent with respect to these new files:

1. Include `../../databases/shared-mime-info/mimedb.mk` (avoid using the `buildlink3.mk` file from this same directory, which is reserved for inclusion from other `buildlink3.mk` files). It takes care of rebuilding the MIME database at installation and deinstallation time, and disallows any access to it directly from the package.

2. Check the PLIST and remove any entries under the `share/mime` directory, except for files saved under `share/mime/packages`. The former are handled automatically by the update-mime-database program, but the latter are package-dependent and must be removed by the package that installed them in the first place.

3. Remove any `share/mime/*` directories from the PLIST. They will be handled by the shared-mime-info package.

### 21.6.13. Packages using intltool

If a package uses intltool during its build, add `intltool` to the `USE_TOOLS`, which forces it to use the intltool package provided by pkgsrc, instead of the one bundled with the distribution file.

This tracks intltool's build-time dependencies and uses the latest available version; this way, the package benefits of any bug fixes that may have appeared since it was released.

### 21.6.14. Packages installing startup scripts

If a package contains an rc.d script, it won't be copied into the startup directory (`/etc/rc.d`) by default, but you can enable copying by setting the option `PKG_RCD_SCRIPTS=YES` in `mk.conf`. With `PKG_RCD_SCRIPTS=YES`, rc.d scripts will be copied into `/etc/rc.d` when a package is installed, but only if the target does not already exist. Copies in `/etc/rc.d` will be automatically removed only if they have not been modified.

Note that this alone does not enable the service: It must still be added to `/etc/rc.conf`.

### 21.6.15. Packages installing TeX modules

If a package installs TeX packages into the texmf tree, the `ls-R` database of the tree needs to be updated.

### Note

Except the main TeX packages such as kpathsea, packages should install files into `${PREFIX}/share/texmf-dist`, not `${PREFIX}/share/texmf`.

1. Include `../../print/kpathsea/texmf.mk`. This takes care of rebuilding the `ls-R` database at installation and deinstallation time.

2. If your package installs files into a texmf tree other than the one at `${PREFIX}/share/texmf-dist`, set `TEX_TEXMF_DIRS` to the list of all texmf trees that need database update.

   If your package also installs font map files that need to be registered using updmap, include `../../print/tex-tetex/map.mk` and set `TEX_MAP_FILES` and/or `TEX_MIXEDMAP_FILES` to the list of all such font map files. Then updmap will be run automatically at installation/deinstallation to enable/disable font map files for TeX output drivers.

3. Make sure that none of `ls-R` databases are included in `PLIST`, as they will be removed only by the kpathsea package.

### 21.6.16. Packages supporting running binaries in emulation

There are some packages that provide libraries and executables for running binaries from a one operating system on a different one (if the latter supports it). One example is running Linux binaries on NetBSD.

The `pkgtools/rpm2pkg` helps in extracting and packaging Linux rpm packages.

The `CHECK_SHLIBS` can be set to no to avoid the check-shlibs target, which tests if all libraries for each installed executable can be found by the dynamic linker. Since the standard dynamic linker is run, this fails for emulation packages, because the libraries used by the emulation are not in the standard directories.

### 21.6.17. Packages installing hicolor icons

If a package installs images under the `share/icons/hicolor` and/or updates the `share/icons/hicolor/icon-theme.cache` database, you need to take some extra steps to make sure that the shared theme directory is handled appropriately and that the cache database is rebuilt:

1. Include `../../graphics/hicolor-icon-theme/buildlink3.mk`.

2. Check the `PLIST` and remove the entry that refers to the theme cache.

3. Ensure that the PLIST does not remove the shared icon directories from the `share/icons/hicolor` hierarchy because they will be handled automatically.

The best way to verify that the PLIST is correct with respect to the last two points is to regenerate it using make print-PLIST.

### 21.6.18. Packages installing desktop files

If a package installs `.desktop` files under `share/applications` and these include MIME information (MimeType key), you need to take extra steps to ensure that they are registered into the MIME database:

1. Include `../../sysutils/desktop-file-utils/desktopdb.mk`.

2. Check the PLIST and remove the entry that refers to the `share/applications/mimeinfo.cache` file. It will be handled automatically.

The best way to verify that the PLIST is correct with respect to the last point is to regenerate it using make print-PLIST.

## 21.7. Marking packages as having problems

In some cases one does not have the time to solve a problem immediately. In this case, one can plainly mark a package as broken. For this, one just sets the variable `BROKEN` to the reason why the package is broken (similar to the `PKG_FAIL_REASON` variable). A user trying to build the package will immediately be shown this message, and the build will not be even tried.

`BROKEN` packages are removed from pkgsrc in irregular intervals.

Prev | Up | Next
Chapter 20. The pkginstall framework | Home | Chapter 22. GNOME packaging and porting