Working directory

independent folder with all build materials, related to your project. It keeps sources, build directories, all temporary files, built binaries, ready to install archives and the copy of build system itself.

The build is designed for non-priveleged user accounts. No impact on filesystem outside working directory is done during the build. Different users may create unlimited quantity of working directories on a single machine.

In case of building in container (the most reliable way) the working directory contains local executable files of host, isolating the build process from outer host system. As the result we have reproducible build environment, portable between different machines. The engine is copied into each working directory, so the configuration of one does not impact the other.

When host container is not applied, the user must support on his own the sane environment from the generic operation system for needed build tasks. One of the examples on no possibility of isolation: the public server does not provide permissions for sudo(1), which is applied for the isolation command chroot(1).

New directory creation for the work with any comes down to the following:

unpacking the main archive with build release of your project:

    tar xzf distro-project.tgz
    mv distro-project your-name
    cd your-name/

if additional stuff is present (host, toolchain), unpack that:

    tar xzf add-files.tgz
    tar xzf more-add-files.tgz
    ...

The configuration of working directory and preparation of the work is described in more details in any-guide(7).

The essential directories of a working directory are:

any/

rdd.conf.d/

rrr/

build/

ports/

release/

The brief overview of each dir:

any/

is the directory with files of any build engine itself. Each working directory has its own copy of that files. They do not interfere with other working directories.

rdd.conf.d/, rdd.def

are the place for configuration files of a project. All settings for build process are assigned through these configs.

rdd.conf.d/ is the directory with generic config set for all purposes. The library-like settings are kept here for different platforms, architectures, toolchains.

rdd.def is the single config file to choose the current settings. You may store configuration name or particular build properties here, which are constantly in use. After that here will be no need to assign them each time in command line.

Essential options from configuration files are described in any-guide(7). All possible options are described in details in any-conf(7).

rrr/

location of build utils, installed locally. There are directories like rrr/bin/ and rrr/share/ inside.

Here is the place for utils from any as well as for third-party programms, if they implement some specific functions for generic build system. Note the generic programs like cp or sed are not installed here.

When the build process is launched, the engine adds the path rrr/bin/ to the variable PATH automatically to make available the programs from that local path by default.

build/

is the directory to write files during the build, as given by default any.conf config.
Here the build directories with sources and object files are stored, compilation takes place, ready package files are installed and binary archives are created. All subdirectories of build/ are created during build process.

The dir must not keep any unique or unproducible data, concerning the engine or input. It is safe to clear it and rebuild the same project from the start. This dir is absent until the first build. The locally saved sources or patches, which are not intended to be available from the network, should not be kept here, ports/ is used for that instead.

build/pack/

is the directory where packaged results of successfull build are stored.

ports/

is the directory with the content of the distro, as given by default any.conf config.
In the first turn, those are scripts, personal for each package, with commands to build them. Etalon input data for checkers is kept here as well. Opposed to that, generic build commands and methods, same for all packages, are stored inside engine any/.

In case of several various distro projects, which share little common between each other, it is possible to keep their data in separate directories, analogues to ports/, and choose the needed one between them with configs.

release/

is the directory with final release results of entire distro project. Its contents depend on the meaning of 'final result', taken by the project. It can be single installable image or some predefined set of packages.

The directory is not normally populated automatically after buildage of single packages.

The entire host environment is copied with new working directory creation by anyinit(1), unless configured otherwise. It includes all executables to enter new shell session in containerised environment and execute generic build procedure: libc, POSIX baseutils, shell implementation, make. It must include toolchain with compiler and linker. Many utils will be needed for documentation generation and other help tasks during the build.

The host files are installed into root of the working directory with the same layout as they have in generic OS: bin/, lib/, usr/lib/, usr/bin/, etc/ and so on. After chroot(1) or other container implementation the build process will see general filesystem, localised inside working dir.

The build process with any is designed to use vast majority of files from build/, which is populated from packages, which are built inside the project or installed by some request. Installations inside build/ are granulated by profiles, can be tweaked and rebuilt by lots of ways.

The purpose of host is bare run-time support for all utilities which handle the build, while introducing minimal impact on compilation and linkage with host libraries and include files. The ideal host from that point of view contains no libraries and includes at all, besides the ones for native toolchain support. Such host is not so easy to reach, especially to get rid off dynamic shared libraries for binaries. Much easier to remove or hide everything from usr/lib/pkgconfig/ and all usr/bin/*-config scripts, and leave only those files from usr/include/ to keep compiler working.

The entire set of engine directories inside a working dir is:

any/bin/

any/corelib/

any/include/

any/lib/

build/context/

build/empty/

build/image/

build/log/

build/pack/

build/patch/

build/src/

build/tmp/

build/work/

ports/packages/

ports/list/

ports/meta/

ports/patch/

ports/srcstatic/

rdd.conf.d/

release/

rrr/bin/

var/dump/

var/lib/

any/bin/

Directory for inner executables of any itself. Such executables are supposed non-usable outside of engine. That can be C binaries, implementing engine methods instead of ones in shell. The path is added to PATH variable inside build environment.

any/corelib/

Directory with main engine libraries, which implement core functional and declare all mandatory interfaces. All libraries from here are always included in the launch disregarding used configuration and modules. Some interfaces contain empty or trivial implementation: that means useful implementation is provided by some additional code and depends on user's choice (one or another control version system, for example). Such support is distributed through additional modules, installed into any/lib/.

any/include/

Directory for shell API constructing beyond core functionality. If some module declares global variables, it should place their assignments in the file inside this dir.

any/lib/

Directory for all possible modules with additional functional. Some of them are included in pack of any, some of them come from anymods(7), still more may be third-party.
Modules within this dir must contain only method implementations. No calls out of methods' bodies are allowed. No declarations of global variables are allowed as well.
The files of the same module may be placed in both any/include/module.sh and any/lib/module.sh. The first file will contain variable declarations, the second one methods.

build/context/

Directory to keep file context for package building. Context is composed from files of other packages, used by the current building one. Composing is done with symbolic links from real installed files of other packages into context dir. Several types of text files are copied, as they are edited by engine. Each package has its own separate context directory. The set of files, used for context, is defined by any-conf(7) settings. The list of packages, needed for current one (build dependencies), is defined by anybuild of current package, as described in any-build(7).

Personal context of each package is stored into dir build/context/PROF/PACKAGENAME/root/.
For example, context of hello-world-1.0 for amd64 will be kept in build/context/amd64/hello-world-1.0/root/ dir inside working directory.

The given scheme means the following:

-

The build environment is controlled explicitly by dependencies. The obscure non-obvious influence on the build process is reduced significantly.

-

Build context is lightweight and quickly composed, as the most files are handled with symbolic links and heavy copies are avoided.

-

Correctness of build dependencies is checked automatically - in case of missing dependency in anybuild the build process will not have access to corresponding files.

-

There is no changing resources, shared between building of several packages. So different packages can be launched in parallel without any synchronisation at all, while they do not depend on each other (in other case synchronisation of dependencies handling comes to a game).

Directory build/context/ does not limit the influence of host environment, i.e. the binaries of container, installed into the root of working directory, or the entire host system without a container.

Interfaces pkg_context(3) and pkg_root(3) write to this dir.

build/context/PROF/platform/

Directory with context, equal for all packages. The context is contained in tar archives. Every archive from this dir is extracted to personal context directory of each package (into build/context/amd64/hello-world-1.0/root/ for the example above).
The example of platform context may be system libc or some minimal filesystem, which is not granulated by packages.

build/context/PROF/static/root/

Directory with single context, shared for all packages, used in staticdeproot mode. That mode makes no personal context for each package, but maintains single shared one, as in traditional file system. See any-do(1) for description of staticdeproot.

build/empty/

Always empty dir. It is used to prevent expansion of meta-symbols, such as '*', in some shell methods.

build/image/

The dir for installed binary files of packages. Each package is installed into separated dir build/image/PROF/PACKAGENAME/.

For example, the package hello-world-1.0 for amd64 will have installed files:

build/image/amd64/hello-world-1.0/etc/

build/image/amd64/hello-world-1.0/usr/bin/

build/image/amd64/hello-world-1.0/usr/lib/

build/image/amd64/hello-world-1.0/usr/include/

build/image/amd64/hello-world-1.0/usr/share/

...

Interfaces src_install(3), pkg_install(3), pkg_rminstall(3), pkg_config(3) write to this dir.

build/log/

Directory to keep logs of build process. Each package PACKAGENAME for profile PROF keeps its log file in build/log/PROF/PACKAGENAME.log.
For example, the package hello-world-1.0 for amd64 will keep log file in build/log/amd64/hello-world-1.0.log.

Under the hood, the logs in this directory are symbolic links to files inside var/dump/, as logs are total output redirect, constructed by rdd(1), and that command uses var/dump/ unconditionally for all its operations.

Each time new build of a package starts its log file is rewritten.

The log file of entire build session is kept in build/log/PROF/PROF.log file. That file is never truncated. Each time new build session starts, summarised information of the build is appended to profile log. That summarised information is the same as outputed to terminal, besides the explicit link to the log in case of error.

build/pack/

Directory for binary archives with packages. When no additional naming convention is included, each built package is saved into installable archive build/pack/PROF/PACKAGENAME-postfix.tgz. postfix part depends on target platform for the package.
For example, the package hello-world-1.0 for amd64 will have simple binary archive build/pack/amd64/hello-world-1.0-x86_64.tgz.
Modules can change the binary format for a package and portion of data in its file name.

When some convention of archive naming takes place, each package has one or more corresponding binary archives with arbitrary names. That names are usually described in anybuild. To match package names in terms of any and names of their binary archives the method pack_name(3) is used.

Interface pkg_pack(3) writes to this dir.

build/patch/

Directory for patches, applied for packages. Each package PACKAGENAME gets his patches from build/patch/PACKAGENAME/ directory.
For example, the package hello-world-1.0 will use patches from build/patch/hello-world-1.0/.

There are different places, where patches may be stored, but all resulting files will be finally copied to build/patch/.

Interface src_fetch(3) writes to this dir.

build/src/

Directory to store sources of packages, retrieved from version control systems. Each package PACKAGENAME stores its sources in build/src/PACKAGENAME/ directory.
For example, the package hello-world-1.0 will fetch development sources into build/src/hello-world-1.0/.

That sources are used to generate some patches, applied before building. That is done by fetch_gen_patches(3) interface. Core any engine does not generate any patches, but modules may implement this interface for git(1), hg(1), svn(1) or another content tracker.
It is possible that sources from this dir are copied 'as is' on top of previously prepared ones.

The default value of that dir can be altered with any_dev_src_dir option.

Interface src_fetch(3) writes to this dir.

build/tmp/

Directory to write and keep all intermediate files during the work of any engine or modules.

build/work/

Directory to do the actual build. The final sources are prepared here, configure, make and make install are launched. Each package has that dir in build/work/PROF/PACKAGENAME/.
For example, the package hello-world-1.0 for amd64 will be built in build/work/amd64/hello-world-1.0/ directory.

The directory is kept after the successfull build, as it contains all intermediate results of the process and irreplaceable for debug.

Interfaces src_fetch(3), src_prepare(3), src_config(3), src_compile(3) write to this dir.

ports/packages/

Directory with individual build scripts, called anybuilds. Each package has its own separate directory, named after package. Anybuild script inside is named as package-name-1.2.3.build. This file is included by default for given package inside any.
For example, anybuild for hello-world-1.0 will be in ports/packages/hello-world-1.0/hello-world-1.0.build file.
More separate shell modules can reside in the directory. That modules are included after data words, saved in rdd_prf_all variable, as described in section Modular code inclusion of any-do(1).

Package directory keeps the config file of rdd(1), personal for package, named atom.conf. That file is analogues to the ones in rdd.conf.d/, but applies only to that package.

The directory ports/packages/package-name-1.2.3/patch, if one exists, is used as source of patches for the package. That dir allows to keep all input data for the package build in a single place (anybuild and patches all together). Besides this dir, patches may be fetched from other places separately from ports/.

When binary archive format for some package manager is used, the various postinstall, preinstall and other scripts alike are needed. They are stored in the package directory inside ports/packages/ as well.
Personal package directory may contain other materials, used by modules (for example, sanity or regression checkers).

The default value of the dir can be altered with rdd_atom_path option.

ports/list/

Directory for package lists of rdd(1). Here text lists are kept with content of some buildable sets, composing final project such as OS distribution.

The convention is to name general hand-written lists with .src ending, and script generated ones with .txt ending.

List all.src has special meaning – it is used to keep all packages, available in project by default. Besides the other, that value is used as data source to resolve build dependencies. The name of the list can be changed through the option any_list_all.

The default value of the dir can be altered with rdd_list_path option.

ports/meta/

Directory for various meta-data, concerning the entire distro project inside ports/: etalon data for sanity checkers, cache for cross-compilation.

The default value of the dir can be altered with any_meta_dir option.

ports/patch/

Directory to keep locally saved patches for packages. Each package has its room in ports/patch/PACKAGE-NAME/.

The default value of the dir can be altered with any_static_patch_dir option.

ports/srcstatic/

Directory to store source archives, or tarballs. All prepared patches will be applied to the sources from a found archive. So these archives are supposed to be original or unchanged upstream sources, while development changes are done through patches or other explicit actions.

The alternative usage is to keep actual full-pledged development sources in ports/srcstatic/ and not to use patches at all. That variant can be handy at inner debug, when still there is no need to share changes done.

In simple case, sources are recognised by expression ports/srcstatic/PACKAGENAME.EXT.
For example, the package hello-world-1.0 will catch source tarball ports/srcstatic/hello-world-1.0.tgz.

That is not the only scheme, though, as sources may have alternate name. This is due to possibly different names of package at authors domain and in naming convention of current distro project. The alternate name for sources is described in any-api(7) and fetch_remote_src(3) method.

If plain directory with matching name is found inside ports/srcstatic/, it will be used. Otherwise the tar archive will be searched for.
Known tarball extensions are:

.tar.xz

.txz

.tar.gz

.tgz

.tar.bz2

.tbz2

The default value of the dir can be altered with any_src_dir option.

Interface src_fetch(3) writes to this dir.

rdd.conf.d/, config file rdd.def

Directory with configuration files of rdd(1). That's the place with all engine properties. Some of them belong to rdd(1) domain, some of them are read by any part of the engine and described in any-conf(7).

Directory with any source code ships example.conf.d/ subdir. If configs from any/example.conf.d/ are added to rdd.conf.d/, working directory will get reasonable default values to begin the work. any.conf is the file with all default values for engine, described in the manuals. Example architecture-specific settings, describing properties of target OS and processor architecture, are stored in build.conf. Arbitrary new files can be added to describe new entities or customise the build.

File rdd.def works as general config, but with higher priority. It is used for current settings, non-saveable for all working directories or situations. It can be used for experiments, usage of testing code, storage of dynamically generated or fetched information.

release/

Directory with release results of distro project. It can be single installable image or some predefined set of packages.

rrr/bin/

rrr/ is the destination directory to install utilities, used by engine. The content of used utilities is determined by anyinit(1) configuration. Changes to utilities should not be done here on a regular basis, better change util at its source and reinstall (though it's handy to edit something right in rrr/ for quick test purposes). Path rrr/bin/ is added to PATH variable inside build engine.

var/dump/

Directory where rdd(1) keeps its intermediate files while constructing the build scripts. The log files of build process are kept here. This dir is the place for dumps – shell files with full any environment and gathered data for some package. The dumps are used at several parts of the engine.

var/lib/

Directory to track installed modules. Each module MOD from anymods(7) leaves text file var/lib/anypkg/MOD, containing list of files of that module.

New working directories can be created with specialized command anyinit(1). It copies data for a new directory from the template, installed into common filesystem.

The template can be installed from the any release archive:

        make install-workdir

Stuff is installed into home directory inside ~/rrr/share/any/workdir by default. If commands of any have been installed by default before, then anyinit(1) will find the stuff from workdir.

Template can be installed into the system:

        sudo env DESTDIR=/usr make install-workdir

The command above will install the stuff into /usr/share/any/workdir. For anyinit(1) to find it, the commands of any must be installed into /usr as well.

After the template installation it is possible to create new working directory like that:

        anyinit my_dir
        cd my_dir

any(1), any-do(1), anyinit(1), any-conf(7), anymods(7), any-build(7), any-api(7)