Skip to content
Snippets Groups Projects
Commit a3847bb9 authored by Simon McVittie's avatar Simon McVittie
Browse files

Document build-aux/many-builds.py

parent a9d16cbd
No related branches found
No related tags found
No related merge requests found
......@@ -7,6 +7,7 @@ Copyright: 2013-2021 Collabora Ltd.
License: MIT
Files:
build-aux/many-builds.md
docs/reference/version.xml.in
helpers/shader.*
tests/template.test.in
......
......@@ -31,20 +31,37 @@ If you want to contribute code to steam-runtime-tools, please include a
[Developer's Certificate of Origin](https://developercertificate.org/)
terms.
## Doing test-builds
For quick tests during development, you can build on any reasonably
modern Linux distribution. Ubuntu 18.04 and Debian 10 are examples of
suitable versions.
To check that everything works in the older Steam Runtime 1 'scout'
environment, either use [Gitlab-CI][], or do a `meson dist` on a modern
distribution and use that to build a package in the SteamRT environment.
The Gitlab-CI does the `meson dist` in the `build` step, and builds the
package on SteamRT in the `autopkgtest` step.
You might find [deb-build-snapshot][] useful for this. smcv uses this
configuration for test-builds:
## Compiling steam-runtime-tools
steam-runtime-tools uses the Meson build system. However, because Steam
supports both 64-bit x86\_64 (amd64) and 32-bit IA32 (x86, i386), to
get a fully functional set of diagnostic tools or a fully functional
version of pressure-vessel that will work in all situations, it is
necessary to build steam-runtime-tools at least twice: once for 64-bit
x86\_64, and once for 32-bit IA32. The 32-bit build can disable
pressure-vessel, but must include at least the "helpers" that are used
to identify and inspect 32-bit libraries. Building for multiple
architectures is outside the scope of the Meson build system, so we
have to do at least two separate Meson builds and combine them.
Also, to make sure that pressure-vessel and the diagnostic tools work
with all the operating systems and runtime environments we support,
production builds of `steam-runtime-tools` need to be compiled in a
Steam Runtime 1 'scout' environment.
The script `build-aux/many-builds.py` can be used to compile and test
steam-runtime-tools in a convenient way. Please see
[build-aux/many-builds.md](build-aux/many-builds.md) for details.
Production builds are done by [Gitlab-CI][], which is also run to test
proposed branches. This does not use `many-builds.py`, because it needs
to run separate parts of the build in separate containers, but the general
principle is the same.
The diagnostic tools provided by steam-runtime-tools are shipped in
Steam Runtime releases via a `.deb` package. We usually build `.deb`
packages via the [deb-build-snapshot][] tool, with a command-line like
this:
```
LC_ALL=C.UTF-8 \
......@@ -127,6 +144,77 @@ Most automated tests run in two phases:
mock implementations of helpers and mock ABI data, it should look
in the directory containing the executable.
## Automated testing for pressure-vessel
Testing a new build of pressure-vessel is relatively complicated, because
several things have to be pulled together:
* We need at least one Steam Linux Runtime container to use for testing,
but for full test coverage, we need several containers. These can be
downloaded by using `pressure-vessel/populate-depot.py`, which is
a copy of the same script that is used to build official
Steam Linux Runtime releases. `build-aux/many-builds.py setup`
automates this, placing a suitable set of containers in
`_build/containers` by default.
* `pressure-vessel-launcher` and `pressure-vessel-adverb` need to be
compiled in a way that will work both on the host system and inside
the test containers. The most reliable way to provide this is to build
them for Steam Linux Runtime 1 'scout'.
`build-aux/many-builds.py install` puts a complete relocatable
installation of pressure-vessel in `_build/containers/pressure-vessel`
by default.
* `pressure-vessel-wrap` needs to be compiled in a way that is compatible
with the host system. In production builds, we use a binary that was
built on scout for maximum compatibility, but when debugging a problem
with pressure-vessel, it can be useful to build it with a newer
compiler to get better warnings, or to build it with debug
instrumentation such as AddressSanitizer or code-coverage
instrumentation. `build-aux/many-builds.py` compiles `_build/host`
with AddressSanitizer by default. It also sets up `_build/coverage`
to be built with code coverage instrumentation, but does not build
that copy by default.
* The pressure-vessel integration test needs to be told where to find
the test containers and the relocatable build of pressure-vessel,
using the `test_containers_dir` Meson option. `build-aux/many-builds.py`
does this for the `_build/host` build by default.
After putting all that together, we can run
`tests/pressure-vessel/containers.py` as an ordinary automated test.
`build-aux/many-builds.py test` will do that for you.
## Manual testing for pressure-vessel
To reproduce issues involving pressure-vessel, it's often necessary to
run a real game in Steam.
`build-aux/many-builds.py install` puts a complete relocatable
installation of pressure-vessel in `_build/containers/pressure-vessel`
by default. This can be copied to a Steam installation for manual
testing, by deleting
`steamapps/common/SteamLinuxRuntime_soldier/pressure-vessel` and
replacing it with a copy of `_build/containers/pressure-vessel`.
If a branch under test has been built on our [Gitlab-CI][], the
artifacts from the `relocatable-install:production` job contain a
complete relocatable installation of pressure-vessel at
`_build/production/pressure-vessel-bin.tar.gz`, and a similar
installation with source code included at
`_build/production/pressure-vessel-bin+src.tar.gz`. These can be
downloaded from the web interface:
* pipeline or merge request
* `relocatable-install:production`
* Job artifacts
* Browse
* navigate into `_build/production`
* click on one of the filenames to view the associated artifact
* Download
* unpack the downloaded archive as a replacement for
`steamapps/common/SteamLinuxRuntime_soldier/pressure-vessel`
## Release procedure
* The version number is *EPOCH*.*YYYYMMDD*.*MICRO* where:
......
# NAME
build-aux/many-builds.py - build steam-runtime-tools and pressure-vessel
# SYNOPSIS
**build-aux/many-builds.py** [*OPTIONS*] **setup** [*MESON SETUP OPTIONS*]
**build-aux/many-builds.py** [*OPTIONS*] **build** [*NINJA OPTIONS*]
**build-aux/many-builds.py** [*OPTIONS*] **install** [*NINJA OPTIONS*]
**build-aux/many-builds.py** [*OPTIONS*] **test** [*MESON TEST OPTIONS*]
**build-aux/many-builds.py** [*OPTIONS*] **clean** [*NINJA OPTIONS*]
# DESCRIPTION
`build-aux/many-builds.py` partially automates developer setup and
testing for steam-runtime-tools and pressure-vessel.
# REQUIREMENTS
By default (when not using the **--docker** or **--podman** options),
`many-builds.py` has
[the same user namespace requirements as Flatpak](https://github.com/flatpak/flatpak/wiki/User-namespace-requirements).
# OPTIONS
**--srcdir** *PATH*
: Relative or absolute path to the `steam-runtime-tools` source
directory. The default is the current working directory.
**--builddir-parent** *PATH*
: Relative or absolute path to the parent directory for all builds.
The default is `_build`. Individual builds and additional required
files will appear in subdirectories such as `_build/host` and
`_build/containers`.
Instead of using this option, it is usually more convenient to make
a symbolic link such as `_build -> ../builds/steam-runtime-tools`
if a separate build location is desired, and then use the default.
**--docker**
: Use **docker**(1) to do builds. The default is to use **bwrap**(1).
Building with Docker requires that the invoking user is either in
the `docker` Unix group or able to run **sudo**(8), either of which
is equivalent to full root privileges, so this mode cannot be used on
machines where gaining full root privileges would be unacceptable.
**--podman**
: Use **podman**(1) to do builds. The default is to use **bwrap**(1).
Building with Podman requires newuidmap, newgidmap and a uid range
configured for the current user in /etc/subuid and /etc/subgid:
see [Troubleshooting](https://github.com/containers/podman/blob/main/troubleshooting.md)
for details.
# STEPS
## setup
The **setup** step is similar to **meson setup**. It also downloads
the necessary Steam Linux Runtime container runtime images to be able
to test **pressure-vessel-wrap**, and the necessary SDK image to be
able to compile for Steam Runtime 1 'scout'.
By default, the same SDK sysroot tarball that will be used for testing
is used for compilation. If the **--docker** or **--podman** options
are used, the official OCI container for the Steam Runtime 1 SDK will
be downloaded and used for compilation.
## build
The **build** step is similar to **meson compile**, but it only builds
a subset of the configurations from the **setup** step.
More specialized configurations such as *${builddir\_parent}*/**coverage**
can be compiled in the usual way with commands like
`meson compile -C _build/coverage` or `ninja -C _build/coverage`.
## install
The **install** step builds a complete relocatable version of the
Steam Runtime 1 'scout' builds of pressure-vessel. This can be
found in `_build/scout-relocatable`, and is also copied into
`_build/containers/pressure-vessel` for testing.
The other builds are not installed.
## test
The **test** step runs unit tests, with as comprehensive a test coverage
as possible. This will take a while.
## clean
The **clean** step is similar to **ninja -C ... clean**.
<!-- vim:set sw=4 sts=4 et: -->
......@@ -123,43 +123,9 @@ to the public. For that, you'll want a relocatable installation: see below.
Building a relocatable install for deployment
---------------------------------------------
To make the built version compatible with older systems, you will need
a SteamRT 1 'scout' environment. Then you can do:
meson --prefix="$(pwd)/_build/prefix" -Dsrcdir=src _build
ninja -C _build
meson test -v -C _build # optional
ninja -C _build install
rm -fr _build/relocatable-install
_build/prefix/bin/pressure-vessel-build-relocatable-install \
--output _build/relocatable-install \
--archive .
./tests/pressure-vessel/relocatable-install.py \
_build/relocatable-install # optional
When building a development version of `pressure-vessel` using a 'scout'
SDK Docker image, if the `pressure-vessel-build-relocatable-install`
script is unable to find some of the required dependencies, it might be
necessary to add apt repositories containing development versions of
those dependencies to `/etc/apt/sources.list`, and run `apt-get update`.
For more convenient use on a development system, if you have a
SteamRT 1 'scout' SDK tarball or an unpacked sysroot, you can place a
tarball at `_build/sysroot.tar.gz` or unpack a sysroot into
`_build/sysroot`, and prefix those commands with
`./build-aux/run-in-sysroot.py`:
./build-aux/run-in-sysroot.py apt-get update
./build-aux/run-in-sysroot.py meson --prefix="$(pwd)/_build/prefix" _build
./build-aux/run-in-sysroot.py ninja -C _build
(etc.)
(Or put them in different locations and pass the `--sysroot` and
`--tarball` options to `./build-aux/run-in-sysroot.py`.)
The relocatable install goes into `relocatable-install` (or
whatever you used as the `--output`), and a compressed version ends
up in the directory passed as an argument to `--archive`.
The script `build-aux/many-builds.py` can be used to compile and test
steam-runtime-tools in a convenient way. Please see
[../build-aux/many-builds.md](../build-aux/many-builds.md) for details.
Instructions for testing
------------------------
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment