Newer
Older
# Contributing to steam-runtime-tools
<!-- This file:
Copyright 2019-2020 Collabora Ltd.
SPDX-License-Identifier: MIT
-->
## Issue tracking
Our main bug tracking system for the whole Steam Runtime is
<https://github.com/ValveSoftware/steam-runtime/issues>. Before reporting
an issue, please take a look at the
[bug reporting information](https://github.com/ValveSoftware/steam-runtime/blob/HEAD/doc/reporting-steamlinuxruntime-bugs.md)
to make sure your issue report contains all the information we need.
The issue tracker on our Gitlab installation, `gitlab.steamos.cloud`, is
primarily used by steam-runtime-tools developers to track issues for
which we already know the technical details of what is happening.
## Contributing code
At the moment our Gitlab installation, `gitlab.steamos.cloud`, is not
set up to receive merge requests from third-party contributors. However,
git is a distributed version control system, so it is possible to push a
clone of the steam-runtime-tools git repository to some other git hosting
platform (such as Github or `gitlab.com`) and send a link to a proposed
branch via an issue report.
If you want to contribute code to steam-runtime-tools, please include a
`Signed-off-by` message in your commits to indicate acceptance of the
[Developer's Certificate of Origin](https://developercertificate.org/)
terms.
## Compiling steam-runtime-tools
steam-runtime-tools uses the Meson build system. However, because Steam
supports both 64-bit x86\_64 (amd64) and 32-bit IA32 (x86, i386), to
get a fully functional set of diagnostic tools or a fully functional
version of pressure-vessel that will work in all situations, it is
necessary to build steam-runtime-tools at least twice: once for 64-bit
x86\_64, and once for 32-bit IA32. The 32-bit build can disable
pressure-vessel, but must include at least the "helpers" that are used
to identify and inspect 32-bit libraries. Building for multiple
architectures is outside the scope of the Meson build system, so we
have to do at least two separate Meson builds and combine them.
Also, to make sure that pressure-vessel and the diagnostic tools work
with all the operating systems and runtime environments we support,
production builds of `steam-runtime-tools` need to be compiled in a
Steam Runtime 1 'scout' environment.
The script `build-aux/many-builds.py` can be used to compile and test
steam-runtime-tools in a convenient way. Please see
[build-aux/many-builds.md](build-aux/many-builds.md) for details.
Production builds are done by [Gitlab-CI][], which is also run to test
proposed branches. This does not use `many-builds.py`, because it needs
to run separate parts of the build in separate containers, but the general
principle is the same.
The diagnostic tools provided by steam-runtime-tools are shipped in
Steam Runtime releases via a `.deb` package. We usually build `.deb`
packages via the [deb-build-snapshot][] tool, with a command-line like
this:
```
LC_ALL=C.UTF-8 \
deb-build-snapshot \
--upstream \
--source \
--download ~/tmp/build-area \
--deb-schroot steamrt_scout_amd64 \
--install-all \
build-vm
```
where `build-vm` is a virtual machine with steam-runtime-tools'
dependencies and a `steamrt_scout_amd64` chroot, that can be accessed
via `ssh build-vm`. If the dependencies and chroot are available on your
development system, use `localhost`, which will make `deb-build-snapshot`
do the build locally instead of using ssh.
[Gitlab-CI]: https://gitlab.steamos.cloud/steamrt/steam-runtime-tools/pipelines
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
## Design notes
* ABIs/architectures are identified by their Debian-style multiarch
tuple, which is a GNU tuple (`x86_64-linux-gnu`, `arm-linux-gnueabi`)
with its CPU part normalized to the oldest/most compatible in a
CPU family (so `i386`, not `i686`, and `arm`, not `armv5tel`).
Helper subprocesses are prefixed with the multiarch tuple,
in the same style used for cross-compilers and cross-tools
(`x86_64-linux-gnu-inspect-library`, `i386-linux-gnu-wflinfo`).
* Be careful when adding dependencies to the main library. GLib 2.32
(Ubuntu 12.04 'precise', SteamRT 1 'scout') can be relied on.
JSON-GLib is OK (we have a backport in SteamRT 1 'scout'). Anything
else is probably bad news.
- In particular, try not to pull in C++ ABIs, because incompatible C++
ABIs are one of the things that steam-runtime-tools should be
able to detect and diagnose. We use C ABIs because they are the
lowest common denominator for compatibility between distros.
* Any checks that can reasonably crash should be in a subprocess. This
means that if they crash, they don't bring down the entire Steam
client with them.
* Any checks that need to look at more than one ABI (32- and 64-bit) must
be in a subprocess, so that the same process can run both.
* The `inspect-library` helper should not have non-libc dependencies -
not even GLib. Other helpers may have library dependencies if it is
useful to do so.
* For system information and checks, new functionality should be accessed
through `SrtSystemInfo` so that its cache lifetime can be managed.
Separate access points like `srt_check_library_presence()` will be
kept until the next ABI break, but don't add new ones.
## Automated tests
New code should have test coverage where feasible. It's OK to have some
"design for test" in the APIs to facilitate this, such as
`srt_system_info_set_environ()`.
If a helper can't be unit-tested (for example because it needs working
OpenGL), we'll understand. The code that calls into the helper should
still be unit-tested, by using a mock implementation of the helper.
Most automated tests run in two phases:
* Build-time tests (`meson test`, `ninja test`, `dh_auto_test`):
`steam-runtime-tools` has just been built, and probably isn't
installed. The environment variables in `tests/meson.build` are set.
The code under test should look for data, helper subprocesses, etc.
relative to those environment variables. We can add more environment
variables like `GI_TYPELIB_PATH` if they become necessary.
* Installed-tests (`autopkgtest`, `debian/tests`): `steam-runtime-tools`
has been built and installed system-wide. The code under test should
look for data, helper subprocesses, etc. in their real installed
locations. For things that get installed next to the tests, such as
mock implementations of helpers and mock ABI data, it should look
in the directory containing the executable.
## Automated testing for pressure-vessel
Testing a new build of pressure-vessel is relatively complicated, because
several things have to be pulled together:
* We need at least one Steam Linux Runtime container to use for testing,
but for full test coverage, we need several containers. These can be
downloaded by using `pressure-vessel/populate-depot.py`, which is
a copy of the same script that is used to build official
Steam Linux Runtime releases. `build-aux/many-builds.py setup`
automates this, placing a suitable set of containers in
`_build/containers` by default.
* `steam-runtime-launcher-service` and `pressure-vessel-adverb` need to be
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
compiled in a way that will work both on the host system and inside
the test containers. The most reliable way to provide this is to build
them for Steam Linux Runtime 1 'scout'.
`build-aux/many-builds.py install` puts a complete relocatable
installation of pressure-vessel in `_build/containers/pressure-vessel`
by default.
* `pressure-vessel-wrap` needs to be compiled in a way that is compatible
with the host system. In production builds, we use a binary that was
built on scout for maximum compatibility, but when debugging a problem
with pressure-vessel, it can be useful to build it with a newer
compiler to get better warnings, or to build it with debug
instrumentation such as AddressSanitizer or code-coverage
instrumentation. `build-aux/many-builds.py` compiles `_build/host`
with AddressSanitizer by default. It also sets up `_build/coverage`
to be built with code coverage instrumentation, but does not build
that copy by default.
* The pressure-vessel integration test needs to be told where to find
the test containers and the relocatable build of pressure-vessel,
using the `test_containers_dir` Meson option. `build-aux/many-builds.py`
does this for the `_build/host` build by default.
After putting all that together, we can run
`tests/pressure-vessel/containers.py` as an ordinary automated test.
`build-aux/many-builds.py test` will do that for you.
## Manual testing for pressure-vessel
To reproduce issues involving pressure-vessel, it's often necessary to
run a real game in Steam.
`build-aux/many-builds.py install` puts a complete relocatable
installation of pressure-vessel in `_build/containers/pressure-vessel`
by default. This can be copied to a Steam installation for manual
testing, by deleting
`steamapps/common/SteamLinuxRuntime_soldier/pressure-vessel` and
replacing it with a copy of `_build/containers/pressure-vessel`.
If a branch under test has been built on our [Gitlab-CI][], the
artifacts from the `relocatable-install:production` job contain a
complete relocatable installation of pressure-vessel at
`_build/production/pressure-vessel-bin.tar.gz`, and a similar
installation with source code included at
`_build/production/pressure-vessel-bin+src.tar.gz`. These can be
downloaded from the web interface:
* pipeline or merge request
* `relocatable-install:production`
* Job artifacts
* Browse
* navigate into `_build/production`
* click on one of the filenames to view the associated artifact
* Download
* unpack the downloaded archive as a replacement for
`steamapps/common/SteamLinuxRuntime_soldier/pressure-vessel`
Developer-oriented information for testing various configurations
is available in the [guide for game developers](docs/slr-for-game-developers.md).
Convenient choices for a reasonably small free-to-play game on various
engines include:
- Floating Point (game ID 302380)
- very small/quick to install, works OK on weak hardware
- UnityPlayer 4.3.4f1
- Life is Strange, episode 1 (game ID 319630)
- Unreal Engine 3
- Team Fortress 2 (game ID 440)
- Source engine
- Unturned (game ID 304930)
- UnityPlayer 5.5.3f1
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
## Release procedure
* The version number is *EPOCH*.*YYYYMMDD*.*MICRO* where:
- *EPOCH* increases on major (incompatible) changes, or if you change
the version-numbering scheme
- *YYYYMMDD* is today's date if you are doing a feature release, or
the date of the version you are branching from if you are applying
"stable branch" hotfixes to an older version
- *MICRO* starts at 0, and increases if you need to apply
"stable branch" hotfixes to an old version (or if you need to do
two feature releases on the same day)
* Look at the Debian package's build log from `deb-build-snapshot`
or [Gitlab-CI][]. If new library ABI has been added, you will see
warnings like these:
```
dpkg-gensymbols: warning: debian/libsteam-runtime-tools-0-0/DEBIAN/symbols doesn't match completely debian/libsteam-runtime-tools-0-0.symbols
--- debian/libsteam-runtime-tools-0-0.symbols (libsteam-runtime-tools-0-0_0.20190806.0+14+g7bda756-0~snapshot_amd64)
+++ dpkg-gensymbolsWIRbYG 2019-08-16 10:34:42.915782703 +0000
@@ -13,9 +13,20 @@
srt_library_get_type@Base 0.20190801.0
srt_library_issues_get_type@Base 0.20190801.0
srt_library_symbols_format_get_type@Base 0.20190801.0
+ srt_runtime_issues_get_type@Base 0.20190806.0+14+g7bda756-0~snapshot
+ srt_steam_issues_get_type@Base 0.20190806.0+14+g7bda756-0~snapshot
srt_system_info_can_run@Base 0.20190801.0
```
Update `debian/libsteam-runtime-tools-0-0.symbols`, adding the symbols
that you have added with the version number that you plan to release,
for example `srt_runtime_issues_get_type@Base 0.20190816.0`.
* Update `debian/changelog`: `gbp dch`, edit to clarify the changelog if
desired, `dch -r`, set the version number.
* Update the version number and ABI minor/micro version in `meson.build`.
* Commit everything.
* Add an annotated git tag v*VERSION*.
* Do a final release build, for example with:
```
deb-build-snapshot -d ~/tmp/build-area --source-only --release localhost
```
* Upload the resulting `.changes` file to your OBS branch to be built.
If it succeeds, submit it to the main OBS project to be included in
the next scout release. If it fails, fix it and prepare a new release
(with the next micro version number).
[deb-build-snapshot]: https://gitlab.collabora.com/smcv/deb-build-snapshot