Skip to content
Snippets Groups Projects
Commit 7c0c45a0 authored by Simon McVittie's avatar Simon McVittie
Browse files

Add a smoke-test for the SteamLinuxRuntime


See the new tests/README.md and tests/depot/README.md files for details.

Signed-off-by: default avatarSimon McVittie <smcv@collabora.com>
parent 58d88ee0
No related branches found
No related tags found
No related merge requests found
/depot/.ref
/depot/com.valvesoftware.SteamRuntime.*-buildid.txt
/depot/com.valvesoftware.SteamRuntime.*-runtime.tar.gz
/depot/pressure-vessel/
/depot/scout/
/depot/scout
/depot/scout_*-0.20*.*/
/depot/scout_0.20*.*/
/pressure-vessel-*-bin+src.tar.gz
/pressure-vessel-*-bin.tar.gz
/pressure-vessel-bin+src.tar.gz
/pressure-vessel-bin.tar.gz
/test-logs/
__pycache__/
all:
check:
prove -v tests/*.sh
steam-container-runtime (0) UNRELEASED; urgency=medium
* This is not a real Debian package. It's only here as a way to
provide automated tests in autopkgtest format.
-- Simon McVittie <smcv@collabora.com> Wed, 06 Mar 2019 11:51:41 +0000
Source: steam-container-runtime
Section: misc
Priority: extra
Maintainer: SteamOS Maintainers <steamos@valvesoftware.com>
Standards-Version: 4.3.0
Package: steam-container-runtime
Architecture: all
Description: Not a real package
This package description is only here as a way to run tests using
Debian's autopkgtest tool. Don't build it.
#!/usr/bin/make -f
build:
@echo "This is not really a Debian package. Don't build it."
@false
clean:
@:
binary-arch: build
binary-indep: build
binary: binary-indep binary-arch
build-arch: build
build-indep: build
Tests: depot
Depends:
bubblewrap,
file,
libc6:i386,
libegl1 | libegl1-mesa,
libgl1 | libgl1-mesa-glx,
libgl1-mesa-dri,
libgl1-mesa-dri:i386,
libgl1:i386 | libgl1-mesa-glx:i386,
libgles2 | libgles2-mesa,
libglx0 | libgl1-mesa-glx (<< 18),
libopengl0 | libgl1-mesa-glx (<< 18),
locales,
python3,
xauth,
xvfb,
Restrictions: allow-stderr, isolation-machine
#!/bin/sh
# Copyright © 2019-2020 Collabora Ltd.
# SPDX-License-Identifier: MIT
export PYTHONUNBUFFERED=1
exec xvfb-run -a -e /proc/self/fd/2 \
tests/depot/pressure-vessel.py
Developer tests
===============
`tests/*.sh` carry out some basic QA checks. Run with:
make check
`python3` and the `prove` tool from Perl are required.
`mypy`, `pycodestyle`, `pyflakes3`, `python3.4`, `python3.5` and
`shellcheck` are required for full coverage, but the relevant checks
are automatically skipped if those utilities are missing.
Non-fatal issues such as failed coding-style checks and other warnings
are reported as a TAP "TODO" diagnostic.
Depot tests
===========
Pleas see [depot/README.md](depot/README.md).
Depot tests
===========
Tests in this directory are designed to be run on any system that is
meant to run Steam. In particular, they have been confirmed to work on
SteamOS 2 'brewmaster', and should work on Debian 8 'jessie' or later
(confirmed on Debian 9) and on Ubuntu 14.04 'trusty' or later.
The depot directory `depot/` must have been pre-populated with at least
`com.valvesoftware.SteamRuntime.Platform-amd64,i386-scout-runtime.tar.gz`
and `com.valvesoftware.SteamRuntime.Platform-amd64,i386-scout-buildid.txt`.
This is normally done by CI infrastructure while building Steam Runtime
releases.
To run the tests manually, use:
./tests/depot/pressure-vessel.py
Note that this will write to the `depot/` directory. If this is not
desired, copy the whole `SteamLinuxRuntime` directory elsewhere and run
the test in the copy.
The debian/ directory provides the basic layout of a Debian source
package (even though it isn't really) and wraps the test in autopkgtest
metadata, to be able to take advantage of the autopkgtest tool's pluggable
virtualization backends, and in particular the qemu backend. Typical use
would resemble this:
rm -fr test-logs
autopkgtest \
--setup-commands "dpkg --add-architecture i386" \
--apt-upgrade \
--no-built-binaries \
--output "$(pwd)/test-logs" \
"$(pwd)" \
-- \
qemu \
--efi \
~/Downloads/brewmaster_amd64.qcow2
Additional arguments like `--debug` can be placed after `autopkgtest`. See
autopkgtest(1) for more details.
Arguments like `--debug`, `--show-boot` and `--qemu-options="-display gtk"`
can be placed after `qemu` for debugging. See autopkgtest-virt-qemu(1)
for more details.
Testing a different runtime
---------------------------
If you have a different runtime unpacked into `depot/`, for example
in `depot/heavy/files`, set the environment variables
`TEST_CONTAINER_RUNTIME_SUITE` (the default is `scout`) and/or
`TEST_CONTAINER_RUNTIME_ARCHITECTURES` (the default is `amd64,i386`).
If using `autopkgtest`, you will need to set the environment variables
in the test system, like this:
autopkgtest \
--env=TEST_CONTAINER_RUNTIME_SUITE=heavy \
--env=TEST_CONTAINER_RUNTIME_ARCHITECTURES=amd64 \
...
Testing inside a `LD_LIBRARY_PATH` Steam Runtime
------------------------------------------------
You can expand test coverage by setting the environment variable
`TEST_CONTAINER_RUNTIME_LD_LIBRARY_PATH_RUNTIME` to the path to an
unpacked `LD_LIBRARY_PATH` Steam Runtime, so that files like
`${TEST_CONTAINER_RUNTIME_LD_LIBRARY_PATH_RUNTIME}/run.sh` and
`${TEST_CONTAINER_RUNTIME_LD_LIBRARY_PATH_RUNTIME}/version.txt` exist.
This emulates what will happen when `pressure-vessel` is run as a
Steam compatibility tool from inside the Steam client, which is
itself running under the `LD_LIBRARY_PATH` Steam Runtime.
If using `autopkgtest`, you will need to ask autopkgtest to copy the
unpacked Steam Runtime into the test system and use the path on the
test system, something like this:
autopkgtest \
--copy "$HOME/.steam/steam/ubuntu12_32/steam-runtime:/tmp/srt" \
--env TEST_CONTAINER_RUNTIME_LD_LIBRARY_PATH_RUNTIME="/tmp/srt" \
...
#!/usr/bin/env python3
# Copyright 2019-2020 Collabora Ltd.
#
# SPDX-License-Identifier: MIT
import contextlib
import json
import logging
import os
import os.path
import shutil
import subprocess
import sys
import tempfile
import unittest
try:
import typing
except ImportError:
pass
logger = logging.getLogger('test-pressure-vessel')
class MyCompletedProcess:
"""
A minimal reimplementation of subprocess.CompletedProcess from
Python 3.5+, so that this test can be run on the Python 3.4
interpreter in Debian 8 'jessie', SteamOS 2 'brewmaster' and
Ubuntu 14.04 'trusty'.
"""
def __init__(
self,
args='', # type: typing.Union[typing.List[str], str]
returncode=-1, # type: int
stdout=None, # type: typing.Optional[bytes]
stderr=None # type: typing.Optional[bytes]
) -> None:
self.args = args
self.returncode = returncode
self.stdout = stdout
self.stderr = stderr
def check_returncode(self) -> None:
if self.returncode != 0:
raise subprocess.CalledProcessError(
self.returncode,
str(self.args),
output=self.stdout,
)
class TestPressureVessel(unittest.TestCase):
def setUp(self) -> None:
# Absolute path to the directory containing pressure-vessel
# and the runtime(s).
self.depot = os.path.abspath('depot')
# Apt suite used for the runtime (scout, heavy or soldier).
# Default: scout
self.runtime_suite = os.getenv('TEST_CONTAINER_RUNTIME_SUITE', 'scout')
# dpkg architectures in the runtime, with primary architecture
# first. Default: amd64, i386
self.dpkg_architectures = os.getenv(
'TEST_CONTAINER_RUNTIME_ARCHITECTURES',
'amd64,i386'
).split(',')
# Path to an unpacked LD_LIBRARY_PATH runtime, or None.
self.ld_library_path_runtime = os.getenv(
'TEST_CONTAINER_RUNTIME_LD_LIBRARY_PATH_RUNTIME',
None,
) # type: typing.Optional[str]
self.tmpdir = tempfile.TemporaryDirectory()
self.addCleanup(self.tmpdir.cleanup)
if self.ld_library_path_runtime is not None:
if os.access(self.ld_library_path_runtime, os.W_OK):
self.ld_library_path_runtime = os.path.abspath(
self.ld_library_path_runtime,
)
else:
old = self.ld_library_path_runtime
new = os.path.join(self.tmpdir.name, 'ldlp')
shutil.copytree(old, new, symlinks=True)
self.ld_library_path_runtime = new
artifacts = os.getenv('AUTOPKGTEST_ARTIFACTS')
if artifacts is not None:
self.artifacts = artifacts
else:
self.artifacts = self.tmpdir.name
self.runtime_build_id = '(unknown)'
@contextlib.contextmanager
def catch(
self,
msg, # type: str
**kwargs # type: typing.Any
):
"""
Run a sub-test, with additional logging.
"""
if kwargs:
logger.info('Starting: %r (%r)', msg, kwargs)
else:
logger.info('Starting: %r', msg)
with self.subTest(msg, **kwargs):
try:
yield
except Exception:
logger.error(msg, exc_info=True)
raise
def get_runtime_build_id(self):
filename = (
'com.valvesoftware.SteamRuntime.Platform-'
'{}-{}-buildid.txt'
).format(
','.join(self.dpkg_architectures),
self.runtime_suite,
)
with self.catch('get build ID'):
with open(os.path.join(self.depot, filename)) as reader:
self.runtime_build_id = reader.read().strip()
logger.info('Build ID: %s', self.runtime_build_id)
return self.runtime_build_id
def run_subprocess(
self,
args, # type: typing.Union[typing.List[str], str]
check=False,
input=None, # type: typing.Optional[bytes]
timeout=None, # type: typing.Optional[int]
**kwargs # type: typing.Any
):
"""
This is basically a reimplementation of subprocess.run()
from Python 3.5+, so that this test can be run on the Python
3.4 interpreter in Debian 8 'jessie', SteamOS 2 'brewmaster'
and Ubuntu 14.04 'trusty'. It also adds extra logging.
"""
logger.info('Running: %r', args)
popen = subprocess.Popen(args, **kwargs) # type: ignore
out, err = popen.communicate(input=input, timeout=timeout)
completed = MyCompletedProcess(
args=args,
returncode=popen.returncode,
stdout=out,
stderr=err,
)
if check:
completed.check_returncode()
return completed
def test_general_info(self) -> None:
with open(
os.path.join(self.artifacts, 'depot-contents.txt'), 'w'
) as writer:
with self.catch('List contents of depot'):
completed = self.run_subprocess(
['find', '.', '-ls'],
check=True,
cwd=self.depot,
stdout=writer,
stderr=subprocess.PIPE,
)
if completed.stderr:
logger.info(
'(stderr) ->\n%s',
completed.stderr.decode('utf-8', 'backslashreplace'),
)
else:
logger.info('(no stderr)')
with self.catch('Read VERSION.txt'):
with open(
os.path.join(
'depot', 'pressure-vessel', 'metadata', 'VERSION.txt',
),
'r'
) as reader:
logger.info(
'pressure-vessel version %s',
reader.read().strip(),
)
for exe in (
'bwrap',
'i386-linux-gnu-capsule-capture-libs',
'x86_64-linux-gnu-capsule-capture-libs',
'pressure-vessel-wrap',
# TODO: Add this when a version with --version has been integrated
# 'steam-runtime-system-info',
):
with self.catch('--version', exe=exe):
completed = self.run_subprocess(
[
os.path.join('pressure-vessel', 'bin', exe),
'--version',
],
check=True,
cwd=self.depot,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
logger.info(
'%s --version ->\n%s',
exe,
completed.stdout.decode('utf-8').strip(),
)
if completed.stderr:
logger.info(
'(stderr) ->\n%s',
completed.stderr.decode('utf-8', 'backslashreplace'),
)
else:
logger.info('(no stderr)')
with open(
os.path.join(self.artifacts, 's-r-s-i-outside.json'),
'w',
) as writer:
with self.catch('steam-runtime-system-info outside container'):
completed = self.run_subprocess(
[
os.path.join(
'pressure-vessel', 'bin',
'steam-runtime-system-info',
),
'--verbose',
],
check=True,
cwd=self.depot,
stdout=writer,
stderr=subprocess.PIPE,
)
if completed.stderr:
logger.info(
'%s --version (stderr) -> %s',
exe, completed.stderr,
)
else:
logger.info('(no stderr)')
self.get_runtime_build_id()
def test_pressure_vessel(
self,
artifact_prefix='s-r-s-i-inside',
ld_library_path_runtime=None # type: typing.Optional[str]
) -> None:
if self.runtime_suite == 'scout':
adverb = [
os.path.join(self.depot, 'run-in-scout'),
'--verbose',
'--',
]
else:
if ld_library_path_runtime is None:
exe = 'pressure-vessel-wrap'
else:
exe = 'pressure-vessel-unruntime'
adverb = [
os.path.join(
self.depot,
'pressure-vessel',
'bin',
exe,
),
'--verbose',
'--runtime-base={}'.format(self.depot),
'--runtime={}/files'.format(self.runtime_suite),
'--',
]
if ld_library_path_runtime is not None:
adverb = [
os.path.join(ld_library_path_runtime, 'run.sh'),
] + adverb
with self.catch('cat /etc/os-release in container'):
# TODO: Ideally this would use check=True,
# but that would need a bubblewrap with
# https://github.com/containers/bubblewrap/issues/336
# fixed, or a pressure-vessel with !18 fixed
completed = self.run_subprocess(
adverb + ['cat', '/etc/os-release'],
cwd=self.tmpdir.name,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
logger.info(
'os-release:\n%s',
completed.stdout.decode('utf-8'),
)
if completed.stderr:
logger.info(
'(stderr) ->\n%s',
completed.stderr.decode('utf-8', 'backslashreplace'),
)
else:
logger.info('(no stderr)')
with open(
os.path.join(self.artifacts, artifact_prefix + '.json'),
'w',
) as writer:
with self.catch('run s-r-s-i in container'):
# TODO: Ideally this would use check=True,
# but that would need a bubblewrap with
# https://github.com/containers/bubblewrap/issues/336
# fixed, or a pressure-vessel with !18 fixed
completed = self.run_subprocess(
adverb + ['steam-runtime-system-info', '--verbose'],
cwd=self.tmpdir.name,
stdout=writer,
stderr=subprocess.PIPE,
)
if completed.stderr:
logger.info(
'(stderr) ->\n%s',
completed.stderr.decode('utf-8', 'backslashreplace'),
)
else:
logger.info('(no stderr)')
with open(
os.path.join(self.artifacts, artifact_prefix + '.json'),
'r',
) as reader:
parsed = json.load(reader)
self.get_runtime_build_id()
self.assertIsInstance(parsed, dict)
self.assertIn('can-write-uinput', parsed)
self.assertIn('steam-installation', parsed)
with self.catch('runtime information'):
self.assertIn('runtime', parsed)
self.assertEqual('/', parsed['runtime'].get('path'))
self.assertIn('version', parsed['runtime'])
issues = parsed['runtime'].get('issues', [])
self.assertNotIn('disabled', issues)
self.assertNotIn('internal-error', issues)
self.assertNotIn('not-in-environment', issues)
self.assertNotIn('not-in-ld-path', issues)
self.assertNotIn('not-in-path', issues)
self.assertNotIn('not-runtime', issues)
self.assertNotIn('not-using-newer-host-libraries', issues)
self.assertNotIn('unexpected-location', issues)
self.assertNotIn('unexpected-version', issues)
# Don't assert whether it contains 'unofficial':
# we want to be able to test unofficial runtimes too
self.assertIn('overrides', parsed['runtime'])
self.assertNotIn('pinned_libs_32', parsed['runtime'])
self.assertNotIn('pinned_libs_64', parsed['runtime'])
with self.catch('os-release information'):
self.assertIn('os-release', parsed)
self.assertEqual('steamrt', parsed['os-release']['id'])
self.assertNotIn(
parsed['os-release']['id'],
parsed['os-release'].get('id_like', [])
)
self.assertIn('name', parsed['os-release'])
self.assertIn('pretty_name', parsed['os-release'])
self.assertIn('version_id', parsed['os-release'])
if self.runtime_suite == 'scout':
self.assertEqual('1', parsed['os-release']['version_id'])
elif self.runtime_suite == 'heavy':
self.assertEqual('1.5', parsed['os-release']['version_id'])
elif self.runtime_suite == 'soldier':
self.assertEqual('2', parsed['os-release']['version_id'])
if self.runtime_suite == 'scout':
# Older versions of base-files didn't have version_codename
if 'version_codename' in parsed['os-release']:
self.assertEqual(
'scout',
parsed['os-release']['version_codename'],
)
else:
self.assertEqual(
self.runtime_suite,
parsed['os-release']['version_codename'],
)
self.assertEqual(
self.runtime_build_id,
parsed['os-release']['build_id'],
)
self.assertIn('architectures', parsed)
for arch in self.dpkg_architectures:
if arch == 'i386':
multiarch = 'i386-linux-gnu'
elif arch == 'amd64':
multiarch = 'x86_64-linux-gnu'
else:
continue
self.assertIn(multiarch, parsed['architectures'])
arch_info = parsed['architectures'][multiarch]
with self.catch('per-architecture information', arch=arch):
self.assertTrue(arch_info['can-run'])
self.assertEqual([], arch_info['library-issues-summary'])
# Graphics driver support depends on the host system, so we
# don't assert that everything is fine, only that we have
# the information.
self.assertIn('graphics-details', arch_info)
self.assertIn('glx/gl', arch_info['graphics-details'])
for soname, details in arch_info['library-details'].items():
with self.catch(
'per-library information',
arch=arch,
soname=soname,
):
self.assertIn('path', details)
self.assertEqual(
[],
details.get('missing-symbols', []),
)
self.assertEqual(
[],
details.get('misversioned-symbols', []),
)
self.assertEqual([], details.get('issues', []))
# Locale support depends on the host system, so we don't assert
# that everything is fine, only that we have the information.
self.assertIn('locale-issues', parsed)
self.assertIn('locales', parsed)
# Graphics driver support depends on the host system, so we
# don't assert that everything is fine, only that we have
# the information.
self.assertIn('egl', parsed)
self.assertIn('vulkan', parsed)
def test_unruntime(
self,
) -> None:
if self.ld_library_path_runtime is not None:
self.run_subprocess(
os.path.join(self.ld_library_path_runtime, 'setup.sh'),
check=True,
stdout=2,
stderr=2,
)
self.test_pressure_vessel(
artifact_prefix='s-r-s-i-inside-unruntime',
ld_library_path_runtime=self.ld_library_path_runtime,
)
else:
self.skipTest(
'TEST_CONTAINER_RUNTIME_LD_LIBRARY_PATH_RUNTIME not provided'
)
def tearDown(self) -> None:
pass
if __name__ == '__main__':
logging.basicConfig(level=logging.DEBUG)
sys.path[:0] = [os.path.join(
os.path.dirname(os.path.dirname(__file__)),
'third-party',
)]
import pycotap
unittest.main(
buffer=False,
testRunner=pycotap.TAPTestRunner,
)
# vi: set sw=4 sts=4 et:
#!/bin/sh
# Copyright © 2016-2018 Simon McVittie
# Copyright © 2019-2020 Collabora Ltd.
#
# SPDX-License-Identifier: MIT
set -e
set -u
export MYPYPATH="${PYTHONPATH:=$(pwd)}"
i=0
for script in \
tests/depot/*.py \
; do
i=$((i + 1))
if [ "x${MYPY:="$(command -v mypy || echo false)"}" = xfalse ]; then
echo "ok $i - $script # SKIP mypy not found"
elif "${MYPY}" \
--python-executable="${PYTHON:=python3}" \
--follow-imports=skip \
--ignore-missing-imports \
"$script"; then
echo "ok $i - $script"
else
echo "not ok $i - $script # TODO mypy issues reported"
fi
done
echo "1..$i"
# vim:set sw=4 sts=4 et:
#!/bin/sh
# Copyright © 2016-2018 Simon McVittie
# Copyright © 2019-2020 Collabora Ltd.
#
# SPDX-License-Identifier: MIT
set -e
set -u
if [ "x${PYCODESTYLE:=pycodestyle}" = xfalse ] || \
[ -z "$(command -v "$PYCODESTYLE")" ]; then
echo "1..0 # SKIP pycodestyle not found"
elif "${PYCODESTYLE}" \
tests/depot/*.py \
>&2; then
echo "1..1"
echo "ok 1 - $PYCODESTYLE reported no issues"
else
echo "1..1"
echo "not ok 1 # TODO $PYCODESTYLE issues reported"
fi
# vim:set sw=4 sts=4 et:
#!/bin/sh
# Copyright © 2016-2018 Simon McVittie
# Copyright © 2019-2020 Collabora Ltd.
#
# SPDX-License-Identifier: MIT
set -e
set -u
if [ "x${PYFLAKES:=pyflakes3}" = xfalse ] || \
[ -z "$(command -v "$PYFLAKES")" ]; then
echo "1..0 # SKIP pyflakes3 not found"
elif "${PYFLAKES}" \
tests/depot/*.py \
>&2; then
echo "1..1"
echo "ok 1 - $PYFLAKES reported no issues"
else
echo "1..1"
echo "not ok 1 # TODO $PYFLAKES issues reported"
fi
# vim:set sw=4 sts=4 et:
#!/bin/sh
# Copyright © 2016-2018 Simon McVittie
# Copyright © 2019-2020 Collabora Ltd.
#
# SPDX-License-Identifier: MIT
set -e
set -u
n=0
fail=
for script in \
tests/depot/*.py \
; do
n=$(( n + 1 ))
if python3 "$script" --help >/dev/null; then
echo "ok $n - $script --help succeeded with python3"
else
echo "not ok $n - $script --help failed with python3"
fail=yes
fi
done
echo "1..$n"
if [ -n "$fail" ]; then
exit 1
fi
# vim:set sw=4 sts=4 et:
#!/bin/sh
# Copyright © 2016-2018 Simon McVittie
# Copyright © 2019-2020 Collabora Ltd.
#
# SPDX-License-Identifier: MIT
set -e
set -u
n=0
fail=
if [ -z "$(command -v python3.4)" ]; then
echo "1..0 # SKIP python3.4 not found"
exit 0
fi
for script in \
tests/depot/*.py \
; do
n=$(( n + 1 ))
if python3.4 "$script" --help >/dev/null; then
echo "ok $n - $script --help succeeded with python3.4"
else
echo "not ok $n - $script --help failed with python3.4"
fail=yes
fi
done
echo "1..$n"
if [ -n "$fail" ]; then
exit 1
fi
# vim:set sw=4 sts=4 et:
#!/bin/sh
# Copyright © 2016-2018 Simon McVittie
# Copyright © 2019-2020 Collabora Ltd.
#
# SPDX-License-Identifier: MIT
set -e
set -u
n=0
fail=
if [ -z "$(command -v python3.5)" ]; then
echo "1..0 # SKIP python3.5 not found"
exit 0
fi
for script in \
tests/depot/*.py \
; do
n=$(( n + 1 ))
if python3.5 "$script" --help >/dev/null; then
echo "ok $n - $script --help succeeded with python3.5"
else
echo "not ok $n - $script --help failed with python3.5"
fail=yes
fi
done
echo "1..$n"
if [ -n "$fail" ]; then
exit 1
fi
# vim:set sw=4 sts=4 et:
#!/bin/sh
#
# Copyright © 2018-2020 Collabora Ltd
# SPDX-License-Identifier: MIT
set -e
set -u
if ! command -v shellcheck >/dev/null 2>&1; then
echo "1..0 # SKIP shellcheck not available"
exit 0
fi
if [ -z "${G_TEST_SRCDIR-}" ]; then
me="$(readlink -f "$0")"
srcdir="${me%/*}"
G_TEST_SRCDIR="${srcdir%/*}"
fi
cd "$G_TEST_SRCDIR"
n=0
for shell_script in \
debian/tests/depot \
tests/*.sh \
; do
n=$((n + 1))
# Ignore SC2039: we assume a Debian-style shell that has 'local'.
if shellcheck --exclude=SC2039 "$shell_script"; then
echo "ok $n - $shell_script"
else
echo "not ok $n # TODO - $shell_script"
fi
done
echo "1..$n"
# vim:set sw=4 sts=4 et ft=sh:
Copyright (c) 2015 Remko Tronçon (https://el-tramo.be)
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
#!/usr/bin/env python
# coding=utf-8
# Copyright (c) 2015 Remko Tronçon (https://el-tramo.be)
# Released under the MIT license
# See COPYING for details
import unittest
import sys
import base64
if sys.hexversion >= 0x03000000:
from io import StringIO
else:
from StringIO import StringIO
# Log modes
class LogMode(object) :
LogToError, LogToDiagnostics, LogToYAML, LogToAttachment = range(4)
class TAPTestResult(unittest.TestResult):
def __init__(self, output_stream, error_stream, message_log, test_output_log):
super(TAPTestResult, self).__init__(self, output_stream)
self.output_stream = output_stream
self.error_stream = error_stream
self.orig_stdout = None
self.orig_stderr = None
self.message = None
self.test_output = None
self.message_log = message_log
self.test_output_log = test_output_log
self.output_stream.write("TAP version 13\n")
self._set_streams()
def printErrors(self):
self.print_raw("1..%d\n" % self.testsRun)
self._reset_streams()
def _set_streams(self):
self.orig_stdout = sys.stdout
self.orig_stderr = sys.stderr
if self.message_log == LogMode.LogToError:
self.message = self.error_stream
else:
self.message = StringIO()
if self.test_output_log == LogMode.LogToError:
self.test_output = self.error_stream
else:
self.test_output = StringIO()
if self.message_log == self.test_output_log:
self.test_output = self.message
sys.stdout = sys.stderr = self.test_output
def _reset_streams(self):
sys.stdout = self.orig_stdout
sys.stderr = self.orig_stderr
def print_raw(self, text):
self.output_stream.write(text)
self.output_stream.flush()
def print_result(self, result, test, directive = None):
self.output_stream.write("%s %d %s" % (result, self.testsRun, test.id()))
if directive:
self.output_stream.write(" # " + directive)
self.output_stream.write("\n")
self.output_stream.flush()
def ok(self, test, directive = None):
self.print_result("ok", test, directive)
def not_ok(self, test):
self.print_result("not ok", test)
def startTest(self, test):
super(TAPTestResult, self).startTest(test)
def stopTest(self, test):
super(TAPTestResult, self).stopTest(test)
if self.message_log == self.test_output_log:
logs = [(self.message_log, self.message, "output")]
else:
logs = [
(self.test_output_log, self.test_output, "test_output"),
(self.message_log, self.message, "message")
]
for log_mode, log, log_name in logs:
if log_mode != LogMode.LogToError:
output = log.getvalue()
if len(output):
if log_mode == LogMode.LogToYAML:
self.print_raw(" ---\n")
self.print_raw(" " + log_name + ": |\n")
self.print_raw(" " + output.rstrip().replace("\n", "\n ") + "\n")
self.print_raw(" ...\n")
elif log_mode == LogMode.LogToAttachment:
self.print_raw(" ---\n")
self.print_raw(" " + log_name + ":\n")
self.print_raw(" File-Name: " + log_name + ".txt\n")
self.print_raw(" File-Type: text/plain\n")
self.print_raw(" File-Content: " + base64.b64encode(output) + "\n")
self.print_raw(" ...\n")
else:
self.print_raw("# " + output.rstrip().replace("\n", "\n# ") + "\n")
log.truncate(0)
def addSuccess(self, test):
super(TAPTestResult, self).addSuccess(test)
self.ok(test)
def addError(self, test, err):
super(TAPTestResult, self).addError(test, err)
self.message.write(self.errors[-1][1] + "\n")
self.not_ok(test)
def addFailure(self, test, err):
super(TAPTestResult, self).addFailure(test, err)
self.message.write(self.failures[-1][1] + "\n")
self.not_ok(test)
def addSkip(self, test, reason):
super(TAPTestResult, self).addSkip(test, reason)
self.ok(test, "SKIP " + reason)
def addExpectedFailure(self, test, err):
super(TAPTestResult, self).addExpectedFailure(test, err)
self.ok(test)
def addUnexpectedSuccess(self, test):
super(TAPTestResult, self).addUnexpectedSuccess(test)
self.message.write("Unexpected success" + "\n")
self.not_ok(test)
class TAPTestRunner(object):
def __init__(self,
message_log = LogMode.LogToYAML,
test_output_log = LogMode.LogToDiagnostics,
output_stream = sys.stdout, error_stream = sys.stderr):
self.output_stream = output_stream
self.error_stream = error_stream
self.message_log = message_log
self.test_output_log = test_output_log
def run(self, test):
result = TAPTestResult(
self.output_stream,
self.error_stream,
self.message_log,
self.test_output_log)
test(result)
result.printErrors()
return result
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment