Translate FEX-Emu rootfs

• pv-runtime: Open host root directory as a file descriptor

• pv-wrap: Handle a larger subset of bwrap command-line syntax

  When we add support for transparent emulation such as FEX-Emu, we'll need to carry out rewriting for bwrap arguments generated from the code we borrow from Flatpak. This will be easier to do if we support more of bwrap's command-line options.

  As before, we classify them by how many arguments they have, and which arguments are paths in the host system.

• pv-wrap: Return new bwrap object from pv_wrap_share_sockets()

  It'll be easier to pass this through transformations like adjust_exports() if we can know that everything in the new bwrap object was produced by pv_wrap_share_sockets().

• pv-wrap: Prototype detection of FEX-Emu "rootfs" filesystem overlay

  A common approach to transparent CPU-architecture emulation is to have an overlay with binaries for the emulated architecture, and transparently use those (as though they were an overlayfs over the real filesystem) when an emulated process accesses their paths.

  For example, FEX-Emu emulates x86 on aarch64 systems. When an aarch64 process opens an architecture-dependent file like /usr/lib64/libc.so.6 or /etc/ld.so.cache, it gets the real file; when an emulated x86 process opens an architecture-dependent file, if that file exists in the FEX-Emu "rootfs", FEX-Emu will transparently redirect the file access to the copy in the "rootfs".

  When we enter the Steam Runtime container, we need two environments inside the container: an aarch64 environment that can run the FEX-Emu interpreter itself, and an x86 environment that can run pressure-vessel-adverb, Steam Runtime executables, and the actual game. To set this up, we have to be able to distinguish between the two environments.

• Use SrtVirtualizationInfo for pv_wrap_detect_interpreter_root()

• pv-wrap: Bypass FEX-Emu overlay for most top-level directories

  The FEX-Emu "rootfs" is conceptually just a /usr (plus parts of /etc and /var), just like the Steam Runtime. However, the rootfs versions that are currently distributed contain some files in /etc, /run and so on.

  When we open files with opendir() or access() or similar, FEX-Emu will transparently substitute files from the "rootfs" if they exist. However, when we're building up a list of bind-mounts for a root filesystem, this is counterproductive: bwrap will do the equivalent of mount --bind /oldroot/run/foo /newroot/run/foo, which is not going to work if /foo is really /home/me/.fex-emu/RootFS/Ubuntu_21_10/run/foo.

  For several categories of file, and in particular /run and other API filesystems, what we really want to do is to ignore the "rootfs" and take the files and directories from the real root. Because FEX-Emu's rewriting is path-based rather than identity-based, we can do this by looking up the paths relative to /proc/self/root.

• pv-wrap: Use the FEX-Emu rootfs as the default graphics stack provider

  We don't yet do anything with FEX-Emu thunks (substitute shared libraries that override certain real libraries), so we need to provide a complete x86 graphics stack, and the FEX-Emu rootfs is a good place to find one of those.

• pv-wrap: Adjust paths set up by Flatpak code for FEX-Emu rootfs

  The code we've borrowed from Flatpak to deal with things like X11 and D-Bus can find files and directories in either the real root filesystem, or the FEX-Emu rootfs. If we naively pass the paths it has discovered to bwrap, we'll find that they don't necessarily exist: for example, the minimal aarch64 system I'm using for initial testing of this code doesn't have /var/cache/fontconfig, but the rootfs does. This makes bwrap fail when it tries to bind-mount /oldroot/var/cache/fontconfig onto /newroot/var/cache/fontconfig.

  To avoid this failure mode, look up the paths in the rootfs, and if necessary rewrite them. For example, we can bind-mount /oldroot/$FEX_ROOTFS/var/cache/fontconfig onto /newroot/var/cache/fontconfig, and that will work fine.

• pv-runtime: Be more compatible with interpreters like FEX-Emu

  A common approach to transparent CPU-architecture emulation is to have an overlay with binaries for the emulated architecture, and transparently use those (as though they were an overlayfs over the real filesystem) when an emulated process accesses their paths.

  For example, FEX-Emu emulates x86 on aarch64 systems. When an aarch64 process opens an architecture-dependent file like /usr/lib64/libc.so.6 or /etc/ld.so.cache, it gets the real file; when an emulated x86 process opens an architecture-dependent file, if that file exists in the FEX-Emu "rootfs", FEX-Emu will transparently redirect the file access to the copy in the "rootfs".

  When we enter the Steam Runtime container, we need two environments inside the container: an aarch64 environment that can run the FEX-Emu interpreter itself, and an x86 environment that can run pressure-vessel-adverb, Steam Runtime executables, and the actual game.

  To arrange for this to happen, we can use a subset of the real aarch64 operating system as the root of the container, and set up a new FEX-Emu rootfs in /run/pressure-vessel/interpreter-root. This means that aarch64 executables in the Steam Runtime container see the real host OS, but x86 executables see the Steam Runtime.

---

This is enough to get a working shell with:

time env \
    PRESSURE_VESSEL_LOG_WITH_TIMESTAMP=1 \
    FEX_ROOTFS=$HOME/.fex-emu/RootFS/Ubuntu_21_10 \
    G_MESSAGES_DEBUG=all STEAM_LINUX_RUNTIME_VERBOSE=1 \
./steam-container-runtime/depot/run \
    --devel --verbose -- \
bash

when run in a Debian 11 cloud image (using qemu-system-aarch64, I don't actually have aarch64 hardware). The emulation-on-emulation makes it pretty slow, but it works.

Known issues, which I think should perhaps be solved as follow-up MRs rather than here:

• steam-runtime-tools is unable to detect the ${PLATFORM}, which means LD_AUDIT, LD_PRELOAD and VDPAU won't work properly. In practice people use Ubuntu sysroots with FEX-Emu, so we could perhaps do something like: if ${LIB} expands to lib/ + the multiarch tuple, then use ${LIB}-based paths, else fall back to ${PLATFORM}-based paths?

• There's no attempt to handle FEX-Emu thunk libraries yet.