pressure-vessel: Use a mtree(5) manifest to set up the runtime

• pv-runtime: GC all deployments if operating from an unpacked directory

  If we are operating from ./scout_platform_x.y.z, self->id will be NULL. Instead of matching on the names of directories, we can just check whether the deployment we are going to use is the same file (device and inode number) as the old deployment we are considering deleting.

• pressure-vessel: Add code to parse and apply a mtree(5) manifest

  This can be used to create directories, symlinks and zero-byte files, create files as a clone (hard-link or copy) of files in a reference directory, assert that files exist, set executable/non-executable permissions, and set files' modification times.

  When cloning files from a reference directory, the default is to use the same filename as the destination, but that can be overridden (for example to deal with inconvenient/non-Windows-friendly filenames). This is also actually slightly faster than pv_cheap_tree_copy(), presumably because the manifest is contiguous on-disk and the directories are not necessarily.

  Setting directories' modification times is implemented, but probably won't work in practice, because any files we create in the directory will alter its mtime.

• runtime: Add support for creating runtime from a mtree(5)

  At the moment we deploy the runtime from a giant tarball to avoid Steam downloader limitations, but that leads to a noticeable delay the first time we launch a game after a new runtime version has been downloaded.

  Now that the Steam download mechanism can deal better with larger numbers of smaller files, we're considering returning to the original design where the runtime depot contains unpacked files. However, the Steam download mechanism doesn't preserve permissions, modification times, or filenames that differ only by case, and has not always preserved empty directories, so we need a way to deal with all of those things.

  By reading a manifest written in a subset of the BSD mtree(5) format, we can create directories and symlinks, and set permissions modification times on regular files. As a bonus, it's actually slightly faster to duplicate a runtime with hard-links (--copy-runtime mode) by reading the manifest than by reading the actual directory tree, because the manifest is more likely to be contiguous on disk.

  In principle the mtree(5) manifest could also be used to validate that the runtime content has not become corrupted by checking files against their sha256sums. This isn't implemented here (and it would have to be done only on demand rather than routinely, because it would be slow), but the parser does at least read the sha256.

  In the tests, we now need to remove the mtree manifest when copying and editing a runtime. When we edit a runtime in-place, it no longer conforms to the manifest, so this can't necesarily be expected to work.

---

/cc @denittis

Part of the space-efficiency improvements from T27130.

To test:

• replace SteamLinuxRuntime/pressure-vessel and SteamLinuxRuntime_soldier/pressure-vessel with this, using a normal SLR build that deploys from archives
  • this confirms that we can upgrade to this pressure-vessel without immediately having to change how we build the runtime depots
• get steamlinuxruntime!39 (merged) and make check (you might want to set IMAGES_SSH_HOST and IMAGES_SSH_PATH so that you get the latest builds)
• replace your SteamLinuxRuntime with depots/test-scout-unpacked
• replace your SteamLinuxRuntime_soldier with depots/test-soldier-unpacked
• replace your SteamLinuxRuntime/pressure-vessel and SteamLinuxRuntime_soldier/pressure-vessel with this, again

For T27914 (a nice-to-have followup) you could also try with something like this:

find SteamLinuxRuntime/scout_platform_0.20210429.0/files -type l -delete
find SteamLinuxRuntime/scout_platform_0.20210429.0/files -empty -delete

which reduces the files/ directory to the bare minimum (just the non-empty regular files, plus their parent directories), and creates the symlinks, empty regular files and empty directories on a just-in-time basis.