Skip to content

Improve compatibility with other mtree implementations

Simon McVittie requested to merge wip/smcv/mtree-enhancements into master

In future we might want to use a mtree manifest to verify that the contents of the runtime depot match what we expect, particularly if we need to detect or investigate implementation issues in Steam's Steampipe CDN downloader (as with the recent issues involving symlinks).

mtree is a better file format for this purpose than sha256sum, because sha256sum is only applicable to regular files, only checks their contents, and normally only detects files that are missing or have changed; whereas mtree can also represent directories, symlinks and other non-regular files, can represent metadata, and normally detects files that are present but should not be (although it can be told to ignore such files, for example in var/). So it would make sense to continue to reuse mtree.

If we want to be able to go that route, it makes sense for pressure-vessel's mtree implementation to be able to understand at least the optional, nochange and ignore flags. These flags are not compatible with assumptions made in pressure-vessel's mtree implementation, so we need to change some things around.

While implementing those flags, I noticed some other incompatibilities between libarchive and pressure-vessel which would also be good to fix.

  • mtree: Accept content= as a synonym for contents=

    libarchive allows both, although FreeBSD and NetBSD mtree do not.

  • mtree: Parse modification times in a compatible way

    time=1.234 has historically meant 1 second + 234 nanoseconds, or what normal people would write as 1.000000234. Parse it as such, but for now treat this ambiguous format as an error, unless the fractional part is exactly ".0" which is unambiguous - we can relax this to a warning later, if we want to.

    libarchive and FreeBSD mtree show mtimes with exactly 9 decimal digits, which makes them mean the same thing in either interpretation.

  • mtree: Parse the ignore, nochange and optional flags

    The nochange flag is fully implemented here: it suppresses changing the modification time or permissions.

    The optional flag is mostly implemented here: it suppresses the error that would ordinarily occur if a file with content does not already exist.

    The ignore flag is not yet implemented, only parsed, because it only really applies when verifying whether a filesystem tree matches a specification.

Merge request reports