Skip to content
Snippets Groups Projects
  • Simon McVittie's avatar
    799b7931
    runtime: Add support for creating runtime from a mtree(5) · 799b7931
    Simon McVittie authored
    
    At the moment we deploy the runtime from a giant tarball to avoid Steam
    downloader limitations, but that leads to a noticeable delay the first
    time we launch a game after a new runtime version has been downloaded.
    
    Now that the Steam download mechanism can deal better with larger
    numbers of smaller files, we're considering returning to the original
    design where the runtime depot contains unpacked files. However, the
    Steam download mechanism doesn't preserve permissions, modification
    times, or filenames that differ only by case, and has not always
    preserved empty directories, so we need a way to deal with all of
    those things.
    
    By reading a manifest written in a subset of the BSD mtree(5) format,
    we can create directories and symlinks, and set permissions modification
    times on regular files. As a bonus, it's actually slightly faster to
    duplicate a runtime with hard-links (--copy-runtime mode) by reading the
    manifest than by reading the actual directory tree, because the manifest
    is more likely to be contiguous on disk.
    
    In principle the mtree(5) manifest could also be used to validate that
    the runtime content has not become corrupted by checking files against
    their sha256sums. This isn't implemented here (and it would have to be
    done only on demand rather than routinely, because it would be slow),
    but the parser does at least read the sha256.
    
    In the tests, we now need to remove the mtree manifest when copying
    and editing a runtime. When we edit a runtime in-place, it no longer
    conforms to the manifest, so this can't necesarily be expected to work.
    
    Signed-off-by: default avatarSimon McVittie <smcv@collabora.com>
    799b7931
    History
    runtime: Add support for creating runtime from a mtree(5)
    Simon McVittie authored
    
    At the moment we deploy the runtime from a giant tarball to avoid Steam
    downloader limitations, but that leads to a noticeable delay the first
    time we launch a game after a new runtime version has been downloaded.
    
    Now that the Steam download mechanism can deal better with larger
    numbers of smaller files, we're considering returning to the original
    design where the runtime depot contains unpacked files. However, the
    Steam download mechanism doesn't preserve permissions, modification
    times, or filenames that differ only by case, and has not always
    preserved empty directories, so we need a way to deal with all of
    those things.
    
    By reading a manifest written in a subset of the BSD mtree(5) format,
    we can create directories and symlinks, and set permissions modification
    times on regular files. As a bonus, it's actually slightly faster to
    duplicate a runtime with hard-links (--copy-runtime mode) by reading the
    manifest than by reading the actual directory tree, because the manifest
    is more likely to be contiguous on disk.
    
    In principle the mtree(5) manifest could also be used to validate that
    the runtime content has not become corrupted by checking files against
    their sha256sums. This isn't implemented here (and it would have to be
    done only on demand rather than routinely, because it would be slow),
    but the parser does at least read the sha256.
    
    In the tests, we now need to remove the mtree manifest when copying
    and editing a runtime. When we edit a runtime in-place, it no longer
    conforms to the manifest, so this can't necesarily be expected to work.
    
    Signed-off-by: default avatarSimon McVittie <smcv@collabora.com>
runtime.c 201.97 KiB