Skip to content
Snippets Groups Projects
  1. Feb 09, 2021
    • Jonathan Lebon's avatar
      glnx_file_copy_at: Add GLNX_FILE_COPY_NOCHOWN · 1345882d
      Jonathan Lebon authored
      In some contexts, we may want to copy a root-owned file but we're not
      running as root so we can't `fchown` it. (The case I'm interested in is
      actually a bit more obscure than this: running in a supermin VM as root,
      and wanting to copy a file we created onto a 9p mount where we don't
      have perms to `fchown`).
      
      Add a `GLNX_FILE_COPY_NOCHOWN` to handle this case.
      1345882d
  2. Nov 03, 2020
    • Alexander Larsson's avatar
      Add GLNX_FILE_REPLACE_INCREASING_MTIME · 013417ea
      Alexander Larsson authored
      This make replaced files have a strictly increasing st_mtime. The main
      usecase I have for this is to ensure the summary file mtime increases
      because the flatpak tests are failing due to the python httpd used
      in the tests rely on st_mtime for the http If-Modified-Since header.
      
      For the tests this breaks all the time since we're just doing a lot of
      summary updates. However, I can see this accidentally happening in the
      wild too, so i think its proper to always ensure the new summary is
      "newer", even though it means it will be timestamped slightly in the
      future. In practice this will not happen regularly, and the times it
      *does* happen we really do need it.
      013417ea
  3. Oct 01, 2020
  4. Sep 23, 2020
    • Rebecca Turner's avatar
      glnx-fdio: try $TMPDIR if /var/tmp doesn't exist · 1ea9158c
      Rebecca Turner authored
      `glnx_open_anonymous_tmpfile` attempts to create an fd in `/var/tmp`
      regardless of the value of `$TMPDIR`.
      
      This is _usually_ okay, but can fail in some contexts, such as in the
      [NixOS][1] build environment, which doesn't have `/var` mapped at all.
      
      To avoid failing in this case, if the inner call to
      `glnx_open_anonymous_tmpfile_full` fails, we retrieve the value of
      `$TMPDIR` and try calling `glnx_open_anonymous_tmpfile_full` again with
      that directory instead.
      
      In the fast path (i.e. where `/var/tmp` exists), functionality is
      unchanged.
      
      [1]: https://nixos.org/
      1ea9158c
  5. Jun 16, 2020
    • Jonathan Lebon's avatar
      glnx-fdio: handle EOPNOTSUPP for copy_file_range · 7e3a1995
      Jonathan Lebon authored
      When using `copy_file_range` to target a source and dest on the same NFS
      mount on some older kernel versions, it's possible that we can get
      `EOPNOTSUPP` e.g. if the NFS server doesn't support server-side copy.
      
      We hit this in the FCOS release pipeline where we run `ostree
      pull-local` to pull content between two repos on the same mount from
      inside an OpenShift cluster on top of RHEL7.
      
      Nowadays, it seems like the kernel itself falls back to a more generic
      version of `copy_file_range()` at least. Though to be compatible with
      older kernels, let's add `EOPNOTSUPP` to the list of errors we interpret
      as "cfr possibly available, but can't be done for this specific
      operation".
      7e3a1995
  6. Jun 03, 2020
  7. Nov 11, 2019
  8. Nov 04, 2019
  9. Oct 09, 2019
  10. Nov 30, 2018
    • Owen W. Taylor's avatar
      Fix docs for glnx_file_replace_contents() · c2dbe181
      Owen W. Taylor authored
      The docs for `glnx_file_replace_contents[_with_perms]` say that the default mode
      is 0666 - umask, but it's actually 0644. Because there's no thread-safe way of
      finding out the current umask without grubbing around in /proc/self/status,
      simply make the docs reflect reality.
      c2dbe181
  11. Oct 24, 2017
  12. Oct 16, 2017
  13. Oct 05, 2017
  14. Oct 02, 2017
  15. Sep 25, 2017
    • Colin Walters's avatar
      fdio: Open target dirname for glnx_file_copy_at() · 5ee2f1be
      Colin Walters authored
      Particularly if `AT_FDCWD` is used, we need to open
      in the target dir, otherwise we can get `EXDEV` when trying
      to do the final link.
      
      (Theoretically we can cross a mountpoint even with fd-relative
       though this is a lot less likely)
      5ee2f1be
  16. Sep 13, 2017
  17. Sep 12, 2017
    • Colin Walters's avatar
    • Colin Walters's avatar
      fdio: Use O_TMPFILE + rename-overwrite for regfile copies · 673f48f6
      Colin Walters authored
      I was working on rpm-ostree unified core, and hit the fact that
      `glnx_file_copy_at()` had the same bug with `fsetxattr()` and files whose mode
      is <= `0400` (e.g. `000` in the case of `/etc/shadow`) that libostree did a
      while ago.  Basically, Linux currently allows `write()` on non-writable open files
      but not `fsetxattr()`.  This situation is masked for privileged (i.e.
      `CAP_DAC_OVERRIDE`) code.
      
      Looking at this, I think it's cleaner to convert to `O_TMPFILE` here,
      since that code already handles setting the tmpfile to mode `0600`.  Now,
      this *is* a behavior change in the corner case of existing files which
      are symbolic links.  Previously we'd do an `open(O_TRUNC)` which would follow
      the link.
      
      But in the big picture, I think the use cases for `open(O_TRUNC)` are really
      rare - I audited all callers of this in ostree/rpm-ostree/flatpak, and all of
      them will be fine with this behavior change. For example, the ostree `/etc`
      merge code already explicitly unlinks the target beforehand. Other cases like
      supporting `repo/pubring.gpg` in an ostree repo being a symlink...eh, just no.
      
      Making this change allows us to convert to new style, and brings all of the
      general benefits of using `O_TMPFILE` too.
      673f48f6
    • Colin Walters's avatar
      fdio: Support taking ownership of tmpfile fd · 9d995a36
      Colin Walters authored
      While reading a strace I noticed a double close in the tests; this was because
      we were missing an assignment to `-1` in the tests. However, let's make
      supporting this clearer by explicitly supporting the fd being `-1` while still
      setting the `initialized` variable to `FALSE`. We also add the `EBADF` assertion
      checking.
      9d995a36
    • Colin Walters's avatar
      fdio: Use O_EXCL for anonymous tmpfiles · 806bb46e
      Colin Walters authored
      I noticed while reading the manpage for `linkat()` that `O_TMPFILE`
      supports `O_EXCL` to mean exactly what we're doing with the anonymous
      tmpfile API.
      
      Change the code to start using it; this required refactoring the internals since
      we had a check to be sure the caller wasn't passing `O_EXCL` for the
      non-anonymous path which we want to keep.
      
      Presumably the storage system could do smarter things if it knows a file will
      always be anonymous, e.g. it doesn't need to journal its data.
      806bb46e
  18. Aug 09, 2017
    • Colin Walters's avatar
      fdio: Merge systemd code to use copy_file_range(), use FICLONE · d18f026e
      Colin Walters authored
      FICLONE is the new alias for the formerly btrfs-specific ioctl; XFS
      has experimental patches to support it.
      
      Further, we should use copy_file_range() for the case where we're only doing a
      limited copy. Both NFS and XFS (with reflink enabled) understand it.
      
      Part of the reason I'm doing this is so that ostree's `/etc` merge will start
      using XFS reflinks. But another major reason is to take the next step after and
      copy this code into GLib as well, so that all of the general GLib users will
      benefit; e.g. Nautilus will transparently do server copy offloads with NFS home
      directories.
      
      See also this coreutils thread about `copy_file_range()`:
      <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24399>. I don't care about file
      holes for our use cases, so it's fine.
      
      Other changes while I'm here:
       - Tweak the sendfile() case to match the newly inlined logic for cfr
       - Add a TEMP_FAILURE_RETRY() around the read()
      d18f026e
  19. Jul 19, 2017
    • Colin Walters's avatar
      fdio: Introduce glnx_openat_read() · 268ae488
      Colin Walters authored
      This is kind of long overdue. Reasons are the same as the other wrappers. I
      debated adding `O_NOFOLLOW` support but the use cases for that are pretty
      obscure, callers who want that can just use the syscall directly for now.
      268ae488
  20. Jul 17, 2017
    • Colin Walters's avatar
      Remove glnx_stream_fstat() · 8b75c8e3
      Colin Walters authored
      There are only two users of this in ostree, and one of them is
      fairly bogus; we can just use `fstat()`.
      8b75c8e3
    • Colin Walters's avatar
      fdio: Add cleanup+flush API for FILE* · e30a773f
      Colin Walters authored
      Mostly in ostree/rpm-ostree, we work in either raw `int fd`, or
      `G{Input,Output}Stream`.  One exception is the rpm-ostree `/etc/passwd`
      handling, which uses `FILE*` since that's what glibc exposes.
      
      And in general, there are use cases for `FILE*`; the raw `GUnixOutputStream` for
      example isn't buffered, and doing so via e.g. `GBufferedOutputStream` means
      allocating *two* GObjects and even worse going through multiple vfuncs for every
      write.
      
      `FILE*` is used heavily in systemd, and provides buffering. It is a bit cheaper
      than gobjects, but has its own trap; by default every operation locks a mutex.
      For more information on that, see `unlocked_stdio(3)`. However, callers can
      avoid that by using e.g. `fwrite_unlocked`, which I plan to do for most users of
      `FILE*` that aren't writing to one of the standard streams like `stdout` etc.
      e30a773f
  21. Jul 10, 2017
  22. Jun 28, 2017
  23. Jun 26, 2017
  24. Jun 14, 2017
    • Colin Walters's avatar
      Add G_IN_SET, patch our internal users via spatch · 9a1b77ef
      Colin Walters authored
      I originally tried to get this into GLib:
      https://bugzilla.gnome.org/show_bug.cgi?id=783751
      
      But that looks like it's going to fail due to MSVC. Let's add it here at least
      so I can start using it tomorrow and not wait for the MSVC team to catch up.
      
      I renamed `glnx-alloca.h` to `glnx-macros.h` as a more natural collective
      home for things from systemd's `macro.h`.
      
      Finally, I used a Coccinelle spatch similar to the one referenced
      in the above BZ to patch our uses.
      9a1b77ef
  25. Jun 13, 2017
  26. May 19, 2017
  27. May 17, 2017
  28. May 15, 2017
    • Colin Walters's avatar
      fdio: Redo tmpfile API with GLnxTmpfile struct · 9929adcd
      Colin Walters authored
      The core problem with the previous tmpfile code
      is we don't have an autocleanup that calls `unlinkat`
      in the non-`O_TMPFILE` case.  And even if we did, it'd
      be awkward still since the `glnx_link_tmpfile_at()` call
      *consumes* the tmpfile.
      
      Fix this by introducing a struct with a cleanup macro. This simplifies a number
      of the callers in libostree - a notable case is where we had two arrays, one of
      fds, one of paths. It makes other places in libostree a bit more complex, but
      that's because some of the commit code paths want to deal with temporary
      *symlinks* too.
      
      Most callers are better though - in libglnx itself, `glnx_file_copy_at()` now
      correctly unlinks on failure for example.
      9929adcd
  29. Apr 28, 2017
    • Colin Walters's avatar
      fdio: Expose glnx_regfile_copy_bytes(), rewrite: GNU style, POSIX errno · 3a4d0f46
      Colin Walters authored
      NOTE: This changes the error handling API of `glnx_loop_write()` to be
      "old school POSIX" instead of "systemd".
      
      In ostree in a few places we use `g_output_stream_splice()`.  I
      thought this would use `splice()`, but actually it doesn't today.
      
      They also, if a cancellable is provided, end up dropping into `poll()` for every
      read and write. (In addition to copying data to/from userspace).
      
      My opinion on this is - for *local files* that's dumb. In the big picture, you
      really only need cancellation when copying gigabytes. Down the line, we could
      perhaps add a `glnx_copy_bytes_cancellable()` that only did that check e.g.
      every gigabyte of copied data. And when we do that we should use
      `g_cancellable_set_error_if_cancelled()` rather than a `poll()` with the regular
      file FD, since regular files are *always* readable and writable.
      
      For my use case with rpm-ostree though, we don't have gigabyte sized files, and
      seeing all of the `poll()` calls in strace is annoying. So let's have the
      non-cancellable file copying API that's modern and uses both reflink and
      `sendfile()` if available, in that order.
      
      My plan at some point once this is tested more is to migrate this code
      into GLib.
      
      Note that in order to keep our APIs consistent, I switched the systemd-imported
      code to "old school POSIX" error conventions. Otherwise we'd have *3* (POSIX,
      systemd, and GError) and particularly given the first two are easily confused,
      it'd be a recipe for bugs.
      3a4d0f46
  30. Apr 25, 2017
    • Colin Walters's avatar
      fdio: Mostly port to new code style · dc1956b2
      Colin Walters authored
      There's one function that did `unlinkat()` in the cleanup section,
      not doing that yet.
      
      Note I uncovered a few bugs in a few places where we didn't preserve errno
      before doing an `unlinkat()` in error paths in a few cases.
      
      I also tried to prefix a few more error cases with the system call name.
      dc1956b2
  31. Mar 23, 2017
    • Jonathan Lebon's avatar
      glnx-errors.h: add glnx_null_throw[_*] variants · 0c52d85e
      Jonathan Lebon authored
      These are equivalent to the non-null throw, except that the returned
      value is a NULL pointer. They can be used in functions where one wants
      to return a pointer. E.g.:
      
      	GKeyFile *foo(GError **error) {
      		return glnx_null_throw (error, "foobar");
      	}
      
      The function call redirections are wrapped around a compound statement
      expression[1] so that they represent a single top-level expression. This
      allows us to avoid -Wunused-value warnings vs using a comma operator if
      the return value isn't used.
      
      I made the 'args...' absorb the fmt argument as well so that callers can
      still use it without always having to specify at least one additional
      variadic argument. I had to check to be sure that the expansion is all
      done by the preprocessor, so we don't need to worry about stack
      intricacies.
      
      [1] https://gcc.gnu.org/onlinedocs/gcc/Statement-Exprs.html
      0c52d85e
  32. Mar 02, 2017
  33. Oct 05, 2016
  34. Aug 04, 2016
  35. Aug 03, 2016
Loading