Skip to content
Snippets Groups Projects
  1. Nov 04, 2019
  2. Oct 09, 2019
  3. Dec 12, 2017
  4. Nov 14, 2017
    • Ray Strode's avatar
      fdio: implement glnx_basename from scratch · 03b48a39
      Ray Strode authored
      At the top of glnx-fdio.h there's this comment:
      
      /* When we include libgen.h because we need
       * dirname() we immediately undefine
       * basename() since libgen.h defines it as
       * a macro to the XDG version which is really
       * broken. */
      
      and then it does #undef basename to try to
      gain access to non-default basename implementation.
      
      The problem is that this trick doesn't work on
      some systems:
      
      ./libglnx/glnx-fdio.h: In function 'glnx_basename':
      ./libglnx/glnx-fdio.h:46:11: error: 'basename'
      undeclared (first use in this function)
         return (basename) (path);
      
      Anyway, basename() is like 3 lines of code to
      implement, so this commit just does that instead
      of relying on glibc for it.
      03b48a39
  5. Oct 11, 2017
    • Colin Walters's avatar
      fdio: Avoid ?: syntax for fstatat_allow_noent() · e627524a
      Colin Walters authored
      `g-ir-scanner` is unaware of this GNUC extension and complains.
      Saw that while building ostree.
      
      While we're here, fix up a few other things:
      
       - Tell the compiler the stat buffer is unused (I didn't see
         a warning, just doing this on general principle)
       - Return from `glnx_throw_errno_prefix()` directly; we do
         preserve errno there, let's feel free to rely on it.
      e627524a
  6. Oct 06, 2017
    • Jonathan Lebon's avatar
      fdio: allow NULL for fstatat_allow_noent stbuf · 5362f6bc
      Jonathan Lebon authored
      Often, the caller doesn't actually care about the details of the stat
      struct itself, but just whether the entry exists or not. It does work
      to just pass `NULL` directly to glibc in a quick test, but given that
      the argument is tagged as `__nonnull` and that the documentation does
      not explicitly specify this is supported, let's do this safely.
      5362f6bc
  7. Sep 07, 2017
    • Colin Walters's avatar
      fdio: Add glnx_fstatat_allow_noent() · 627d4e2f
      Colin Walters authored
      This is a very common pattern in both ostree/rpm-ostree. Make a better API for
      this. I thought a lot about simply zeroing out `struct stat` but that feels
      dangerous; none of the values have seem obviously `cannot be zero`.
      627d4e2f
  8. Aug 09, 2017
    • Colin Walters's avatar
      fdio: Merge systemd code to use copy_file_range(), use FICLONE · d18f026e
      Colin Walters authored
      FICLONE is the new alias for the formerly btrfs-specific ioctl; XFS
      has experimental patches to support it.
      
      Further, we should use copy_file_range() for the case where we're only doing a
      limited copy. Both NFS and XFS (with reflink enabled) understand it.
      
      Part of the reason I'm doing this is so that ostree's `/etc` merge will start
      using XFS reflinks. But another major reason is to take the next step after and
      copy this code into GLib as well, so that all of the general GLib users will
      benefit; e.g. Nautilus will transparently do server copy offloads with NFS home
      directories.
      
      See also this coreutils thread about `copy_file_range()`:
      <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24399>. I don't care about file
      holes for our use cases, so it's fine.
      
      Other changes while I'm here:
       - Tweak the sendfile() case to match the newly inlined logic for cfr
       - Add a TEMP_FAILURE_RETRY() around the read()
      d18f026e
  9. Jul 19, 2017
    • Colin Walters's avatar
      fdio: Introduce glnx_openat_read() · 268ae488
      Colin Walters authored
      This is kind of long overdue. Reasons are the same as the other wrappers. I
      debated adding `O_NOFOLLOW` support but the use cases for that are pretty
      obscure, callers who want that can just use the syscall directly for now.
      268ae488
  10. Jul 17, 2017
    • Colin Walters's avatar
      fdio: Add string prefix for glnx_fstat() · 61ef326a
      Colin Walters authored
      For consistency.
      61ef326a
    • Colin Walters's avatar
      fdio: Add a fchmod wrapper · 547bcea2
      Colin Walters authored
      There are a number of versions of this in ostree at least, might as well wrap
      it.
      547bcea2
    • Colin Walters's avatar
      Remove glnx_stream_fstat() · 8b75c8e3
      Colin Walters authored
      There are only two users of this in ostree, and one of them is
      fairly bogus; we can just use `fstat()`.
      8b75c8e3
    • Colin Walters's avatar
      fdio: Add cleanup+flush API for FILE* · e30a773f
      Colin Walters authored
      Mostly in ostree/rpm-ostree, we work in either raw `int fd`, or
      `G{Input,Output}Stream`.  One exception is the rpm-ostree `/etc/passwd`
      handling, which uses `FILE*` since that's what glibc exposes.
      
      And in general, there are use cases for `FILE*`; the raw `GUnixOutputStream` for
      example isn't buffered, and doing so via e.g. `GBufferedOutputStream` means
      allocating *two* GObjects and even worse going through multiple vfuncs for every
      write.
      
      `FILE*` is used heavily in systemd, and provides buffering. It is a bit cheaper
      than gobjects, but has its own trap; by default every operation locks a mutex.
      For more information on that, see `unlocked_stdio(3)`. However, callers can
      avoid that by using e.g. `fwrite_unlocked`, which I plan to do for most users of
      `FILE*` that aren't writing to one of the standard streams like `stdout` etc.
      e30a773f
  11. Jun 28, 2017
  12. Jun 26, 2017
  13. Jun 14, 2017
    • Colin Walters's avatar
      Add G_IN_SET, patch our internal users via spatch · 9a1b77ef
      Colin Walters authored
      I originally tried to get this into GLib:
      https://bugzilla.gnome.org/show_bug.cgi?id=783751
      
      But that looks like it's going to fail due to MSVC. Let's add it here at least
      so I can start using it tomorrow and not wait for the MSVC team to catch up.
      
      I renamed `glnx-alloca.h` to `glnx-macros.h` as a more natural collective
      home for things from systemd's `macro.h`.
      
      Finally, I used a Coccinelle spatch similar to the one referenced
      in the above BZ to patch our uses.
      9a1b77ef
  14. Jun 13, 2017
  15. May 19, 2017
  16. May 15, 2017
    • Colin Walters's avatar
      fdio: Redo tmpfile API with GLnxTmpfile struct · 9929adcd
      Colin Walters authored
      The core problem with the previous tmpfile code
      is we don't have an autocleanup that calls `unlinkat`
      in the non-`O_TMPFILE` case.  And even if we did, it'd
      be awkward still since the `glnx_link_tmpfile_at()` call
      *consumes* the tmpfile.
      
      Fix this by introducing a struct with a cleanup macro. This simplifies a number
      of the callers in libostree - a notable case is where we had two arrays, one of
      fds, one of paths. It makes other places in libostree a bit more complex, but
      that's because some of the commit code paths want to deal with temporary
      *symlinks* too.
      
      Most callers are better though - in libglnx itself, `glnx_file_copy_at()` now
      correctly unlinks on failure for example.
      9929adcd
  17. Apr 28, 2017
    • Colin Walters's avatar
      fdio: Expose glnx_regfile_copy_bytes(), rewrite: GNU style, POSIX errno · 3a4d0f46
      Colin Walters authored
      NOTE: This changes the error handling API of `glnx_loop_write()` to be
      "old school POSIX" instead of "systemd".
      
      In ostree in a few places we use `g_output_stream_splice()`.  I
      thought this would use `splice()`, but actually it doesn't today.
      
      They also, if a cancellable is provided, end up dropping into `poll()` for every
      read and write. (In addition to copying data to/from userspace).
      
      My opinion on this is - for *local files* that's dumb. In the big picture, you
      really only need cancellation when copying gigabytes. Down the line, we could
      perhaps add a `glnx_copy_bytes_cancellable()` that only did that check e.g.
      every gigabyte of copied data. And when we do that we should use
      `g_cancellable_set_error_if_cancelled()` rather than a `poll()` with the regular
      file FD, since regular files are *always* readable and writable.
      
      For my use case with rpm-ostree though, we don't have gigabyte sized files, and
      seeing all of the `poll()` calls in strace is annoying. So let's have the
      non-cancellable file copying API that's modern and uses both reflink and
      `sendfile()` if available, in that order.
      
      My plan at some point once this is tested more is to migrate this code
      into GLib.
      
      Note that in order to keep our APIs consistent, I switched the systemd-imported
      code to "old school POSIX" error conventions. Otherwise we'd have *3* (POSIX,
      systemd, and GError) and particularly given the first two are easily confused,
      it'd be a recipe for bugs.
      3a4d0f46
  18. Apr 21, 2017
  19. Mar 02, 2017
  20. Oct 25, 2016
    • Colin Walters's avatar
      fdio: Make GLnxFileCopyFlags actually flags · 7d2f577d
      Colin Walters authored
      I wanted to add a new one, and realized it was wrong.  Luckily,
      I think we were safe until now, since the set of bits for `(0, 1, 2)`
      is actually distinct.
      
      Although, hm, callers specifying `GLNX_FILE_COPY_OVERWRITE` may
      have not actually been getting that.
      7d2f577d
  21. Jul 01, 2016
    • Colin Walters's avatar
      fdio: Add open_tmpfile_linkable() and link_tmpfile_at() · 113c770d
      Colin Walters authored
      We had a bug previously where we failed to clean up a temporary file
      in an error path.  This is a classic case where the new `O_TMPFILE`
      API in Linux is nicer.
      
      To implement this, as usual we start with some original bits from
      systemd.  But in this case I ended up having to heavily modify it
      because systemd doesn't support "link into place and overwrite".  They
      don't actually use their tempfile code much at all in fact - as far as
      I can tell, just in the coredump code.
      
      Whereas in many apps, ostree included, a very common use case is
      atomically updating an existing file, which is
      `glnx_file_replace_contents_at()`, including subtleties like doing an
      `fdatasync()` if the file already existed.
      
      Implementing this then is slightly weird since we need to link() the
      file into place, then rename() after.
      
      It's still better though because if we e.g. hit `ENOSPC` halfway
      through, we'll clean up the file automatically.
      
      We still do keep the mode where we error out if the file exists.
      Finally, the ostree core though does have a more unusual case where we
      want to ignore EEXIST (allow concurrent object writers), so add
      support for that now.
      
      Note: One really confusing bug I had here was that `O_TMPFILE` ignores
      the provided mode, and this caused ostree to write refs that weren't
      world readable.
      
      Rework things so we always call `fchmod()`, but as a consequence we're
      no longer honoring umask in the default case.  I doubt anyone will
      care, and if they do we should probably fix ostree to consistently use
      a mode inherited from the repo or something.
      113c770d
  22. May 03, 2016
  23. Jan 11, 2016
    • Colin Walters's avatar
      fdio: Export loop_write · 3c470803
      Colin Walters authored
      I plan to use this in rpm-ostree.  Sad how many times this gets
      reinvented.  Should probably stick a copy in `glib-unix.h` or so.
      3c470803
  24. Apr 09, 2015
  25. Apr 07, 2015
  26. Apr 01, 2015
  27. Mar 05, 2015
    • Colin Walters's avatar
      Add glnx_basename() · 175502e5
      Colin Walters authored
      We have to wrap the glibc version to ensure we get the right version,
      otherwise depending on the variance of includes we may end up crashing
      if we get the POSIX version.
      175502e5
  28. Mar 03, 2015
    • Colin Walters's avatar
      fdio: Add glnx_file_copy_at() · 162d1f6b
      Colin Walters authored
      This will allow deleting some code from OSTree for the config file
      merging.  We're reusing some code from systemd, which a nice modern
      clean codebase, and among other things this gets us BTRFS reflinking
      (if available) again.
      162d1f6b
  29. Feb 20, 2015
    • Colin Walters's avatar
      fdio: New APIs to read/write on fds, fd-relative · 1ebfefa5
      Colin Walters authored
      We don't have this really in GLib, unfortunately.  We do want
      GCancellable, but we also want to operate on raw fds where possible.
      
      The "read a file and validate as UTF-8" is a common use case of mine,
      and this combines that with openat().
      1ebfefa5
  30. Feb 15, 2015
  31. Feb 14, 2015
Loading