- Jun 03, 2020
-
-
Simon McVittie authored
We don't modify this struct (if non-NULL), so it can be const. In particular, this is helpful if calling glnx_file_copy_at() from nftw() to implement the equivalent of `cp -a --reflink=auto`. Signed-off-by:
Simon McVittie <smcv@collabora.com>
-
- Nov 11, 2019
-
-
Simon McVittie authored
This will be necessary if targeting GLib versions older than 2.34, such as GLib 2.32 in Ubuntu 12.04 and the Steam Runtime. Signed-off-by:
Simon McVittie <smcv@collabora.com>
-
- Nov 04, 2019
-
-
Colin Walters authored
For fs-verity.
-
- Oct 09, 2019
-
-
Alexander Larsson authored
This is useful if you need the file to be on a particular filesystem. In particular, flatpak wants this to make tempfiles on /tmp for things we need to write during flatpak run, such as the libseccomp output fd. We've had "flatpak run" stop working in low disk situations without this, so its nice to be able to fix it.
-
- Nov 30, 2018
-
-
Owen W. Taylor authored
The docs for `glnx_file_replace_contents[_with_perms]` say that the default mode is 0666 - umask, but it's actually 0644. Because there's no thread-safe way of finding out the current umask without grubbing around in /proc/self/status, simply make the docs reflect reality.
-
- Oct 24, 2017
-
-
Colin Walters authored
Use decl-after-stmt where applicable.
-
- Oct 16, 2017
-
-
Colin Walters authored
Port to `glnx_autofd` tree wide, and add one missed `glnx_close_fd()` use in the tmpfile code.
-
- Oct 05, 2017
-
-
Colin Walters authored
Oops.
-
- Oct 02, 2017
-
-
Colin Walters authored
I was using this in rpm-ostree and glanced at the code. This was clearly the intent, but isn't a full fix. See code comments for more details.
-
- Sep 25, 2017
-
-
Colin Walters authored
Particularly if `AT_FDCWD` is used, we need to open in the target dir, otherwise we can get `EXDEV` when trying to do the final link. (Theoretically we can cross a mountpoint even with fd-relative though this is a lot less likely)
-
- Sep 13, 2017
-
-
Colin Walters authored
Followup to similar commits in the ostree stack recently.
-
- Sep 12, 2017
-
-
Colin Walters authored
I was working on rpm-ostree unified core, and hit the fact that `glnx_file_copy_at()` had the same bug with `fsetxattr()` and files whose mode is <= `0400` (e.g. `000` in the case of `/etc/shadow`) that libostree did a while ago. Basically, Linux currently allows `write()` on non-writable open files but not `fsetxattr()`. This situation is masked for privileged (i.e. `CAP_DAC_OVERRIDE`) code. Looking at this, I think it's cleaner to convert to `O_TMPFILE` here, since that code already handles setting the tmpfile to mode `0600`. Now, this *is* a behavior change in the corner case of existing files which are symbolic links. Previously we'd do an `open(O_TRUNC)` which would follow the link. But in the big picture, I think the use cases for `open(O_TRUNC)` are really rare - I audited all callers of this in ostree/rpm-ostree/flatpak, and all of them will be fine with this behavior change. For example, the ostree `/etc` merge code already explicitly unlinks the target beforehand. Other cases like supporting `repo/pubring.gpg` in an ostree repo being a symlink...eh, just no. Making this change allows us to convert to new style, and brings all of the general benefits of using `O_TMPFILE` too.
-
Colin Walters authored
While reading a strace I noticed a double close in the tests; this was because we were missing an assignment to `-1` in the tests. However, let's make supporting this clearer by explicitly supporting the fd being `-1` while still setting the `initialized` variable to `FALSE`. We also add the `EBADF` assertion checking.
-
Colin Walters authored
I noticed while reading the manpage for `linkat()` that `O_TMPFILE` supports `O_EXCL` to mean exactly what we're doing with the anonymous tmpfile API. Change the code to start using it; this required refactoring the internals since we had a check to be sure the caller wasn't passing `O_EXCL` for the non-anonymous path which we want to keep. Presumably the storage system could do smarter things if it knows a file will always be anonymous, e.g. it doesn't need to journal its data.
- Aug 09, 2017
-
-
Colin Walters authored
FICLONE is the new alias for the formerly btrfs-specific ioctl; XFS has experimental patches to support it. Further, we should use copy_file_range() for the case where we're only doing a limited copy. Both NFS and XFS (with reflink enabled) understand it. Part of the reason I'm doing this is so that ostree's `/etc` merge will start using XFS reflinks. But another major reason is to take the next step after and copy this code into GLib as well, so that all of the general GLib users will benefit; e.g. Nautilus will transparently do server copy offloads with NFS home directories. See also this coreutils thread about `copy_file_range()`: <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24399>. I don't care about file holes for our use cases, so it's fine. Other changes while I'm here: - Tweak the sendfile() case to match the newly inlined logic for cfr - Add a TEMP_FAILURE_RETRY() around the read()
-
- Jul 19, 2017
-
-
Colin Walters authored
This is kind of long overdue. Reasons are the same as the other wrappers. I debated adding `O_NOFOLLOW` support but the use cases for that are pretty obscure, callers who want that can just use the syscall directly for now.
-
- Jul 17, 2017
-
-
Colin Walters authored
There are only two users of this in ostree, and one of them is fairly bogus; we can just use `fstat()`.
-
Colin Walters authored
Mostly in ostree/rpm-ostree, we work in either raw `int fd`, or `G{Input,Output}Stream`. One exception is the rpm-ostree `/etc/passwd` handling, which uses `FILE*` since that's what glibc exposes. And in general, there are use cases for `FILE*`; the raw `GUnixOutputStream` for example isn't buffered, and doing so via e.g. `GBufferedOutputStream` means allocating *two* GObjects and even worse going through multiple vfuncs for every write. `FILE*` is used heavily in systemd, and provides buffering. It is a bit cheaper than gobjects, but has its own trap; by default every operation locks a mutex. For more information on that, see `unlocked_stdio(3)`. However, callers can avoid that by using e.g. `fwrite_unlocked`, which I plan to do for most users of `FILE*` that aren't writing to one of the standard streams like `stdout` etc.
-
- Jul 10, 2017
-
-
Colin Walters authored
Work around an older glibc bug.
-
- Jun 28, 2017
-
-
Colin Walters authored
There was a user of this in the libostree static delta code.
-
Colin Walters authored
I'm not aware of a problem in practice here, but we should do this on general principle. Writing this patch now because I hit a fd leak in the ostree static delta processing that was introduced in the tmpfile prep code, but fixed in the final port.
-
- Jun 26, 2017
-
-
Colin Walters authored
Besides doing `TEMP_FAILURE_RETRY` and `GError` conversion, these also prefix the error with arguments.
-
- Jun 14, 2017
-
-
Colin Walters authored
I originally tried to get this into GLib: https://bugzilla.gnome.org/show_bug.cgi?id=783751 But that looks like it's going to fail due to MSVC. Let's add it here at least so I can start using it tomorrow and not wait for the MSVC team to catch up. I renamed `glnx-alloca.h` to `glnx-macros.h` as a more natural collective home for things from systemd's `macro.h`. Finally, I used a Coccinelle spatch similar to the one referenced in the above BZ to patch our uses.
-
- Jun 13, 2017
-
-
Colin Walters authored
The glibc `posix_fallocate()` implementation has a bad fallback, and further we need to handle `EOPNOTSUPP` for musl. https://github.com/flatpak/flatpak/issues/802
-
- May 19, 2017
-
-
Alexander Larsson authored
Add an `initialized` member which means we work by default in structs allocated with `g_new0` etc. and don't need a special initializer. This also fixes a bug where we need to support `src_dfd == -1` or `AT_FDCWD`. This fixes flatpak which uses AT_FDCWD. Modified-by:
Colin Walters <walters@verbum.org>
-
- May 17, 2017
-
-
Colin Walters authored
Just noticed this while reading the code.
-
- May 15, 2017
-
-
Colin Walters authored
The core problem with the previous tmpfile code is we don't have an autocleanup that calls `unlinkat` in the non-`O_TMPFILE` case. And even if we did, it'd be awkward still since the `glnx_link_tmpfile_at()` call *consumes* the tmpfile. Fix this by introducing a struct with a cleanup macro. This simplifies a number of the callers in libostree - a notable case is where we had two arrays, one of fds, one of paths. It makes other places in libostree a bit more complex, but that's because some of the commit code paths want to deal with temporary *symlinks* too. Most callers are better though - in libglnx itself, `glnx_file_copy_at()` now correctly unlinks on failure for example.
-
- Apr 28, 2017
-
-
Colin Walters authored
NOTE: This changes the error handling API of `glnx_loop_write()` to be "old school POSIX" instead of "systemd". In ostree in a few places we use `g_output_stream_splice()`. I thought this would use `splice()`, but actually it doesn't today. They also, if a cancellable is provided, end up dropping into `poll()` for every read and write. (In addition to copying data to/from userspace). My opinion on this is - for *local files* that's dumb. In the big picture, you really only need cancellation when copying gigabytes. Down the line, we could perhaps add a `glnx_copy_bytes_cancellable()` that only did that check e.g. every gigabyte of copied data. And when we do that we should use `g_cancellable_set_error_if_cancelled()` rather than a `poll()` with the regular file FD, since regular files are *always* readable and writable. For my use case with rpm-ostree though, we don't have gigabyte sized files, and seeing all of the `poll()` calls in strace is annoying. So let's have the non-cancellable file copying API that's modern and uses both reflink and `sendfile()` if available, in that order. My plan at some point once this is tested more is to migrate this code into GLib. Note that in order to keep our APIs consistent, I switched the systemd-imported code to "old school POSIX" error conventions. Otherwise we'd have *3* (POSIX, systemd, and GError) and particularly given the first two are easily confused, it'd be a recipe for bugs.
-
- Apr 25, 2017
-
-
Colin Walters authored
There's one function that did `unlinkat()` in the cleanup section, not doing that yet. Note I uncovered a few bugs in a few places where we didn't preserve errno before doing an `unlinkat()` in error paths in a few cases. I also tried to prefix a few more error cases with the system call name.
-
- Mar 23, 2017
-
-
Jonathan Lebon authored
These are equivalent to the non-null throw, except that the returned value is a NULL pointer. They can be used in functions where one wants to return a pointer. E.g.: GKeyFile *foo(GError **error) { return glnx_null_throw (error, "foobar"); } The function call redirections are wrapped around a compound statement expression[1] so that they represent a single top-level expression. This allows us to avoid -Wunused-value warnings vs using a comma operator if the return value isn't used. I made the 'args...' absorb the fmt argument as well so that callers can still use it without always having to specify at least one additional variadic argument. I had to check to be sure that the expansion is all done by the preprocessor, so we don't need to worry about stack intricacies. [1] https://gcc.gnu.org/onlinedocs/gcc/Statement-Exprs.html
-
- Mar 02, 2017
-
-
Colin Walters authored
I want the `RENAME_EXCHANGE` version for rpm-ostree, to atomically swap `/usr/share/rpm` (a directory) with a new verison. While we're here we might as well expose `RENAME_NOREPLACE` in case something else wants it. These both have fallbacks to the non-atomic version. Closes: https://github.com/GNOME/libglnx/pull/36
-
- Oct 05, 2016
-
-
Colin Walters authored
See https://mail.gnome.org/archives/ostree-list/2016-October/msg00003.html Basically https://github.com/wrpseudo/pseudo doesn't implement newer APIs like renameat2() and O_TMPFILE, so on the host side (as potentially opposed to the target system) we want to be able to disable them.
-
- Aug 04, 2016
-
-
Colin Walters authored
In some cases we want to replace with zero size, and `posix_fallocate()` is documented to return `EINVAL` in this case. Making this change since I noticed it elsewhere.
-
- Aug 03, 2016
-
-
Colin Walters authored
Some systems have bugs with it, so let's allow downstreams to easily disable it. https://bugzilla.gnome.org/show_bug.cgi?id=769453 https://github.com/ostreedev/ostree/issues/421
-
- Jul 08, 2016
-
-
Colin Walters authored
This is kind of an ABI change but it's for the better I think; on error we consistently clean up the temp file. This is obviously necessary without `O_TMPFILE`. With it, we still need an error cleanup in the case where we're trying to replace an existing file. I noticed this in ostree's `tests/test-refs.sh` which intentionally tries to rename a file over a directory path.
-
Colin Walters authored
While auditing this code to figure out why ostree's `tests/test-refs.sh` was failing, while the bug turned out to be different, I noticed that in the case where `dfd != target_dfd`, we failed to do the right `renameat()`. (No code I'm aware of does this now).
-
- Jul 01, 2016
-
-
Colin Walters authored
We had a bug previously where we failed to clean up a temporary file in an error path. This is a classic case where the new `O_TMPFILE` API in Linux is nicer. To implement this, as usual we start with some original bits from systemd. But in this case I ended up having to heavily modify it because systemd doesn't support "link into place and overwrite". They don't actually use their tempfile code much at all in fact - as far as I can tell, just in the coredump code. Whereas in many apps, ostree included, a very common use case is atomically updating an existing file, which is `glnx_file_replace_contents_at()`, including subtleties like doing an `fdatasync()` if the file already existed. Implementing this then is slightly weird since we need to link() the file into place, then rename() after. It's still better though because if we e.g. hit `ENOSPC` halfway through, we'll clean up the file automatically. We still do keep the mode where we error out if the file exists. Finally, the ostree core though does have a more unusual case where we want to ignore EEXIST (allow concurrent object writers), so add support for that now. Note: One really confusing bug I had here was that `O_TMPFILE` ignores the provided mode, and this caused ostree to write refs that weren't world readable. Rework things so we always call `fchmod()`, but as a consequence we're no longer honoring umask in the default case. I doubt anyone will care, and if they do we should probably fix ostree to consistently use a mode inherited from the repo or something.
-
- Jun 16, 2016
-
-
Yu Qi Zhang authored
We noticed the temp files being left over in ostree when (mistakenly) trying to replace the contents of a subpath that wasn't a directory. In the future we should look at the systemd code using `O_TMPFILE` here.
-
- May 03, 2016
-
-
Colin Walters authored
Migrated from libgsystem's `gs_stream_fstat()`. It's a small function but I end up using it in OSTree a fair bit.
-