Follow-up for Vulkan layer support: figure out how to deal with 32- and 64-bit versions
Part of https://github.com/ValveSoftware/steam-runtime/issues/295, tracked internally as T25064.
The Vulkan layer support in 0.20201124.0 only partially works. The maintainer of vkBasalt reports that it's working for 64-bit but not for 32-bit:
ERROR: /overrides/lib/x86_64-linux-gnu/vulkan_imp_layer/10/steamoverlayvulkanlayer.so: wrong ELF class: ELFCLASS64
DEBUG: Loading layer library /overrides/lib/i386-linux-gnu/vulkan_imp_layer/9/steamoverlayvulkanlayer.so
INFO: Insert instance layer VK_LAYER_VALVE_steam_overlay_32 (/overrides/lib/i386-linux-gnu/vulkan_imp_layer/9/steamoverlayvulkanlayer.so)
ERROR: /overrides/lib/x86_64-linux-gnu/vulkan_imp_layer/8/libVkLayer_steam_fossilize.so: wrong ELF class: ELFCLASS64
DEBUG: Loading layer library /overrides/lib/i386-linux-gnu/vulkan_imp_layer/7/libVkLayer_steam_fossilize.so
INFO: Insert instance layer VK_LAYER_VALVE_steam_fossilize_32 (/overrides/lib/i386-linux-gnu/vulkan_imp_layer/7/libVkLayer_steam_fossilize.so)
ERROR: /overrides/lib/x86_64-linux-gnu/vulkan_imp_layer/6/libbathingshots.so: wrong ELF class: ELFCLASS64
ERROR: /overrides/lib/x86_64-linux-gnu/vulkan_imp_layer/4/libVkLayer_MESA_device_select.so: wrong ELF class: ELFCLASS64
ERROR: /overrides/lib/x86_64-linux-gnu/vulkan_imp_layer/1/libvkbasalt.so: wrong ELF class: ELFCLASS64
One theory for why this happens is that the Vulkan loader deduplicates layers by their name
field, but pressure-vessel is creating two layer JSON manifests: one for x86_64 and one for i386. The Vulkan layer sees that they have the same name
and ignores one of them.
This is not trivial to deal with in general, because we have to put both architectures' versions of the layer at distinct filenames in /overrides
, then write something into the JSON that is valid for both word sizes simultaneously and will make libdl
load the correct layer.
For the special case of layers like vkBasalt where the library_path
is a bare basename to be looked up in the system-wide library search path, we could put the layer library in /overrides/lib/<multiarch>
just like we do for libc.so.6
and libGL.so.1
, and write out a single JSON file shared between both word sizes that uses the basename too, like "library_path": "libvkbasalt.so"
. This is the low-hanging-fruit partial solution to that bug. However, this won't work for MangoHud, which is known not to be set up like this.
For layers more generally, and in particular MangoHud (which uses $LIB
), this is going to be a pain. We can't put "library_path": "/overrides/$LIB/foo/bar.so"
, because we can't know that $LIB
will expand to the rigth thing. We might either be using the Steam Runtime's glibc (which is Debian-style multiarch: $LIB
is lib/x86_64-linux-gnu
or lib/i386-linux-gnu
), or the host system's glibc (which could be using Debian-style multiarch, or could be using lib64
and lib
, or could be using lib
and lib32
, or could be something arbitrarily different).
We could try using the same hack that we do for VDPAU: use ${PLATFORM}
, and have symlinks to account for the fact that ${PLATFORM}
for 32-bit is variously i686
, i586
, i486
or i386
. I'm reluctant to let this hack spread further than it has to, because it's pretty horrible, and I'm concerned that when we start getting newer "functionality levels" in x86_64, we might start seeing ${PLATFORM}
expand to x86_64-2022
or ssse4
or something - but it's doable for now. If we do this, we should probably stop pretending it has anything to do with multiarch tuples, and use paths like /overrides/platform-${PLATFORM}/vulkan-layers/1/libvkbasalt.so
.
A second approach would be to change the layer's name
, which is not allowed for explicit layers, but might be OK for implicit layers.
A third, longer-term approach would be to get the Vulkan-Loader changed so that it does not deduplicate in this way, and then backport that version into the runtime. We will use ours if it appears to be newer than the host version, so it is OK to include and rely on a newer upstream release.
/cc @denittis