XR glasses
XR glasses 4.0
I own a pair of XR glasses for a while now. Initially I got myself the Viture Pro for productivity work and maybe watch some movies. This was before finding out about their Side-By-Side (SBS) mode where each eye is fed with a slightly different camera position resulting in 3D depth perception similar to VR. Unlike VR I don’t get motion sick from these though, because I can always see a part of the real world.
The glasses also allow access to the head tracking data which make them a great addition to my home cockpit.
Breezy
For daily work or movies I often make use of the Breezy GNOME project. This is entirely optional though, since the XR glasses are “just” another display for the PC. The only requirements are USB-C + DP ALT allowing to get a DisplayPort signal over USB-C. Not many desktop PCs offer this already, but there are various HDMI adapters and laptops have this feature often already. The Steam Deck has it built in and works out of the box. Here does the Breezy Vulkan project all the heavy lifting that I do manually on my desktop PC.
It’s a great project that I can totally recommend - please do support it!
Side-By-Side with ReShade
I love to play many games with my XR glasses in Side-By-Side mode where each eye is fed with a slightly different camera position resulting in 3D depth perception.
This is similar to VR but does not offer e.g. a backchannel for head tracking. Some games, like Elite Dangerous, support this natively. Other games can be forced into a SBS like mode with ReShade and a plugin like SuperDepth3D or Rendepth Reshade. In theory are Reshade shaders compatible with vkBasalt but the depth stuff is apparently exempt from this rule so that is not an option. The approach works nicely with Proton though.
The reshade-linux repo is very useful to get you started but the required steps can be done manually too, of course.
You will need gamescope on top though, because the output has usually to be rescaled or the display ratio is completely off. This also depends on the glasses. Mine do FULL SBS so a resolution of 3840x1080 is excepted in the end.
For Elite Dangerous the correct settings would e.g. be:
gamescope -h 2160 -w 3840 -H 1080 -W 3840 –scaler stretch -e – %command%
Other games may require slightly different settings here. This is an example for Ace Combat 7: Skies Unknown with Reshade and the SuperDepth3D shader:
WINEDLLOVERRIDES=“d3dcompiler_47=n;dxgi=n,b;” gamescope -h 1080 -w 1920 -H 1080 -W 3840 –scaler stretch -b -r 60 -e – %command%
YMMV but the general idea should work with almost any game.
Head tracking
XR doesn’t offer a built in way for head tracking, like VR does. We can however access their IMU data in most cases and from here it’s some “simple” Euler math to convert this into a format OpenTrack understands. And since OpenTrack accepts UDP input we don’t even need a fancy plugin - just a “bridge” tool, which can be written in basically anything. I wrote a proof of concept in PHP reading the shared memory data from the Breezy xr-driver on Linux - so that works in theory with any glasses fully(!) supported by the Breezy project
The idea was picked up by someone else and resulted in a standalone version written in Rust that can also access the IMU directly but is restricted to the Viture glasses [for now]. And since such glasses experience naturally some drift it even features a quick way to reset the tracking via cli command, that can be bound e.g. to a joystick button with AntiMicroX.
Both projects are available on GitHub.
Demo
Demo of Breezy Desktop on GNOME with Viture Pro XR glasses. Breezy has now support for virtual displays on the XR desktop. It’s basically only limited by system resources but who is counting at this point. The most taxing job is the recording (beside gaming, of course) anyway. It’s perfect for me when e.g. modding a game!
Watch this video on YouTube PeerTube