This is because unifying the display manager, compositor, and window manager allows for each implementation to be far more integrated. For example, this opens up the door to virtual-reality or mixed-reality Wayland display managers, that can run desktop apps without the usual overhead of screen capture. Xorg can't do that. Trying to make Xorg do that will only end in misery.
Being much more integrated isn't actually that valuable. VR compositors already render a full screen frame that goes straight to the display. The window manager for VR is typically in a separate process from the compositor.
You could add Wayland support to an existing VR compositor, but that wouldn't force you to combine the window manager into the same process.
Excuse me, but try to actually use any VR window manager, today, on an OS like Windows. It will bog down and eventually crash your GPU when you open too many windows, because it has to capture those windows from the regular desktop. A Wayland display manager on Linux would not have this issue, because it would already be the window server, so it wouldn't have to perform any extra captures. I only know about this because I have experience trying to do this exact thing.
Wayland would have the same exact problem as you described if it was implemented by copying a window's surface to the surface you are submitting to the compositor.
The solution in both cases is to just submit the original surface to the compositor.
>I only know about this because I have experience trying to do this exact thing.
Did you use Windows Mixed Reality? If you want deep integration with the operating system, you should use the implementation that is part of the operating system.
> Wayland would have the same exact problem as you described if it was implemented by copying a window's surface to the surface you are submitting to the compositor.
Sure, but as far as I can tell, Wayland seems to be the only architecture that would allow you to not do this. Windows does not hand you the original surface. Xorg does not hand you the original surface. But if you are the Wayland display server, you have the original surface because you're the one who created it. So if you start drawing to an HMD, you can place existing windows on it with no extra overhead.
> Did you use Windows Mixed Reality? If you want deep integration with the operating system, you should use the implementation that is part of the operating system.
Yes. I do use Windows Mixed Reality. Which does not support opening desktop windows into the virtual space; it only supports mirroring an entire display onto a surface, and not even capturing individual windows. For that purpose I tried XSOverlay and OVR Toolkit running on SteamVR, which have that capturing issue.
>Wayland seems to be the only architecture that would allow you to not do this
All modern operating systems use this same architecture where apps render into a texture which the compositor can use. Wayland is not unique in having applications just render to a surface which the compositor composites into the final ouput for the display. We have been doing things this way for decades. Saying that xorg cannot do it doesn't mean much because xorgs design is bad and decades behind. This ability has nothing to do with why wayland compoaitors have a wm built in instead being able to easily interface many different ones.
>Which does not support opening desktop windows into the virtual space
Look at this video. He has blender open without mirroring a display.
> All modern operating systems use this same architecture where apps render into a texture which the compositor can use.
I know, but Windows does not allow anyone to just have that texture, AFAIK.
> All modern operating systems use this same architecture where apps render into a texture which the compositor can use.
...I'm going to have to try that. You can even use your mouse with the HMD? I didn't expect that.
Edit: Okay, I've tried it.
- You can open some small portion of your desktop apps in WMR, but nine times out of ten they are just going to ignore WMR, and open on the actual desktop instead, giving you no feedback whatsoever.
- The apps that you can load often don't work properly, for example the File Explorer is a completely different version that doesn't allow you to actually view decently sized thumbnails of anything, nor open them in any sort of photo viewer.
- Once something is on the desktop, it's impossible to get it into WMR. You can only use it through the desktop view. So, apps that can't open into the WMR environment (i.e. Discord) will be impossible to view or interact with from WMR.
- WMR has this delusion that all windows have to be perfectly vertical, so it's impossible to use the headset while laying down, because a mouse cursor doesn't have enough degrees of freedom to communicate that a window should be tilted. (But even using the controllers has this issue.)
But hey, at least it seems not to have that window capturing problem. If you can bear the fact that nothing actually opens, and nothing that does open even works.
