I've largely come to the conclusion that 1) X got as bad as it did not for technical reasons but because of designers stuck in the past (see below), 2) Wayland inherited many of the same architecture astronaut attitudes.
You see this if you look at e.g. XRender. XRender was a big improvement for X, by providing a set of rendering primitives. Yet at the same time they repeated mistake after mistake from the core protocol. E.g. the flexibility in visuals that mattered when we did have monochrome, 8-bit palettes, 15-bit direct color, 16-bit direct color and 24-bit truecolor graphics output in the same computer lab no longer matters when you can assume that every server can provide RGB24 and RGBA32 visuals, not least because XRender demands this so if a server wants to support XRender, it must support RGB24 and RGBA32 whether or not that's a natural fit for the backend or not.
Yet it still makes you iterate through a list of every weird format known to man (and some that feel like they must be alien) even when you want to look up one of the 5 standard formats Xrender dictates that every server must support. Xrender could have just made the relevant request return 5 ints.
It's not hard to deal with the current solution, but it's pointless and it makes the server unnecessarily complicated to still support all of those extra visuals that almost nobody will be using, and nobody needs to use.
If that was all, it wouldn't be so bad, but this is typical for XRender, and this is why X became unmanageable: They both left the core protocol intact, but each extension they added was just as bloated as the core was.
Nothing would stop you from fixing this by bumping a version number and start deprecating requests and introducing replacements. But that only works if you can get buy-in into changes that actually simplifies things, and the XRender extension is a good indication that this was the real problem for X: You'd get buy-in for changes, but changes that only ever added complexity.
This was particularly idiotic because Xorg came to pretty much control this space: Almost everyone used the same X server and almost everyone used the same Xlib. They could have bumped the protocol version and started deprecating old functionality whenever they wanted while optionally papering it over in the client lib to give people time to update their clients.
So Wayland became a way of escaping that and starting over. And it could have been great if they weren't so afraid of repeating the same complexity mistakes that they repeated the same complexity mistake by proxy by forcing even basic functionality into extensions where there was no ability to reign it in. The Wayland design reliance on extensions made an explosion in complexity virtually guaranteed.
I wish they'd have forked Xorg instead, and had the guts to gut it and clean it up: Bump the major version to 12. Set out an aggressive deprecation schedule and ensure a sufficient set of apps would keep working with various calls and visuals yanked out. Optionally add a shim to xlib/xcb to temporarily handle some of the deprecated functionality client side (e.g. most old rendering calls and bitmap font handling could easily be transitioned this way). Then you'd have a base to iterate from where you could iterate step by step from a working starting point. Then maybe we'd all be on something new and working instead of having this ongoing schism that won't be resolved anytime soon.
They could not drop core functionality from X11; somewhere, there is a client that expects it and dropping it would break it.
That's exactly the reason why everything is an extension is Wayland: so the client is forced to enumerate, what features are available and adjust its behavior when something is not. It is exactly this, that allows Wayland to drop features, once they became obsolete. It is the lesson from not being able to drop core X11 primitives.
> They could not drop core functionality from X11; somewhere, there is a client that expects it and dropping it would break it.
Of course they can. Dropping it from new versions won't wipe old servers off the planet, and as we can see from the existence of Wayland, the OSS world is very much prepared ditch far more backwards compatibility than just breaking an occasional obsolete client.
If anything, it's a far easier transition because shimming some Xlib calls is far easier than migrating applications to a whole different display model.
> That's exactly the reason why everything is an extension is Wayland: so the client is forced to enumerate, what features are available and adjust its behavior when something is not. It is exactly this, that allows Wayland to drop features, once they became obsolete. It is the lesson from not being able to drop core X11 primitives.
That doesn't work that way once something becomes reasonably ubiquitous. Experience from X is that people will expect a given extension to be there (heck, most of my own X software will categorically not work on an X server without XRender, so if that extension is ever dropped from X, my software will stop working and need to be updated - that's a tradeoff I'm perfectly fine with), not have older alternatives that make sense, and obviously not be able to predict future alternatives that do not yet exist.
In other words: You'll still need to either support those APIs, or deprecate them and update the clients. Doesn't matter if they're part of the core or extensions.
You see this if you look at e.g. XRender. XRender was a big improvement for X, by providing a set of rendering primitives. Yet at the same time they repeated mistake after mistake from the core protocol. E.g. the flexibility in visuals that mattered when we did have monochrome, 8-bit palettes, 15-bit direct color, 16-bit direct color and 24-bit truecolor graphics output in the same computer lab no longer matters when you can assume that every server can provide RGB24 and RGBA32 visuals, not least because XRender demands this so if a server wants to support XRender, it must support RGB24 and RGBA32 whether or not that's a natural fit for the backend or not.
Yet it still makes you iterate through a list of every weird format known to man (and some that feel like they must be alien) even when you want to look up one of the 5 standard formats Xrender dictates that every server must support. Xrender could have just made the relevant request return 5 ints.
It's not hard to deal with the current solution, but it's pointless and it makes the server unnecessarily complicated to still support all of those extra visuals that almost nobody will be using, and nobody needs to use.
If that was all, it wouldn't be so bad, but this is typical for XRender, and this is why X became unmanageable: They both left the core protocol intact, but each extension they added was just as bloated as the core was.
Nothing would stop you from fixing this by bumping a version number and start deprecating requests and introducing replacements. But that only works if you can get buy-in into changes that actually simplifies things, and the XRender extension is a good indication that this was the real problem for X: You'd get buy-in for changes, but changes that only ever added complexity.
This was particularly idiotic because Xorg came to pretty much control this space: Almost everyone used the same X server and almost everyone used the same Xlib. They could have bumped the protocol version and started deprecating old functionality whenever they wanted while optionally papering it over in the client lib to give people time to update their clients.
So Wayland became a way of escaping that and starting over. And it could have been great if they weren't so afraid of repeating the same complexity mistakes that they repeated the same complexity mistake by proxy by forcing even basic functionality into extensions where there was no ability to reign it in. The Wayland design reliance on extensions made an explosion in complexity virtually guaranteed.
I wish they'd have forked Xorg instead, and had the guts to gut it and clean it up: Bump the major version to 12. Set out an aggressive deprecation schedule and ensure a sufficient set of apps would keep working with various calls and visuals yanked out. Optionally add a shim to xlib/xcb to temporarily handle some of the deprecated functionality client side (e.g. most old rendering calls and bitmap font handling could easily be transitioned this way). Then you'd have a base to iterate from where you could iterate step by step from a working starting point. Then maybe we'd all be on something new and working instead of having this ongoing schism that won't be resolved anytime soon.