It's really strange to me that display servers/window managers (Xorg, Wayland, etc) aren't locked to given desktop environments (GNOME, KDE, Unity, Cinnamon, etc). It doesn't make sense to me that they are separate and not optimized together.

I might be mistaken, but I think the desktop environment (DE) talks to the window compositor. All window compositors are built to an standard, either the X standard or the Wayland standard, and then there's Ubuntu Mir. So, the window compositors can have X and Wayland implementations. So you might have Metacity (Gnome's window compositor) for X and another for Wayland.

In open systems complying to standards, you can typically swap out different pieces without having to worry about optimization. X and Wayland run equally well (drivers aside) on Linux, FreeBSD, and many other operating systems that employ the POSIX standard. Uaing the standard means that for little to no extra work Gnome and KDE can reach many more operating systems.