I would have never considered it but several people thought: why not train our diffusion models on mappings between latent spaces themselves instead of on say, raw data like pixels?

It's a palm-to-face moment because of how obvious it is in hindsight.

Details in the following link (or just google 'latent diffusion models')

https://huggingface.co/docs/...