I think of the foundational model like CPUs. They're the core of powerful, general-purpose computers, and will likely remain popular and common for most computing solutions. But we also have GPUs, microcontrollers, FPGAs, etc. that don't just act as the core of a wide variety of solutions, but are also paired alongside CPUs for specific use cases that need specialization.

Foundational models are not great for many specific tasks. Assuming that one architecture will eventually work for everything is like saying that x86/amd64/ARM will be all we ever need for processors.