The embedding revolution has transformed how we represent meaning in machines. But for all their power, embeddings have a fundamental limitation: they collapse relationships into distance.
What Embeddings Miss
Consider the relationship between 'doctor' and 'patient.' In embedding space, these concepts are close—they co-occur frequently. But the nature of their relationship—who treats whom, who depends on whom—is lost. This matters when we need to reason, not just retrieve.
