#probing-classifiers

The Illusion of Linearity in High-Dimensional Embeddings

TL;DR: High-dimensional embeddings fail to form linear subspaces for semantic concepts, revealing the limitations of probing classifiers.

High-dimensional embeddings are not the panacea for semantic representation that many claim. Despite the allure of probing classifiers suggesting that semantic concepts align neatly into linear subspaces, the reality is far more complex and less flattering.

The linear representation hypothesis posits that semantic concepts can be captured as linear subspaces within high-dimensional embeddings. However, this assumption crumbles under scrutiny. The rank and spectral properties of weight matrices used in these embeddings reveal a stark truth: linear read-outs often achieve spurious accuracy, not through genuine semantic understanding, but by exploiting dataset artifacts. This is a critical flaw, as it suggests that what we perceive as semantic alignment is often just a mirage created by statistical noise.

Moreover, the curse of dimensionality distorts cosine similarity in these embedding spaces. As dimensionality increases, all points tend to become equidistant, rendering cosine similarity a poor measure of semantic closeness. This phenomenon undermines the very foundation of using high-dimensional spaces for semantic tasks.

To further complicate matters, the Johnson-Lindenstrauss lemma provides a mathematical basis for why dimensionality reduction, often employed to manage these high-dimensional spaces, destroys semantic relationships. By projecting data into lower dimensions, we inadvertently lose the very nuances that are crucial for maintaining semantic integrity.

Key Point One: High-dimensional embeddings fail to form linear subspaces for semantic concepts.

Key Point Two: Linear read-outs achieve spurious accuracy through dataset artifacts, not genuine semantic understanding.

Key Point Three: Dimensionality reduction via Johnson-Lindenstrauss lemma destroys semantic relationships.