The Two Decisions
A label space answers two questions at once, and every routine comparison answers them together.
Where the points are, and who sits where
Write the label geometry as a matrix. Instead of a one-hot vector, the target for class is the -th row of some , with free. One-hot is the case . The object a distance-based decoder actually consumes, the prototype classifier of Snell et al. (2017) being the canonical one, is the centred Gram matrix,
because every score it computes is a squared distance, and squared distances are inner products in disguise.
Inside that one matrix there are two independent decisions.
The arrangement is the point cloud stripped of its names: the multiset of rows of , the shape, the constellation. It says how far apart things are without saying what any of them is. This is a labelled point configuration in the sense of Kendall (1984), whose shape theory studies exactly what survives after quotienting out location, scale and rotation.
The assignment is the bijection from labels onto the points of that cloud: which class gets which position. It says what sits where without changing any distance in the configuration.
The distinction is exactly the action of the symmetric group. Relabelling by a permutation sends
which moves the assignment and leaves the arrangement untouched. The orbit of under is the arrangement; the position within the orbit is the assignment. Every geometry decomposes this way, and the two pieces are logically independent: you can hold one fixed and vary the other.
One-hot is the case where the second decision does not exist
Return to the centred one-hot Gram matrix from the previous article. Up to scale it is
and commutes with every permutation matrix, so
The orbit is a single point. Relabelling the simplex gives back the simplex, exactly, every time.
That is the structural fact that makes one-hot peculiar rather than merely conventional. Under the default design, the assignment decision carries no information, because there is nothing to decide. All ways of hanging the class names on the vertices produce the same geometry. Nothing in the target expresses a belief about which confusions are cheap, because the target is invariant to every possible such belief.
For any other geometry, the fibre of an arrangement, its set of distinct labellings, has up to elements. At that is roughly . The default design collapses all of them into one.
Three literatures already separate them
The distinction is not novel, and it is worth naming who has used it and how.
| Work | What it does with the split |
|---|---|
| Dietterich & Bakiri (1995) | Permute codeword-to-class assignments as a null control, and report that performance does not change |
| Evron et al. (2023) | Vary the assignment at a fixed arrangement, and find empirically that it does matter |
| Saadabadi et al. (2024) | Split classification into a prototype arrangement plus a dynamic label-to-prototype assignment, and optimise the assignment |
Three uses: as a control, as an empirical effect, as a design variable. Note that the first two disagree, which is itself informative. The answer plausibly depends on the arrangement, the task, and the decoder, and none of the three puts the two contributions on a common scale.
Why the split is hard to use in practice
Here is the obstacle. Every practical comparison between label spaces moves both decisions simultaneously.
Swap a one-hot target for class-name embeddings, and you have changed the shape of the cloud and the pairing of names to positions. Compare the resulting accuracies and you get one number attributable to the pair. Nothing in the comparison tells you whether the gain came from the fact that the classes are now at unequal distances, or from the fact that the particular classes the encoder considers similar are the ones now placed together.
The two readings have opposite implications for what to do next:
- If the gain is in the arrangement, then what mattered was the shape, how spread out the classes are and in how many effective dimensions. Any reasonably shaped geometry of the same scale would have done comparably.
- If the gain is in the assignment, then what mattered was semantic: the map knew which classes confuse, and a shape-matched geometry with the names shuffled would have lost the benefit.
The one family of methods that isolates the assignment cleanly is random relabelling: draw each example’s label at random, keep everything else. But that destroys the arrangement’s relationship to the data entirely; there is no metric structure left to preserve, so it answers a different question.
What this asks for
To attribute a gain to one of the two decisions, you need a comparison that holds the other fixed. That is what a null model is: a distribution that keeps some declared statistic and draws the rest. Permuting labels at a fixed cloud holds the arrangement and randomises the assignment. Drawing a fresh cloud entirely holds nothing but the size of the problem.
Two prerequisites follow. A comparison needs a scale with a declared zero, or the differences it produces do not add up and do not travel between tasks. And it needs a stated null, because the answer depends entirely on what you agreed to concede for free. Those are the next two articles.
References
- Dietterich, T. G., & Bakiri, G. (1995). Solving Multiclass Learning Problems via Error-Correcting Output Codes. JAIR, 2, 263–286.
- Evron, I., Onn, O., Weiss Orzech, T., Azeroual, H., & Soudry, D. (2023). The Role of Codeword-to-Class Assignments in Error-Correcting Codes: An Empirical Study. AISTATS, PMLR 206, 8053–8077.
- Kendall, D. G. (1984). Shape Manifolds, Procrustean Metrics, and Complex Projective Spaces. Bulletin of the LMS, 16(2), 81–121.
- Saadabadi, M. S. E., Dabouei, A., Rahimi Malakshan, S., & Nasrabadi, N. M. (2024). Hyperspherical Classification with Dynamic Label-to-Prototype Assignment. CVPR.
- Snell, J., Swersky, K., & Zemel, R. (2017). Prototypical Networks for Few-shot Learning. NeurIPS.