Mixed Materials
On Admissible Rank-based Input Normalization Operators
| Title: | On Admissible Rank-based Input Normalization Operators |
|---|---|
| Authors: | Kim, Taeyun |
| Publisher Information: | 2025-12-27 |
| Document Type: | Mixed materials |
| Abstract: | Rank-based input normalization is a workhorse of modern machine learning, prized for its robustness to scale, monotone transformations, and batch-to-batch variation. In many real systems, the ordering of feature values matters far more than their raw magnitudes - yet the structural conditions that a rank-based normalization operator must satisfy to remain stable under these invariances have never been formally pinned down. We show that widely used differentiable sorting and ranking operators fundamentally fail these criteria. Because they rely on value gaps and batch-level pairwise interactions, they are intrinsically unstable under strictly monotone transformations, shifts in mini-batch composition, and even tiny input perturbations. Crucially, these failures stem from the operators' structural design, not from incidental implementation choices. To address this, we propose three axioms that formalize the minimal invariance and stability properties required of rank-based input normalization. We prove that any operator satisfying these axioms must factor into (i) a feature-wise rank representation and (ii) a scalarization map that is both monotone and Lipschitz-continuous. We then construct a minimal operator that meets these criteria and empirically show that the resulting constraints are non-trivial in realistic setups. Together, our results sharply delineate the design space of valid rank-based normalization operators and formally separate them from existing continuous-relaxation-based sorting methods. 31 pages, 2 figures |
| Index Terms: | Machine Learning, 68Q32 (Primary) 68T05 (Secondary), text |
| URL: | |
| Availability: | Open access content. Open access content |
| Other Numbers: | COO oai:arXiv.org:2512.22587 1577672055 |
| Contributing Source: | CORNELL UNIV From OAIster®, provided by the OCLC Cooperative. |
| Accession Number: | edsoai.on1577672055 |
| Database: | OAIster |
| Description not available. |