The Illusion of Replacement: Rethinking Specialized Machine Learning Models in the Foundation Model Era

תקציר מקורי באנגליתarXiv:2608.28980v3 Announce Type: replace-cross Abstract: Specialized machine learning architectures encode structural assumptions (equivariance, permutation invariance, relational structure) that language-based foundation models lack by design. This review asks whether such assumptions can instead be acquired through language, drawing on a corpus of 186 papers published between 2016 and 2026 across nine modalities: tabular data, graphs, time series, vision, chemistry, code, knowledge graphs, point clouds, and protein structure. Methods are organized into eight representational regimes, ranging from language-only prompting to fully specialized architectures, and are assessed not only on predictive accuracy but on whether structural information is preserved by the representation and compute