What Do Tabular Foundation Models Compute In Context? In-Situ Representation Refinement through Attention-Gated Updates

תקציר מקורי באנגליתarXiv:2609.27679v2 Announce Type: replace Abstract: A tabular foundation model must discover which distinctions matter for each new table without updating its parameters. We develop in-situ representation refinement: support labels guide changes to the episode's representations, improving the information available to later queries. A regularized leave-one-out objective yields a support correction and its query extension. The leading term separates attention-based reading from state-dependent scaling, motivating RefineICL: an attention-gated, FFN-free contextual stack with selected low-rank feature interaction and typed memory. A direct intervention tests the role of evolving support states: removing one intermediate support update while preserving the block's query output increases final q