What You See Is What You Get: Observation-Aligned Supervision for Chart-to-Code Generation

תקציר מקורי באנגליתarXiv:2607.04726v2 Announce Type: replace-cross Abstract: Chart-to-code generation is commonly trained with supervised fine-tuning on reference plotting scripts, implicitly treating the gold code as a fully observable target. We argue that this assumption is often invalid: many chart programs contain latent raw variables that cannot be uniquely recovered from the rendered image. We identify this systematic latent--observation mismatch with three forms: aggregation-induced mismatch, where raw samples are reduced to summary statistics or bin-level mass; normalization-induced mismatch, where absolute scale is removed; and projection-induced mismatch, where higher-dimensional information is lost through visual projection. These mismatches introduce target ambiguity and require models to comple