Evaluating Neural Decompilation of Dart AOT Binaries: Fine-Tuning, Metric Validity, Specification Leakage, and Reliability

תקציר מקורי באנגליתarXiv:2607.06125v2 Announce Type: replace-cross Abstract: We present an execution-based evaluation of neural decompilation for Dart ahead-of-time binaries and an audit of what its scores measure. Across six archived adapter-baseline comparisons, paired tests of pass@k at k = 1, 5, and 10, with Holm adjustment over 18 endpoints, identify functional regressions in both Qwen3-8B adapters at every k. The other four comparisons are inconclusive. On 141 reference-certified, contract-valid tasks, three independently trained graph-prefix systems score the same candidates. Best CodeBLEU has modest association with pass@10 ($\rho$ = .218-.246), compile@10 has weak association ($\rho$ = .072-.082), and only 21.0-23.3% of compiling candidates pass. A paired single-seed intervention that removes semant