Fewer yet critical: Reducing Redundant Token Dependencies for Transformer-based Time Series Forecasting

תקציר מקורי באנגליתarXiv:2503.06867v2 Announce Type: replace Abstract: Time series forecasting (TSF) is important in real-world applications. Recently, Transformer-based methods have achieved strong performance by modeling token dependencies through attention mechanisms. However, existing methods are usually trained mainly with prediction error losses, which may cause models to exploit both critical and redundant token dependencies. Such redundant dependencies can introduce irrelevant information and weaken generalization. To address this issue, we propose a simple yet effective token dependency selection strategy. Specifically, by jointly introducing the attention entropy constraint and prediction error constraint, the model can identify fewer but more critical inter-token dependencies and perform forecasti