בחירת הדברים החשובים: צינור עיבוי-מחקרי לבחירה סלקטיבית של חבישה לארוכה

תקציר מקורי באנגליתarXiv:2609.37782v1 Announce Type: new Abstract: Large language models (LLMs) have shown strong potential as training-free text encoders for long-context embeddings. Existing approaches primarily improve information flow under causal attention and typically construct embeddings by uniformly averaging all token representations. However, for long documents, such mean pooling can dilute salient semantic information with abundant redundant or weakly informative content. To this end, we propose SCSP, a training-free framework that leverages semantic compression for informative token selection in long-context embedding. Specifically, SCSP first partitions a document into sentence-aware chunks and appends a semantic compression prompt to each chunk. A prompt-isolated attention mask preserves infor