מילון חזרה של שפת סימנים: הכרה של שפת סימנים פתוחה-לשונית מאמירה רציפה דרך תיאור תמונה וחיפוש תיאור

תקציר מקורי באנגליתarXiv:2609.03788v1 Announce Type: cross Abstract: Isolated Sign Language Recognition (ISLR) is conventionally cast as closed-set classification over gloss labels, which cannot generalize to signs unseen in training and ties every deployment to a gloss-annotated lexicon. We instead recognize signs extracted from continuous signing by (1) captioning a sign-level clip into a free-form procedural description of the articulation with an open-weight vision-language model, and (2) retrieving the closest entry from a vocabulary of target descriptions with a multilingual sentence encoder: a reverse sign language dictionary that needs no gloss supervision and admits an open vocabulary. On 1,300 sign-level segments from a Japanese Sign Language (JSL) dialogue corpus annotated with procedural descript