Enhancing LLMs for Identifying and Prioritizing Important Medical Jargons from Electronic Health Record Notes Utilizing Data Augmentation: A Comparative Study

תקציר מקורי באנגליתarXiv:2502.16022v3 Announce Type: replace Abstract: OpenNotes gives patients access to their EHR notes, but dense medical jargon limits comprehension. We evaluate closed-source and open-source LLMs for extracting and prioritizing the jargon terms most relevant to individual patients, using 90 expert-annotated EHR notes. We test combinations of general vs. structured prompts, zero-shot vs. few-shot prompting, fine-tuning, and GPT-4o-based data augmentation, the last paired with a ranking technique to refine training in low-resource settings. To assess the effect of dataset size, we fine-tune on augmented datasets scaled from 10 to 9,995 examples. All settings are evaluated with 10-fold cross-validation, reporting F1 and Mean Reciprocal Rank (MRR) under two string-matching criteria (relaxed