Measuring Human Value Expression in Social Media Texts: Calibrated LLM Annotation and Encoder Transfer

תקציר מקורי באנגליתarXiv:2606.11018v3 Announce Type: replace Abstract: Measuring subjective constructs in naturally occurring social media text requires annotation procedures that are theoretically grounded, empirically validated, and transferable to an encoder model for scalable prediction. Using posts annotated according to Schwartz's theory of basic human values, we investigate how different LLMs and prompting strategies, which we call annotation regimes, operationalize the expression of values in text. Beyond standard classification metrics, we evaluate structural alignment, annotation ambiguity, error patterns, and stability across repeated runs. We find that different LLMs produce different value interpretations, and iterative prompt calibration through error analysis reduces misattributions and improv