{rfName}
Us

License and Use

Icono OpenAccess

Altmetrics

Impact on the Sustainable Development Goals (SDGs)

Analysis of institutional authors

Molero, JdAuthorConde, JAuthorReviriego, PAuthor

Share

February 3, 2025
Publications
>
Article
Bronze

Using large language models to estimate features of multi-word expressions: Concreteness, valence, arousal

Publicated to: Behavior Research Methods. 57 (1): 5- - 2025-01-01 57(1), DOI: 10.3758/s13428-024-02515-z

Authors:

Martinez, Gonzalo; Molero, Juan Diego; Gonzalez, Sandra; Conde, Javier; Brysbaert, Marc; Reviriego, Pedro
[+]

Affiliations

Univ Carlos Iii Madrid - Author
Univ Ghent, Dept Expt Psychol - Author
Univ Politecn Madrid, ETSI Telecomunicac - Author
See more

Abstract

This study investigates the potential of large language models (LLMs) to provide accurate estimates of concreteness, valence, and arousal for multi-word expressions. Unlike previous artificial intelligence (AI) methods, LLMs can capture the nuanced meanings of multi-word expressions. We systematically evaluated GPT-4o's ability to predict concreteness, valence, and arousal. In Study 1, GPT-4o showed strong correlations with human concreteness ratings (r = .8) for multi-word expressions. In Study 2, these findings were repeated for valence and arousal ratings of individual words, matching or outperforming previous AI models. Studies 3-5 extended the valence and arousal analysis to multi-word expressions and showed good validity of the LLM-generated estimates for these stimuli as well. To help researchers with stimulus selection, we provide datasets with LLM-generated norms of concreteness, valence, and arousal for 126,397 English single words and 63,680 multi-word expressions.
[+]

Keywords

Affective normsArousalArtificial intelligenceConcretenessCooccurrenceEnglishHumanHumansLanguageLarge language modelMulti-word expressionsPhysiologyProceduresPsycholinguisticsRatingsReduced inequalitiesSemanticsValenceWord norms

Quality index

Bibliometric impact. Analysis of the contribution and dissemination channel

The work has been published in the journal Behavior Research Methods due to its progression and the good impact it has achieved in recent years, according to the agency WoS (JCR), it has become a reference in its field. In the year of publication of the work, 2025, it was in position 1/13, thus managing to position itself as a Q1 (Primer Cuartil), in the category Psychology, Mathematical. Notably, the journal is positioned above the 90th percentile.

Independientemente del impacto esperado determinado por el canal de difusión, es importante destacar el impacto real observado de la propia aportación.

Según las diferentes agencias de indexación, el número de citas acumuladas por esta publicación hasta la fecha 2026-04-24:

  • Google Scholar: 6
  • WoS: 16
  • Scopus: 18
[+]

Impact and social visibility

From the perspective of influence or social adoption, and based on metrics associated with mentions and interactions provided by agencies specializing in calculating the so-called "Alternative or Social Metrics," we can highlight as of 2026-04-24:

  • The use, from an academic perspective evidenced by the Altmetric agency indicator referring to aggregations made by the personal bibliographic manager Mendeley, gives us a total of: 17.
  • The use of this contribution in bookmarks, code forks, additions to favorite lists for recurrent reading, as well as general views, indicates that someone is using the publication as a basis for their current work. This may be a notable indicator of future more formal and academic citations. This claim is supported by the result of the "Capture" indicator, which yields a total of: 17 (PlumX).

With a more dissemination-oriented intent and targeting more general audiences, we can observe other more global scores such as:

  • The Total Score from Altmetric: 3.
  • The number of mentions on the social network X (formerly Twitter): 4 (Altmetric).

It is essential to present evidence supporting full alignment with institutional principles and guidelines on Open Science and the Conservation and Dissemination of Intellectual Heritage. A clear example of this is:

  • The work has been submitted to a journal whose editorial policy allows open Open Access publication.
  • Assignment of a Handle/URN as an identifier within the deposit in the Institutional Repository: https://oa.upm.es/85232/

As a result of the publication of the work in the institutional repository, statistical usage data has been obtained that reflects its impact. In terms of dissemination, we can state that, as of

  • Views: 190
  • Downloads: 14
Continuing with the social impact of the work, it is important to emphasize that, due to its content, it can be assigned to the area of interest of ODS 10 - Reduce inequality within and among countries, with a probability of 46% according to the mBERT algorithm developed by Aurora University.
[+]

Leadership analysis of institutional authors

This work has been carried out with international collaboration, specifically with researchers from: Belgium.

There is a significant leadership presence as some of the institution’s authors appear as the first or last signer, detailed as follows: Last Author (REVIRIEGO VASALLO, PEDRO).

[+]

Project objectives

La aportación persigue los siguientes objetivos: analizar la capacidad de los grandes modelos de lenguaje (LLMs) para estimar concreción, valencia y activación en expresiones de varias palabras; evaluar la precisión de GPT-4o en la predicción de concreción mediante correlaciones con valoraciones humanas; determinar la validez de las estimaciones de valencia y activación para palabras individuales y expresiones múltiples; comparar el rendimiento de GPT-4o con modelos de inteligencia artificial previos; y proporcionar conjuntos de datos con normas generadas por LLM para 126,397 palabras y 63,680 expresiones múltiples en inglés, facilitando la selección de estímulos en investigaciones futuras.
[+]

Most relevant results

El estudio evaluó la capacidad de los grandes modelos de lenguaje (LLMs) para estimar concreción, valencia y activación en expresiones de varias palabras. Los resultados más relevantes son: (1) GPT-4o mostró una alta correlación con las valoraciones humanas de concreción en expresiones multi-palabra (r = 0.8); (2) se replicaron estos resultados para valencia y activación en palabras individuales, igualando o superando modelos de IA previos; (3) la valencia y activación también fueron validadas en expresiones multi-palabra en estudios posteriores; (4) se generaron y facilitaron datasets con normas LLM para 126,397 palabras individuales y 63,680 expresiones multi-palabra en inglés.
[+]