Lexical analysis of texts generated by language models: reflection of their world models
DOI:
https://doi.org/10.15381/lengsoc.v23i2.28336Keywords:
world model, language model, artificial intelligence, lexical diversity, lexical densityAbstract
Artificial intelligence (AI) has transformed numerous fields, including linguistics. Large Language Models (LLMs) have revolutionized interaction with text by providing responses that mimic human language. These models not only generate text, but also reflect their interpretation of the world. However, these models' understanding of the world is limited, which has led to the proposal of developing Large World Models (LWMs), which integrate textual, visual, and auditory data for a more complete understanding. This article employs a lexicostatistical perspective to analyze how LLMs articulate responses based on their world models. A comparative quasi-experimental design was utilized to evaluate six different LLMs. The methodology focused on measuring the diversity and lexical density of the texts generated by these models. The results demonstrated that ChatGPT-4 has high lexical density and moderate lexical diversity, while Copilot has the highest lexical diversity but lower lexical density. This analysis is of great importance for understanding the capabilities and limitations of LLMs, with implications for their applications in various areas. The concepts and methodology are presented, the findings are discussed , and the paper concludes with reflections on future research and practical applications.
References
Altmann, E. G., Pierrehumbert, J. B. y Motter, A. E. (2009). Beyond Word Frequency: Bursts, Lulls, and Scaling in the Temporal Distributions of Words. PLoS ONE, 4(11), 1-7. https://doi.org/10.1371/journal.pone.0007678
Asensi, M. (2016). Teoría de los modelos de mundo y teoría de los mundos posibles. Actio Nova: Revista de teoría de la literatura y literatura comparada, (0), 38-55. https://doi.org/10.15366/actionova2016.0.003
Carvajal, Á. (2013). Teorías y modelos: formas de representación de la realidad. Revista Comunicación, 12(1), 33-46.
Cheng, M., Durmus, E. y Jurafsky, D. (2023). Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 1504–1532). Toronto, Canada. Association for Computational Linguistics.
Deutscher, G. (2011). El prisma del lenguaje. Cómo las palabras colorean el mundo. Editorial Ariel.
Goodwin, A. P. y Ahn, S. (2010). A meta-analysis of morphological interventions: effects on literacy achievement of children with literacy difficulties. Annals of dyslexia, 60(2), 183–208. https://doi.org/10.1007/s11881-010-0041-x
Hirschman, M. (2000). Language repair via metalinguistic means. International Journal of Language & Communication Disorders, 35(2), 252–268. https://doi.org/10.1080/136828200247179
Johansson, V. (2008). Lexical diversity and lexical density in speech and writing: a develop-mental perspective. Working Papers in Linguistics, 53, 61-79. https://journals.lub.lu.se/index.php/LWPL/article/view/2273/1848
Joly, F. (1988). La cartografía. Oikos-Tau.
Justice, L. M., Schmitt, M. B., Murphy, K. A., Pratt, A. y Biancone, T. (2014). The 'robustness' of vocabulary intervention in the public schools: targets and techniques employed in speech-language therapy. International journal of language & communication disorders, 49(3), 288–303. https://doi.org/10.1111/1460-6984.12072
Kamhi, A. G. y Koenig, L. A. (1985). Metalinguistic Awareness in Normal and Language-Disordered Children. Language Speech and Hearing Services in Schools, 16, 199-210. https://doi.org/10.1044/0161-1461.1603.199
Kubát, M. y Milička, J. (2013). Vocabulary Richness Measure in Genres. Journal of Quantitative Linguistics, 20(4), 339-349. https://doi.org/10.1080/09296174.2013.830552
Larsen, J. A. y Nippold, M. A. (2007). Morphological analysis in school-age children: dynamic assessment of a word learning strategy. Language, speech, and hearing services in schools, 38(3), 201–212. https://doi.org/10.1044/0161-1461(2007/021)
López Morales, H. (2002). Los índices de ‘Riqueza léxica’ y la enseñanza de lenguas. En Del texto a la lengua: la aplicación de los textos a la enseñanza-aprendizaje del español (pp. 15-28). https://dialnet.unirioja.es/descarga/articulo/5419218.pdf
Matthews, P. (2014). Sapir-Whorf hypothesis. En P. H. Matthews (Hrsg.), The Concise Oxford Dictionary of Linguistics (3.° ed.). Oxford University Press.
Mccarthy, P. y Jarvis, S. (2010). MTLD, vocd-D, and HD-D: a validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42, 381-392.
McGregor, K. K., Oleson, J., Bahnsen, A. y Duff, D. (2013). Children with developmental language impairment have vocabulary deficits characterized by limited breadth and depth. International journal of language & communication disorders, 48(3), 307–319. https://doi.org/10.1111/1460-6984.12008
Minaee, S., Mikolov, T., Nikzad, N., Chenaghlu, M., Socher, R., Amatriain, X. y Jianfeng, G. (2024). Large Language Models: A Survey. Arxiv. https://doi.org/10.48550/arXiv.2402.06196
Nippold, M. A. (2014). Language sampling with adolescents: Implications for intervention. (2nd ed.). Plural Publishing.
Nippold, M. A. (2017). Building a literate lexicon in adolescents: Selecting words for language intervention. 14th International Congress for the Study of Child Language (IASCL), University of Lyon, France.
Ozdemir, S. (2023). Quick Start Guide to Large Language Models: Strategies and Best Practices for Using ChatGPT and Other LLMs. Addison-Wesley Professional.
Pape, C. (2010). Sprachliche Relativität und ihre Implikationen für den Englischunterricht [Tesis de licenciatura, Humboldt-University of Berlin]. GRIN Verlag, https://www.grin.com/document/169274
Pérez Parejo, R. (2004). Modelos de mundo y tópicos literarios: la construcción ficcional al servicio de la ideología del poder. Revista De Literatura, 66(131), 49–76. https://doi.org/10.3989/revliteratura.2004.v66.i131.140
Read, J. (2010) Assessing vocabulary. (9. ed). Cambridge University Press.
Riffo, K. F., Osuna, S. H. y Lagos, P. S. (2019). Descripción de la diversidad y densidad léxicas en noticias escritas por estudiantes de periodismo. Revista Brasileira de Linguística Aplicada, 19(3), 499-528. https://doi.org/10.1590/1984-6398201914113
Torres-Bravo, L. (2020). Teoría de los modelos de mundo: lectura literaria y posibilidades didácticas. Enunciación, 25(2), 292-305. https://doi.org/https://doi.org/10.14483/22486798.16634
Vine, A., Fuentes Riffo, K., Neira Martínez, A. y Poza Molina, C. (2021). Lexical richness on written texts by foreign spanish language learners: A contribution to the specific notions of the curricular plan of the Cervantes Institute. Nueva revista del Pacífico, (75), 201-226.
von Humboldt, W. (1991). Escritos sobre el lenguaje. (A. Sánchez Pascual, Trad.). Península.
Wittgenstein, L. (1922). Tractatus Logius-Philosophicus, con una introducción de Bertrand Russell. Trench Trübner.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Gabriela Kotz, Pedro Salcedo, Karina Fuentes
This work is licensed under a Creative Commons Attribution 4.0 International License.
AUTHORS RETAIN THEIR RIGHTS
a. Authors retain their trade mark rights and patent, and also on any process or procedure described in the article.
b. Authors can submit to the journal Lengua y Sociedad, papers disseminated as pre-print in repositories. This should be made known in the cover letter.
c. Authors retain their right to share, copy, distribute, perform and publicly communicate their article (eg, to place their article in an institutional repository or publish it in a book), with an acknowledgment of its initial publication in the journal Lengua y Sociedad.
d. Authors retain theirs right to make a subsequent publication of their work, to use the article or any part thereof (eg a compilation of his papers, lecture notes, thesis, or a book), always indicating its initial publication in the journal Lengua y Sociedad (the originator of the work, journal, volume, number and date).
