Pragmatic Implicature Processing in ChatGPT

Published: May 12, 2023

Recent large language models (LLMs) and LLM-driven chatbots, such as ChatGPT, have sparked debate regarding whether these artificial systems can develop human-like linguistic capacities. We examined this issue by investigating ChatGPT resembles humans in its ability to enrich literal meanings of utterances with pragmatic implicatures. Humans not only distinguish implicatures from truth-conditional but also compute contingent on the communicative context. In three preregistered experiments (https://osf.io/4bcx9/), we assessed computation Experiment 1 investigated generalized conversational (GCIs); for example, utterance “She walked into bathroom. The window was open.” has implicature that is located bathroom, while (literal) meaning allows possibility elsewhere. demonstrate their GCIs inhibiting when explicitly instructed focus sense utterances. tested could inhibit do. 2 3 context modulates how computes a specific type GCIs, namely scalar (SIs). For humans, sentence “Julie had found crab or starfish” implies Julie did find both starfish, even though sentence’s possibility. Moreover, argued be more available word “or” information focus, e.g. reply question “What found?” than background, “Who starfish?”. shows similar sensitivity structure computing SIs. focused different contextual aspect, face-threatening face-boosting contexts effects Previous research shown human interlocutors SIs contexts, interpreting “Some people loved your poem.” saying “Not all so much they are contexts; exhibits tendency. experiments, display flexibility switching between semantic processing failed show well-established SI rate. Overall, our although parallels surpasses many tasks, it still does closely resemble beings GCIs. attribute discrepancy differences acquisition computational resources machines.

Language: Английский

---

Grammaticality Representation in ChatGPT as Compared to Linguists and Laypeople

Published: March 21, 2024

Large language models (LLMs) have demonstrated exceptional performance across various linguistic tasks. However, it remains uncertain whether LLMs developed human-like fine-grained grammatical intuition. This preregistered study (https://osf.io/t5nes) presents the first large-scale investigation of ChatGPT’s intuition, building upon a previous that collected laypeople’s judgments on 148 phenomena linguists judged to be grammatical, ungrammatical, or marginally (Sprouse, Schütze, & Almeida, 2013). Our primary focus was compare ChatGPT with both laypeople and in judgement these constructions. In Experiment 1, assigned ratings sentences based given reference sentence. 2 involved rating 7-point scale, 3 asked choose more sentence from pair. Overall, our findings demonstrate convergence rates ranging 73% 95% between linguists, an overall point-estimate 89%. Significant correlations were also found all tasks, though correlation strength varied by task. We attribute results psychometric nature judgment tasks differences processing styles humans LLMs.

Language: Английский

---

Characterizing English Preposing in PP constructions

Journal of Linguistics, Journal Year: 2024, Volume and Issue: unknown, P. 1 - 39

Published: Oct. 8, 2024

The English Preposing in PP construction (PiPP; e.g., H appy though / as we were ) is extremely rare but displays an intricate set of stable syntactic properties. How do people become proficient with this despite such limited evidence? It tempting to posit innate learning mechanisms, present-day large language models seem learn represent PiPPs well, even employ only very general mechanisms and experience few instances the during training. This suggests alternative hypothesis on which knowledge more frequent constructions helps shape PiPPs. I seek make idea precise using model-theoretic syntax (MTS). In MTS, a grammar essentially constraints forms. context, can be seen arising from mix construction-specific general-purpose constraints, all inferable linguistic experience.

Language: Английский

---

Neural Networks as Cognitive Models of the Processing of Syntactic Constraints

Open Mind, Journal Year: 2024, Volume and Issue: 8, P. 558 - 614

Published: Jan. 1, 2024

Abstract Languages are governed by syntactic constraints—structural rules that determine which sentences grammatical in the language. In English, one such constraint is subject-verb agreement, dictates number of a verb must match its corresponding subject: “the dogs run”, but dog runs”. While this appears to be simple, practice speakers make agreement errors, particularly when noun phrase near differs from subject (for example, speaker might produce ungrammatical sentence key cabinets rusty”). This phenomenon, referred as attraction, sensitive wide range properties sentence; no single existing model able generate predictions for variety materials studied human experimental literature. We explore viability neural network language models—broad-coverage systems trained predict next word corpus—as framework addressing limitation. analyze errors made Long Short-Term Memory (LSTM) networks and compare them those humans. The models successfully simulate certain results, so-called asymmetry difference between attraction strength sentences, failed others, effect distance or notional (conceptual) number. further evaluate with explicit supervision, find form supervision does not always lead more human-like behavior. Finally, we show corpus used train significantly affects pattern produced network, discuss strengths limitations tool understanding processing.

Language: Английский

---

Creative minds like ours? Large Language Models and the creative aspect of language use

Biolinguistics, Journal Year: 2024, Volume and Issue: 18

Published: Oct. 29, 2024

Descartes famously constructed a language test to determine the existence of other minds. The made critical observations about how humans use that purportedly distinguishes them from animals and machines. These were carried into generative (and later biolinguistic) enterprise under what Chomsky in his Cartesian Linguistics, terms “creative aspect use” (CALU). CALU refers stimulus - free, unbounded, yet appropriate language—a tripartite depiction whose function biolinguistics is highlight species-specific form intellectual freedom. This paper argues provides set facts have significant downstream effects on explanatory theory-construction. include internalist orientation linguistics, invocation competence-performance distinction, postulation faculty makes possible—but does not explain—CALU. It contrasts biolinguistic approach with recent wave enthusiasm for Transformer-based Large Language Models (LLMs) as tools, models, or theories human language, arguing such uses neglect these fundamental insights their detriment. that, absence replication, identification, accounting CALU, LLMs do match depth framework, thereby limiting theoretical usefulness.

Language: Английский

---

Prompting Metalinguistic Awareness in Large Language Models: ChatGPT and Bias Effects on the Grammar of Italian and Italian Varieties

Verbum, Journal Year: 2023, Volume and Issue: 14, P. 1 - 11

Published: Dec. 20, 2023

We explore ChatGPT’s handling of left-peripheral phenomena in Italian and varieties through prompt engineering to investigate 1) forms syntactic bias the model, 2) model’s metalinguistic awareness relation reorderings canonical clauses (e.g., Topics) certain grammatical categories (object clitics). A further question concerns content sources training data: how are minor languages included training? The results our investigation show that model seems be biased against reorderings, labelling them as archaic even though it is not case; have difficulties with coindexed elements such clitics their anaphoric status, labeling ‘not referring any element phrase’, 3) major still seem dominant, overshadowing positive effects including training.

Language: Английский

---

A new way to identify if variation in children’s input could be developmentally meaningful: Using computational cognitive modeling to assess input across socio-economic status for syntactic islands

Journal of Child Language, Journal Year: 2022, Volume and Issue: 51(4), P. 800 - 833

Published: Nov. 24, 2022

While there are always differences in children's input, it is unclear how often these impact language development - that is, developmentally meaningful and why they do (or not) so. We describe a new approach using computational cognitive modeling links input to predicted outcomes, can identify if potentially meaningful. use this investigate developmentally-meaningful variation across socio-economic status (

Language: Английский

---

A-maze of Natural Stories: Comprehension and surprisal in the Maze task

Published: Aug. 25, 2022

Behavioral measures of word-by-word reading time provide experimental evidence to test theories language processing. A-maze is a recent method for measuring incremental sentence processing that can localize slowdowns related syntactic ambiguities in individual sentences. We adapted use on longer passages and tested it the Natural Stories corpus. Participants were able comprehend these text they read via Maze task. Moreover, task yielded useable reaction data with word predictability effects linearly surprisal, same pattern found other methods. Crucially, times show tight relationship properties current word, little spillover effects from previous words. This superior localization an advantage compared Overall, we expanded scope materials, thus theoretical questions, be studied

Language: Английский

---

What language models can tell us about learning adjectives

Proceedings of the Linguistic Society of America, Journal Year: 2024, Volume and Issue: 9(1), P. 5693 - 5693

Published: May 15, 2024

It has been argued that language models (LMs) inform our knowledge of acquisition. While LMs are claimed to replicate aspects grammatical knowledge, it remains unclear how this translates acquisition directly. We ask if a model trained specifically on child-directed speech (CDS) is able capture adjectives. Ultimately, results reveal what the “learning” adjectives distributed in CDS, and not properties different adjective classes. highlighting ability learn distributional information, these findings suggest alone cannot explain children generalize beyond their input.

Language: Английский

---

Acquiring Constraints on Filler-Gap Dependencies from Structural Collocations: Assessing a Computational Learning Model of Island-Insensitivity in Norwegian

Published: Nov. 25, 2024

Children induce complex syntactic knowledge from their native language input. A long-standing discussion focuses on types of learning biases that help them arrive at correct generalization and solve induction problems posed by impoverished Studies employing computational models for specific phenomena serve as testing grounds evaluating required successful acquisition. Recent work Pearl Sprouse (2013b) demonstrates a distributional learner tracks trigrams over structurally annotated input can acquire wh-filler-gap dependencies island constraints in English. While intriguing, it is unclear yet whether similar model viable mechanism facts other languages given the possibility cross-linguistic variation. In this study, we explore wh- relative clause filler-gap Norwegian child-directed text. We find proposed strategy capture some patterns island-insensitivity while failing to learn others due lack relevant data Our findings suggest limited data, simple n-gram-based structured representations may not be sufficient fully recover human-like dependency relations cross-linguistically.

Language: Английский

---