Embodied Processing at Six Linguistic Granularity Levels: A Consensus Paper DOI Creative Commons
Anita Körner, Mauricio Castillo, Linda Drijvers

et al.

Journal of Cognition, Journal Year: 2023, Volume and Issue: 6(1)

Published: Jan. 1, 2023

Language processing is influenced by sensorimotor experiences. Here, we review behavioral evidence for embodied and grounded influences in language across six linguistic levels of granularity. We examine (a) sub-word features, discussing on iconicity (systematic associations between word form meaning); (b) words, boundary conditions generalizations the simulation color, sensory modality, spatial position; (c) sentences, applications action direction simulation; (d) texts, how teaching can improve comprehension beginning readers; (e) conversations, multi-modal cues turn taking alignment; (f) text corpora, distributional semantic models reveal knowledge encoded texts. These approaches are converging a convincing account psychology language, but at same time, there important criticisms approach specific experimental paradigms. The surest way forward requires adoption wide array scientific methods. By providing complimentary evidence, combination multiple methods various granularity help us gain more complete understanding role embodiment grounding processing.

Language: Английский

Over-reliance on English hinders cognitive science DOI Creative Commons
Damián E. Blasí, Joseph Henrich, Evangelia Adamou

et al.

Trends in Cognitive Sciences, Journal Year: 2022, Volume and Issue: 26(12), P. 1153 - 1170

Published: Oct. 14, 2022

English is the dominant language in study of human cognition and behavior: individuals studied by cognitive scientists, as well most scientists themselves, are frequently speakers. However, differs from other languages ways that have consequences for whole sciences, reaching far beyond itself. Here, we review an emerging body evidence highlights how particular characteristics linguistic habits speakers bias field both warping research programs (e.g., overemphasizing features mechanisms present over others) overgeneralizing observations speakers' behaviors, brains, to our entire species. We propose mitigating strategies could help avoid some these pitfalls.

Language: Английский

Citations

240

The Border Between Seeing and Thinking DOI Open Access
Ned Block

Oxford University Press eBooks, Journal Year: 2023, Volume and Issue: unknown

Published: Feb. 16, 2023

Abstract This book argues that there is a joint in nature between seeing and thinking, perception, cognition. Perception constitutively iconic, nonconceptual, nonpropositional, whereas cognition does not have these properties constitutively. The appeal to “intuitions,” as common philosophy, but empirical evidence, including experiments neuroscience psychology. affects i.e., perception cognitively penetrable, this impugn the nature. A key part of argument we perceive only low-level like colors, shapes, textures also high-level such faces causation. Along way, explains difference perceptual memory, differences format content, whether probabilistic despite our lack awareness properties. for categories are concepts, need be singular, attribution discrimination equally fundamental, basic features mind known “core cognition” third category chapter on consciousness leverages results argue against some most widely accepted theories consciousness. Although one about consciousness, much rest repurposes work isolate scientific basis perception.

Language: Английский

Citations

162

Dual coding of knowledge in the human brain DOI Creative Commons
Yanchao Bi

Trends in Cognitive Sciences, Journal Year: 2021, Volume and Issue: 25(10), P. 883 - 895

Published: Sept. 8, 2021

Neural systems have been assumed to represent knowledge through the embodiment of sensory experiences. This type representation is considered fundamentally different from artificial intelligence approach, which implements using 'symbolic' approach.Recent behavioral and neuroimaging studies color (and other visual knowledge) with without experience provide empirical evidence for neural coding non-sensory, language-derived knowledge, along sensory-derived representation, in brain systems.The proposed dual-coding framework motivated by these recent findings offers nature's solution classical challenges associated either only symbolic tradition or embodied approach neuroscience. How does human code about world? While disciplines such as world based on language, neurocognitive models dominated embodiment, derived sensory/motor supported high-level association cortices. The correlates an alternative disembodied system had previously difficult establish. A line exploring properties, color, visually deprived individuals converge positive, compelling language-derived, dorsal anterior temporal lobe extended language network, addition representations, leading a sketch framework. (A android): There was thought experiment they gave us. It's called 'Mary Black White room.' Mary scientist, her special subject color. She knows everything there know it […] But she lives black white room. born there, raised One day… walks out sees blue sky. And at that moment learns something all couldn't tell her. what feels see show students difference between computer mind. when out. – Ex Machina (a Movie, 2014), citing [1.Jackson F. Epiphenomenal qualia.Philos. Q. 1982; 32: 127-136Crossref Google Scholar]. What mean, know? Is knowing relations versus (see Glossary) (current) machines humans, suggested earlier movie dialogue? stores vast amounts world, roses, shape earth, mechanisms evolution, functions jury. Various sorts guide broad range behaviors, recognizing objects understanding words, reasoning, decision making. Understanding how represented one oldest most-debated questions any discipline interested mind brain. discussions every field, including philosophy, psychology, cognitive science, center two notions representing knowledge: (sensory/motor-experience-derived). granted room Scholar], prevalent way intelligence. Beyond computers, similar also various (e.g., dual both verbal nonverbal [2.Paivio A. Mental Representations: Dual Coding Approach. Oxford University Press, 1986Google Scholar,3.Paivio Mind Its Evolution: Theoretical Lawrence Erlbaum Associates Publishers, 2007Google Scholar]) philosophical theories dualism [4.Papineau D. Thinking About Consciousness. 2002Crossref Scholar]). Despite this popularity some areas, (biological) implementation are more strongly influenced notion 'sensory embodiment', start assumptions least physical apple), acquired grounded experiences stored higher-order cortices [5.Binder J.R. Desai R.H. neurobiology semantic memory.Trends Cogn. Sci. 2011; 15: 527-536Abstract Full Text PDF PubMed Scopus (1067) Scholar, 6.Martin GRAPES—grounding representations action, perception, emotion systems: object properties categories brain.Psychon. Bull. Rev. 2016; 23: 979-990Crossref (140) 7.Pulvermüller neurons make meaning: Brain abstract-symbolic semantics.Trends 2013; 17: 458-470Abstract (305) 8.Barsalou L.W. On staying avoiding quixotic dead ends.Psychon. 1122-1142Crossref (128) 9.Lambon Ralph M.A. et al.The computational bases cognition.Nat. Neurosci. 2017; 18: 42-55Crossref (650) 10.Patterson K. al.Where do you brain.Nat. 2007; 8: 976-987Crossref (1690) 11.Binder In defense abstract conceptual representations.Psychon. 1096-1108Crossref (96) 12.Barsalou Grounded cognition.Annu. Psychol. 2008; 59: 617-645Crossref (3619) review, I discuss identifies non-sensory form representation. To explain rationales help readers appreciate their implications first briefly present current views embodiment' stems combination considerations: biological entities acquire information interaction world; long identified brain; numerous during over decades shown retrieving activates regions territory corresponding (primary associative) For instance, roses) induces activation close V4, region critical perception [13.Martin al.Discrete cortical action.Science. 1995; 270: 102-105Crossref (833) Scholar,14.Simmons W.K. al.A common substrate perceiving color.Neuropsychologia. 45: 2802-2810Crossref (280) processing words whose meanings strong associations tends modulate activities [15.Fernandino L. al.Concept reflects multimodal abstraction: semantics.Cereb. Cortex. 26: 2018-2034Crossref (125) Thus, often assume, extents, can be perceived modalities, abstracted experiences/systems connections set combinational It axiomatic humans learn via things not personally experienced sensorily someone 'roses red' reading/hearing seeing red roses), yet, quoting [16.Binder al.Toward brain-based componential representation.Cogn. Neuropsychol. 33: 130-174Crossref (126) Scholar] 'In cognition theories, however, our model maintains concepts ultimately experience, whether directly indirectly refers experience'. research has away specific given [17.Mahon B.Z. Caramazza look hypothesis new proposal grounding content.J. Physiol. Paris. 102: 59-70Crossref (841) Scholar]), isolate, articulate, identify supporting non-sensory-derived knowledge. because typically developed brain, evoked even justice) cannot perceive quark). cases deprivation input modality, congenital blindness, most still gained modalities apple obtained touch vision). difficulty cognitively excluding leads further mapping activity cognition. clean test would involve testing happens particular modality absent. exactly why philosophers Scholar,18.Locke J. An Essay Concerning Human Understanding. 1690Crossref fascinated Color sensation exclusively visual, driven wavelength light, no senses it. Colors congenitally blind individual mirror What, how, brains sight colors visual-exclusive knowledge)? Answers opportunity isolate following sections, will review provided answers compelling, positive existence highly sensory-related wave empirically started almost four ago. showed people types reviewed [19.Bedny M. Saxe R. Insights into origins neuroscience blindness.Cogn. 2012; 29: 56-84Crossref (31) Briefly, early work found children use adjectives names speech, perceptual qualities felt touch, name [20.Landau B. Gleitman Language Experience: Evidence Blind Child. Harvard 1985Google 21.Connolly A.C. al.Effect blindness everyday concepts.Proc. Natl. Acad. U. S. 104: 8241-8246Crossref (42) 22.Mills A.E. Acquisition Child: Normal Deficient. College-Hill 1983Google group free-recall tasks [23.Zimler Keenan J.M. Imagery blind: images?.J. Exp. Learn. Mem. 1983; 9: 269-282Crossref (91) recover Newtonian circle labels [24.Shepard R.N. Cooper L.A. Representation blind, color-blind, normally sighted.Psychol. 1992; 3: 97-104Crossref (123) Scholar,25.Marmor G.S. Age onset development semantics names.J. Child 1978; 25: 267-278Crossref (44) Given work's relevance larger question recently rigorously investigated, detailed benchmark comparisons, detail below. comparison sighted individuals, multiple (basic descriptive terms) tested similarity rating task paired terms (congenital noncongenital) control conditions [26.Saysani al.Colour envisioned: colour sighted.Vis. 2018; 382-392Crossref (15) space overall strikingly (Figure 1A ), subjects indistinguishable sighted, greater variation group. high degree correlation (r = 0.93) term spaces replicated, relationship lower than results action 0.99) [27.Bottini al.Brain involved retrieval people.J. 2020; 1009-1025Crossref (8) Compared intriguing cross-modal reported (red heavier yellow), color-weight associations, but object-taste (boulder-sour, strength benchmarks), suggesting contributes [28.Barilari al.Is yellow blind?.Iperception. 1-4Google Another colors. individuals' fruits vegetables probed [29.Wang X. al.Two forms brain.Neuron. 107: 383-393Abstract (22) again yielded 1B) relative individuals. Furthermore, strengths typical (nonvisual) (including shape, taste), well general (both continuously rated categorically measured), were groups, indicating differently relying derive relations. investigation carried visual-specific [30.Bedny al.There's "sparkle" meets eye: vision light verbs among individuals.Cognition. 2019; 189: 105-115Crossref Scholar,31.Bedny al.Growing up change theory mind.Proc. 2009; 106: 11312-11317Crossref (68) peek feel/perceive; shimmer boom/roll), controls (by means word judgment). Multidimensional scaling analyses distinguish, like emission sound 1C). suggests concepts, blindness. Over dozen spanning decades, across paradigms (generation, sorting, rating, association) languages (English, Italian, Mandarin Chinese) paint consistent picture complete deprivation. (i) Without actually sensorially experiencing relational structures, patterns those sighted; (ii) level, non-visual-specific [32.Kim J.S. al.Knowledge animal appearance adults.Proc. 166: 11213-11222Crossref (25) Scholar]); (iii) within population fully quite different, additional may reduce idiosyncrasy [33.Wang, Bi, Y. Idiosyncratic tower babel: Individual differences meaning increase abstractness. (accepted) (bioRxiv preprint, doi: https://doi.org/10.1101/2020.08.28.272997)Google point property absence ways perfectly probing Turing scenario [34.Turing A.M. Computing machinery intelligence.Mind. 1950; 49: 433-460Crossref basis visual-concept vision? Recent fMRI groups important clues, described below 2). responses examined depicting dominant phenomena, 'red', 'rainbow', 'island'), features (abstract e.g., 'freedom'), non-visual accessible 'square', 'rain', 'cup') [35.Striem-Amit E. al.Neural blind.Nat. Commun. 5250Crossref (24) cluster (ATL), superior gyrus/sulcus (STG/STS) pole (TP), significant interaction, stronger 'rainbow' (or 'red') compared 'rain' 'square') ('rainbow'/'red' 'rain'/'square' is), (all accessible; Figure 2 another study adaptation paradigm, judgments produced repetition suppression effects V4 STG (along precentral cortex) group, peak portion These convergently ATL sensitive uncovered representational analysis (RSA) [36.Kriegeskorte N. al.Representational - connecting branches neuroscience.Front. Syst. 2: 1-28PubMed Participants listened retrieved matrix (correlation pattern items) computed correlations taken Two 2): posterior fusiform color-perception-sensitive (close V4) similarity; left (anterior TP), object-color after controlled for. sets intrinsically, functionally connected Note under lenient threshold, ventral/medial observed showing convergence related [9.Lambon Scholar,35.Striem-Amit Although handful addressed question, appear clear. (univariate contrasts, adaptation, RSA) (Italian, (for concepts) while confirming cortex sighted. Importantly, RSA revealed occurs available representations. Are tested, reflect mechanism ATL? studying (i.e., referents, 'justice' 'rationality'). imaging consistently concrete 'cup'), requiring facts facts, elicit ATL, centering around dorsal-lateral aspects, encompassing network [37.Fedorenko al.New method investigations language: defining ROIs subjects.J. Neurophysiol. 2010; 1177-1194Crossref (276) inferior frontal gyrus (IFG) middle (pMTG) [38.Wang concepts: meta-analysis studies.Hum. Mapp. 31: 1459-1468Crossref (253) 39.Wang al.Close yet independent: dissociation social valence dimensions lobe.Hum. 40: 4759-4776Crossref (19) 40.Hoffman P. al.Differing contributions prefrontal knowledge.Cortex. 2015; 63: 250-266Crossref (115) 41.Binder system? 120 functional studies.Cereb. 19: 2767-2796Crossref (2414) 42.Sabsevitz D.S. al.Modulation imageability.Neuroimage. 2005; 27: 188-200Crossref (236) interpretations controversial researchers debated relate involvement Scholar,42.Sabsevitz emotional [43.Zahn al.Social cortex.Proc. 6430-6435Crossref (344) Scholar,44.Mellem M.S. al.Sentence shows social-emotional bias.Neuropsychologia. 89: 217-224Crossref (30) argued (external and/or internal) due nuanced lexical confounds age acquisition familiarity. observations parsimoniously explained potentially its network) Intriguingly, if perceptible properties? might richly distributed thus diluted network. earlier, primarily focusing jointly considering allows us 3, Key figure ). well-establ

Language: Английский

Citations

82

Cultural evolution of genetic heritability DOI
Ryutaro Uchiyama, Rachel Spicer, Michael Muthukrishna

et al.

Behavioral and Brain Sciences, Journal Year: 2021, Volume and Issue: 45

Published: May 21, 2021

Behavioral genetics and cultural evolution have both revolutionized our understanding of human behavior - largely independent each other. Here, we reconcile these two fields under a dual inheritance framework, offering more nuanced the interaction between genes culture. Going beyond typical analyses gene-environment interactions, describe dynamics that shape interactions by shaping environment population structure. A evolutionary approach can explain, for example, how factors such as rates innovation diffusion, density subgroups, tolerance behavioral diversity impact heritability estimates, thus yielding predictions different social contexts. Moreover, when cumulative culture functionally overlaps with genes, genetic effects become masked, unmasked, or even reversed, causal an identified gene confounded features environment. The manner confounding is specific to particular society at time, but WEIRD (western, educated, industrialized, rich, democratic) sampling problem obscures this boundedness. Cultural are typically missing from models gene-to-phenotype causality, hindering generalizability across societies time. We lay out reconciled framework use it predict ways in which should differ societies, socioeconomic levels, other groupings within some not others, over life course. An integrated cuts through nature-nurture debate helps resolve controversies topics IQ.

Language: Английский

Citations

71

Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Features DOI
Changde Du, Kaicheng Fu,

Jinpeng Li

et al.

IEEE Transactions on Pattern Analysis and Machine Intelligence, Journal Year: 2023, Volume and Issue: 45(9), P. 10760 - 10777

Published: March 30, 2023

Decoding human visual neural representations is a challenging task with great scientific significance in revealing vision-processing mechanisms and developing brain-like intelligent machines. Most existing methods are difficult to generalize novel categories that have no corresponding data for training. The two main reasons 1) the under-exploitation of multimodal semantic knowledge underlying 2) small number paired (stimuli-responses) training data. To overcome these limitations, this paper presents generic decoding method called BraVL uses learning brain-visual-linguistic features. We focus on modeling relationships between brain, linguistic features via deep generative models. Specifically, we leverage mixture-of-product-of-experts formulation infer latent code enables coherent joint generation all three modalities. learn more consistent representation improve efficiency case limited brain activity data, exploit both intra- inter-modality mutual information maximization regularization terms. In particular, our model can be trained under various semi-supervised scenarios incorporate textual obtained from extra categories. Finally, construct trimodal matching datasets, extensive experiments lead some interesting conclusions cognitive insights: practically possible good accuracy; models using combination perform much better than those either them alone; 3) perception may accompanied by influences represent semantics stimuli.

Language: Английский

Citations

37

From language development to language evolution: A unified view of human lexical creativity DOI
Thomas Brochhagen, Gemma Boleda,

Eleonora Gualdoni

et al.

Science, Journal Year: 2023, Volume and Issue: 381(6656), P. 431 - 436

Published: July 27, 2023

A defining property of human language is the creative use words to express multiple meanings through word meaning extension. Such lexical creativity manifested at different timescales, ranging from development in children evolution over history. We explored whether manifestations build on a common foundation. Using computational models, we show that parsimonious set semantic knowledge types characterize developmental data as well evolutionary products extension spanning 1400 languages. Models for account very data, and vice versa. These findings suggest unified foundation underlying both fleeting individual ontogeny phylogeny across

Language: Английский

Citations

36

Inner speech as language process and cognitive tool DOI Creative Commons
Charles Fernyhough, Anna M. Borghi

Trends in Cognitive Sciences, Journal Year: 2023, Volume and Issue: 27(12), P. 1180 - 1193

Published: Sept. 26, 2023

Many people report a form of internal language known as inner speech (IS). This review examines recent growth research interest in the phenomenon, which has broadly supported theoretical model IS is functional process that can confer benefits for cognition range domains. A key insight to have emerged years an embodied experience characterized by varied subjective qualities, be usefully modeled artificial systems and whose neural signals potential decoded through advancing brain–computer interface technologies. Challenges future include understanding individual differences mapping function across subtypes.

Language: Английский

Citations

31

The ‘L-factor’: Language as a transdiagnostic dimension in psychopathology DOI Creative Commons
Wolfram Hinzen, Lena Palaniyappan

Progress in Neuro-Psychopharmacology and Biological Psychiatry, Journal Year: 2024, Volume and Issue: 131, P. 110952 - 110952

Published: Jan. 26, 2024

Thoughts and moods constituting our mental life incessantly change. When the steady flow of this dynamics diverges in clinical directions, possible pathways involved are captured through discrete diagnostic labels. Yet a single vulnerable neurocognitive system may be causally psychopathological deviations transdiagnostically. We argue that language viewed as integrating cortical functions is best current candidate, whose forms breakdown along its different dimensions then manifest symptoms – from prosodic abnormalities rumination depression to distortions speech perception verbal hallucinations, meaning content delusions, or disorganized formal thought disorder. Spontaneous connected provides continuous objective readouts generating highly accessible bio-behavioral marker with potential revolutionizing neuropsychological measurement. This argument turns into transdiagnostic ‘L-factor’ providing an analytical mechanistic substrate for previously proposed latent general factors psychopathology (‘p-factor’) cognitive functioning (‘c-factor’). Together immense practical opportunities afforded by rapidly advancing natural processing (NLP) technologies abundantly available data, suggests new era translational psychiatry, which both rethought together.

Language: Английский

Citations

11

Developing the Mental Effort and Load–Translingual Scale (MEL-TS) as a Foundation for Translingual Research in Self-Regulated Learning DOI Creative Commons
Tino Endres, Lisa Bender, Stoo Sepp

et al.

Educational Psychology Review, Journal Year: 2025, Volume and Issue: 37(1)

Published: Jan. 10, 2025

Abstract Assessing cognitive demand is crucial for research on self-regulated learning; however, discrepancies in translating essential concepts across languages can hinder the comparison of findings. Different often emphasize various components and interpret certain constructs differently. This paper aims to develop a translingual set items distinguishing between intentionally invested mental effort passively perceived load as key differentiations broad range learning situations, they occur learning. Using mixed-methods approach, we evaluated content, criterion, convergent, incremental validity this scale different languages. To establish content validity, conducted qualitative interviews with bilingual participants who discussed their understanding load. These translated back-translated established new from cognitive-demand literature into English, Dutch, Spanish, German, Chinese, French. criterion preregistered experiments using German versions scale. Within those experiments, validated manipulations first-language participants. In within-subjects design eight measurements ( N = 131), demonstrated scale’s by showing sensitivity differences task complexity, extraneous manipulation, motivation complex tasks. We found evidence convergent shown medium-size correlations measures. offer common foundation research. As best practice, recommend four within reference point evaluation.

Language: Английский

Citations

1

The influence of three-gendered grammatical systems on simultaneous bilingual cognition: The case of Ukrainian-Russian bilinguals DOI Creative Commons
Oleksandra Osypenko, Silke Brandt, Panos Athanasopoulos

et al.

Language and Cognition, Journal Year: 2025, Volume and Issue: 17

Published: Jan. 1, 2025

Abstract This paper examines the linguistic relativity principle (Whorf, 1956) by investigating impact of grammatical gender on cognition in simultaneous bilinguals three-gendered Ukrainian and Russian. It whether speakers languages show effects categorisation, empirically addressing claims that such are insignificant due to presence neuter (Sera et al., 2002). We conducted two experiments using a similarity judgement paradigm while manipulating stimuli (Phillips & Boroditsky, 2003). Experiment 1, including gender, revealed no significant effects, compatible with earlier studies languages. Conversely, 2, excluding stimuli, showed language effects. Bilingual participants rated pairs as more similar when genders both were congruent biological sex character. Significant also found for mismatching Participants higher proficiency noun was character’s sex, incongruent Our findings thus provide first empirical demonstration exclusion online induces

Language: Английский

Citations

1