An Empirical Study on the Effectiveness of Gesture in L2 Listening Class

doi:10.12002/j.bisu.134

Abstract

Abstract:

Gestures are gaining attention in second language acquisition for their close connection with speech, despite the fact that empirical research on gestures’ validity is in deficiency. Researchers have been especially interested in the input effect of gestures, i.e. whether gestures play a positive role in L2 learners’ comprehension. To study gestures’ validity in listening comprehension, this research selected subjects from 183 L2 learners and divided them into high and low score groups, testing their listening skills under three conditions, audio, audio & face, and audio, face & hand/arm gestures. Subjects achieved highest score after watching videos containing hand/arm gestures, and subjects in low score group behaved significantly better. And gestures helped subjects in low score group understand both concrete things and abstract metaphors. From the perspective of the Sociocultural Theory, this research verifies that gestures enhance language learners’ competence in constructing meaning during communication, and gestures are valid mediation in students’ listening comprehension. Gestures help listeners to accurately segment speech flows and draw visual attention to important information. Gestures also represent, explain and complement the meaning of speech, making the intended meaning more concrete and imaginable, and subjects can understand the meaning of unfamiliar words by watching gestures. Besides, the appearance of gestures enables subjects to code information in both phonological and visual channels, and avoid cognitive overload in working memory. Our research results also indicate that gestures have significant mediation effect. Gestures are useful communicative tools, and L2 learners often rely too much on speech information while ignore the effect of gestures. Gestures also positively mediate in comprehension of metaphoric expressions, providing listeners with dynamic and motoric comprehension process. L2 teachers and learners should pay more attention to the use of gestures, so that L2 learners can more efficiently participate in social activities.The research results offer a method of enriching teaching mediation tools and constructing multimodal L2 classrooms.

Keywords: listening comprehension; gesture; mediation; listening teaching

摘要：

手势与言语密切相关,在二语习得领域逐渐受到关注,但对其有效性的实证研究相对匮乏。为考察手势语在听力理解中的作用,本研究从183名英语学习者中选取高、低分两组受试,在音频、音频+面部、音频+面部+手/臂三种实验条件下进行听力测试。结果显示,观看含有手/臂手势视频的受试成绩最高,低分组受试成绩提高更为显著,且手势对低分组受试理解具体概念和抽象隐喻的能力均有所促进。结合社会文化理论,本文认为,手势语能够提高英语学习者在交际中构建意义的能力,是听力理解的有效中介。本研究的结果为丰富教学中介工具、构建多模态外语课堂提供了参考。

关键词: 听力理解, 手势语, 中介, 听力教学

Xiaoyun ZHONG,Yuan GAO. An Empirical Study on the Effectiveness of Gesture in L2 Listening Class[J]. Journal of Beijing International Studies University, 2018, 40(4): 115-130.

钟晓云,高原. 英语听力教学中手势语有效性的实证研究[J]. 北京第二外国语学院学报, 2018, 40(4): 115-130.

Figures/Tables 8

References

[1]	Adams T W. Gesture in Foreigner Talk [M]. Boston: University of Pennsylvania Press, 1998.
[2]	Allen L Q . The effects of emblematic gestures on the development and access of mental representations of french expressions[J]. The Modern Language Journal, 1995,79(4):521~ 529.
[3]	Allen L Q . Nonverbal accommodations in foreign language teacher talk[J]. Applied Language Learning, 2000,11(1):155~ 176.
[4]	Bancroft W J . Nonverbal communication: How important is it for the language teacher?[J]. Mosaic, 1997,4(4):3~ 12.
[5]	Barnett M A . Replacing teacher talk with gestures: Nonverbal communication in the foreign language classroom[J]. Foreign Language Annals, 1983,16(3):173~ 176.
[6]	Beattie G&Shovelton H. Mapping the range of information contained in the iconic hand gestures that accompany spontaneous speech[J]. Journal of Language and Social Psychology, 1999,34(4):438~ 462.
[7]	Belhiah H . Using the hand to choreograph instruction: On the functional role of gesture in definition talk[J]. The Modern Language Journal, 2013,97(2):417~ 434.
[8]	Cabrera M P&Martinez P B. The effects of repetition, comprehension checks and gestures on primary school children in an EFL situation[J]. ELT Journal, 2001,55(3):281~ 288.
[9]	Canale M&Swain M. Theoretical basis of communicative approaches to second language teaching and testing[J]. Applied Linguistics, 1980,1(1):1~ 47.
[10]	Cekaite A . Soliciting teacher attention in an L2 classroom:Affect displays, classroom artefacts, and embodied action[J]. Applied Linguistics, 2009,30(1):26~ 48.
[11]	Cohen L&Otterbein N. The mnemonic effect of speech gestures: Pantomimic and non-pantomimic gestures compared[J]. European Journal of Cognitive Psychology, 1992,4(2):113~ 139.
[12]	Cook S W&Goldin-Meadow S. The role of gesture in learning: Do children use their hands to change their minds?[J]. Journal of Cognition and Development, 2006,7(2):211~ 232.
[13]	Crowder E M . Gestures at work in sense-making science talk[J]. The Journal of the Learning Sciences, 1996,5(3):173~ 208.
[14]	Flege J. Assessing constraints on second-language segmental production and perception [C]// Meyer A & Schiller N. Phonetics and Phonology in Language Comprehension and Production: Differences and Similarities. Berlin: Mouton de Gruyter, 2003.
[15]	Galati A&Brennan S E. Speakers adapt gestures to addressees’ knowledge: Implications for models of co-speech gesture[J]. Language, Cognition, and Neuroscience, 2014,29(4):435~ 451.
[16]	Gibbs R W. Embodiment and Cognitive Science [M]. Cambridge: Cambridge University Press, 2006.
[17]	Goldin-Meadow S&Wagner S M. How our hands help us learn[J]. Trends in Cognitive Sciences, 2005,9(5):234~ 241.
[18]	Gregersen T Olivares-Cuhat G&Storm J..An examination of L1 and L2 gesture use: What role does proficiency play?[J]. The Modern Language Journal, 2009,93(2):195~ 208.
[19]	Gullberg M . Handling discourse: Gestures, reference tracking, and communication strategies in early L2[J]. Language Learning, 2006a,56(1):155~ 196.
[20]	Gullberg M . Some reasons for studying gesture and second language acquisition[J]. IRAL, 2006b,44(2):103~ 124.
[21]	Gullberg M&McCafferty S G. Introduction to gesture and SLA: Toward an integrated approach[J]. SSLA, 2008,30(2):133~ 146.
[22]	Gullberg M de Bot K&Volterra V. Gestures and some key issues in the study of language development[C]// Gullberg M & de Bot K. Gestures in Language Development. Amsterdam: John Benjamins Publishing Company, 2010.
[23]	Guvendir E . The role of non-verbal behavior of teachers in providing students corrective feedback and their consequences[J]. Sino-US English Teaching, 2011(9):577~ 591.
[24]	Harris T . Listening with your eyes:The importance of speech-related gestures in the language classroom[J]. Foreign Language Annuals, 2003(2):180~ 187.
[25]	Hudson N . Teacher Gesture in a Post-secondary English as a Second Language Classroom: A Sociocultural Approach[D]. Reno: University of Nevada, 2011.
[26]	Kellerman S . “I see what you mean”: The role of kinesic behaviour in listening and implications for foreign and second language learning.[J]. Applied Linguistics, 1992,13(3):239~ 257.
[27]	Kellogg P&Lawson B. Envoy: Your Personal Guide to Classroom Management[M]. Battle Ground: Michael Grinder, 1993.
[28]	Kelly S D & Barr D J, Church R B&Lynch K. Offering a hand to pragmatic understanding: The role of speech and gesture in comprehension and memory[J]. Journal of Memory and Language, 1999,40(4):577~ 592.
[29]	Kita S&Davies T S. Competing conceptual representations trigger co-speech representational gestures[J]. Language and Cognitive Processes, 2009,24(5):795~ 804.
[30]	Lakoff G&Johnson M. Metaphors We Live by [M]. Chicago: University of Chicago Press, 1980.
[31]	Lantolf J P . Second language learning as a meditational process[J]. Language Teaching, 2000,33(2):79~ 96.
[32]	Lazaraton A . Gesture and speech in the vocabulary explanations of one ESL teacher: A microanalytic inquiry[J]. Language Learning, 2004,54(1):79~ 117.
[33]	Le H T X&Gonzales C. A microanalysis of gestures in classroom talk[J]. Hawaii Pacific University TESOL Working Paper Series, 2012,10:13~ 29.
[34]	Lee J . Gesture and private speech in second language acquisition[J]. SSLA, 2008,30(2):169~ 190.
[35]	Macedonia M&Kn?sche T R. Body in mind: How gestures empower foreign language learning[J]. Mind, Brain and Education, 2011,8(4):196~ 211.
[36]	Macedonia M & Kriegstein K. Gestures enhance foreign language learning[J]. Biolinguistics, 2012(64):393~ 416.
[37]	Macedonia M Müller K&Friederici A D..The impact of iconic gestures on foreign language word learning and its neural substrate[J]. Human Brain Mapping, 2011,32(6):982~ 998.
[38]	Marstaller L&Burianová H. A common functional neural network for overt production of speech and gesture[J]. Neuroscience, 2015,284:29~ 41.
[39]	Mayberry R I&Nicoladis E. Gesture reflects language development: Evidence from bilingual children[J]. Current Directions in Psychological Science, 2000,9(6):192~ 196.
[40]	McCafferty S G . Nonverbal expression and L2 private speech[J]. Applied Linguistics, 1998,19(1):73~ 96.
[41]	McCafferty S G . Gesture and creating zones of proximal development for second language learning[J]. The Modern Language Journal, 2002,86(2):192~ 203.
[42]	McCafferty S G . Space for cognition: Gesture and second language learning[J]. International Journal of Applied Linguistics, 2004,14(1):148~ 165.
[43]	McCafferty S G . Gesture and the materialization of second language prosody[J]. IRAL, 2006,44(2):197~ 209.
[44]	McCafferty S G . Mimesis and second language acquisition: A sociocultural perspective[J]. SSLA, 2008a,30(30):147~ 167.
[45]	McCafferty S G. Material foundations for second language acquisition: Gesture, metaphor and internalization [C]// McCafferty S G & Stam G . Gesture: Second Language Acquisition and Classroom Research. New York: Routledge, 2008b.
[46]	McCafferty S G&Stam G. Preface [C]// McCafferty S G & Stam G . Gesture :Second Language Acquisition and Classroom Research. New York :Routledge, 2008.
[47]	McNeill D . Hand and Mind: What Gestures Reveal about Thought [M]. Chicago: University of Chicago Press, 1992.
[48]	McNeill D . Gesture and Thought [M]. Chicago: University of Chicago Press, 2005.
[49]	Melinger A&Levelt W J M. Gesture and the communicative intention of the speaker[J]. Gesture, 2004,4(2):119~ 141.
[50]	Mohan B&Helmer S. Context and second language development: Preschooler’s comprehension of gestures[J]. Applied Linguistics, 1988,9(3):275~ 292.
[51]	Mori J&Hayashi M. The achievement of intersubjectivity through embodied completions: A study of interactions between first and second language speakers[J]. Applied Linguistics, 2006,27(2):195~ 219.
[52]	Negi J S . The role of teachers’ non-verbal communication in ELT classroom[J]. Journal of NELTA, 2009(1-2):101~ 110.
[53]	Negueruela E&Lantolf J P. The dialectics of gesture in the construction of meaning in second language oral narratives [C]// McCafferty S G & Stam G. Gesture: Second Language Acquisition and Classroom Research. New York: Routledge, 2008.
[54]	Novack N&Goldin-Meadow S. Learning from gesture: How our hands change our minds[J]. Educational Psychology Review, 2015,27(3):405~ 412.
[55]	Parrill F&Kimbara I. Seeing and hearing double: The influence of mimicry in speech and gesture on observers[J]. Journal of Nonverbal behavior, 2006 ( 4):157~ 166.
[56]	Paivio A. Mental Representations [M]. New York: Oxford University Press, 1986.
[57]	Ping R&Goldin-Meadow S. Gesturing saves cognitive resources when talking about nonpresent objects[J]. Cognitive Science, 2010,34(4):602~ 619.
[58]	Richards J C&Schmidt R. Longman Dictionary of Language Teaching and Applied Linguistics[M]. London: Pearson Education, 2002.
[59]	Roth W . Gestures: Their role in teaching and learning[J]. Review of Educational Research, 2001,71(3):365~ 392.
[60]	Roth W&Lawless D. Scientific investigations, metaphorical gestures, and the emergence of abstract scientific concepts[J]. Learning and Instruction, 2002,12(3):285~ 304.
[61]	Rowe M Silverman R&Mullan B..The role of pictures and gestures as nonverbal aids in preschooler’s word learning in a novel language[J]. Contemporary Educational Psychology, 2013,38(2):109~ 117.
[62]	Sainsbury P&Wood E. Measuring gesture: Its cultural and clinical correlates[J]. Psychological Medicine, 1977,7(1):63~ 72.
[63]	Seo M S&Koshik I. A conversation analytic study of gesture that engender repair in ESL conversational tutoring[J]. Journal of Pragmatics, 2010,42(8):2219~ 2239.
[64]	Sime D . What do learners make of teachers’ gestures in the language classroom?[J]. International Review of Applied Linguistics in Language Teaching, 2006,44(2):211~ 230.
[65]	Smotrova T&Lantolf J P. The function of gesture in lexically focused L2 instructional conversations[J]. The Modern Language Journal, 2013 ( 2):397~ 416.
[66]	Straube B, Green A Bromberger B&Kircher T..The differentiation of iconic and metaphoric gestures: Common and unique integration processes[J]. Human Brain Mapping, 2011,32(4):520~ 533.
[67]	Sueyoshi A&Hardison D M. The role of gestures and facial cues in second language listening comprehension[J]. Language Learning, 2005,55(4):661~ 699.
[68]	Tarone E. Conscious communication strategies in interlanguage: A progress report [C]// Brown H, Yorio C & Crymes R . On TESOL ’77 Teaching and Learning English as a Second Language . Washington, DC: TESOL, 1977.
[69]	Vygotsky L S. Mind, consciousness, the unconscious [C]// In Vygotsky L S (Ed.). The Collected Works of L. S. Vygotsky (Vol. 1 : Problems of General Psychology). New York: Plenum Press, 1997.
[70]	Wagner P,Malisz Z & Kopps. Gesture and speech in interaction: An overview[J]. Speech Communication, 2014,57(1):209~ 232.
[71]	Wu Y&Coulson S. Co-speech iconic gestures and visuo-spatial working memory[J]. Acta Psychologica, 2014,153:39~ 50.
[72]	Yasui E . Collaborative idea construction: Repetition of gestures and talk in joint brainstorming[J]. Journal of Pragmatics, 2013,46(1):157~ 172.
[73]	Zhao J . Metaphors and Gestures for Abstract Concepts in Academic English Writing[D]. Tuscon: University of Arizona, 2007.
[74]	高原, 钟晓云 . 英语学习者时间语伴隐喻手势的维度研究[J]. 外语教学与研究, 2014(6):879~ 888.
[75]	江桂英, 王容花 . 英语演讲中言语一手势多模态隐喻的融合研究[J]. 外语研究, 2013(5):9~ 16.
[76]	任庆梅 . 英语听力教学[M]. 北京: 外语教学与研究出版社, 2011.
[77]	王艳 . 英语听力教学与研究[M]. 北京: 外语教学与研究出版社, 2012.
[78]	夏宁满 . 多媒体信息输入、认知负荷与英语听力解码[J]. 外语电化教学, 2014(1):33~ 38.
[79]	杨红燕 . 外语课堂教师自发手势研究[J]. 北京第二外国语学院学报, 2015(8):68~ 74.

结果条件	平均分	标准差
音频	7.48	3.87
音频+面部	7.14	3.72
音频+面部+手/臂	9.29	3.63

结果条件	平均分	标准差
音频	7.48	3.87
音频+面部	7.14	3.72
音频+面部+手/臂	9.29	3.63

组别结果条件	低分组		高分组
组别结果条件	平均分	标准差	平均分	标准差
音频	4.29	1.14	10.92	2.50
音频+面部	4.43	2.06	9.86	2.93
音频+面部+手/臂	7.14	2.77	11.43	3.13

组别结果条件	低分组		高分组
组别结果条件	平均分	标准差	平均分	标准差
音频	4.29	1.14	10.92	2.50
音频+面部	4.43	2.06	9.86	2.93
音频+面部+手/臂	7.14	2.77	11.43	3.13

条件条件	条件	平均差
音频条件	面部条件	—0.15
音频条件	面部+手/臂条件	—2.86
面部条件	面部+手/臂条件	—2.71