Marta Fernández-Villanueva (Universitat de Barcelona)

Multimodal annotation of discourse markers in transcultural Spanish-German encounters

DMs – understood in its broadest sense including modal particles, responsives and discursive connectives (Blühdorn, Deppermann, Helmer, & Spranz-Fogasy, 2017) – have many functions and therefore operate at different metafunctional levels (Crible & Degand, 2017). Among others, they draw the attention to some important constructions for the discourse organization and the interaction, offering indications regarding information flow, turn taking and discourse organization, managing it by denoting the speaker’s attitude in talk in interaction.
In transcultural encounters, defined as attentional focused gatherings (Goffman 1961, p.18), they are specially tricky, because they implicitly indicate a potential interpretation regarding the speakers relationship (Fraser, 2009), playing therefore a major role in specific intercultural competences such as achieving understanding and rapport management (Spencer-Oatey & Franklin, 2009; Spencer-Oatey, 2011). When participants in transcultural encounters are not able to notice the different functions DMs have, they will have difficulties to successfully interpret speaker’s stance, positioning and interpersonal comments in the ongoing interaction (Tracy, Randles, Steckler, Crockett, & Cuddy, 2015) and to react accordingly to negotiate meaning and “find ways in which people’s subjective perceptions of (dis)harmony or smoothness-turbulence in interpersonal relations(…) are (mis)managed” (Spencer-Oatey & Franklin, 2009, p. 102).
Nevertheless, as Hata (2016) points out, few attention has been paid to the fact that human communication is multimodal and any discourse analysis should include all the resources participants are using: spoken language, but also gesture, gaze and body arrangements in interactional spaces. In the case of DMs, that multimodal approach is used scarcely but increasingly, see for instance (Ferré, 2011; Baiat, Coler, Pullen, Tienkouw, & Hunyadi, 2013; Fernández-Villanueva & Jungbluth, 2016).
The multimodality maximizes the participants’ mutual monitoring of their “attentional focused gathering” on two different but equally important axes: the horizontal axis of sequentially successive moments, which have received primary research attention, both in conceptualization and in transcription practice, and the vertical axis of simultaneity of participants’ actions (Erickson, 2017:60). Moreover, Mondada (2016, 2018) highlights the diversity of multimodal practices involved: movements not only of the upper (head, gesture) but also the lower (feet, legs, posterior) parts of the body, haptic contacts touching objects and coparticipants and so on, increasing the complexity of the transcription and annotation. Fixing the relevant details in this complex multilayered finely tuned coordination is already an invaluable analytical exercise, therefore reflecting on the process of making a multimodal transcript is a matter of vital importance (Slembrouck, 2007; Bezemer, 2014)).
The aim of this contribution is twofold: firstly briefly summarize the importance of multimodal analysis in the study of discourse markers according to recent literature, and secondly, using sequences of achieving understanding and rapport management in transcultural encounters in Spanish, Catalan and German from the VARKOM and the InCriT corpora (Marta Fernández-Villanueva & Strunk, 2009), describe and discuss the possibilities of their multimodal transcription and analysis: from reviewing multimodal frameworks and considering rhetorical status of the transcript, to defining purpose and focus of transcript and creating the transcript template in ELAN, in order to annotate the transcript and draw conclusions.

• Baiat, G. E., Coler, M., Pullen, M., Tienkouw, S., & Hunyadi, L. (2013). Multimodal Analysis of "well" as Discourse Marker in Conversation: A Pilot Study. In CogInfoCom 2013 • 4th IEEE International Conference on Cognitive Infocommunications (pp. 283–287). Budapest.
• Bezemer, J. (2014). Multimodal Transcription: A case study. In S. Norris & C. D. Maier (Eds.), Interactions, images and texts: A reader in multimodality (pp. 155–170). Berlin [etc.] : De Gruyter Mouton.
• Blühdorn, H., Deppermann, A., Helmer, H., & Spranz-Fogasy, T. (Eds.). (2017). Diskursmarker im Deutschen. Reflexionen und Analysen. Göttingen: Verlag für Gesprächsforschung. Retrieved from http://www.verlag-gespraechsforschung.de/2017/pdf/ diskursmarker.pdf.
• Crible, L., & Degand, L. (2017). Polysemy and polyfunctionality: A two-dimensional account of discourse markers. Retrieved from https://dial.uclouvain.be/pr/boreal/object/boreal:185465#?
• Erickson, F. (2017). Conceiving, noticing, and transcribing multi-modality in the study of social interaction as a learning environment. Linguistics and Education, 41, 59–61.
• Fernández-Villanueva, M., & Jungbluth, K. (2016). Beyond language boundaries: Multimodal use in multilingual contexts. Berlin [etc.]: De Gruyter Mouton.
• Fernández-Villanueva, M., & Strunk, O. (2009). Das Korpus Varkom – Variation und Kommunikation in der gesprochenen Sprache. Deutsch Als Fremdsprache, 46(2), 67–73.
• Ferré, G. (2011). Multimodal Analysis of Discourse Marker “donc”, “alors” and “en fait” in Conversational French. In ICPhS VXII (pp. 671–674). Hong Kong.
• Fraser, B. (2009). An Account of Discourse Markers. International Review of Pragmatics, 1(2), 293–320.
• Hata, K. (2016). On the Importance of the Multimodal Approach to Discourse Markers. International Review of Pragmatics, 8(1), 36–54.
• Mondada, L. (2016). Challenges of multimodality: Language and the body in social interaction. Journal of Sociolinguistics, 20(3), 336–366.
• Mondada, L. (2018). Multiple Temporalities of Language and Body in Interaction: Challenges for Transcribing Multimodality. Research on Language and Social Interaction, 51(1), 85–106.
• Slembrouck, S. (2007). Transcription — the extended directions of data histories: a response to M. Bucholtz’s “Variation in Transcription.” Discourse Studies, 9(6), 822–827.
• Spencer-Oatey, H. (2011). Conceptualising “the relational” in pragmatics: Insights from metapragmatic emotion and (im)politeness comments. Journal of Pragmatics, 43(14), 3565–3578.
• Spencer-Oatey, H., & Franklin, P. (2009). Intercultural Interaction. A Multidisciplinary Approach to Intercultural Communication. Basingstoke: Palgrave Macmillan.
• Tracy, J. L., Randles, D., Steckler, C. M., Crockett, M. J., & Cuddy, A. (2015). The nonverbal communication of emotions. Current Opinion in Behavioral Sciences, 3, 25–30.

Biodata

Marta Fernández-Villanueva és Doctora en Filologia Alemanya i professora titular de Lingüística i Gramàtica Alemanya al Departament de Filologia Anglesa i Alemanya. Les seves línies principals de recerca són la lingüística textual, l’anàlisi del discurs i la interfície entre gramàtica i lèxic, especialment pel que fa a la negociació de sentit en la comunicació transcultural alemany-català-castellà i les seves implicacions per a l’adquisició de l’alemany com a llengua estrangera.