International Research Institute MICA, Hanoi University of Science and Technology

Vietnamese social affect: expression and perception

 Attitudes or social affects are strongly implied in face-to-face interaction, and specifically into the socio-cultural aspects of language. In speech interaction, social affects are expressed through prosodic patterns, called expressive speech. While Vietnamese is increasingly well-endowed in terms of digital linguistic resources, there remains considerable room for progress in the field of expressive speech. This paper presents our work on Vietnamese social affects.
The first part of this paper describes the construction of an audio-visual corpus of Vietnamese attitudes which contains sixteen different expressions. Based on this corpus, a series of perceptual tests was carried out with native and non-native (French) listeners to study Vietnamese attitudinal perception and expressions in a cross-cultural context.
In the second part of the paper, we present two perception tests. The first perception test is to study the role and integration of the audio, visual, and audio-visual information in the listener’s perception of the speaker’s attitudes. Experimental results reveal the fact that the influential factors on attitude recognition include the modality of presentation (audio, visual and audio-visual) and the attitudinal expression itself. These results also allow us to investigate the similarities and cross-cultural specificities between Vietnamese and French attitudes. A further perception test was carried out using sentences with different lexical tone sequences to study possible interactions between Vietnamese tones and the perception of attitudes. The results show that non-native listeners interpret the local phonetic characteristics of tones as cues to speaker attitude, along with the global cues of attitude patterns.