Multimodal speech synthesis - 5MMPLTVS

Goal(s)

This course gives an introduction to speech technologies (speech coding, synthesis and recognition) that process audible (acoustic signal) and visible (lip movements, etc.) consequences of underlying articulatory movements (produced by the jaw, the tongue, the larynx, the velum, etc.). We first introduce the basic knowledge in physiology, phonetics, phonology and linguistics necessary to understand the mechanisms underlying speech production, perception and comprehension. Then fundamentals in signal processing, representation and modeling are presented. We go on with a review of current systems that enable spoken interaction embodied by anthropoids or virtual conversational agents.

Content(s)

• Multimodal speech production and perception
• Phonological structures of world’s languages. Example of French
• Phonetic representations and speech processing
• Text-to-speech systems and facial animation
• Audiovisual speech recognition
• Systems for situated verbal interaction

None