Can you tell if tongue movements are real or synthetic?
2009 (English)In: Proceedings of AVSP, 2009Conference paper (Refereed)
We have investigated if subjects are aware of what natural tongue movements look like, by showing them animations based on either measurements or rule-based synthesis. The issue is of interest since a previous audiovisual speech perception study recently showed that the word recognition rate in sentences with degraded audio was signiﬁcantly better with real tongue movements than with synthesized. The subjects in the current study could as a group not tell which movements were real, with a classiﬁcation score at chance level. About half of the subjects were signiﬁcantly better at discriminating between the two types of animations, but their classiﬁcation score was as often well below chance as above. The correlation between classiﬁcation score and word recognition rate for subjects who also participated in the perception study was very weak, suggesting that the higher recognition score for real tongue movements may be due to subconscious, rather than conscious, processes. This ﬁnding could potentially be interpreted as an indication that audiovisual speech perception is based onarticulatory gestures.
Place, publisher, year, edition, pages
augmented reality, tongue reading, visual speech synthesis, data-driven animation
Computer Science Language Technology (Computational Linguistics)
IdentifiersURN: urn:nbn:se:kth:diva-52102OAI: oai:DiVA.org:kth-52102DiVA: diva2:465397
International Conference on Auditory-Visual Speech Processing 2009, 10-13 September 2009, University of East Anglia, Norwich, UK
tmh_import_11_12_14. QC 201112292011-12-142011-12-142011-12-29Bibliographically approved