A Review of the Roll of Lip Reading in Verbal Communication and Lip Reading Techniques

Document Type : Review Article

Authors

1 Student Research Committee. MSc in Audiology. Department of Audiology, School of Rehabilitation, Shahid Beheshti University of Medical Sciences, Tehran, Iran

2 Physiotherapy Research Center, Department of Audiology, School of Rehabilitation, Shahid Beheshti University of Medical Sciences, Tehran, Iran

Abstract

Background and Aim: Speech is a multi-dimensional process. Human speech perception mechanism uses visual and auditory signals to decode the produced speech. While a normal speech is heard by a person, it is also simultaneously visualized, as the processing areas in the brain combine the visual and auditory data related to the speech. The visual data received from the face and mouth of the speaker has an important role in speech comprehension. Lip reading in word means the cognition of spoken words using data such as lip movements. The lips provide more than half of the visual data. This is considered as a normal skill in individuals with normal hearing. In fact, the goal of lip reading is to increase the independence level in those with hearing defect. When lip reading is required (e.g. a sudden deafness), lip reading softwares can be used to teach lip reading. The current review was an attempt to reveal the importance of lip reading in verbal communication (speech recognition) and to propound lip reading techniques and softwares based on the latest clinical researches.
Materials and Methods: The present review study used the latest articles and books issued from 1979 to 2015 in the field of lip reading in verbal communication and speech recognition selected from Googlescholar, SID, Scopus, and Pubmed data banks.
Conclusion: It was found that visual data of the speaker's mouth and face plays an important role in speech comprehension by the audience; in fact, lip reading provides the visual data. It is adived that the role of vision in decoding speech lip reading be used in auditory rehabilitation and especially in verbal communication enhancement. There are also some techniques and softwares for teaching this skill to the individuals suffering from auditory deficit.

Keywords

Main Subjects

References

  1. Schwartz J-L, Berthommier F, Savariaux C. Seeing to hear better: evidence for early audio-visual interactions in speech identification. Cognition. 2004;93(2):B69-B78.##
  2. Ortiz IdlRR. Lipreading in the prelingually deaf: what makes a skilled speechreader? The Spanish journal of psychology. 2008;11(02):488-502.##
  3. Nureddini Seyyedeh Zeynab, Mohammadzadeh Ali, Ashrafi majid,Tabatabai Seyyed Mehdi, Jalilvand karimi Leyla.Recognition of stop consonants in babble noise in normal hearing individuals. Aud Vest Res (2015); 24 (1): 31-37.##
  4. Nureddini Seyyedeh Zeynab, Mohammadzadeh Ali,Tabatabai Seyyed Mehdi. Comparsion the recognition score of stop and fricative consonants in babble noise. J Rehab Med. 2015; 4(1): 134-141.##
  5. Deypir M, Alizadeh S, Zoughi T, Boostani R. Boosting a multi-linear classifier with application to visual lip reading. Expert Systems with Applications. 2011;38(1):941-8.##
  6. Movallali, G., review of half century of studies about diary speech reading(in persion). Audiology, 2002. 11(1): p. 65-76.##
  7. Meier U, Stiefelhagen R, Yang J, Waibel A. Towards unrestricted lip reading. International Journal of Pattern Recognition and Artificial Intelligence. 2000;14(05):571-85.##
  8. Maidment DW, Macken B, Jones DM. Modalities of memory: Is reading lips like hearing voices? Cognition. 2013;129(3):471-93.##
  9. Feld J, Sommers M. There goes the neighborhood: Lipreading and the structure of the mental lexicon. Speech communication. 2011;53(2):220-8.##
  10. Movalleli, G., sara lip reading tests(in persion). 1st.Iran:University of social welfare and rehabilitation sciences;2014. p. 5-16.##
  11. Utley J. A test of lip reading ability. Journal of Speech Disorders. 1946;11(2):109-16.##
  12. Bernstein LE, Auer Jr ET, Takayanagi S. Auditory speech detection in noise enhanced by lipreading. Speech Communication. 2004;44(1):5-18.##
  13. Ma WJ, Zhou X, Ross LA, Foxe JJ, Parra LC. Lip-reading aids word recognition most in moderate noise: a Bayesian explanation using high-dimensional feature space. PloS one. 2009;4(3):e4638.##
  14. Dodd B. Lip readingin infants: Attention to speech presented in-and out-of-synchrony. Cognitive Psychology. 1979;11(4):478-84.##
  15. Smith RC, Kitchen DW. Lipreading performance and contextual cues. Journal of Communication Disorders. 1972;5(1):86-90.##
  16. Movalleli, G., Lip reading(speech reading) instructing(in persion). Audiology, 2004. 13(1): p. 45-50.##
  17. Devergie A, Grimault N, Gaudrain E, Healy EW, Berthommier F. The effect of lip-reading on primary stream segregation. The Journal of the Acoustical Society of America. 2011;130(1):91-283##
  18. Feld JE, Sommers MS. Lipreading, processing speed, and working memory in younger andolder adults. Journal of Speech, Language, and Hearing Research. 2009;52(6):1555-65.##
  19. Baddeley A. Working memory and language: An overview. Journal of communication disorders. 2003;36(3):189-208.##
  20. Lander K, Capek C. Investigating the impact of lip visibility and talking style onspeechreading performance. Speech Communication. 2013;55(5):600-5.##
  21. Putzar L, Goerendt I, Heed T, Richard G, Büchel C, Röder B. The neural basis of lip-reading capabilities is altered by early visual deprivation. Neuropsychologia. 2010;48(7):2158-66.##
  22. Baart M, Vroomen J. Do you see what you are hearing? Cross-modal effects of speech sounds on lipreading. Neuroscience letters. 2010;471(2):100-3.##
  23. Bernstein LE, Tucker PE, Demorest ME. Speech perception without hearing. Perception & Psychophysics. 2000;62(2):233-52.##
  24. Alegria J. The origin and functions of phonological representations in deaf people. Reading and spelling: Development and disorders. 1998:263-86.##
  25. Alegria J, Charlier BL, Mattys S. The role of lip-reading and cued speech in the processing of phonological information in French-educated deaf children. European Journal of Cognitive Psychology. 1999;11(4):451-72.##
  26. Neef NA, Iwata BA. The development of generative lipreading skills in deaf persons using cued speech training. Analysis and intervention in developmental disabilities. 1985;5(4):289-305.##
  27. Yun-Long Lay a, Chung-Ho Tsai a, Hui-Jen Yang b,*, Chern-Sheng Lin c, Chuan-Zhao Lai. The application of extension neuro-network on computer-assisted lip-reading recognition for hearing impaired. Expert Systems with Applications 34 (2008) 1465–1473.##
  28. Wai Chee Yau, Dinesh Kant Kumar, and Tharangini Chinnadurai. Lip-Reading Technique UsingSpatio-Temporal Templates and Support VectorMachines.Lecture Notes in Computer Science Volume 5197, 2008, pp 610-617.##
  29. Sharmila Sengupta, Arpita Bhattacharya, Pranita Desai, Aarti Gupta. Automated Lip Reading Technique for Password Authentication. International Journal of Applied Information Systems (IJAIS); Volume 4– No.3, September 2012.##
  30. M.W. Mak,W.G.Allen.Lip-motion analysis for speech segmentation in noise.speech communication 14(1994) 279-296.##
  31. Salah Werda, Walid Mahdi and Abdelmajid Ben Hamadou. Lip Localization and Viseme Classificationfor Visual Speech Recognition. International Journal of Computing & Information Sciences Vol.5, No.1, April 2007, On-Line.##
  32. Massaro Dominic, W. and D.G. Stork, Speech recognition and sensory integration. Computer Standards & Interfaces, 1999. 20(6): p. 450.##
  33. Campbell, R., B. Dodd, and D.K. Burnham, Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech. Vol. 2. 1998: Psychology Press.##
  34. Stork, D.G. and M.E. Hennecke, Speechreading by humans and machines: models, systems, and applications. Vol. 150. 1996: Springer.##
  35. Barbara Dodd, Geoff Plant, and Mark Gregory. Teaching lip-reading: The efficacy of lessons on video. 1989, Vol. 23, No. 3, Pages 229-238.##
  36. Black ,J.W.Self-administered training in lip-reading.Journal of Speech and Hearing Disorders.1963,28,183-186##
  37. Jamie Berke. Lipreading Software. Updated June 18, 2014. Availableat:URL:http://deafness.about.com/od/accessibility/qt/lipreading_software.htm

##

The Scientific Journal of Rehabilitation Medicine
Volume 5, Issue 4
January and February 2017
Pages 250-259

Files

History

  • Receive Date: 25 June 2015
  • Revise Date: 12 January 2016
  • Accept Date: 02 July 2016
  • First Publish Date: 21 December 2016

Share

How to cite

Statistics