発話シーンからのキーフレーム検出とキーフレームに基づく単語読唇  [in Japanese] Keyframe Extraction from Utterance Scene and Keyframe-based Word Lip Reading  [in Japanese]

Access this Article

Search this Article

Author(s)

Abstract

In this paper, we propose the new keyframe-based lip reading method which does not need the advanced registration of an utterance scene. To extract keyframe, we apply the frame subtraction method and extract frame which the difference value is the local minimum as the keyframe. We compute thirteen shape features from the five lip regions of the extracted keyframe. Then we apply a discriminant analysis to mouth shape recognition. We generate a code sequence based on a mouth shape recognition result. Moreover, in accordance with several rules, we generate candidate code sequences. Finally, we apply DP matching using two kinds of code sequence of based on keyframe and candidate, and select the similar code sequence as the result word. We set Japanese 19 words as the target. We took four speakers' utterance scene. We carried out three experiments of the keyframe extraction, the mouth shape recognition, and the word recognition. As a result, we obtained average recognition rate of 53.9%. Although there was individual difference, one speaker obtained 72.1% of the highest recognition rate.

Journal

  • IEEJ Transactions on Electronics, Information and Systems

    IEEJ Transactions on Electronics, Information and Systems 131(2), 418-424, 2011-02-01

    The Institute of Electrical Engineers of Japan

References:  21

Cited by:  3

Codes

  • NII Article ID (NAID)
    10027804091
  • NII NACSIS-CAT ID (NCID)
    AN10065950
  • Text Lang
    JPN
  • Article Type
    Journal Article
  • ISSN
    03854221
  • NDL Article ID
    10952974
  • NDL Source Classification
    ZN31(科学技術--電気工学・電気機械工業)
  • NDL Call No.
    Z16-795
  • Data Source
    CJP  CJPref  NDL  J-STAGE 
Page Top