Keyframe Extraction from Utterance Scene and Keyframe-based Word Lip Reading

DOI Web Site Web Site 1 Citations 26 References

Bibliographic Information

Other Title
  • 発話シーンからのキーフレーム検出とキーフレームに基づく単語読唇
  • ハツワ シーン カラ ノ キーフレーム ケンシュツ ト キーフレーム ニ モトズク タンゴドクシン

Search this article

Abstract

In this paper, we propose the new keyframe-based lip reading method which does not need the advanced registration of an utterance scene. To extract keyframe, we apply the frame subtraction method and extract frame which the difference value is the local minimum as the keyframe. We compute thirteen shape features from the five lip regions of the extracted keyframe. Then we apply a discriminant analysis to mouth shape recognition. We generate a code sequence based on a mouth shape recognition result. Moreover, in accordance with several rules, we generate candidate code sequences. Finally, we apply DP matching using two kinds of code sequence of based on keyframe and candidate, and select the similar code sequence as the result word. We set Japanese 19 words as the target. We took four speakers' utterance scene. We carried out three experiments of the keyframe extraction, the mouth shape recognition, and the word recognition. As a result, we obtained average recognition rate of 53.9%. Although there was individual difference, one speaker obtained 72.1% of the highest recognition rate.

Journal

Citations (1)*help

See more

References(26)*help

See more

Related Projects

See more

Keywords

Details 詳細情報について

Report a problem

Back to top