Modifying Existing Analogy-based N-gram Language Model

• Meng Tian

  IPS, Waseda University

• Yves Lepage

  IPS, Waseda University

抄録

By investigating the occurrence of different proportional analogies in corpora, this paper describes an approach to increase the performance of existing analogy-based N-gram language models evaluated by perplexity. Our approach consists in using analogy to reconstruct N-grams from the test data so as to give higher probabilities to these N-grams. By giving different weights to different patterns, we also except that some N-grams which can be reconstructed by different patterns will get more accurate probabilities. The use of suffix arrays for data searching leads to a lesser computation time on text scoring tasks.

収録刊行物

• 情報処理学会研究報告. 自然言語処理研究会報告

  情報処理学会研究報告. 自然言語処理研究会報告 2014 (2), 1-4, 2014-01-30

  一般社団法人情報処理学会

キーワード

• 言語モデル・統計処理

詳細情報

• CRID: 1573950402681238016
• NII論文ID: 110009659641
• NII書誌ID: AN10115061
• 本文言語コード: en
• データソース種別:
  • CiNii Articles