日本語情報検索システム評価用テストコレクションの構築 Construction of a Test Collection for the Evaluation of Japanese Information Retrieval Systems

この論文にアクセスする

この論文をさがす

著者

抄録

本論文では 情報処理学会内のワーキンググループ活動によって構築し 1996年3月からモニタ公開している日本語情報検索システム評価用テストコレクションBMIR-J1の設計・構築・利用・評価について報告する. BMIR-J1は日本語情報検索システムのための初めてのテストコレクションであると向時に 情報検索システムに求められる 自然言語で表現された種々の検索要求を解釈・処理するためのファンクションの観点から検索要求を分類して提供している点が特長である. ファンクション分類したことにより システム間の比較評価として 全検索要求を用いた総合的/平均的な比較だけでなく 個別のファンクションごとの分析的な比較も行えるようになった. 同時に 高精度な情報検索システムの実現へ向けて 自然言語処理技術/知識処理技術を活用した取組みに期待される課題を 具体的な検索要求文の形で示したものでもある. 公開後 多くのモニタユーザによる活発な研究発表がなされ アンケート回答も多く寄せられている. これらからBMIR-J1の有効性と次期バージョンへの課題などが明らかになった.This paper describes BMIR-J1 (Benchmark for Information Retrieval Systems for Japanese texts Ver.1), which is designed as a test collection for Japanese information retrieval systems. The design principles, construction, user reports and the evaluation of BMIR-J1 are described. BMIR-J1, which was released for monitoring use in March 1996, was developed by a working group under the Special Interest Group on Database Systems in Information Processing Society of Japan. BMlR-J1 is the first test collection for Japanese information retrieval systems, although there are a couple of test collections for English information retrieval systems. One of the unique features of BMIR-J1 is that the queries are classified into five groups according to interpretation levels needed for processing these queries which are represented as natural language sentences. These query groups enable more analytical comparison of systems, while total comparison of systems using whole queries is possible. English test collections do not have these groups in themselves. The query groups presented by their queries themes to be studied and functions to be developed using natural language processing techniques and knowledge processing techniques for a next generation information retrieval system which will be more effective and user friendly. Since the release of BMIR-J1, many researchers and developers are using BMIR-J1, and there are a couple of research reports on the evaluation of each information retrieval system using BMlR-J1 as a test collection. Many BMIR-J1 user opinions show that BMIR-J1 is effective as a test collection for Japanese information retrieval systems, and also these opinions show future works for developing a next version of BMIR-J1.

This paper describes BMIR-J1 (Benchmark for Information Retrieval Systems for Japanese texts Ver.1), which is designed as a test collection for Japanese information retrieval systems. The design principles, construction, user reports and the evaluation of BMIR-J1 are described. BMIR-J1, which was released for monitoring use in March 1996, was developed by a working group under the Special Interest Group on Database Systems in Information Processing Society of Japan. BMlR-J1 is the first test collection for Japanese information retrieval systems, although there are a couple of test collections for English information retrieval systems. One of the unique features of BMIR-J1 is that the queries are classified into five groups according to interpretation levels needed for processing these queries which are represented as natural language sentences. These query groups enable more analytical comparison of systems, while total comparison of systems using whole queries is possible. English test collections do not have these groups in themselves. The query groups presented by their queries themes to be studied and functions to be developed using natural language processing techniques and knowledge processing techniques for a next generation information retrieval system which will be more effective and user friendly. Since the release of BMIR-J1, many researchers and developers are using BMIR-J1, and there are a couple of research reports on the evaluation of each information retrieval system using BMlR-J1 as a test collection. Many BMIR-J1 user opinions show that BMIR-J1 is effective as a test collection for Japanese information retrieval systems, and also these opinions show future works for developing a next version of BMIR-J1.

収録刊行物

  • 情報処理学会論文誌

    情報処理学会論文誌 40(9), 3537-3553, 1999-09-15

    一般社団法人情報処理学会

参考文献:  38件中 1-38件 を表示

各種コード

  • NII論文ID(NAID)
    110002725073
  • NII書誌ID(NCID)
    AN00116647
  • 本文言語コード
    JPN
  • 資料種別
    Journal Article
  • ISSN
    1882-7764
  • NDL 記事登録ID
    4851585
  • NDL 雑誌分類
    ZM13(科学技術--科学技術一般--データ処理・計算機)
  • NDL 請求記号
    Z14-741
  • データ提供元
    CJP書誌  NDL  NII-ELS  IPSJ 
ページトップへ