Rephrasing Visual Questions by Specifying the Entropy of the Answer Distribution

• TERAO Kento

  Hiroshima University

• TAMAKI Toru

  Hiroshima University

• RAYTCHEV Bisser

  Hiroshima University

• KANEDA Kazufumi

  Hiroshima University

• SATOH Shin'ichi

  National Institute of Informatics

抄録

Visual question answering (VQA) is a task of answering a visual question that is a pair of question and image. Some visual questions are ambiguous and some are clear, and it may be appropriate to change the ambiguity of questions from situation to situation. However, this issue has not been addressed by any prior work. We propose a novel task, rephrasing the questions by controlling the ambiguity of the questions. The ambiguity of a visual question is defined by the use of the entropy of the answer distribution predicted by a VQA model. The proposed model rephrases a source question given with an image so that the rephrased question has the ambiguity (or entropy) specified by users. We propose two learning strategies to train the proposed model with the VQA v2 dataset, which has no ambiguity information. We demonstrate the advantage of our approach that can control the ambiguity of the rephrased questions, and an interesting observation that it is harder to increase than to reduce ambiguity.

収録刊行物

• IEICE Transactions on Information and Systems

  IEICE Transactions on Information and Systems E103.D (11), 2362-2370, 2020-11-01

  一般社団法人 電子情報通信学会

キーワード

• visual question answering
• visual question generation
• rephrasing

詳細情報

• CRID: 1391412326420459264
• NII論文ID: 130007933953
• DOI: 10.1587/transinf.2020edp7089
• ISSN: 17451361
• ISSN: 09168532
• Web Site: https://www.jstage.jst.go.jp/article/transinf/E103.D/11/E103.D_2020EDP7089/_pdf
• 本文言語コード: en
• データソース種別:
  • JaLC
  • Crossref
  • CiNii Articles
  • KAKEN
• 抄録ライセンスフラグ: 使用不可