Constrained Least-Squares Density-Difference Estimation

• NGUYEN Tuan Duong

  Tokyo Institute of Technology

• DU PLESSIS Marthinus Christoffel

  Tokyo Institute of Technology

• KANAMORI Takafumi

  Nagoya University

• SUGIYAMA Masashi

  Tokyo Institute of Technology

Abstract

We address the problem of estimating the difference between two probability densities. A naive approach is a two-step procedure that first estimates two densities separately and then computes their difference. However, such a two-step procedure does not necessarily work well because the first step is performed without regard to the second step and thus a small error in the first stage can cause a big error in the second stage. Recently, a single-shot method called the least-squares density-difference (LSDD) estimator has been proposed. LSDD directly estimates the density difference without separately estimating two densities, and it was demonstrated to outperform the two-step approach. In this paper, we propose a variation of LSDD called the constrained least-squares density-difference (CLSDD) estimator, and theoretically prove that CLSDD improves the accuracy of density difference estimation for correctly specified parametric models. The usefulness of the proposed method is also demonstrated experimentally.

Journal

• IEICE Transactions on Information and Systems

  IEICE Transactions on Information and Systems E97.D (7), 1822-1829, 2014

  The Institute of Electronics, Information and Communication Engineers

Keywords

• density difference
• asymptotic variance
• <i>L</i>²-distance
• bias
• class-balance change
• two-sample homogeneity test

Details

• CRID: 1390001204379946752
• NII Article ID: 130004519278
• DOI: 10.1587/transinf.e97.d.1822
• ISSN: 17451361
• ISSN: 09168532
• Web Site: https://www.jstage.jst.go.jp/article/transinf/E97.D/7/E97.D_1822/_pdf
• Text Lang: en
• Data Source:
  • JaLC
  • Crossref
  • CiNii Articles
• Abstract License Flag: Disallowed