Feature Selection via <i>l</i>₁-Penalized Squared-Loss Mutual Information

• JITKRITTUM Wittawat

  Tokyo Institute of Technology

• HACHIYA Hirotaka

  Tokyo Institute of Technology

• SUGIYAMA Masashi

  Tokyo Institute of Technology

Abstract

Feature selection is a technique to screen out less important features. Many existing supervised feature selection algorithms use redundancy and relevancy as the main criteria to select features. However, feature interaction, potentially a key characteristic in real-world problems, has not received much attention. As an attempt to take feature interaction into account, we propose l₁-LSMI, an l₁-regularization based algorithm that maximizes a squared-loss variant of mutual information between selected features and outputs. Numerical results show that l₁-LSMI performs well in handling redundancy, detecting non-linear dependency, and considering feature interaction.

Journal

• IEICE Transactions on Information and Systems

  IEICE Transactions on Information and Systems E96.D (7), 1513-1524, 2013

  The Institute of Electronics, Information and Communication Engineers

Keywords

• feature selection
• <i>l</i>₁-regularization
• squared-loss mutual information
• density-ratio estimation
• dimensionality reduction

Details

• CRID: 1390282679355165824
• NII Article ID: 130003370928
• DOI: 10.1587/transinf.e96.d.1513
• ISSN: 17451361
• ISSN: 09168532
• Web Site: https://www.jstage.jst.go.jp/article/transinf/E96.D/7/E96.D_1513/_pdf
• Text Lang: en
• Data Source:
  • JaLC
  • Crossref
  • CiNii Articles
• Abstract License Flag: Disallowed