Modeling Content Structures of Domain-Specific Texts with RUP-HDP-HSMM and Its Applications

  • LU Youwei
    Department of Computational Intelligence and Systems Science, Tokyo Institute of Technology
  • OKADA Shogo
    School of Information Science, Japan Advanced Institute of Science and Technology
  • NITTA Katsumi
    School of Computing, Tokyo Institute of Technology

Abstract

<p>We propose a novel method, built upon the hierarchical Dirichlet process hidden semi-Markov model, to reveal the content structures of unstructured domain-specific texts. The content structures of texts consisting of sequential local contexts are useful for tasks, such as text retrieval, classification, and text mining. The prominent feature of our model is the use of the recursive uniform partitioning, a stochastic process taking a view different from existing HSMMs in modeling state duration. We show that the recursive uniform partitioning plays an important role in avoiding the rapid switching between hidden states. Remarkably, our method greatly outperforms others in terms of ranking performance in our text retrieval experiments, and provides more accurate features for SVM to achieve higher F1 scores in our text classification experiments. These experiment results suggest that our method can yield improved representations of domain-specific texts. Furthermore, we present a method of automatically discovering the local contexts that serve to account for why a text is classified as a positive instance, in the supervised learning settings.</p>

Journal

References(12)*help

See more

Related Projects

See more

Details 詳細情報について

Report a problem

Back to top