Automatic Extraction of Useful Facet Hierarchies from Text Databases

Databases of text and text-annotated data constitute a significant fraction of the information available in electronic form. Searching and browsing are the typical ways that users locate items of interest in such databases. Faceted interfaces represent a new powerful paradigm that proved to be a successful complement to searching. Thus far the identification of the facets was either a manual procedure or relied on apriori knowledge of the facets that can potentially appear in the underlying collection. In this paper we present an unsupervised technique for automatic extraction of facets useful for browsing text databases. In particular we observe through a pilot study that facet terms rarely appear in text documents showing that we need external resources to identify useful facet terms. For this we first identify important phrases in each document. Then we expand each phrase with "context" phrases using external resources such as WordNet and Wikipedia causing facet terms to appear in the expanded database. Finally we compare the term distributions in the original database and the expanded database to identify the terms that can be used to construct browsing facets. Our extensive user studies using the Amazon Mechanical Turk service show that our techniques produce facets with high precision and recall that are superior to existing approaches and help users locate interesting items faster.