Date of Award

6-7-2019

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Computer Science

First Advisor

Xiaowei Xu

Abstract

Deep language models learning a hierarchical representation proved to be a powerful tool for natural language processing, text mining, and information retrieval tasks. However, more specifically, representations that perform well for ad-hoc retrieval must capture semantic meaning at different levels of abstraction or context-scopes. The primary goal of ad-hoc retrieval is to find relevant documents satisfying the information need posted in a natural language query. It requires a good understanding of the query and all the documents in a corpus, which is difficult because the meaning of natural language texts depends on the context, syntax, and semantics. In this dissertation, two multi-resolution techniques for improving learning representations in deep ad-hoc retrievals are proposed: (1) Multi-Resolution Word Embedding is composed of cascaded operations and generates multi-resolution word embeddings that represent documents at multiple resolutions in term of context-scopes. To this end, we first compare various text embedding methods for retrieval performance and provide an extensive empirical comparison with the performance of different non-augmented base embeddings with and without multi-resolution representation. (2) Multi-Resolution Neural Network (MRNN) is devised to leverage the whole hierarchy of representations for ad-hoc retrieval. The proposed MRNN model is capable of deriving a representation that integrates representations of different levels of abstraction from all the layers of the learned hierarchical representation. Moreover, a duplex attention component is designed to refine the multi-resolution representation so that an optimal context for matching the query and document can be determined. To investigate performances of proposed techniques, we use the SQuAD, WikiQA, QUASAR, and TrecQA datasets in an open-domain question-answering setting, where the first task is to find useful documents useful for answering a given question. Finally, we present that multi-resolution word embeddings are consistently superior to the original counterparts and deep residual neural models specifically trained for retrieval purposes can yield further significant gains when they are used for augmenting those embeddings. MRNN with the duplex attention significantly outperforms the existing models used for ad-hoc retrieval on benchmark datasets. The results show that all our multiresolution techniques and algorithms work efficiently for real-world natural language processing, text mining and information retrieval problems.

Download

Share

COinS