Committee: Dejing Dou (chair), Daniel Lowd, Reza Rejaie
Directed Research Project(Dec 2014)
Keywords: Relation Classification, Recursive Neural Networks, Dependency Parsing
Deep Learning is a new area of Machine Learning research, which mainly addresses the problem of time consuming, often incomplete feature engineering in machine learning. Recursive Neural Network (RNN) is a new deep learning architecture that has been highly successful in several Natural Language Processing tasks.
We propose a new approach for relation classification, using an RNN, based on the shortest path between two entities in the dependency graph. Most previous works on RNN are based on constituency-based parsing because phrasal nodes in a parse tree can capture compositionality in a sentence. Compared with constituency-based parse trees, dependency graphs can represent the relation more compactly. This is particularly important in sentences with distant entities, where the parse tree spans words that are not relevant to the relation. In such cases RNN cannot be trained effectively in a timely manner. On the other hand, dependency graphs lack phrasal nodes that complicates the application of RNN. In order to tackle this problem, we employ dependency constituent units called chains. Further, we devise two methods to incorporate chains into an RNN. The first model uses a fixed tree structure based on a heuristic, while the second one predicts the structure by means of a recursive autoencoder.
Chain based RNN provides a smaller network which performs considerably faster, and achieves better classification results. Experiments on SemEval 2010 relation classification task and SemEval 2013 drug drug interaction task demonstrate the effectiveness of our approach compared with the state-of-the-art models.