Geometric relational embeddings

Abstract

In classical AI, symbolic knowledge is typically represented as relational data within a graph-structured framework, a.k.a., relational knowledge bases (KBs). Relational KBs suffer from incompleteness and numerous efforts have been dedicated to KB completion. One prevalent approach involves mapping relational data into continuous representations within a low-dimensional vector space, referred to as relational representation learning. This facilitates the preservation of relational structures, allowing for effective inference of missing knowledge from the embedding space. Nevertheless, existing methods employ pure-vector embeddings and map each relational object, such as entities, concepts, or relations, as a simple point in a vector space (typically Euclidean. While these pure-vector embeddings are simple and adept at capturing object similarities, they fall short in capturing various discrete and symbolic properties inherent in relational data. This thesis surpasses conventional vector embeddings by embracing geometric embeddings to more effectively capture the relational structures and underlying discrete semantics of relational data. Geometric embeddings map data objects as geometric elements, such as points in hyperbolic space with constant negative curvature or convex regions (e.g., boxes, disks) in Euclidean vector space, offering superior modeling of discrete properties present in relational data. Specifically, this dissertation introduces various geometric relational embedding models capable of capturing: 1) complex structured patterns like hierarchies and cycles in networks and knowledge graphs; 2) intricate relational/logical patterns in knowledge graphs; 3) logical structures in ontologies and logical constraints applicable for constraining machine learning model outputs; and 4) high-order complex relationships between entities and relations. Our results obtained from benchmark and real-world datasets demonstrate the efficacy of geometric relational embeddings in adeptly capturing these discrete, symbolic, and structured properties inherent in relational data, which leads to performance improvements over various relational reasoning tasks.