RI: Small: Reliable Machine Learning in Hyperbolic Spaces

Cornell University, Ithaca NY

Investigators

Abstract

Many artificial intelligence (AI) methods reason about things in the world by assigning each thing to a point in space and processing them using geometry. The way such an AI reasons about words is analogous to writing down many words on a piece of paper, such that words with similar or related meanings are located close to each other on the page: this map is called an "embedding." Often, AI perform better when they map things onto a curved surface (such as the surface of a sphere), rather than a flat one. A human might find this hard because of the need to write a large number of words into a small space on the paper; an analogous problem with the way points are represented on computers presently limits the performance of AI that use certain curved embeddings. This project will build tools to alleviate this problem, letting practitioners use curved embeddings without having to worry about how their points are represented as bits on a computer. This will promote the progress of AI science by making this powerful class of AI techniques more accessible to humans and improving the accuracy of AI on fundamental tasks such as processing natural language and learning from social networks. The project will focus on "hyperbolic space," which is a homogeneous geometry with constant negative curvature. Hyperbolic embeddings can significantly improve the performance of many AI applications, but the same properties that make it attractive for learning also create serious numerical reliability problems for existing machine learning frameworks, which were designed with Euclidean geometry in mind. This project will fix this by building a new model of hyperbolic space that provably avoids the numerical issues of existing approaches while still leveraging the hardware acceleration capabilities of GPUs for learning algorithms. Accomplishing this would improve the performance of learning in hyperbolic spaces and enable more people to train AIs using non-Euclidean space. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.