EAGER: Additive Parts-based Data Representation with Nonnegative Sparse Autoencoders

University Of Louisville Research Foundation Inc, Louisville KY

Investigators

Abstract

One of the long-standing open problems of computational learning is its inability to produce solutions that are intuitively understandable. Because of the limited transparency, most computed predictions are not easily justifiable by humans who need to render final decisions. This project addresses this shortcoming of machine learning predictions by investigating a class of machine learning algorithms that mimics natural processing and leads to more interpretable representations of data. Such processing decomposes visual patterns or other data into parts through unsupervised learning and produces non-negative parts only. Since this approach allows only additive recombination of parts to reconstruct the original data, it mimics natural processing in human perception and cognition. The project advances novel data representation beyond standard computational learning approaches. Tests are conducted with planar images or tabulated data that describe specific domain of interest. The tests objectives are to produce useful and understandable features, logic rules or verbal explanations in lower dimensional space. This work aims at evaluating how rich data can be explored in order to be better understood. The novel paradigm is to generate non-negative, sparse and localized features and receptive fields within hierarchies of features. This is achieved through autoencoder-based transformations of visual images or of typical non-negative data matrices. The novel autoencoders are constrained to have non-negative weights. Specific conditions to be tested include pooling, rectifying-type activation functions of neurons and select norms of activations sparsity. The project advances the following transformational challenges at the intersection of computational and human systems: (1) Representation of data with non-negative encodings only, (2) Complexity of layers and of receptive filters (more simpler filters vs. fewer complex filters), (3) Choice of the number of layers, also in the context of hyper-parameter tuning, (4) Pooling for non-negative processing, (5) Connections between the autoencoder-based learning and related biological evidence, and finally (6) Ability to generate explanations or understandable rules for select domains