SoD: Algorithmic Differentiation of Functional Programs

Purdue University, West Lafayette IN

Investigators

Abstract

CCF-0438806 Jeffrey Mark Siskind Purdue University Algorithmic Differentiation of Functional Programs Automatic Differentiation (AD) is an established enterprise that seeks to take the derivatives of functions specified as imperative programs through symbolic manipulation rather than finite differencing. A framework for applying AD techniques to modern functional-programming languages, instead of imperative programs, will be investigated. This framework will treat AD operators as higher-order functions that map function objects to function objects. These operators will be compositional and will apply to more general functions than the maps between real vectors to which traditional AD transformations are limited. This framework offers several advantages: - a callee can perform the necessary AD rather than insisting that callers provide appropriately transformed functions, making it practical to compositionally build an efficient library of functions based on derivatives - higher-order derivatives can be constructed naturally, without special mechanisms - gradients can be taken through processes that themselves involve AD-based optimization or approximate iterate-to-fixedpoint operators such as PDE solvers The proposed work will have broad impact in such diverse fields as engineering design, scientific computing, and machine learning, where researchers regularly need to take derivatives of complex programs. It has intellectual merit as it offers a novel synthesis of the differential calculus with the lambda calculus.