Approximate Dynamic Programming Using Random Sampling

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

Abstract Proposal Number: ECCS-0824077 Proposal Title: Approximate Dynamic Programming Using Random Sampling PI Name: Atkeson, Christopher G. PI Institution: Carnegie-Mellon University The objective of this research is to develop approximate dynamic programming methods for the control of high performance nonlinear systems. The approach is to use spatially local models of key functions such as a function that represents future costs over an entire task or mission (the value function), and a function that tells the system what to do in each situation (the policy). Humanoid robots will be used to evaluate the nonlinear control design techniques and compare them with other approximate dynamic programming approaches. Intellectual Merit Key contributions of this work will be: 1) Developing an integrated approach for dynamic planning based on approximate dynamic programming for high performance systems. 2) Showing how to combine many fast greedy local planners to produce globally optimal solutions. 3) Showing how to represent knowledge along explicit trajectories, which increases the feasible sparseness of this approximate dynamic programming method. 4) Showing how to create an adaptive representation using random sampling of states. 5) Showing how to use local models, and compare them to global parametric function approximators. 6) Showing how to use fast trajectory optimization to speed up approximate dynamic programming. Broader impacts A specific impact of this work will be to enable high performance (near optimal) control in areas like robotics, transportation, and energy. A more general societal payoff is industrial processes, machines, and robots that are easier to program, perform better and waste fewer resources.