Recurrent Deep Learning Machines

University Of Maryland Baltimore County, Baltimore MD

Investigators

Abstract

The objective of this research is to develop a new paradigm of deep learning machine - those with a feedback structure. Feedbacks bring to computing nodes current or past information contained in neighboring or larger receptive fields of other computing nodes from the same or higher layers for forming better local representations or features. Such information is required for processing dynamical data and for maximizing generalization capabilities on static data. The approach of this research is to select or design deep and recurrent architectures, develop generative and discriminative learning techniques, and integrating the risk-averting method of convexifying training criteria into training recurrent deep learning machines. Intellectual Merit: Recurrent neural networks are irreplaceable for applications involving dynamical data and are fundamentally better than feedforward networks even on static data. However, difficulty in training recurrent networks has stifled development and understanding of them. The proposed research is expected to help remove this difficulty, bring forth the full power of recurrent neural networks, and boost interests in neural networks in general, which have unfortunately and undeservedly fallen out of favor in recent years. Broader Impact: Recurrent deep learning machines are powerful for static and dynamical classification and regression, including image and video recognition, analysis and compression; nonlinear system identification/control; signal processing/filtering; and critical system health/fault monitoring/detection. Therefore, the proposed work will contribute greatly to medical instrumentation, computer/robot/information technology, wireless telecommunication, national defense, and homeland security. Recurrent deep learning machines will ecome an important component in the graduate education in engineering and computer science