Theory for the Practice of Deep Learning: Insights into Autoencoder, LLM Fine-Tuning, and Transfer Learning

University Of California-Berkeley, Berkeley CA

Investigators

Abstract

AI entered a new and accelerated phase with the public rollout in Nov. 2022, of the generative AI large language model (LLM) ChatGPT, which is a transformer deep learning (DL) model with 1.5 billion parameters trained on 570GB of data. The potential impact of ChatGPT and other chatbots in scientific research, teaching, medicine, government, business, and society at large is enormous. Currently, the dominant empirical paradigm for solving tasks using deep learning is to first pretrain massive models in an unsupervised manner on large data corpora and then fine-tune them on specific tasks of interest. For example, the standard practice for pretraining modern LLMs like ChatGPT is to train models to predict the next token on datasets scraped from the internet, which allows the model to learn meaningful and general representations about language. Although models learn extensively about language during the pre-training phase, they are typically not immediately useful for tasks of interest, and transfer learning must be applied by fine-tuning on a specific downstream task, such as coding, math, chat-botting, etc. There are many important questions about this fine-tuning process, such as how the hyperparameters should be set and when different algorithms can be expected to generalize well. In this project, a theoretical study will be taken to address different aspects of finetuning and transfer learning with the aim of producing practically relevant guidance for improving efficiency and generalization performance for these settings. This project includes financial support and mentorship for graduate students. Concretely, the proposed research will focus on the following two directions: developing methods that help choose the learning rate and rank in a near-optimal manner for a popular finetuning method known as Low Rank Adapters (LoRA). Insights from the large width scaling theory of neural networks will be used to guide how to select hyperparameters appropriately. Successful methods have the potential to greatly reduce the computing cost of hyperparameter tuning when finetuning large models. The second thrust involves studying transfer learning in the context of over-parametrized linear regression. The setting in over-parametrized linear regression is rich enough to provide conceptual insights into modern deep learning yet simplified enough for a rigorous mathematical study of generalization performance. Building upon previous works analyzing in-distribution generalization performance in over-parametrized linear regression and using similar random matrix theoretic tools, extensions will be made to the out-of-distribution transfer learning setting. By rigorously characterizing how transfer learning affects generalization, intuition will be provided for practitioners seeking to predict how various shifts will affect the performance of their models. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.