CAREER: Score-Based Diffusion Models for Probabilistic Forecasting of Weather and Climate

University Of Hawaii, Honolulu

Investigators

Abstract

This project will develop machine learning methods for estimating the risk of adverse weather and climate events. Estimating the probability of rare events in high-dimensional data is a fundamental problem in data science. Advances in generative artificial intelligence will be used to develop new data-driven computational methods for modeling risk and apply these methods to weather applications. In particular, these models will be applied to forecasting solar irradiance and precipitation, two applications that are particularly important for tropical islands such as Hawaii. Estimating the risk of rapid changes in solar power generation (ramp events) is necessary for managing energy grids that are seeing a rapid increase in variable renewable sources, and floods claim hundreds of lives and billions in property damage each year in the United States alone. This research will be paired with an educational outreach program that includes a summer data science course for high school students and a workshop to share data science teaching materials with local K-12 teachers. Generative artificial intelligence methods have led to rapid progress in text-to-image models, image super-resolution, and video prediction. The key development is in the neural network models used to learn joint probability distributions over high-dimensional data such as images and video. These include score-based diffusion models, which solve many of the limitations of previous generative models including other flow-based models, autoregressive models, variational autoencoders, and generative adversarial networks. This project will investigate and improve the ability of score-based diffusion models to efficiently learn the probability of rare events from finite training data. Applications to solar irradiance and precipitation forecasting will serve as motivating case-studies, as these problems require modeling the joint distributions over spatiotemporal weather data. Experiments will leverage existing data from numerical simulations of atmospheric variables and observations from satellites and ground-based sensor networks. The machine learning methods developed by this project will complement existing physics-based numerical weather prediction models by providing location-specific forecasts with increased computational speed, spatiotemporal resolution, and probabilistic prediction accuracy. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.