A Statistics-Based Geographic Bias Quantification and Debiasing Framework for GeoAI and Foundation Models

University Of Texas At Austin, Austin TX

Investigators

Abstract

This proposal addresses geographic bias foundation models. Geographic bias refers to a phenomenon in which an AI model performs differently across geographic regions. This project develops a statistics-informed framework to measure and mitigate geographic bias for both task-specific GeoAI model and foundation models. Open-source software and benchmarks are developed and made freely available to help researchers and practitioners from both academia and industry and to facilitate the translation of this research into immediate societal benefit. The goal of this project is to develop a statistics-informed geographic bias and debiasing framework for both task-specific GeoAI models and foundation models with three major contributions. First, the project proposes a set of geographic bias metrics that are grounded in spatial point pattern analysis and are widely applicable to numerous geospatial tasks and models. These metrics can handle continuous space and are numerically comparable across models. Second, a geographic bias evaluation benchmark is established that consists of various geospatial datasets, tasks, and newly developed geo-bias metrics, as a quality and ethical control tool for different Geospatial AI (GeoAI) and foundation models. Third, a suite of novel geographic debiasing algorithms is developed for both task-specific GeoAI models and task-agnostic foundation models that go beyond traditional approaches during the supervised fine-tuning stage. The framework can mitigate foundation models’ geographic bias during the inference stage and pre-training stage. The former provides a lightweight solution for debiasing foundation models, while the latter improves the task generalizability of the model. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.