Collaborative Research: Developing a research hub for automated bee identification, data sharing, and citizen science using computer vision

University Of Wisconsin-Madison, Madison WI

Investigators

Abstract

Bees have great value as pollinators for crops, natural plant communities, and backyard gardens. Understanding the factors that affect bee diversity and function are therefore important subjects of basic ecology, conservation, and ensuring the continued provision of pollination services. The global bee decline highlights the need for effective conservation research. One essential yet challenging step in many bee studies is to identify bees so that species can be counted and population sizes measured. However, the identification process is time-consuming, expensive, and requires a high degree of expertise. But new technology in the field of artificial intelligence (AI) and computer vision is making automated identification from images a realistic goal. To develop the computer vision algorithms, we will need to generate a large dataset of bee images with known species IDs. Previous work has utilized public image repositories comprised mainly of bee images taken in natural settings. However, such images are often not amenable to identification by humans because the subtle characteristics that differentiate species are not visible and without a known ID, they cannot be used for training AI models. THe project will therefore use museum collections to generate a large image database from pinned bee specimens that have been expertly identified, representing at least 1,000 of the estimated 4,000 species in North America. This will allow researchers to develop a computer vision algorithm capable of identifying the most commonly seen species and all of the bee genera in North America. The resulting algorithm will be incorporated into a website (https://beemachine.ai), which will serve as a free identification resource and research hub. Users will be able to upload images of bees for identification and be able to discuss and validate machine-generated identifications. This research hub will therefore contribute to large-scale data collection and sharing among both scientists and citizen scientists. These large-scale data collection efforts are important for understanding bee trends that can span continental scales. Reliable identification of pollinators, such as bees, is critical for basic ecological research, conservation, and maintaining pollination services. However, species-level bee identification is difficult and requires specialized taxonomic knowledge because of their great diversity and often subtle morphological differences between species. The identification process results in a bottleneck that is expensive and time consuming, which slows the pace of research and adoption of new applications. Moreover, setbacks for pollinator research can result from errors based on misidentification if experts are unavailable or if funds are insufficient to employ them. However, cutting-edge technology in the fields of artificial intelligence (AI) and computer vision are enabling fast and reliable detection and classification of bees from images. Our project will cover three objectives aimed at greatly expanding the use of this technology for automated bee species identification and data sharing: (1) The project will develop a large and expertly labeled image dataset for model training that includes a minimum of 1,000 North American bee species. (2) The project will develop an AI-based classification model using convolutional neural networks to identify bee species in images. (3) The project will develop a website for novices to expert users that utilizes our computer vision algorithm to identify bees to the genus- and species-level. The website will be a substantial expansion of a previously released computer vision work, BeeMachine, and serve as a research hub for bee identification, data sharing, and communication among researchers. The current project will have bee identification capabilities far beyond superficially similar platforms, such as iNaturalist. Thus, automated identifications of user-contributed bee images will enable researchers to collect and share large-scale data more effectively and on a vastly greater diversity of bee species than is currently possible. Expert users will be able to verify AI-generated identifications so that contributed observations can be used to update and improve computer vision models. This project will provide a novel and transformative platform for research with high potential for transfer to other taxa beyond bees. Reducing the cost and time necessary to perform species-level identifications will allow researchers to expand the number of studies as well as the spatiotemporal scale of research so that the drivers of population and diversity change can be determined and so that effective conservation strategies can be enacted. The web-based research hub will increase access to expert-level identification capability, communication among researchers, and provide for a large-scale and ever-growing open dataset of bee distributions that can be used for research. The project website will be available at https://beemachine.ai This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.