BIGDATA: Collaborative Research: IA: Quantifying Plankton Diversity with Taxonomy and Attribute Based Classifiers of Underwater Microscope Images

University Of California-San Diego Scripps Inst Of Oceanography, La Jolla CA

Investigators

Abstract

Plankton play an essential role in the global ecosystem, forming the base of marine food webs, linking the atmosphere to the deep ocean, and regulating a myriad of ecologically and climatologically important processes. Despite their importance, however, the technology to assess abundances and distributions of plankton has been limited. Changes in abundances of individual species are particularly poorly resolved; this includes the harmful algal blooms that have profound economic, societal, and ecosystem effects in many coastal systems. Traditional tools such as nets and bottles can destroy fragile organisms during sampling. Underwater microscopes, on the other hand, allow observation of the organisms undisturbed, and in their natural setting. New underwater microscopes are generating many thousands of high-resolution images of individual plankton each day. Before these images can be used for scientific analyses, the imaged organisms must be identified and classified. However, the vast number of images generated by such microscopes has led to a serious bottleneck: identification and classification of the images takes an impossibly long time for individuals to accomplish. Fortunately, advances in computer vision science have shown great promise in accurately performing such classification tasks. The main goal of this award is to explore and develop computer vision approaches for plankton image classification. A team of instrumentation specialists, an ocean ecologist, and a computer scientist, including two graduate students and one post doctoral student, will formulate, implement, and test methods to advance the goal of efficient and accurate automated plankton image classification. The advances made in this award will enable both improved classification algorithms in computer science, and vast new data streams for plankton ecology. Plankton form the base of marine food webs, link the atmosphere to the deep ocean, and regulate global biogeochemical cycles. Plankton are often studied either through bulk measures, or by manual enumeration of individual taxa. Novel underwater microscope systems such as the Scripps Plankton Camera System (SPCS) are generating tens of thousands of images of individual plankton daily. However, without accurate annotation of the images, the potential science is limited. This project will explore the use of many-layer, deep Convolutional Neural Nets (CNN) as automated computer recognition methods; these techniques hold promise for classifying the nearly one trillion underwater microscope images that have been collected by a variety of research groups around the globe. The primary source of images will be a pair of microscopes that have been operating for 2 years from the Scripps Inst. of Oceanography's pier, yielding 200 million regions of interest. The project will build a large data base of training sets using a novel approach: a bench-top imaging system that is capable of rapidly producing thousands of annotated images showing organisms in all orientations and configurations identical to that in the field. Based on these automatically collected training sets, and hand annotation of in situ images from experts, a deep (many layer) CNN will embed taxonomic and attribute constraints, and will be used to classify the organisms imaged. With success, this massive, growing, taxonomically classified dataset will enable unprecedented, transformative, taxon-specific explorations of the dynamics of the planktonic ecosystem on time scales from hours to decades.