SI2-SSE: The Next Generation of The Montage Image Mosaic Engine: Beyond Mosaics

California Institute Of Technology, Pasadena CA

Investigators

Abstract

Data sets produced by modern space missions and ground-based telescopes are so large and complex that they threaten to overwhelm astronomers' ability to analyze them. In response, the astronomy and IT communities are actively engaged in developing powerful new tools that can process these data. The Montage Image Mosaic Engine is a participant in this community of next generation tool development. Up to now, Montage has been used for high-performance processing of images, so that astronomers can take small images and create a mosaic that reveals the structure of a large region of the sky. With this new funding, Montage will be extended from a processing engine into a "one-stop shopping" toolkit that will enable astronomers to manage their image data as well as process them. They will be able to analyze their data and repair defects found in them, discover and access datasets released worldwide that will make analysis of their own data more valuable, compare and integrate their data sets with other data in web browsers, create their own data archives that can be accessed, as required, by team members or the public, and process large scale image mosaics through an on-demand service that will be hosted by a new astronomy data sharing platform called SciServer. All these features will dramatically reduce the overhead needed in managing data and will lead to a corresponding increase in productivity. Montage will at the same time retain all its existing capabilities. Montage is written as a toolkit rather than as an application, so astronomers can use each of its functions independently of each other and embed them in their processing environments. It is written in a language, C, that optimizes processing, runs on all Unix based operating systems used by all astronomers, and functions just as well on single machines as on massive computational facilities. These features enable Montage to be used by astronomers analyzing data on their desktops as well as by teams of scientists creating complex, massive new data sets for distribution to the astronomy community. One example is its use in the analysis of images to detect new near-Earth asteroids. It has also been taken up by the computer science community, who use it as an example of a real-world application in driving the development of powerful new cyberinfrastructure platforms for processing massive scientific data sets. One of the important current topics in scientific software is how to make powerful new tools sustainable so that they can be updated and used over many years. Montage is a potential model of sustainable software, and is active in national debates on approaches to sustainability. A version of Montage will be developed that will operate on Windows platforms, used by educators and students. This brings Montage to high-school students, youth and amateur astronomers. Partnerships have been built with two groups of educators who will work to integrate Montage into tools used by high-school students. For example, Montage will be integrated into tools used by students and youth to process images through a network of educational telescopes called SkyNet. This project will deploy new functionality currently existing as proof-of-concept prototypes, developed as a result of community input, that will transform Montage into a "data and metadata access, data management and curation toolkit in a box," while retaining all its current image processing functionality. The new functions are: discovery and acquisition of data at scale in space and in time, using R-tree based indexing schemes; tools for analyzing and repairing metadata; tools for visualizing sky coverage of multiple data sets; migration of an on-demand image mosaic service from IPAC to operate at scale on the SciServer platform; investigation of processing images in browsers with Javascript-enabled tools, and the development of custom image processing environments. Finally, a Windows-enabled version of Montage will be delivered. Montage will continue to have all the benefits of its architecture. It is written in C for portability and sustainability, is highly scalable and delivered as a toolkit that is easy to incorporate into processing environments. Montage is the only mosaic engine with all these characteristics. It is delivered with an Open Source BSD 3-clause license and accessible on GitHub. Development uses rigorous software engineering methods, and exploits a hybrid of the evolutionary prototyping and staged delivery development models. Astronomy data sets are becoming much larger and at the same time more complex. Montage will provide the tools to access, discover and manage data sets and their metadata. Many more datasets will be made accessible in this way, and astronomers will be able to develop their own data access services without the overhead of maintaining a database. Because it is written in ANSI-C and has structured inputs and outputs, astronomers will be able to integrate it into any *nix-based platform or environment to support data management and integration. The capability to search in space and time will be of immense value in querying upcoming time-domain data sets such as ZTF and LSST. The on-demand service will be an exemplar of deploying image processing services at scale. Thus, Montage will be of value to large missions and projects, as well as the long-tail of scientists working in small groups. The impact of Montage will be measured by tracking growth in citations to Montage, and to the creation of data sets that use the software, and by tracking usage of services and environments that use Montage as part of their underpinnings. Montage is having considerable impact on broad intellectual and educational areas outside astronomical research: (1) Advancing Learning and Discovery- It is used in developing Education and Public Outreach products such as a 5-color Galactic Plane image of Herschel data. It has found applicability in undergraduate research projects. The Windows version will bring Montage to high-school students, youth and amateur astronomers. Browser processing of images may lead to more sophisticated Citizen Science projects; (2) Dissemination to Enhance Scientific and Technological Understanding- It is recognized as an example of sustainable software, and the development team is active in national debates on the issues of software transparency and sustainability; (3) Enhancing Infrastructure for Research and Education- Montage is an exemplar application used to develop national cyber-infrastructure.