CDI-Type I: Automated Documentation and Illustration of Material Culture through the Collaborative Algorithmic Rendering Engine (CARE)

Princeton University, Princeton NJ

Investigators

Abstract

The ability of computers to unify visual information from multiple imaging modes into comprehensible illustrations will revolutionize the ability of scientists, engineers, and humanities scholars to gain and communicate knowledge about the visual world. Achieving this goal, however, will require a joint focus on developing novel shape and image analysis methods, and designing collaborative user interfaces that allow multiple domain experts and illustrators to bring together their expertise. The Collaborative Algorithmic Rendering Engine (CARE) will be an open-source tool for extracting and merging visual details available only under certain lighting conditions, certain wavelengths, or certain imaging modalities. By focusing on minimal user effort, cross-site collaborative visualization design, and integrated archiving and process history (provenance) tracking, the CARE tool is specifically designed to remove existing obstacles to widespread adoption of digital tools for visual analysis and communication. As part of the project, investigators are developing novel image analysis techniques that build upon existing technologies such as Reflectance Transformation Imaging (RTI) and non-photorealistic rendering using images with normals (RGBN NPR), which have already received enormous interest within the cultural heritage community. The research includes methods for: (1) analyzing the collection of images to decompose them into "maps" of color, orientation, and material at each pixel; (2) performing an arbitrary sequence or combination of image-processing operations on some or all of the maps separately; and (3) combining several maps into the final illustration. The whole process is driven by (4) a user interface designed for interactive response and including special features that enable collaborative illustration design. The project involves a close collaboration between a university-based research group, responsible for development of new technologies, and a non-profit company with a demonstrated track record of working with museums and archaeological sites to deploy novel imaging and computational photography systems. This joint development will ensure that the underlying technologies will have immediate high impact in the field: cultural heritage scholars and scientists will be able to generate high-quality, comprehensible illustrations for scientific papers and textbooks, with control over selective emphasis, contrast, attention, and abstraction, at lower cost and greater flexibility than generating such figures by hand. The subject matter of art history also offers the unique opportunity to stimulate the interest of students who would not normally take courses in computer science, broadening the class of students exposed to the tools and capabilities of computing.