SHF: Small: Declaratively Creating Semantics-driven Visualizations

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

Visual diagrams are essential for communicating difficult technical concepts; for instance, there is ample evidence that students learn more effectively and can solve problems more efficiently when using diagrams rather than plain text. Effective communication of scientific concepts is limited by the fact that good diagrams remain hard to create: abstract logical or mathematical concepts are difficult for non-experts to translate into compelling graphical figures, and traditional software tools produce static diagrams that do not easily facilitate interaction or exploration. This project develops a next-generation framework called Penrose, where high-quality diagrams can be created by simply typing expressions in domain-specific languages that reflect the way students and scientists already talk about mathematics. The project's novelties are to systematically encode the relationships between abstract mathematical statements and their visual representations, and to automatically generate diagrams satisfying these relationships via constrained nonlinear optimization. The project's impacts are to significantly lower the barrier to creating effective diagrams, and to accelerate the rate at which complex technical ideas are communicated---especially in the domain of scientific education and research. The project takes an interdisciplinary approach between programming languages and computer graphics. At the language level, Penrose enforces a clean separation between mathematical content and its visual representation via two extensible specification languages, Substance and Style, akin to HTML and CSS. On the graphics side, programs are compiled into a constrained optimization program whose solutions describe a family of possible diagrams. Automatic or user-assisted tools can then be used to select or interactively explore specific diagrams. The systematic encodings provided by Penrose can also be used to intelligently search for useful examples, special cases, or counterexamples. A free web-based Penrose interface encourages community development of new user-designed modules; integration of these modules into an interactive mathematical tutor system helps to enrich and accelerate the training of the next generation of scientists and engineers in the US. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.