Network Embedded Storage and Compute (NESCO)

University Of Maryland, College Park, College Park MD

Investigators

Abstract

The Network Embedded Storage and Compute (NESCO) project is motivated by a premise that future Research and Education networks should include "embedded compute and storage" functions as a mechanism to provide advanced services for domain science application workflows. This is a natural follow-on to the current cyberinfrastructure evolution revolving around "Cloud" and "Edge" computing. The NESCO project is developing the next important trend based on a distributed ecosystem of compute and storage facilities which are embedded deeply within networks. These will be similar in function to on-premise edge cloud deployments but will be customized for placement within networks. Application workflow developers will have access to a new class of services which can be rapidly tailored to specific flows and use cases. One result anticipated is the "democratization of middlebox" functions in a manner where application workflow developers can customize their workflows and data processing on real-time basis with flow level granularity. This project includes a climate science workflow as a prototype use case. It is anticipated that these new services will be of interest to a broad range of domain science applications. Individual and small research groups who cannot afford access to dedicated resources may find these services of particular value. A NESCO facility is defined which can be placed inside of Wide Area Networks, Regional Networks, and Exchange Points. These facilities will contain advanced programmable hardware-based network functions, compute and storage resources, and be architected to provide services on a highly customizable per flow basis. An open source orchestration system known as StackV is being modified to develop a new class of realtime "Flow Management" services. The StackV system includes a realtime modeling framework to "datafy" infrastructure, modular pluggable computation elements, and intelligent workflow processes to build on-demand topologies across heterogeneous resource types and service providers. This system also includes a user facing application programming interface which allows workflow agents to request resource topologies which span network, compute, and storage resources via a single integrated request. This project is developing a NESCO reference implementation which will be deployed for testing and experimentation. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.