Elements: HPN-SSH

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

The transfer of large data sets between computing resources is an integral component of the scientific workflow. Multiple tools have been developed to aid in this task but, despite significant performance bottlenecks, secure shell (SSH) based tools like secure copy protocol (SCP) and secure file transfer protocol (SFTP) remain popular due to ubiquity, ease of use, and minimal administrative burdens. HPN-SSH, the project tool, was initially developed in 2004 to address this need. This award will allow the PIs to expand on the original work of HPN-SSH and address new challenges created by advances in computer technology and the needs of users. They will incorporate hardware accelerated encryption; efficiently use modern CPUs by dynamically sizing the number of threads; accelerate the cryptographic workflow by allowing it to work in parallel; investigate making the default open source secure shell OpenSSH cipher use multiple cores; create a ‘resume on failure’ feature enabling users to restart transfers from the point of failure; and incorporate networking metrics to aid in troubleshooting and performance analysis of HPN-SSH. HPN-SSH will support and enhance research efforts across a wide range of scientific domains by lowering the costs of entry to big data and remote computation without compromising security or functionality. These benefits will extend to business and industry, educational communities, and the general public as well. The transfer of large data sets between computing resources is an integral component of the scientific workflow. Multiple tools have been developed to aid in this task but, despite significant performance bottlenecks, SSH based tools remain popular due to ubiquity, ease of use, and minimal associated costs. To address these bottlenecks we developed HPN-SSH; a series of patches that enable high performance throughput for the OpenSSH application. These patches were initially released in 2004 and have become widely used throughout the research, academic, financial, and technology communities. This award gives the PIs opportunity to foster innovative development in HPN-SSH that will benefit the community by significantly increasing performance. With this grant the PIs will: incorporate on-die hardware accelerated encryption in their multithreaded AES counter cipher; efficiently use multicore CPUs by dynamically sizing the number of threads; introduce pipelining and parallelization into the cryptographic workflow; investigate the parallelization of the default OpenSSH cipher CHACHA20; create a ‘resume on failure’ feature enabling users to restart transfers from the point of failure; and incorporate inline network telemetry to aid in troubleshooting and performance analysis. This work will also advance the field of computer science through the development and improvement of parallelization methods to enhance the performance of cryptographic routines. As most widely used cryptographic libraries and methods are highly serial in nature they are unable to take advantage of multicore processors. As processor speed has remained relatively stable over the past ten years we must distribute the cryptographic workload over multiple cores in order to significantly increase throughput. HPN-SSH will democratize access and extend the reach of the national cyberinfrastructure by lowering the costs of entry without compromising security or functionality. These benefits will extend to business and industry, educational communities, and the general public. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.