THE THOUSAND-FOLD INCREASE IN COMPUTATIONAL CAPABILITIES EXPECTED OVER THE NEXT DECADE WILL CHANGE THE WAY IN WHICH BOTH ENGINEERING AND SCIENTIFIC DISCOVERY ARE PURSUED. BUT WE WILL ONLY BE ABLE TO ACHIEVE SUCH NEW METHODS OF SCIENTIFIC INQUIRY IF WE ARE ABLE TO HARNESS THE POWER OF FUTURE SUPERCOMPUTERS FOR AMONG OTHER AREAS COMPLEX NAVIER-STOKES COMPUTATIONS OF TURBULENT FLUID FLOW USING SCALABLE PRECONDITIONERS AND SOLVERS THAT DO NOT EXIST TODAY. THE YET-TO-BE-DETERMINED ARCHITECTURES OF EXASCALE SYSTEMS ARE EXPECTED TO BE RADICALLY DIFFERENT FROM EXISTING SYSTEMS. ALTHOUGH WE DO NOT KNOW WHAT THESE COMPUTERS MAY LOOK LIKE ONE CAN IDENTIFY COMMON CHARACTERISTICS OF THESE EXASCALE SYSTEMS THAT WILL REQUIRE SERIOUS RETHINKING OF TODAY'S SOLVERS FOR LARGE-SCALE TURBULENT FLOW SIMULATIONS. THE ARCHITECTURAL FEATURES THAT ARE THE MOST SALIENT TO THE DESIGN IMPLEMENTATION AND DEPLOYMENT OF SCALABLE AND EFFICIENT SOLVERS FOR PARALLEL HIGH-PERFORMANCE SCIENTIFIC COMPUTING INCLUDE THE FOLLOWING: (A) EXTREME PARALLELISM -- ESTIMATES OF 2 TO 3 ORDERS OF MAGNITUDE OF PARALLELISM OVER TODAY'S LEVELS WILL REQUIRE SOLVERS TO PAY PARTICULAR ATTENTION TO AMDAHL'S LAW AND NEW WAYS OF ACHIEVING NEAR-PERFECT LOAD BALANCE MOST LIKELY IN A DYNAMIC FASHION (B) DATA PLACEMENT AND MOVEMENT -- OPTIMIZING DATA PLACEMENT AND MOVEMENT WILL BE KEY TO PERFORMANCE AS WELL AS A PRIMARY WAY FOR SOLVERS TO REDUCE POWER CONSUMPTION A MAJOR CONCERN IN FUTURE ARCHITECTURES (C) HETEROGENEOUS ARCHITECTURES -- HETEROGENEITY WILL BE NEEDED TO MEET THE POWER REQUIREMENTS OF EXASCALE SYSTEMS; BUT LITTLE HAS BEEN DONE TO DEVELOP PORTABLE HYBRID SOLVERS THAT CAN RUN ACROSS DIFFERENT TYPES OF ARCHITECTURES WITH COMPLEX MEMORY HIERARCHIES AND (D) RESILIENCE -- THE NUMBER OF FAILURES IS EXPECTED TO INCREASE WITH CONCURRENCY REQUIRING SOLVERS THAT CAN RUN THROUGH OR DETECT AND RECOVER FROM FAULTS. IN THIS PROPOSAL WE OFFER A PLAN TO IMPROVE ON THE STATE OF THE ART FOR PRECONDITIONED ITERATIVE SOLVERS FOR CFD WITH THE FOLLOWING KEY INGREDIENTS: 1. DEVELOP A PROMISING CLASS OF PRECONDITIONERS BASED ON LOW-RANK COMPRESSION AND HIERARCHICAL MATRICES AND EMBED THEM INTO AN APPROACH BASED ON NESTED DISSECTION AND MULTIFRONTAL ELIMINATION. 2. DEVELOP PIPELINED S-STEP COMMUNICATION-HIDING AND COMMUNICATION-REDUCING STRATEGIES FOR ITERATIVE KRYLOV SOLVERS THAT WILL BE PRECONDITIONED TO ENABLE SCALABILITY AT VERY LARGE NUMBERS OF PROCESSING CORES AND IN HETEROGENEOUS COMPUTING ARCHITECTURES. 3. PROTOTYPE THIS NEW CLASS OF SCALABLE PRECONDITIONED SOLVERS USING TWO ADVANCED PROGRAMMING APPROACHES INCLUDING LEGION/ REGENT AND QTHREADS AND DEMONSTRATE SCALABILITY IN THE LARGEST COMPUTATIONAL RESOURCES THAT WE HAVE ACCESS TO DURING THE PERIOD OF EXECUTION. 4. DEMONSTRATE OUR SOLVER STRATEGIES IN THE OPEN-SOURCE SU2 DISCONTINUOUS-GALERKIN FEM HIGHER-ORDER SOLVER FOR LES AND WMLES SOLUTIONS AND DEMONSTRATE APPLICABILITY OF OUR SOLVER APPROACH TO PROBLEMS OF ORDER HIGHER THAN P=3. 5. DEMONSTRATE LARGE-SCALE SYSTEM ALGORITHMIC SCALABILITY IN O(N LN N^P) AND GUARANTEED CONVERGENCE OF ITERATIVE METHODS IN O(1) STEPS WITH O(N LN N^P) PRECONDITIONER COST. THE RESULT WILL BE A NEW PRECONDITIONED SOLVER APPROACH FOR THE NAVIER-STOKES EQUATIONS THAT WILL BE AVAILABLE IN THE OPEN-SOURCE AND THAT COULD BE INCORPORATED INTO NASA TOOLS IN THE FUTURE. THE SOLVER WILL DEMONSTRATE IN THE CONTEXT OF THE SU2 DG-FEM HIGHER-ORDER SOLVER ALGORITHMIC SCALABILITY AND ROBUSTNESS OF NAVIER-STOKES LES AND WMLES SIMULATIONS.