grand challenges of advanced computing for energy innovation€¦ · laboratory-developed advanced...

DISCLAIMER This report was prepared as an account of a workshop sponsored by the U.S. Department of Energy. Neither the United States Government nor any agency thereof, nor any of their employees or officers, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of document authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof. Copyrights to portions of this report (including graphics) are reserved by original copyright holders or their assignees, and are used by the Government’s license and by permission. Requests to use any images must be made to the provider identified in the image credits.

ON THE COVER Advanced computing enhances U.S. energy innovation in the ways the country produces, moves, stores, and uses energy. The following examples are illustrated in the three panels of the cover.

• PRODUCE – The Consortium for Advanced Simulation of Light Water Reactors (CASL), a U.S. Department of Energy (DOE) Energy Innovation Hub, is developing modeling and simulation capabilities for commercial nuclear power reactors. The CASL vision is to predict, with confidence, the performance of nuclear reactors through comprehensive, science-based modeling and simulation technology that is deployed and applied broadly throughout the nuclear energy industry to enhance safety, reliability, and economics. CASL employs High Performance Computing, science-based models, state-of-the-art numerical methods, modern computational science and engineering practices, and uncertainty quantification and validation against data from operating pressurized water reactors, single-effect experiments, and integral tests.

• TRANSPORT – Applying High Performance Computing to the energy sector promises to revolutionize the way we think about energy once more. Supported by the DOE Advanced Grid Modeling Program, researchers are developing new computational tools for the power grid. These tools vastly improve the ability to respond to emergencies, predict grid behaviors for operation and control, and transport more power using existing transmission lines. The tools include fast state estimation, faster-than-real-time dynamic simulation, advanced market optimization, massive contingency analysis, and graphical contingency analysis. The graphical contingency analysis shows potential to reduce emergency response times by 30 percent.

• USE – Enormous potential exists to improve the fuel economy of internal combustion engines for transportation. To fully realize this potential, automotive and heavy-duty engine Original Equipment Manufacturers (OEMs) strongly agree that there is a pressing need for advanced combustion computational fluid dynamics (CFD) tools to both better design advanced high-efficiency, clean engines and dramatically increase the pace of their development. The DOE Basic Energy Science and Vehicle Technologies Programs are collaborating to develop advanced high-performance computational tools, such as Large Eddy Simulation. These tools will be used to help develop the science underpinnings of advanced combustion strategies that will enable higher efficiency engines, and to advance, calibrate, and validate the engineering CFD tools used by industry to design engines. They will also form the basis for new levels of advanced engine design tools.

GRAND CHALLENGES OF ADVANCED COMPUTING FOR ENERGY INNOVATION

Report from the Workshop Held July 31–August 2, 2012 Sponsored by the U.S. Department of Energy, Office of Nuclear Energy, Advanced Modeling and Simulation Office

Lead Author, Alex Larzelere U.S. Department of Energy

Co-Chair, Steven Ashby Pacific Northwest National Laboratory

Co-Chair, Dana Christensen National Renewable Energy Laboratory

Co-Chair, Dona Crawford Lawrence Livermore National Laboratory

Grand Challenges of Advanced Computing for Energy Innovation iii

EXECUTIVE SUMMARY

On July 31–August 2 of 2012, the U.S. Department of Energy (DOE) held a workshop entitled Grand Challenges of Advanced Computing for Energy Innovation. This workshop built on three earlier workshops that clearly identified the potential for the Department and its national laboratories to enable energy innovation. The specific goal of the workshop was to identify the key challenges that the nation must overcome to apply the full benefit of taxpayer-funded advanced computing technologies to U.S. energy innovation in the ways that the country produces, moves, stores, and uses energy. Perhaps more importantly, the workshop also developed a set of recommendations to help the Department overcome those challenges. These recommendations provide an “action plan” for what the Department can do in the coming years to improve the nation’s energy future.

Three Types of Grand Challenges – Workshop discussions among a wide range of participants—from the energy industry, advanced computing technology companies, independent software vendors, universities, the national laboratories, and DOE—identified three distinct types of challenges. This is particularly significant because addressing these challenges requires different actions and different sets of people in broad engagement across the energy industry, advanced computing, and DOE communities:

• Technical Grand Challenges: These are the technical inadequacies of advanced computing and the need for building better and more accurate modeling and simulation capabilities that require more powerful computers.

• Structural Grand Challenges: These challenges pertain to the transfer and use of DOE advanced computing resources (hardware and software) for energy innovation in commercial environments.

• Incentive Grand Challenges: Even if technically useful advanced computing capabilities exist and the means of leveraging those capabilities are in place, private companies and the DOE national laboratories need to have proper incentives to make the investments required to implement the use of advanced computing for energy innovation.

Key Recommendation – Based on the observations, findings and recommendations of this workshop, the workshop chairs provide the following overall recommendation:

• Establish an Advanced Computing for Energy (ACE) program within the Department of Energy to transform industrial innovation in energy. As a central point of contact that would be responsible for implementing the DOE outcomes recommended by the workshop, this program would do for energy innovation what the Advanced Simulation and Computing (ASC) program did for stockpile stewardship and what the Scientific Discovery through Advanced Computing (SciDAC) program did for scientific discovery. ACE would complement ASC and SciDAC by focusing on the commercial “hardening” and dissemination of modeling, simulation, and analytics software—and access to mid-range computing resources—to demonstrably speed the testing, evaluation, and deployment of new energy products and services to the marketplace.

The workshop definitions, grand challenge findings, and specific recommendations that are the foundation for this overall recommendation are summarized in the following pages.

iv Grand Challenges of Advanced Computing for Energy Innovation

Definitions – For purposes of the workshop, the following definitions were used:

• Advanced computing—advanced development and use of modeling and simulation software, enabling middleware and hardware, and information analytics

• Energy innovation—creation and adoption of better products, processes, services, and technologies, to produce, move, store, and use energy.

Summary of Grand Challenge Findings – The distinctions between the three identified types of grand challenges are important because the actions needed to overcome one type of grand challenge will be different from the actions needed for another type. Understanding the different types of grand challenges may be one of the significant findings of the workshop.

• Technical Grand Challenges: These challenges concern the technical ability of advanced computing to address energy innovation needs. Issues include the following:

• The advanced computing capabilities developed by the DOE national laboratories offer tremendous opportunities to enhance U.S. energy innovation. However, these tools are currently inaccessible to many innovators because of the level of expertise needed to productively use them to address energy innovation problems.

• DOE-developed advanced computing technologies are not immediately suitable for a particular use by energy innovators. These capabilities must be tailored to a particular use. Also, most importantly, the capabilities must be validated to ensure that results can be trusted.

• As advanced computing transitions from peta-scale to exa-scale computers, a number of challenges must be considered. These include new programming models and the ability to adapt existing modeling and simulation software. This transition will undoubtedly impact the ability of energy innovators to take advantage of the power of the new computing architectures.

• Structural Grand Challenges: These challenges pertain to the transfer and use of DOE advanced computing resources (hardware and software) for energy innovation in commercial environments. Issues include these:

• Energy innovators and their supporting advanced computing technology vendors (software and hardware) are interested in learning what is being done at the national laboratories. However, they have found it difficult to understand what is happening at the DOE laboratories and how they might participate in the research and development activities.

• The working relationship between the Independent Software Vendors (ISVs) and the national laboratories needs to be improved. This is an important relationship because ISVs offer a means to adapt and maintain DOE-developed advanced computing capabilities for targeted commercial uses by energy innovators.

• The ability of DOE-developed advanced computing to assist energy innovators has been limited by the process and pace of completing technology transfer agreements. Companies are finding that the technology transfer process takes too long and that there is too much uncertainty in the resulting agreement. Also, companies are finding that each laboratory and DOE Operations Office has its own unique technology transfer process.

Grand Challenges of Advanced Computing for Energy Innovation v

• Incentive Grand Challenges: Even if technically useful advanced computing capabilities exist and the means of leveraging those capabilities are in place, private companies and the laboratories need to have proper incentives to make investments required to implement the use of advanced computing for energy innovation. Issues include the following:

• The workshop participants found that in many cases, energy innovation companies were interested in using advanced computing. However, given the large initial investment required, the companies were unwilling to adopt unproven technology and wanted short-term access to DOE capabilities to determine the value of the technology for their own application.

• Companies generally do not have the “in-house” expertise to take advantage of DOE national laboratory-developed advanced computing capabilities and are unwilling to make investments in hiring that expertise without a firm understanding of the return on the investment.

• U.S. energy innovators need a better understanding of the benefits and value of advanced computing technologies.

• DOE and its national laboratories need a better understanding of the opportunities to impact energy innovators through advanced computing.

Summary of Workshop Recommendations – The breakout sessions generated a great deal of discussion about what should be done to address the workshop findings. The numerous suggestions from the workshop have been consolidated into four major recommendations, including the overall recommendation to establish an Advanced Computing for Energy program within the DOE to serve as a central, enabling force:

1. Improve the usability and availability of DOE-developed advanced computing solutions.

The advanced computing modeling, simulation, and analytics software developed by the DOE national laboratories is most often developed for expert users in support of research activities. This software must be “hardened” for commercial use by non-experts with special attention to usability, extensibility, and incorporation in innovation workflow processes. Here the DOE should

• simplify access to computing resources at the laboratories, including allowance of in-kind contributions to meet full cost recovery requirements

• create a few primary points-of-contact (web portal, email, and telephone) to facilitate identification by industry of relevant advanced computing capabilities and resources within the DOE and the DOE laboratories

• engage relevant and qualified ISVs to partner with the labs to mature and harden their software (see the next recommendation).

2. Engage the Independent Software Vendor community to promote energy innovation.

Few companies have the resources to develop and maintain their own modeling, simulation, and analytics software. The DOE should design and implement an outreach program to relevant and qualified ISVs, who can help bridge the gap between the research capabilities of the laboratories and the commercial needs of companies by hardening, adapting, and customizing laboratory-developed software for use by industry. Elements of this outreach program might include

• regular forums that bring together industry and laboratory personnel engaged in the development and use of advanced computing solutions for energy innovation

vi Grand Challenges of Advanced Computing for Energy Innovation

• incentives to engage ISVs in the hardening and deployment of DOE-developed advanced computing capabilities

• incentives to encourage ISVs to modify their licensing model to facilitate greater use of parallel processing power via mid-range computing resources

• representation of the ISV community in the proposed Advanced Computing subcommittee of the Secretary of Energy Advisory Board (see next recommendation).

3. Implement policies that will facilitate adoption of advanced computing for energy innovation.

Industry participants at this workshop, as well as all three industry-laboratory workshops held this year, cited numerous bureaucratic and policy barriers to working with the national laboratories and DOE. It is essential that DOE review and revise policies related to technology transfer, access to computing resources, and engagement of laboratory expertise. For example:

• Develop a standard, simplified software licensing agreement.

• Either direct fund technology transfer activities or increase (e.g., double) the amount of overhead funding that laboratories can devote to technology transfer activities, and encourage the increased flexibility to be used for maturation and hardening of laboratory-developed advanced computing software.

• Work with ISVs to develop a software licensing business model that will encourage the use of parallel processing power.

• Establish an Advanced Computing subcommittee of the Secretary of Energy Advisory Board (SEAB) to guide DOE advanced computing policy.

4. Establish an Advanced Computing for Energy (ACE) program within the Department of Energy.

The purpose of this overall recommendation would be to create a program that focuses on the commercial “hardening” and dissemination of modeling, simulation, and analytics software—and access to mid-range computing resources—to demonstrably speed the testing, evaluation, and deployment of new energy products and services to the marketplace. Key elements of this program might include

• establishment of easily-accessible advanced computing centers focused on energy innovation

• access to national laboratory-based advanced computing resources, including hardware, software, and consulting expertise

• funding to improve the usability of DOE-developed modeling, simulation, and analytics software

• aforementioned primary web portal into DOE advanced computing capabilities

• aforementioned ISV engagement program.

The workshop chairs recognize that without a single point of responsibility within the Department, many of the recommendations provided by the workshop would not succeed. As the chairs had learned through experiences with the Advanced Simulation and Computing (ASC) and Advanced Scientific Computing Research (ASCR) programs, progress in changing the way advanced computing was used for nuclear weapons and science required a focused program. Therefore, they recommend a similar approach for the DOE applied energy programs.

Grand Challenges of Advanced Computing for Energy Innovation vii

Recommended Immediate Next Step – The DOE is fortunate in that there are already some current activities that can provide “testbeds” to learn how to overcome the grand challenges identified in this report. These current activities include the following:

• Combustion Research Facility (CRF) at Sandia National Laboratories

• High Performance Computing Innovation Center (HPCIC) at Lawrence Livermore National Laboratory

• Consortium for Advanced Simulation of Light Water Reactors (CASL) led by Oak Ridge National Laboratory.

Since these three activities are dealing with questions about how advanced computing technologies can be made useful to energy innovators (technical challenges), they also are required to deal with the technology transfer issues (structural challenges) and the need to attract industry users (incentive challenges). So, as an immediate next step, these three examples should be studied closely to understand what is working well and what is not. These “lessons learned” should be shared and incorporated into any future Department-wide Advanced Computing for Energy program.

Summary of Conclusions – As a result of this workshop, we now have a better understanding of what challenges lie ahead and what actions we must take to achieve the vision of enabling energy innovation through advanced computing. Participants at the Workshop on the Grand Challenges of Advanced Computing for Energy Innovation were positive and encouraging about the importance and potential impact of the workshop but recognized that the impact will only be felt when the DOE is able to act upon the recommendations provided. This workshop and the previous three workshops have confirmed the importance of DOE advanced computing and its potential to improve the energy future of the United States. The workshop chairs strongly support the actions needed to make that vision a reality for the country.

viii Grand Challenges of Advanced Computing for Energy Innovation

ACKNOWLEDGMENTS

The authors would like to acknowledge the outstanding contributions to the workshop and this report of the following collaborators:

Lee Ann Dudney, Pacific Northwest National Laboratory

Noah Goldstein, Lawrence Livermore National Laboratory

John Grosh, Lawrence Livermore National Laboratory

Benjamin Grover, Lawrence Livermore National Laboratory

Steve Hammond, National Renewable Energy Laboratory

Moe Khaleel, Pacific Northwest National Laboratory

Steven Lee, U.S. Department of Energy

Rob Neely, Lawrence Livermore National Laboratory

Jim Peltz, U.S. Department of Energy

Ray Stults, National Renewable Energy Laboratory

Jack Wells, Oak Ridge National Laboratory

The authors are also grateful for the active engagement, thoughtful discussions, and insights provided by the workshop participants identified in Appendix A.

Grand Challenges of Advanced Computing for Energy Innovation ix

TABLE OF CONTENTS EXECUTIVE SUMMARY .......................................................................................................... III ACKNOWLEDGMENTS ............................................................................................................ VIII WORKSHOP BACKGROUND AND GOAL ............................................................................. 1

The Central Question ........................................................................................................... 1 Working Definitions ............................................................................................................. 2 Grand Challenges Strategy ................................................................................................... 2 Recommendations Strategy .................................................................................................. 3 The Final Imperative ............................................................................................................ 3

WORKSHOP STRUCTURE ....................................................................................................... 5 Invitees ................................................................................................................................. 5 Discussion Tracks ................................................................................................................ 5 Session Organization ............................................................................................................ 6

SUMMARY OF COLLOQUIA ................................................................................................... 9 Advanced Computing for Energy – Challenges, Successes, and Opportunities .................. 9 DOE Advanced Computing Resources for Energy Applications ......................................... 9 Accessing Commercially Available Advanced Computing Resources and Expertise ......... 10

SUMMARY OF PANEL SESSIONS .......................................................................................... 11 DOE Assistant Secretaries Panel on DOE Applied Technology Programs – Advanced

Computing for Energy – Promises and Challenges ...................................................... 11 Panel on Energy Innovation through Advanced Computing: Success Stories .................... 14 Panel on Energy Innovation through Advanced Computing: Potential and Challenges ..... 16

BREAKOUT 1: DOE APPLIED PROGRAMS, MODELING AND SIMULATION – GRAND CHALLENGES .................................................................................................... 19 Introduction .......................................................................................................................... 19 Challenges, Recommendations, Impacts .............................................................................. 21 Detailed Recommendations for Wind Energy and Next-Generation Grid ........................... 24 Breakout Conclusions .......................................................................................................... 26

BREAKOUT 2: CURRENT USERS – GRAND CHALLENGES ............................................. 27 Introduction .......................................................................................................................... 27 Opportunities ........................................................................................................................ 28 Challenges ............................................................................................................................ 28 Recommendations ................................................................................................................ 30 Impacts ................................................................................................................................. 33

BREAKOUT 3: POTENTIAL USERS – GRAND CHALLENGES .......................................... 35 Introduction .......................................................................................................................... 35 Challenges - Barriers to Adoption ........................................................................................ 36 Recommendations - Path Forward ....................................................................................... 37

x Grand Challenges of Advanced Computing for Energy Innovation

Breakout Conclusions .......................................................................................................... 39 SUMMARY OF FINDINGS AND RECOMMENDATIONS ..................................................... 41 CONCLUSIONS .......................................................................................................................... 45

The Central Question ........................................................................................................... 45 Getting Some Answers ......................................................................................................... 45 Recommending Some Actions ............................................................................................. 45 Need for A Single Point of Responsibility ........................................................................... 45 Immediate Next Steps .......................................................................................................... 46 Working Toward The Future ................................................................................................ 47

APPENDIX A: WORKSHOP PARTICIPANTS ........................................................................ A-1 APPENDIX B: WORKSHOP AGENDA ................................................................................... B-1 APPENDIX C: ACRONYMS AND ABBREVIATIONS .......................................................... C-1

WORKSHOP BACKGROUND AND GOAL

Grand Challenges of Advanced Computing for Energy Innovation 1


THE CENTRAL QUESTION

On May 8, 2012, Forbes had an exclusive interview with Secretary Steven Chu on the subject of supercomputing and energy. The first question was about why the Secretary was excited about the possibility of applying supercomputing to changing the energy future for the United States. Secretary Chu replied:

There has been tremendous progress in supercomputing over the last decade. In fact, today’s supercomputers are over a million times more powerful than the average desktop computer. Because of those advances, we can now use those computers to achieve qualitatively different objectives.

Those objectives are to improve the energy security of the United States. There are many different definitions of “energy security,” but most boil down to improving three aspects of the country’s ability to produce, move, store, and use energy. These aspects are as follows:

• National Security: the need to protect the security of the sources of supplies of energy and the routes used to get it to users within the U.S.

• Economic Security: the need for energy to remain readily available and at affordable prices to protect the economic interests of the country

• Environmental Security: considerations of the environmental impact of energy on the U.S and the rest of the world.

Secretary Chu’s vision of supercomputing being used to change the country’s energy future was informed by three earlier workshops:

• The Simulations Summit held on October 13, 2010 in Washington, DC

• The National Summit on Advancing Clean Energy Technologies held on May 16 to 17, 2011 in Washington, DC

• The Industry-National Laboratory Workshop on Modeling and Simulation held on March 7 to 8, 2012 in Austin, Texas.

All of these previous workshops confirmed the potential (and in some cases proven) value of supercomputing—also known as advanced computing or High Performance Computing (HPC)—to spur energy innovation. A detailed report was prepared from the findings of the National Summit held in May 2011. One of the findings reported was the following:

HPC has been shown to substantially reduce the time and cost to design, develop, prototype, and deploy new energy materials, components, and systems. As already seen in other sectors like defense, aerospace, and pharmaceuticals, HPC enables innovation, lessens uncertainty, and improves options, thereby allowing companies to become more facile, responsive, and technologically advanced. Through virtual prototyping, hundreds


2 Grand Challenges of Advanced Computing for Energy Innovation

of clean energy-related designs and new materials can be tested and optimized before manufacturing begins.

Clearly, the potential of advanced computing is well understood. A central question remains: what is holding the DOE back from exploiting its advanced computing investments for energy innovation? That was the central focus of this workshop and is now the focus of this report.

WORKING DEFINITIONS

For the purposes of the workshop, the following definitions were used. As noted earlier in this discussion, unfortunately many different words (super, high performance, advanced) are used to describe essentially the same thing. Also, the meaning of these terms often appears to be relative to the listener’s current perspective about computing.

• Advanced Computing – advanced development and use of modeling and simulation software, enabling middleware and hardware, and information analytics. Details include

• Modeling and simulation

• Improved dimensionality, resolution, and fidelity

• Data analytics

• Extracting knowledge out of large data sets

• Enabled by High Performance Computing

• Hardware, middleware, applications software

• Capabilities potentially deployable on a full range of systems (laptops and up)

• Energy innovation – creation and adoption of better products, processes, services, and technologies, to produce, move, store, and use energy. Details include:

• Improving the pace and quality of the innovation process

• Getting answers to “what if” questions faster

• Avoiding misunderstandings and mistakes.

During the workshop, the participants were encouraged to take as broad a perspective as possible when thinking about advanced computing and energy innovation.

GRAND CHALLENGES STRATEGY

As stated in its title, the focus was on grand challenges associated with advanced computing in achieving the Secretary’s vision of achieving qualitatively different kinds of objectives for Energy Innovation. Typically, workshops like this tend to focus on

• Technical Grand Challenges: the technical inadequacies of advanced computing and the need for building better and more accurate modeling and simulation capabilities that require bigger and more powerful computers.



However, this workshop recognized that better technology is not the only thing holding the country back from fully exploiting its advanced computing capabilities for energy innovation. The workshop specifically asked the participants to consider two other important factors:

• Structural Grand Challenges: These challenges pertain to the transfer and use of DOE advanced computing resources (hardware and software) for energy innovation in commercial environments.

• Incentive Grand Challenges: Even if technically useful advanced computing capabilities exist and the means of leveraging those capabilities are in place, private companies and the DOE national laboratories need to have proper incentives to make investments needed to implement the use of advanced computing for energy innovation.

RECOMMENDATIONS STRATEGY

Many times identifying challenges are easy. In the areas of high technology like advanced computing the problems and limitations are well known and at times may seem insurmountable. For that reason, the workshop participants also spent considerable time thinking about recommendations to overcome the challenges. They were encouraged to be inventive in developing recommendations that were

• actionable (preferably doable by the DOE)

• quantifiable (if possible)

• bounded by a completion date

• accompanied by a description of the expected impact.

This way the workshop would result in a “To Do” list for the Department of Energy that provided not only an understanding of the advanced computing grand challenges but also a set of actionable recommendations with the understanding of the impact if those actions were taken.

THE FINAL IMPERATIVE

The Forbes interview with Secretary Chu ended with questions, “How is the Department of Energy playing a role in this emerging space?” Secretary Chu replied:

In every way possible. We have been the leader in the development of supercomputers in the United States for decades. We also help to develop the software to work with these machines, developing new algorithms to increase the calculation ability and make them easier to work with. The DOE is really playing a critical role by maintaining large user facilities and employing a core of computer scientists and applied mathematicians who help industry take full advantage of our supercomputing capabilities.

The question for the workshop and for the DOE efforts is: How can we do better? The previous workshops on advanced computing for energy clearly demonstrated the potential and proven value of the technologies. However, those workshops also provided some glimpses of where the Department could do better. This workshop and this report help to answer the imperative: Where do we go from here?

WORKSHOP STRUCTURE


WORKSHOP STRUCTURE

INVITEES

Addressing the central question of this workshop—Where do we go from here?—involved a very diverse group. This included people who understand what energy innovation means. Adding to the complexity of this group is that energy innovation often means different things to different energy sectors. So that means even more diversity. Another important group was the people who understand the world of advanced computing and those particular challenges. This included advanced computing specialists who appreciate the challenges of using the tools for particular energy uses. Finally, the workshop needed people who run the government programs that would implement the recommendations from the workshop.

Well over 300 people were considered as invitees to the workshop. However, given the need for active discussion, the workshop organizers wanted to limit attendance to about 100. A complete list of attendees appears in Appendix A of this report. The attendees represented

• commercial energy/analysis companies

• advanced computing system developers

• independent software vendors

• universities

• national laboratories

• Department of Energy leadership

• Department of Energy program managers.

DISCUSSION TRACKS

Part of the answer to the question “Where do we go from here?” is the realization that everyone is not starting from the same place. In some cases, energy companies have been using advanced computing for years to innovate. In other cases, energy companies are only beginning to understand the potential. Therefore the workshop was organized around three discussion tracks to make sure that the full range of grand challenges and possible recommendations would be considered:

• Current Advanced Computing Users: This track focused on energy companies who are current users of advanced computing and have realized its potential value. The focus of the discussions was on what challenges those companies faced and how were they addressed.

• Potential Advanced Computing Users: This track involved energy companies who could see the potential value of advanced computing but are not currently using it. This discussion was centered on the questions, “What challenges are holding back potential users, and how could those challenges be addressed?”

• DOE Applied Energy Programs: This track centered on the DOE programs that are sponsoring energy innovation activities. Some of these programs are currently using advanced computing, but others are not. Because these government programs face special challenges, this discussion track identified what has worked in the past, what has not, and recommendations for future activities.

WORKSHOP STRUCTURE


SESSION ORGANIZATION

The workshop was structured to gather representatives from these groups and to provide them with some basic information that helped establish a common foundation for the discussions to make them as productive as possible. Also, the sessions were designed to help build on each other, leading to the creation of findings and recommendations that would live beyond the workshop sessions. This led to three types of sessions described here and in the workshop agenda provided in Appendix B of this report:

• Colloquia: Given the diverse group invited to the workshop, there was a high likelihood that participants would not have a common understanding of the current challenges and opportunities of advanced computing or energy innovation. At the suggestion of Secretary Chu, the first day of the workshop was organized into colloquia that provided that background information.

These colloquia were optional, but many of the attendees took the time to attend. Attendees were given the freedom to attend any two of the following three colloquium sessions:

• Advanced Computing for Energy — Challenges, Successes and Opportunities: This colloquium featured experts in the application of DOE Advanced Computing resources presenting a variety of projects, demonstrating that High Performance Computing can have significant impact on improving fundamental understanding, advancing technologies, and accelerating new products to market. The intent was to provide examples from a variety of energy topical areas in an effort to demonstrate a broad base of application. Attendees saw a variety of applications and engaged in conversations about software packages and computing resources that can be applied to their specific challenges. The overall goal was for attendees to become familiar with computing tools that are available and to stimulate interest in accessing those tools and establishing partnerships with DOE.

• DOE Advanced Computing Resources for Energy Applications: This colloquium reached out to energy innovators, especially those working in companies and DOE Applied Technology Programs, who are intrigued by the possibility of improving energy technologies through advanced computing (modeling, simulation, and analytics). This colloquium provided information on a wealth of software tools and computing resources that are available at the DOE national laboratories. The presenters described these resources, their design and purpose, the target platforms, and future plans.

• Accessing Commercially Available Advanced Computing Resources and Expertise: The intent of this colloquium was to provide an overview of software and platforms available in the commercial marketplace. Commercial software provides users with robust, easy-to-use, well-documented tools with a range of options for consulting support to help end users get started quickly with design. The advent of cloud computing is meanwhile making access to computing power as easy as having internet access and a credit card. In this colloquium, providers of commercial tools gave a survey of their capabilities as those capabilities apply to energy innovation through modeling and simulation.

• Panel Sessions: These sessions were provided to set the stage for the workshop participants for the three discussion tracks. The panels consisted of three or four speakers and were moderated by the three workshop chairs. The panelists were asked to spend just a few minutes talking about their experiences with advanced computing for energy innovation, the challenges they faced, and the extent to which they were able to address or overcome those challenges. The detailed

WORKSHOP STRUCTURE


workshop agenda in Appendix B of this report identifies the speakers and their talk titles. The three panels were as follows:

• DOE Assistant Secretaries Panel on DOE Applied Technology Programs – Advanced Computing for Energy – Promises and Challenges

• Panel on Energy Innovation through Advanced Computing: Success Stories

• Panel on Energy Innovation through Advanced Computing: Potential and Challenges

These panel sessions were followed by a keynote address from Secretary Chu. His talk and subsequent question-and-answer session reinforced the messages from his May 8, 2012 Forbes interview. He made the point again that the U.S. has a unique resource in the advanced computing capabilities in the national laboratories and universities as well as the urgent need to apply them to the challenges of energy security.

• Breakout Sessions: The heart of the workshop was the breakout sessions. In these sessions the participants were organized along the lines of the discussion tracks. Participants were specifically assigned to break out based on their experiences with those tracks. Each breakout was led by a representative from one of the national laboratories with specific experience in dealing with the challenges of applying advanced computing for energy innovation. Those leads were given the freedom to organize the discussions as they thought best.

The point of the breakout sessions was to take an independent look at understanding the three types of grand challenges (technical, structural, and incentive) and then to develop recommendations for the DOE to address them. The results served well to capture the unique perspective of each discussion track and to provide a comprehensive look at the challenges and ways to address them.

WORKSHOP STRUCTURE


SUMMARY OF COLLOQUIA



The first day of the workshop provided three colloquia that provided information to the attendees about challenges of using advanced computing for energy innovation and existing resources to help overcome those challenges. The colloquia were organized into three half-day sessions, which allowed participants the choice of which two they wanted to attend.

ADVANCED COMPUTING FOR ENERGY – CHALLENGES, SUCCESSES, AND OPPORTUNITIES

One of the colloquia sessions provided information about current use within the DOE programs of how of advanced computing is being used for energy innovation. It was organized by Steve Hammond of the National Renewable Energy Laboratory and invited speakers who talked about current DOE advanced computing projects and how they were impacting the ability of national laboratory and industry energy innovators to come up with new technology and methods to improve U.S. energy security.

The presentations in this session were the following:

Presentation Title Given By Organization Electrons flow fast. Can power grid computation keep up? Henry Huang Pacific Northwest National

Laboratory

Design, discovery, and detailed theory for photovoltaic materials using electronic structure methods

Stephan Lany National Renewable Energy Laboratory

Direct Numerical Simulation of Turbulent Combustion: Fundamental Science towards Predictive Models

Ramanan Sankaran Oak Ridge National Laboratory

Advanced Bioenergy, Biomass, and Biofuels: Molecular Level Design of Solutions Using Advanced Simulation and Modeling

Mike Crowley National Renewable Energy Laboratory

Windplant Aerodynamics and Loads Fort Felker National Renewable Energy Laboratory

The Consortium for Advanced Simulation of Light Water Reactors: A DOE Energy Innovation Hub

Doug Kothe Oak Ridge National Laboratory

DOE ADVANCED COMPUTING RESOURCES FOR ENERGY APPLICATIONS

Another session focused on resources that were available to the attendees to help them confront the challenges of advanced computing. This session was organized by Steven Lee from the Office of Science Advanced Scientific Computing Research program. He organized the session to highlight programs that could be accessed by advanced computing developers and users to deal with the grand challenges.




Presentation Title Given By Organization Overview of DOE Modeling, Simulation and Analytics for Energy Applications

Steven Lee Department of Energy-SC

DOE Applied Mathematics Software for High Performance Computing

Lori Diachin Lawrence Livermore National Laboratory

Scalable Data Management, Analysis, and Visualization Tools for Computational Science

Arie Shoshani Lawrence Berkeley National Laboratory

Advanced Simulation and Analysis Software for Power Distribution Systems

Rob Pratt Pacific Northwest National Laboratory

Advanced Software for Radiation Physics and Safety Tim Valentine Oak Ridge National Laboratory

DOE Advanced Computing Platforms and Programs Dan Hitchcock Department of Energy-SC

ACCESSING COMMERCIALLY AVAILABLE ADVANCED COMPUTING RESOURCES AND EXPERTISE

The third session of the colloquia added the dimension of commercially available capabilities to allow energy innovators to use advanced computing. This was a very interesting set of talks because it involved people and organizations not normally involved with Department of Energy technology workshops. The colloquium was organized by Benjy Grover and Rob Neely of Lawrence Livermore National Laboratory with presentations provided by a number of Independent Software Vendors (ISVs) and companies offering cloud-based advanced computing services.


Presentation Title Given By Organization Complexity, Reliability, and Competition: Energy Innovation through Engineering Simulation in the Cloud Computing Era

Barbara Hutchings

ANSYS, Inc.

Addressing Energy Challenges by Going Beyond Realistic Engineering Simulations

Mahesh Kailasam

Dassault Systèmes

Computational Methods In Energy Related Materials Science Nick Reynolds Accelrys, Inc

The Cloud - Lowering Barriers to Computing, Software and Expertise Bob Graybill Nimbis Services, Inc.

Advanced Computing for Energy: Multi-Physics Analysis, Design Optimization, and Data Analytics

Dave Corson and Shing Pan

Altair

SUMMARY OF PANEL SESSIONS



DOE ASSISTANT SECRETARIES PANEL ON DOE APPLIED TECHNOLOGY PROGRAMS – ADVANCED COMPUTING FOR ENERGY – PROMISES AND CHALLENGES

Moderator: Dana Christensen, National Renewable Energy Laboratory

Panelists:

• David Danielson, DOE Assistant Secretary for Energy Efficiency and Renewable Energy

• Peter Lyons: DOE Assistant Secretary for Nuclear Energy

• Charles McConnell, DOE Assistant Secretary for Fossil Energy

• Patricia Hoffman, DOE Assistant Secretary for Electricity Delivery and Energy Reliability

Overview

The Department of Energy has been a leader in advancing the role of High-Performance Computing toward solving technically challenging problems. This began with the Office of Defense Programs in the early days of the Nuclear Weapons Program but advanced rapidly with the initiation of the Advanced Scientific Computing Initiative (ASCI) in the 1990s. The Office of Science further accelerated its High Performance Computing initiative with the advent of the Leadership Computing Facilities in the 2000s. But, until only recently, the offices reporting to the Undersecretary of Energy have not experienced the need for High Performance Computing in addressing energy technology challenges. This situation is changing rapidly. Energy challenges are becoming increasingly complex, large quantities of data must now be handled, and increasingly, integration of systems is becoming essential. These issues require that High Performance Computing become a central tool within the research portfolios.

Office of Energy Efficiency and Renewable Energy

The office understands that energy challenges require broader integration across all of the stakeholder areas. For instance, for solar photovoltaics to move at-scale into the marketplace, we need to have a much stronger understanding of not only the performance of conversion materials but also of the “soft costs,” which include siting, licensing, installation and maintenance, and grid connectivity. In addition, power electronics and overall system manufacturing must be included. All of this must be accomplished at a cost competitive with traditional sources of energy. Similar challenges exist in the wind energy industry, biofuels and chemicals industry, and other renewable sources. Additional challenges exist, depending on regional locations. Energy resources vary depending on location, laws and ordinances vary, and public acceptance varies. The complexity of addressing all dimensions of the challenges for any one technology demands that we be able to manage large quantities data in a way that decision makers can arrive at decisions much more rapidly. High Performance Computing is one of the essential tools necessary to manage the challenge of analytics of massive amounts of data to extract meaning (also known as the “Big Data” challenge).

Beyond the individual technology areas is the question of system integration and optimization. We envision a future with increasing penetration of renewable energy, and much of this adoption will occur in



the distribution side of our electricity grid. The future will also involve the adoption of significant energy efficiency improvements in our buildings and in the adoption of more electric transportation. These energy efficiency improvements also exist at the demand end of the grid. This means that the perspective of the grid will change from one of large power plants, transmission and distribution, and demand centers to a paradigm of multidirectional power flow, increasing demand response for load control, a mixture of large and small power generators, and significantly increasing public control over how electricity is used. The only mechanism to understand and manage the complexity of such a system is through High Performance Computing and significant modeling and simulation.

Office of Nuclear Energy

The office of nuclear energy has relied on computing for a number of years as essentially all of the nuclear power plants are designed using sophisticated computer models. Indeed, the Nuclear Regulatory Commission relies on design codes as a primary mechanism to assess and approve safety for both new designs and any changes to existing systems. Uprating of nuclear power plants have relied on computational results, and the entire issue of life extension of existing power plants relies on a combination of experimental and computational results.

More recently, the Office of Nuclear Energy has initiated one of the new integration concepts, the Energy Innovation Hubs, by establishing the Consortium for Advanced Simulation of Light Water Reactors (CASL), which connects fundamental research and technology development through an integrated partnership of government, academia, and industry that extends across the nuclear energy enterprise. The Consortium is composed of five DOE national laboratories, three universities, the Electric Power Research Institute, Westinghouse, and the Tennessee Valley Authority with the goal of being able to design, certify, and operate a nuclear power plant in virtual space.

Finally, the Office of Nuclear Energy has also established the Advanced Modeling and Simulation Office. This office focuses more broadly on nuclear modeling and simulation tools. The mission of the office is to set computer modeling of nuclear energy systems on a modern footing to simulate nuclear systems with much higher fidelity and well-defined and validated prediction capabilities. Included are reactors, fuel fabrication plants, used-fuel processing plants, and waste disposition systems. The approach is to employ modern high performance computers to create a set of engineering-level codes in which the performance of fuels and materials are informed by first-principles modeling of materials on atomistic and mesoscale scales, experimentally validated. These tools aim to achieve scalability in terms of computing power and the types and couplings of the physics that dominates the system behavior.

Office of Fossil Energy

The scope of work in this office is very broad to include:

• exploration and extraction of coal, petroleum, natural gas, and methane hydrates

• clean coal technologies

• carbon capture, utilization, and storage

• hydrogen and other clean fuels

• combustion technologies and combustion efficiency

• managing the U.S. Petroleum Reserves.



High Performance Computing is increasingly playing a role in each area of the mission space. Improved understanding of the earth subsurface assists in the exploration for fuels, such as natural gas, as well as the improvement of both enhanced oil recovery and potential for storage of combustion by-products. Modeling and simulation is assisting in improving the understanding of the earth subsurface. In particular, advanced computing is playing a critical role to improve the processes for capturing the carbon that results from burning fossil fuels. The use of advanced computing in the Carbon Capture Simulation Initiative provides a good example of how industry, universities, and laboratories can work together on important uses of advanced modeling and simulation. This technology is also being used to model carbon sequestration in deep geological formations.

Improving combustion efficiency involves understanding the behavior of fuels in engines, including fuel mixing and combustion dynamics, both of which require complex understanding of gas hydrodynamics. Advanced modeling and simulation is the pathway to achieving improved combustion efficiency and therefore reduced emissions.

The Clean Coal program focuses on improving the efficiency of coal-based power systems, enabling affordable carbon dioxide (CO2) capture, increasing plant availability, and maintaining the highest environmental standards. The program supports gasification-related research and development (R&D) to convert coal into synthesis gas (syngas) that can be converted into electricity, chemicals, hydrogen, and liquid fuels. In addition, this program advances hydrogen turbine designs to improve the performance of pre-combustion CO2 capture systems and supports the development of advanced combustion systems through research focused on new high-temperature materials and the continued development of oxy-combustion technologies. Across this entire thrust area, High Performance Computing is beginning to bring significant improvement of understanding of the complex set of technologies.

Office of Electricity Delivery and Energy Reliability

The mission of the Office is to lead national efforts to modernize the electric grid, enhance security and reliability of the infrastructure, and facilitate recovery from disruptions to energy supply. The area where High Performance Computing is beginning to have a strong impact is within the R&D activities where clean energy transmission and reliability, smart grid, energy storage, and cyber security work is performed.

Clean energy transmission and reliability includes next-generation cables and conductors to increase the delivery capacity of electricity systems, to improve the affordability of electric services, and to enhance efficiency by reducing energy losses. It also enhances understanding of the power system and enables response to changing system and market conditions, which is critical for ensuring reliable and efficient grid operations under high penetration of variable generation (such as solar and wind turbines). It also advances our analytical ability to upgrade, extend, and replace existing grid modeling and analysis, visualization, and decision tools.

Smart grid R&D includes adoption of digital technology to modernize the nation’s electric delivery network for enhanced operational intelligence and connectivity. The enhanced connectivity will allow different applications, systems, and devices to be interoperable with one another through a combined use of open system architecture, as an integration platform, and commonly-shared technical standards and protocols for communications and information systems. Computing and Big Data management are keys to success.



Energy storage includes materials research for battery, electrolytic capacitor, and flywheel systems as well as advanced component development and field testing of storage systems in diverse market applications. The goals are lower life-cycle costs, improved performance, and fewer siting issues because of reduced size and environmental impact. Computing at the atomistic level up to the system level is leading to successful outcomes.

Cyber security for energy delivery systems includes next-generation control systems for hardening control systems with built-in security. It includes vulnerability assessments that reveal exploitable system vulnerabilities to encourage development of system fixes, and integrated risk analysis, which helps stakeholders assess their security posture and hasten their ability to mitigate potential risks. Computing, data analysis and visualization, and Big Data management are keys to solving cyber security issues.

General Conclusions (Panel On Doe Applied Technology Programs – Advanced Computing For Energy – Promises and Challenges)

The DOE Energy Program Offices vary in their rate of adoption of High Performance Computing tools, but all of the Assistant Secretaries acknowledge that the complexity of the growing energy system requires the efficient use of High Performance Computing. All of the Offices have a strong link to industry, and all support research in the pre-competitive areas that contribute to the industry as a whole. The high cost of High Performance Computing makes support to this technical area a natural role for government, but the results of both the research and computing must find their way into the marketplace. Possible solutions include:

• increased participation by industry in both the physical and computational research

• industry input in the definition of the problems and challenges that represent barriers to technology adoption

• identification of the high-impact applications common to multiple market areas where government investments can be leveraged into multiple fields

• easier access to computational tools with partnerships involving industry, academia, and the national laboratories

• building of additional capacity to use High Performance Computing for energy innovation through workforce training and expanded university programs.

PANEL ON ENERGY INNOVATION THROUGH ADVANCED COMPUTING: SUCCESS STORIES

Moderator: Steve Ashby, Pacific Northwest National Laboratory

Panelists:

• Dave Turek, IBM

• Gary Leonard, General Electric

• Wayne Eckerle, Cummins

• Steve Gravante, Navistar



Overview

Combustion engines consume energy and generate pollution. All of the manufacturing companies represented on the panel are using computer simulation to improve engine product efficiency and reduce the emission of pollutants. From being an ad hoc reactive discipline in the 1990s, High Performance Computing (HPC) has become a predictive, proactive tool that is fully integrated into the product development process of these companies. Challenges remain. There is a desire for greater simulation accuracy to reduce risks and allow more analysis of the available design space. There is a need for multi-physics coupled models as well as better post-processing and data management tools. The need for continued software development for HPC architectures was addressed as well as better HPC training for engineers and design staff.

Bringing High Performance Computing to Energy Problems (Dave Turek, IBM)

The increased power of HPC offers industries a new path in which to move from relatively crude models used in product design to higher-fidelity models that enable better designs and shape the design and manufacturing processes. EXA Corporation, for example, uses an IBM Intelligent Cluster solution, which includes System x iDataPlex and IBM Platform Computing LSF workload management software to run their computation fluid dynamics (CFD) software. This optimized solution design allows for precise testing of aerodynamic simulation concepts, enabling EXA to quickly and cost-effectively help manufacturers plan and test models of new transport trucks. The result is an approximate aerodynamic drag reduction of 24 percent, which translates into an average company saving around 12 percent in fuel consumption and $6000 per year in operational costs for a single truck.

From EXA's original focus on digital wind tunnels as design tools, the company now applies computational fluid and thermal dynamics to many design aspects of moving vehicles, using highly detailed models of components such as the vehicle’s engine compartment and undercarriage. This capability for simulation-based design has also driven the design and manufacturing processes, resulting in fewer prototypes needed, faster turnaround, and increased automation.

Another example of applying HPC in simulation-based design is the development of the water-cooled design for the IBM BlueGene supercomputer. The simulation-based design achieved a two times improvement in heat dissipation compared to the prior air-cooled design, increasing the energy efficiency of operation by requiring less air conditioning.

High Performance Computing at GE (Gary Leonard, GE)

HPC is one of GE’s most important technologies. Computer simulation is seen as the equivalent of a microscope whose magnification is dependent on the amount of computational horsepower that can be brought to bear on a problem. HPC is used to optimize the design space leading to products with greater value and efficiency. GE products are estimated to consume $150 billion of oil and gas fuel a year; thus designing for improved fuel efficiency is critical. GE maintains that it already has excellent collaborations with DOE national laboratories and its core hour usage at DOE facilities is growing rapidly. The major challenges that need to be addressed for HPC to play an even bigger role at GE have to do with software development, scalable software, code validation, post-processing and Big Data



management. GE uses third-party software. Such software needs to be affordable at scale when combined with HPC. Employee training in HPC was also noted as key.

The Importance of Analysis in the Cummins Engine Development Process (Wayne Eckerle, Cummins)

Analysis-led design replaced component-based design at Cummins more than a decade ago, changing a design regime from reactive to proactive and accelerating the development process while improving productivity. In retrospect, this approach, analysis-led design, is seen as having been a necessity in order for Cummins to remain competitive in its global market. HPC is used in the design process for structural and dynamic analysis, combustion system design (such as predicting emission levels) and 3D computational fluid dynamics (CFD). While key advances due to HPC have been made, there are perceived gaps. There is a need for better multi-physics tools, a better understanding of component fatigue, predictive noise simulations, and so on. For example, Cummins would like to do a Fully-Coupled Large Eddy Simulation with sprays and chemistry. This is not currently feasible. Cummins sees significant opportunities for expanded HPC simulation, particularly in controls and catalyst modeling.

Analysis-Led Product Development (Steve Gravante, Navistar)

Computer Aided Engineering (CAE) is crucial to Navistar’s product development, and advanced analysis and simulation is fully integrated into the product design and verification process. CAE at Navistar is a highly iterative and highly serial process as designs move from initial concept to final product. CAE is used to explore the design space and ultimately narrow it down to the most optimal design for customer satisfaction. Navistar’s goal is to move the discovery phase upstream in product development through analysis, discarding undesirable designs early in the process. The cost of changes becomes higher the later they are made in the design process. Simulation is used to test and verify designs and shorten the development cycle. Navistar sees a need for technology transfer of models and software, especially for computer systems in the 100- to 500-core range. Simulation can predict failures and recommend design improvements, but expansion of simulation’s role is dependent on its cost and the user perception of its risks and accuracy.

PANEL ON ENERGY INNOVATION THROUGH ADVANCED COMPUTING: POTENTIAL AND CHALLENGES

Moderator: Dona Crawford, Lawrence Livermore National Laboratory

Panelists:

• Sumit Ray, Westinghouse

• David Sun, Alstom

• Satish Narayanan, United Technologies Research Center

Overview

The advantage of using advanced computing resources by energy companies is not always clear. Energy companies may not see a clear analytical need or convincing evidence in the business case for using advanced computing. To address these concerns, a series of leading energy companies discussed the



nuances of their decisions to not take advantage of advanced computing or delay a significant investment in expanding, noting the context of computing approaches currently in place.

Supercomputing Applications for Nuclear Reactors – a Westinghouse Perspective (Sumit Ray, Westinghouse)

Westinghouse understands that advancing solutions in their domain needs to be balanced with solving current problems. There is room for innovative thinking and approaches if current and future problems can be solved or averted. Throughout their production and facility lifecycle, there are a variety of applications that could benefit from advanced computing. As an example, Westinghouse has created the “Virtual Reactor” to provide better understanding of a number of processes and potential concerns; this has proven to be a valuable tool. However, Westinghouse has identified the following challenges to pursuing an advanced computing initiative:

1. Advanced computing can be expensive, and that expense and value need to be clearly demonstrated to allocate funds for the effort.

2. There is an internal lack of critical skills and expertise at Westinghouse.

3. The need for validation is critical, yet with Westinghouse’s nuclear energy domain, getting suitable experimental data at very high resolution is very expensive.

Innovation Opportunities and Challenges for Advanced Computing in Smart Grid Operation (David Sun, Alstom)

Alstom presented how the electric utility infrastructure has evolved from a physically based, vertically integrated system to a distributed decision-making system, where markets are much more dynamic. This evolution has resulted in new challenges, including maintaining reliability and security, integrating volatile components such as renewables, and a vast increase in the number of decision makers and data flows. These changes necessitate new business protocols. The challenges to incorporating advanced computing include the following:

1. the challenge of workflow from data to simulations to results because of the lack of standardization of analytical approaches and data management

2. concerns with the ability of current and future software interoperability and the long-term permanence of computing resources

3. a lack of consensus as to the benefit to drastically improve or change current modeling and simulation approaches

4. a lack of clear business case for the regulated electric utilities.

There is potential for applications of advanced computing for the electrical grid in the following areas:

1. decision support tools for grid operators

2. the implementation of the self-healing grid

3. workforce training through explicit simulation and scenario analysis

4. validation of business practices.



Role of Computations and Mathematics in Delivering High Performance, Energy Efficient Buildings: Some Industry Perspectives (Satish Narayanan, United Technologies Research Center)

United Technologies Research Center presented how there is great transformative potential for advanced computing to aid in the reduction of energy used by the building sector. There is a range of needs for computation in the industry, from optimization modeling to uncertainty quantification and data assimilation. If incorporated, advanced computing simulation and modeling could significantly speed up the design cycle; reduce model, performance, and operational uncertainty; and reduce commissioning and operation and maintenance costs by a factor of five.

Within the building life cycle, there are multiple challenges for incorporating advanced computing resources. Primarily, the industry is focused on decision making (in design, controls, operations, etc.) and not explicitly on prediction; the value that simulation can provide is not inherently clear. In addition, the industry is highly diverse, with firms of many sizes and computational tools, and a clear computational pipeline is hard to identify. That said, the industry can benefit from tools that help current and future (dynamic) decision making, from improved quality of model-based design to the creation of reduced-order models for testing and verifying control schema.

General Conclusions – Panel on Energy Innovation Through Advanced Computing: Potential and Challenges

The companies represented on this panel were not new or adverse to advanced computing. They all currently use complex computational resources to model and simulate, visualize, and render solutions to solve problems for their customers. This makes their hesitation and barriers even more salient; these are strong “next adopters” if the business case can be made.

From the discussion, the following were highlighted as possible solutions to lowering the barriers for their deeper investment in advanced computing:

• demonstration, through examples, of a short-term return on investment (ROI) or business case for adopting advanced computing approaches and simulation

• enumeration of high-impact applications across the energy domain

• the creation of easy collaboration tools between industry, academia, and the national laboratories

• life-cycle analysis of advanced computing platforms—from user/developer perspectives— to establish technology and economic targets

• workforce training on current and future computational approaches.

BREAKOUT 1: DOE APPLIED PROGRAMS, MODELING AND SIMULATION – GRAND CHALLENGES



Lead: Ray Stults (National Renewable Energy Laboratory)

Panel Members: Dana Christensen (National Renewable Energy Laboratory), Mike Crowley (National Renewable Energy Laboratory), Fort Felker (National Renewable Energy Laboratory), Steve Hammond (National Renewable Energy Laboratory), Mike Knotek (National Renewable Energy Laboratory), Stephan Lany (National Renewable Energy Laboratory), Gary Leonard (General Electric), David Miller (National Energy Technology Laboratory), Martin Ossowski (North Dakota State University), Arthur Pontau (Sandia National Laboratories), Mike Robinson (Department of Energy-EERE), Doug Rotman (Lawrence Livermore National Laboratory), Alex Simpson (General Electric), Ned Stetson (Department of Energy-EERE), Jack Wells (Oak Ridge National Laboratory)

INTRODUCTION

Across the broad research community and especially at the DOE National Nuclear Security Administration (NNSA) and Office of Science laboratories, computation is firmly established as an equal and indispensable partner, along with theory and experiment, in the advance of scientific knowledge and engineering practice. In fact, numerical simulation is the only viable way to advance our knowledge and understanding in many areas of scientific pursuit. For example, the design of practical and efficient catalysts for biomass conversion requires the ability to predict, at the molecular level, the detailed behavior of the large, complex molecules and materials involved in catalytic processes. Since even the most sophisticated experimental techniques are unable to provide the fidelity necessary to resolve chemical reactions occurring at the surface of a catalyst, advancing our understanding of the underlying phenomenology is only possible by taking advantage of high performance computational hardware and software.

Numerical simulation enables the study of complex systems and natural phenomena that would be too expensive, too dangerous, impractical, or even impossible to study by direct experimentation. Historically, numerical simulation was used as a qualitative and more holistic guide in the design and control of complex systems, and not necessarily expected to provide quantitative results. Simulation is now used in a more quantitative way, as an integral part of the manufacturing, design and decision-making processes, and as a fundamental tool for scientific research. Advances in High Performance Computing (HPC), numerical methods, algorithms, and software design, now enable scientists and engineers to solve large complex multi-scale and multi-physics problems that were once thought intractable. Similarly, advances in online digital instruments, massive data storage capacities, and powerful “data mining” techniques are enabling the full exploitation of rich, data-intensive, computational results that advance new physical insight and scientific discovery. At the same time, high-speed networks provide virtual proximity between scientists, instruments, and data, facilitating unprecedented dynamic interaction and collaboration.

Detailed numerical simulations utilizing HPC modeling and analyses will enable the exploration and quantification of renewable resources and assist in the design and performance optimization of energy extraction systems based on first principles. Extracting energy from renewable wind, solar, and marine hydrokinetic resources requires large aggregations of physical devices to achieve production parity with conventional generation. The complexities of quantifying the resource, device/plant interaction, and



subsequent micro and macro environmental impacts requires an assessment across a wide variety of configurations, alternatives, and possible designs to identify optimal components, processes, and systems. HPC modeling offers the only viable assessment and validation method over traditional experimentation or trial and error. In particular, these efforts will save time and money, significantly improve the likelihood of breakthroughs and useful advances, and reduce the risks and uncertainty in individual components to fully integrated systems that are often barriers to industry adopting new and innovative technologies.

The breakout group charged with examining the role of modeling and simulation for the DOE energy programs examined opportunities to impact the six strategies (Figure 1) identified in the Quadrennial Technology Review for addressing our national energy challenges.1

Figure 1. Quadrennial Technology Review – Six Strategies to Address National Energy Challenges

At a high level, the group viewed modeling and simulation as playing a strong role in advancing all six strategies. To provide more detail to the role of modeling and simulation, the group developed a set of quad charts to define a set of goals/opportunities, challenges/barriers, approach to overcome barriers and potential impacts in specific technology areas. The group examined seven areas of energy R&D ranging from materials for advanced energy to deployment of renewable energy on the grid. In numerous energy technology areas, strong modeling and simulation programs are ongoing, including many of the Energy Frontier Research Centers (EFRCs), the Consortium for Advanced Light Water Reactors (CASL), and the SunShot Program. In this brief report we present five of the quad charts for energy technologies that have strong potential to improve our energy future if commitment and resources are forthcoming in the future.

1 DOE – U.S. Department of Energy. 2011. QTR Report on the First Quadrennial Technology Review. DOE/S-001, Washington, DC. Accessed January 21, 2013 at http://energy.gov/sites/prod/files/ReportOnTheFirstQTR.pdf



The five high-impact areas are the following:

• Wind Energy

• Electric Grid

• Fossil Energy – Geologic Formations

• Fossil Energy – Carbon Capture

• Vehicles Innovation.

CHALLENGES, RECOMMENDATIONS, IMPACTS

The following quad charts highlight opportunities and goals, challenges and barriers, an approach and actions to overcome barriers, and potential impacts for each of the five high-impact energy technology areas.

Modeling and Simulation for Energy Innovation: Wind Energy

Opportunities/Goals

• Delivering wind energy at ½ cost

• Transitioning from turbines to plants and facilitating high-penetration scenarios

• Seamless/automated plant-to-grid interconnection, transmission, and dispatch strategy

Approach to Overcome Barriers – Actions

• Focused effort, e.g., Wind Technology Innovation Hub, to achieve 50% cost reduction and facilitate high penetration—integrating across DOE, industry, and academia

• Multi-scale, multi-physics computational solutions

• Field observations and validation data at multiple scales and multiple sites

Challenges/Barriers

• Increasing the current power generation capacity to increase energy capture

• Defining the “in-situ” wind plant operating environment driving loads

• Forecasting and integrated control of turbines as a plan system to optimize power and performance

• Achieving grid integration, dispatch, and control of high-penetration renewables

• Quantifying the potential macro and micro environmental impacts

Impact

• Drive cost reduction.

• Accelerate market deployment to high renewable penetration.

• Reduce greenhouse gas emissions while delivering power in a reliable grid.

• Enable expanded public-private partnerships.



Modeling and Simulation for Energy Innovation: Grid

Opportunities/Goals

• Dynamically optimize grid operations and resources.

• Fully integrate dynamic intermittent sources, demand response, and consumer participation into grid resource planning and operations.


• Measurement and Controls: Acquire, share, and process data throughout the electric system.

• Communication and Security: Establish a secure, resilient information backbone.

• Modeling and Analysis: Facilitate system understanding to support better grid operations, planning, markets, and policy decision-making.

Challenges/Barriers

• Future grid mix is unknown.

• The intermittent nature of wind/solar requires new ways to operate the grid ecosystem.

• Changes in technologies and policy

• Transforming the grid has been likened to swapping an aircraft engine in flight.

• Public perception and acceptance

Impact

• More efficiently match supply and demand

• Improved long-term infrastructure planning

• A secure, reliable, and resilient grid

Fossil Energy – Geologic Formations (Fracking, Enhanced Oil Recovery, Carbon Storage)

Opportunities/Goals

• Answer question: Is CO2 storage safe and permanent?

• Answer question: Is fracking safe?

• Ensure science-based understanding to build fracking best practices.

• Enable better waste management and disposal practices.

• Answer question: Can we design novel, science-based technologies for unlocking oil from oil shale/tar sands?


• Design environmentally benign "smart" fracking fluids.

• Improve simulation of fracture propagation.

• Develop a better understanding of emission dynamics.

• Develop infrastructure to enable remote execution of simulation codes to answer questions in the field.

• Develop multi-scale approach to simulate kerogen nanoparticle-/clay/rock systems.

• Design methods to quantify uncertainties associated with large-scale geologic simulations.



Challenges/Barriers

• Tie physics and chemistry of fluids (fracking and CO2) with geomechanics – multi-scale/multi-physics.

• Validation of codes

• Need to characterize and understand failure mechanism of the well bore

• Geological formations have diverse characteristics.

• Interaction of hydrocarbon nanomoyeties (e.g., in kerogen) with minerals

• Oil/gas industry not broadly participating in HPC

Impact

• Ensure the long-term viability of shale gas/oil resources.

• Kerogen is an abundant source of hydrocarbons in the U.S.

• Ensure safety and viability of carbon storage.

• Provide scientific foundations for regulations of fracking, carbon storage, etc.

Fossil Energy – Carbon Capture

Opportunities/Goals

• Accelerate development and commercial deployment of capture technology.

• Reduce the cost of capture to $10/ton captured.


• Develop and apply validated multi-scale models for predicting behavior of whole systems at large scales (requires data).

• Develop and apply uncertainty quantification methods for multi-scale models. Identify sources of uncertainty.

• Integrate complex simulations with system-wide, multi-scale optimization (i.e., screening new materials in context of potential processes).

Challenges/Barriers

• Larger scale than ever before deployed, resulting in technical and enterprise risk

• Long scale-up period typically required

• Need for development of materials, equipment and processes

• Complex processes (chemistry, multi-phase flow, multi-scale phenomena, systems integration)

• Lack of large-scale validation data

Impact

• Reduced time and risk for scale-up and deployment

• Enable cost-effective carbon capture (will allow Enhanced Oil Recovery [EOR], which will facilitate more adoption).

• Enable environmentally responsible use of fossil fuels to support energy security while also mitigating climate change for both natural gas and coal.



Modeling and Simulation for Vehicle Innovation

Opportunities/Goals

• Improved combustion engine performance

• Materials: catalysis, hydrogen storage, batteries, structural

• End-to-end batteries, fuel cell systems, total vehicle

• Alternative fuels


• Develop the infrastructure, both hardware and software, to implement the next generation of advanced multi-physics models.

• Integrated experimental and modeling/simulation program

• Coordinated vehicle industry, DOE Lab, and software vendor effort

Challenges/Barriers

• Breadth of length/time scales, complexity

• Understanding of underlying physics/chemistry/kinetics

• Modeling/simulation development “ecosystem” inefficient

• High experimental costs

Impact

• Improved vehicle efficiency

• Shorter, cheaper development times

• Reduced petroleum use

• Enabling alternative vehicles

• Lower environmental impact

DETAILED RECOMMENDATIONS FOR WIND ENERGY AND NEXT-GENERATION GRID

Each quad chart provides brief recommendations for the specific energy technology. In this report we are providing expanded recommendations for two areas: wind energy and next-generation grid. Wind energy is chosen as a representative of a bigger set of renewable energy technologies that will be key as we address energy security and climate change. Next-generation grid was chosen because of its importance for our energy future and our ability to deal with natural disasters.

Wind Energy Recommendations

The United States brought roughly 6.8 GW of new wind energy capacity onto the electric grid in 2011, bringing the total wind capacity to over 50 GW. In order to provide 20% of U.S. electricity from wind energy by 2030, installed wind capacity must increase to over 300 GW. Simulation can bring innovative solutions to achieve this 6-fold leap in national wind energy capacity and energy security. Success is centered on an integrated approach combining simulation and experiment from resource/plant/turbine/grid interaction to 1) cut by half, the cost of delivering wind energy and 2) ensure seamless and efficient integration of wind energy into the national electrical transmission and distribution system. Key modeling and simulation opportunities include

• quantifying the wind resource as a national, strategic energy reserve

• modeling multi-turbine arrays and control strategies for optimized wind plant performance



• designing the next generation of “wind plant” optimized turbines, components, and control systems for improved reliability, lowered cost, and optimized energy generation

• forecasting wind energy generation for reliable smart grid integration at high renewable penetration

• quantifying and reducing all macro- and micro-environmental impacts from high renewable penetration across the United States.

Partnership across the public and private research and business communities is the surest and fastest pathway to achieving our national goals. In particular, the creation of a focused Wind Technology Innovation Hub can integrate national research laboratories, science experts in academia, and industrial leaders to apply the power of High Performance Computing and Simulation to essential multi-scale and multi-physics computational solutions. Included in this Hub would be field observations and validation data sets at multiple scales and multiple geographic sites to examine, understand, and ensure the quality of our simulation results. The Hub and its rich set of partnerships would be the innovation vehicle to ensure that wind energy generation meets our national goals.

Next-Generation Electrical Grid Modeling and Simulation with High Performance Computing

Today’s electrical system is undergoing a period of rapid change that requires a new generation of modeling and simulation tools using advanced High Performance Computing. Many changes are being introduced into the electrical system, including changes in load; transmission; two-way communication; availability of ubiquitous, real-time, high-quality data from an array of new devices; and dramatic innovations in the use of computing and software for grid planning and operations. To support these changes, computational, mathematical, and scientific understanding is needed to transform the tools and techniques that underpin the planning and operation of the electric system to allow for advancements in the following areas:

• Accelerate Performance – improving grid resilience to fast-time-scale phenomena that drive cascading network failures and blackouts

• Enable Predictive Capability – relying on real-time measurements and improved modeling and simulation techniques to more accurately represent the operational attributes of the electric system, enabling better prediction of system behavior and thus reducing margins and equipment redundancies needed to cover uncertainties

• Integrate Modeling Platforms (across temporal and spatial scales) – capturing the interdependencies and interactions that will allow development of new control techniques and technologies and better integrate variable-output renewable energy.

For example, the size of many grid operations today motivates the need for energy management systems (EMSs) to perform massive contingency analysis for assessing the dynamic security of the power grid. Massive Contingency Analysis (MCA) provides useful information for timely, preventive, and corrective actions and thus is a crucial requirement for the proper functioning and maintenance of the modern EMS. However, the computational intensity of time-domain simulations limits the number of contingencies that can be simulated in faster than real-time to support grid operations in an online environment.



Computational centers for the power grid and regional electrical grid hubs are needed to demonstrate the transformation of the electrical grid from a reactive to a predictive paradigm by accelerating performance and predictive capability. These centers would be formed by partnerships with key electrical grid stakeholders, national laboratories, experts from academia, and members from utilities and grid operators. Utility operators, by their very nature of having to “keep the lights on,” are very conservative in their operational approach; hence they require demonstration projects to reduce their risk. Hubs and regional centers would reduce the risks for the new electrical grid operating paradigm and help transform the electrical grid to the next-generation system.

BREAKOUT CONCLUSIONS

The purpose of this breakout session was to explore the grand challenges associated with using advanced computing for innovation within the DOE applied technology programs. The breakout discussed demonstrated successes and explored areas where advanced computing could have major impacts. The group concluded that there were several of these areas as shown above.

The breakout group also found that significant barriers stand in the way of achieving the potential impact that advanced computing could have on the DOE applied technology programs. These included the complications of properly simulating the physics involved with the energy application and the need for strong validation that the simulation results provide a good reflection of that physics. However, given these concerns, the breakout group believes that there are excellent opportunities where advanced computing could make a significant impact and be a potential “game changer.”

BREAKOUT 2: CURRENT USERS – GRAND CHALLENGES



Co-Leads: Moe Khaleel (Pacific Northwest National Laboratory), Steven Lee (Department of Energy-SC)

Panel Members: Steve Ashby (Pacific Northwest National Laboratory), Charles Barbour (Sandia National Laboratories), Philippe Bardet (George Washington University), Phil Colella (Lawrence Berkeley National Laboratory), Lee Ann Dudney (Pacific Northwest National Laboratory), Kevin Duffy (Caterpillar), Thomas Halsey (Exxon Mobil), Henry Huang (Pacific Northwest National Laboratory), Barbara Hutchings (ANSYS), Doug Kothe (Oak Ridge National Laboratory), John Kuzan (Exxon Mobil), Terrence Liao (Total Oil), John Magerlein (IBM), Mark O'Riley (IBM), Bruce Robinson (Los Alamos National Laboratory), Tariq Samad (Honeywell), Ramanan Sankaran (Oak Ridge National Laboratory), Gene Schultz (Boeing), Alex Simpson (General Electric), Suzy Tichenor (Oak Ridge National Laboratory), John Turner (Oak Ridge National Laboratory), Timothy Valentine (Oak Ridge National Laboratory), Chap Wong (Chevron)

INTRODUCTION

As shown in the Figure 2 chart, among the 25 panel participants from 16 institutions, almost half represent industry. The industrial participants were all large enterprise companies. The small to medium-size market sector not represented in this panel is often described as the “The Missing Middle”2 (manufacturing companies employing fewer than 500 people). There is a perception that the “missing middle” issue impacts the pace of innovation in companies addressing the nation’s critical energy problems. As the “new torchbearers of innovation,” these companies need 21st century tools to compete and more rapidly move their technologies into the marketplace.

Figure 2. Makeup of Panel Participants

General agreement exists that advanced computing is crucial to industry, is an essential part of the design process, and that the companies represented could readily justify advanced computing investments.

2 Riley J. 2011. The Invisible Innovators: What is the Missing Middle? And Why Do We Need Them? Accessed November 8, 2012 at http://www.digitalmanufacturingreport.com/dmr/2011-06-15/the_invisible_innovators:_what_is_the_missing_middle_and_why_do_we_need_them.html (last updated June 17, 2011)



While lack of access to advanced computing hardware is not generally a concern for large enterprises, there exists a further need for hard data and evidence (profitability, jobs, competitiveness) to support DOE facilitating wider adoption of advanced computing for a wider range of commercial organizations. Where is the evidence to support pushing advanced computing to small and mid-sized organizations? What should the government’s role be?

This panel report focuses on what it took for current users of advanced computing to attain the proven impact of advanced computing and includes recommendations based on lessons learned.

OPPORTUNITIES

The opportunities available to users of advanced computing provided many compelling use cases. Organizations obtained better answers and made faster decisions, reducing time to market. They were able to simulate with higher fidelity and with reduced uncertainty, resulting in safer products. They could solve problems at a larger scale and/or with greater details (using finer meshing, for example). Consequential decisions could be made based on “predictive” results. Complex problems could be solved that were beyond the means of historic data and established knowledge. Advanced computing could be used as a virtual microscope and as an enabler, allowing development by simulations rather than expensive and/or risky testing. New systems could be designed from first principles.

Examples where advanced computing has been used to address challenges and solve problems include upstream oil and gas assessment, seismic data image processing and reconstruction, 3D thermal modeling, hydrocarbon visualization, heat transfer modeling, reservoir simulation, combustion modeling and simulation, engine building, metal casting simulation, aerodynamics characteristics and signatures prediction, and modeling the flow of water in a nuclear reactor at ten times finer resolution than prior methods.

Justifications for advanced computing investments range from being better able to discover oil, to reducing the testing time needed for new products, to addressing problems that are impractical to solve in other ways (such as physical methods). For the oil industry, advanced computing is an inexpensive investment relative to the cost of drilling wells in the Gulf Coast. It enables dealing with more complicated models and improved quality of images used to determine where to drill. For the transportation industry, the cost of testing (such as for mechanical fatigue) is well understood from years of experience, so the use of advanced computing for testing is financially beneficial by comparison. Reduced testing time results in getting products to market faster.

CHALLENGES

Establishing a Clear Business Case for Advanced Computing

Given the large initial investment required for advanced computing, companies are unwilling to adopt unproven technology, so identifying the return on investment (ROI) is important. If it makes business sense and if advanced computing can be used to solve problems for less cost than more traditional methods, then simple ROI calculations can justify its use. The ROI calculation must include all the hardware, software, and support costs, and include comparison to known benchmarks (such as cost and time of design, testing, and prototyping). Identifying what can be achieved that cannot be done via



traditional means or scale—including identifying (and quantifying, where possible) product improvements that can be realized—also helps. Testing an HPC system with a DOE laboratory or other technology provider has also been used to justify the need for advanced computing. The justification should clearly articulate the business value that supports the company’s mission.

Accessing HPC Expertise

Industry, in general, understands the importance of HPC but needs to be clear about its business value, available capabilities (including what and from whom), and the problems that HPC is expected to solve. Without access to HPC expertise, it is difficult for industry to achieve those understandings. Industry must be vocal about the need for software and support.

For example, in the area of wind turbine development, challenges exist in knowing the level of fidelity needed to deal with a specific problem and in specifying what computational resources would be needed to study such problems. Such questions could be addressed and perhaps answered via in-depth discussions with HPC experts and computational scientists.

Lack of access to supercomputers makes industry’s use of advanced computing a challenge, especially in the area of developing HPC code (that meets industry-specific needs). Universities can more readily partner with national laboratories to get access to HPC hardware resources to help overcome this challenge. Expanding on existing programs like the Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program and the discretionary computer time allocations available on the leadership-class machines and National Energy Research Scientific Computing Center (NERSC) could also be beneficial.

Determining Effective Computing Life Cycle Models

Challenges exist with respect to optimizing hardware utilization, developing software internally versus outsourcing software development versus buying commercial software, and deploying, supporting, and maintaining software. Most companies use a variety of models in these areas.

Hardware that spans high-end desktops to clusters to supercomputers is purchased and maintained in-house (for production use and in-house development, for example), and leased or rented (for research purposes, hosted cloud, or use of a national laboratory user facility, for example). Industrial companies who maintain their own internal code go to national laboratories to scale, refresh, test, and/or benchmark their codes, to help determine the feasibility of doing this before they make an internal hardware investment.

Industry does whatever is most appropriate to obtain software. Codes are developed in-house, bought externally, outsourced to third party developers, or obtained from open source, depending on the circumstances. One practice by industry is to use commercial software in production and develop its own research codes. It may be preferable to develop software internally because of better access to validation data; however, commercial firms can offer better tools and potentially better support. Industry might have a proprietary software technology to protect or might see competitive advantage in developing particular intellectual property (IP).



Industry is open to using open source software from national laboratories and universities. However, a few important national laboratory-developed codes are not freely available to industry or are not publicly available. Advantages of open source code are seen as the ability to release it for broad impact and the ability to modify it as needed with different algorithms, making changes to enable coupling, scalability, and so on. However, some companies avoid using open source code in applications used to make business decisions.

Software maintenance and support are also provided by a variety of mechanisms. Common practices are to support in-house developed codes internally (sometimes via a central help desk and in some situations via individual programs) and to use vendors and/or consultants to support commercial software.

Engaging and Resolving IP Issues with the National Laboratories

Industrial companies do not have good visibility of advanced computing capabilities available within the national laboratory system. Also, industry perception exists that the culture of competition between national laboratories is excessive, as evidenced by different laboratories developing competing codes, which has led to poor compatibility and confusion.

The cost of working with the national laboratories is viewed as overly expensive, with contracting mechanisms that are considered onerous and cumbersome. Technology transfer agreements take too long and entail too much uncertainty. There exists a need for more openness and clarity on commercialization, given broad concerns about engaging with the national laboratories because of IP issues. Administrative barriers in DOE’s contract language are seen as a major obstacle, which are typically overcome only with lengthy, labor-intensive negotiations (which improve in subsequent Cooperative Research and Development Agreements—CRADAs). In addition, multiple mechanisms and interpretations across different national laboratories regarding the technology transfer process lead to inconsistencies in the ways that industry is able to interact with the laboratories.

RECOMMENDATIONS

To accomplish the recommendations offered below, DOE should develop a roadmap for advanced computing for energy innovation, working with individual program offices and taking into account the needs of specific target industries. Assistant Secretaries should prioritize their program activities with respect to the modeling and simulation advancements needed to support energy innovations in those industries.

What Can DOE Do to Help?

DOE could create an ecosystem to better spur energy and computational innovation, providing the vision, leadership, and technical capabilities. DOE could help accelerate the use of simulation by demonstrating the ability to get “right answers” to tough problems, including model validation and best practices. This might be done by a DOE-funded hub that could also provide trial use of advanced computing, particularly to “The Missing Middle.” Also, DOE might partner with commercial code suppliers to assess capability in these codes and identify gaps.

DOE could work with universities to help produce more highly skilled computational scientists to use advanced models. At a minimum, making national laboratory codes easier to use would be helpful.



Make National Laboratory Technologies More Accessible to Industry

This is both a legal and a technical challenge. On the legal front, DOE should seek to develop a single, unified (yet tailorable) approach to advanced computing technology transfer. This might include a standard agreement with standard terms and conditions with respect to non-disclosure and IP protection. Given the historical difficulty of establishing CRADAS, industry is interested in large “umbrella” agreements with national laboratories. While standardized IP agreements have been created by DOE-HQ, industrial companies do not necessarily want to accept them and often require modifications, and different contractors who run the national laboratories may have their own business models and corporate legal requirements. It is unclear whether DOE would need to modify its Management and Operations contracts to require a uniform approach.

On the technical front, a clearinghouse of national laboratory capabilities should be made available to industry. This could include available algorithms and software tools that are a long-time strength of the national laboratories. DOE should establish an institutional model by which national laboratories can collaborate with various industrial sectors to do the necessary adaptation/customization and support of laboratory-developed software and algorithms to make them usable by those commercial sectors. DOE should offer public application programming interfaces (APIs) that provide a means to allow DOE codes to plug and play with other technologies. Also, DOE should encourage advanced computing technologies developed in the national laboratories to coexist and be delivered through independent software vendors (ISVs) to the wider innovation market.

Create Incentives for Greater Adoption and Use of Energy Innovation

DOE should establish an easily accessible means for energy innovators to conduct quick engagements with laboratories to determine whether advanced computing could make a difference to their business. The National Digital Engineering and Manufacturing Consortium (NDEMC) public-private partnership pilot program led by the Council on Competitiveness could offer useful results and lessons learned toward this.

DOE should establish procedures for temporary exchanges of staff between national laboratories and industry, to help companies use and understand the benefits of technologies. For example, industry and national laboratories might exchange experts on sabbaticals for 3- to 6-month assignments. Companies could get training on how to make the best use of advanced computing, and laboratory staff would get a better understanding of industry’s requirements.

DOE could better facilitate and fund laboratory staff to take national laboratory innovation (including advanced computing) into the marketplace directly. DOE can provide incentives for greater adoption via the existing DOE grant process (Small Business Technology Transfer [STTR], Small Business Innovation Research [SBIR]) if preference is given to research proposals that take an advanced computing approach (this is done to varying degrees in applied programs). Such proposals could also encourage collaborations with national laboratory supercomputer centers. Similarly, in its calls for proposals DOE could emphasize modeling and simulation approaches to drive advanced computing usage in energy challenge areas.

Finally, DOE could develop policies and standards to create incentives for adoption in those industries like the electrical power grid that have yet to move into HPC.



Be Proactive in Outreach/Education to Industry and Key Stakeholders

DOE can help companies understand how to navigate the national laboratories to get access to HPC capabilities. DOE should create a central website that consolidates information about DOE advanced computing programs and R&D activities at its national laboratories, including how to engage with them and access capabilities. Additional ways of doing this could include laboratory/industry fairs/open houses/tours to showcase tools and capabilities, forums focused on computing tools, and publication of success stories. Similarly, DOE could use existing industry domain forums as venues for proactive outreach. DOE could also make good use of technology transfer/partnership offices for outreach in advanced computing to convene a “best practices” workshop and/or establish a website on CRADA development, addressing business, IP, economies of scale, etc.

DOE should provide information that helps small/mid-size energy innovation companies understand the competitive advantage that could be gained by the use of advanced computing capabilities, working with ISVs and the Council on Competitiveness to facilitate this.

Create a Program for Advanced Modeling and Simulation for Energy Applications

Although the Advanced Simulation and Computing (ASC) program stewards modeling and simulation capabilities for the DOE National Nuclear Security Administration (NNSA), and the Advanced Scientific Computing Research (ASCR) program provides computational capabilities for scientific modeling and simulation for the DOE Office of Science, there is a need for tighter integration among applied energy program offices with respect to providing modeling and simulation capabilities for energy applications. DOE should create an Advanced Modeling and Simulation for Energy (AMS-E) program to facilitate and integrate planning for development of the core computational technologies that would enable a variety of energy applications—as illustrated in Figure 3.

Figure 3. Energy Applications Would Benefit from Integrated Development of Core Enabling Technologies



IMPACTS

Advantages that have been realized from advanced computing include reducing the time to design products and build prototypes, which reduced cost. In some cases, advanced computing was the only practical means available to solve a problem or conduct investigations (for example, handling and analyzing large amounts of data). Greater reliability, safety, and utility have been realized, and in some cases the use of advanced computing led directly to greater profit.

BREAKOUT 3: POTENTIAL USERS – GRAND CHALLENGES



Lead: John Grosh (Lawrence Livermore Laboratory)

Panel Members: Gil Bindewald (Department of Energy-OE), John Burns (Virginia Tech), Steven Chu (Department of Energy), Trevor Cook (Department of Energy-NE), Dave Corson (Altair), Dona Crawford (Lawrence Livermore National Laboratory), Tom Cully (Dassault Systèmes),Michael Derby (Department of Energy-EERE), Noah Goldstein (Lawrence Livermore National Laboratory), Steve Gravante (Navistar), Robert Graybill (Nimbis Services, Inc.), Benjy Grover (Lawrence Livermore National Laboratory), Ahmad Haidari (ANSYS), Kirk Jordan (IBM), Mahesh Kailasam (Dassault Systèmes), Walt Kirchner (Council on Competitiveness), Scott Kruger (Tech-X Corporation), Bob LaBarre (United Technologies Research Center), Eugene Litvinov (ISO New England), Justin Marble (Department of Energy-EM), Colin McCormick (Department of Energy), Satish Narayanan (United Technologies Research Center), Sumit Ray (Westinghouse Electric Corporation), Steve Smith (Lawrence Livermore National Laboratory), Fred Streitz (Lawrence Livermore National Laboratory), David Sun (Alstom), Paul Turinsky (North Carolina State University), Paul Van Slooten (United Technologies Research Center), Tim Wagner (United Technologies Research Center), Gil Weigand (Oak Ridge National Laboratory)

INTRODUCTION

The goal of this session was to examine the barriers to adoption of advanced computing technology and opportunities for companies and business sectors where advanced computing could be applied but is not. Important example sectors include electric grid and building energy systems, where simulations and analytics are run on desktop or small server computer systems. In these communities, adoption of leading-edge computational science is very limited. The crux of the challenge is that advanced computing separated from the business context has little or no value for U.S. economic competiveness. For example, it has been recognized by early adopters in the electric grid community that advanced computing would have a major impact in understanding and designing the next-generation “smart grid.” The challenge is that advanced computing approaches have not been adopted because there is no industry-wide understanding of benefits and acceptable business cases. The major barriers in such industrial sectors are driven by the business culture, one that is risk-adverse and where advanced computing solutions have not been clearly demonstrated to having value to the businesses who could use tools.

The breakout session attendees included a mix of industry, laboratory, and academic participants who have not adopted advanced computing approaches and those who have. To accomplish the charge of the workshop, the session working group adopted a three-phase strategy. The first was to develop a lengthy list of barriers to adoption. The session organizers felt that it was important to examine in depth the reasons why advanced computing was not valued. The second was to develop a set of recommendations that addressed these barriers. These would represent a mix of short- and long-term actions, some focused on research and development with others focused on policy. Finally, we attempted to address the “so what” questions: if we move forward with a recommendation, what does this buy for a business sector, company, etc.



CHALLENGES - BARRIERS TO ADOPTION

The cultural and technical divide between the advanced computing community and potential users can be very significant and forms the basis for why non-traditional users have such a difficulty. At a high level, these barriers can be viewed in three dimensions.

• The first is lack of understanding. The national laboratories do not understand how industry operates. Industry does not understand the capabilities and business models within the 17 DOE national laboratories.

• The second is lack of access and difficulty of transition. Gaining access to appropriate computing environments, software technologies, and expertise can be very daunting due to a lack of contracting mechanisms, policies, etc. to facilitate such interactions.

• The third is a lack of DOE strategy. DOE lacks a coherent approach to nurturing potential communities of users, reducing their risk of adoption, promoting the value of HPC, and pursuing R&D to develop products that have value and impact.

Understanding is a first important step towards bridging the two communities. For industry, the DOE laboratories are a vast and confusing community of organizations with both competing and collaborative interests. Companies would like to quickly understand laboratory capabilities and identify the right person to talk to within the laboratory system to explore opportunities—but this is difficult because of the structure of the labs. If a connection is actually made, then there are frequent mismatches between expectations and understanding of the business models and motivations. Industry may require short-term solutions, which does not necessarily match the long-term focus of some of the researchers in the laboratories. Laboratories may not understand the ROI-driven culture that drives decision making in companies. Such lack of understanding creates frustration and inhibits effective engagement. As one of the industry workshop attendee commented, “Technical links are easy. Business links are hard.”

The second challenge centers on access to HPC resources and the difficulty of transition from tools using desktop computers to those utilizing advanced computing approaches. For many companies, there is a fairly steep learning curve for using advanced computing. To reduce this risk of adoption, trial access to HPC resources and expertise—possibly subsidized by government for initial engagements—would allow industry to gain confidence in the value of these technologies and how the technologies would fit into their business models. Frequently, companies use very industry-specific software (such as electric power grid modeling) that is only available on desktop systems and not compatible with HPC systems. Conversely, many DOE tools are very difficult to use and lack the robustness of industry tools. As such, the lack of relevant software inhibits access to the benefits of advanced computing. For independent software vendors (ISVs) and companies who develop their own applications software, there are many occasions where DOE lab libraries, algorithms, and other software technologies could be adopted and integrated into commercial software. What inhibits this transition is two-fold: inconsistent laboratory intellectual property policies and the risk and cost of adapting laboratory solutions into industry software.

The third barrier is associated with gaps in the DOE’s advanced computing strategy. Traditionally, computational and computer science investments in DOE have focused on large-scale science and engineering applications, solving large-scale partial differential equations for either science or nuclear weapons simulations. While this addresses a subclass of industrial applications, there are large classes of modeling, simulation, and analytic problems (such as building energy simulations) that are dominated by



computational approaches that fall outside this traditional area. An example of a specific area historically ignored has been approaches for design-optimize-control found in many industrial applications of computing. Recently, the Advanced Scientific Computing Research (ASCR) program office has started to make basic research investments that are applicable to electric grid planning and operations (such as Multifaceted Mathematics for Complex Energy Systems). To ensure that such research traverses the “valley of death” from basic research to development to commercialization, the DOE applied energy offices must also have corresponding efforts to “catch” such research and transition it into industrial usage.

RECOMMENDATIONS - PATH FORWARD

Given the set of barriers identified, the breakout session members developed a corresponding set of recommendations (along with description of benefits) to open a pathway to potential users of advanced computing. Improving communications and relationships between DOE and industry is a key element. In addition, the DOE laboratories as a group need to reduce the confusion and level of effort required to develop meaningful technical interactions with industry. For industry, policy and funding mechanisms are needed to reduce risk and to develop and encourage industry/laboratory collaborations that promote advanced computing for energy innovation.

The first set of recommendations revolves around “getting the word out,” helping to building understanding and strong relationships between DOE and industry in advanced computing. The DOE Technology Transfer Office working with the various stakeholders in DOE would be a key player in the implementation of each recommendation below:

• Create a DOE Advanced Computing Marketing Strategy: Develop a DOE-facilitated industry/laboratory marketing strategy to enable industry to clearly identify how to work with the DOE laboratories, what laboratories do, mechanisms of engagement, and laboratory core competencies. This could be achieved through such means as workshops and newsletters.

• Develop Website/Service to Match DOE and Industry Partners: The analogy was to develop a website/service along the lines of those that facilitate dating. The concept is to improve the ability of business to interact with the laboratories for research and development support by providing multi-level matchmaking that enables company/laboratory connections and the identification of capabilities and points of contact.

• Exchange Staff between Industry and DOE: One of the most effective ways of transferring technology is face-to-face technical collaboration. This enables laboratory staff to better understand industry’s needs and challenges while industry staff can see what laboratory expertise can do for their competiveness.

• Provide Incentives for DOE Site Offices: DOE site offices provide a high-level management for DOE laboratories. Those activities range from serving as key enablers for innovations in laboratory relationships with industry to creating roadblocks. DOE should provide support and rewards for site offices who facilitate laboratory engagements with industry.

To facilitate access and transition, DOE and the laboratories need to strengthen the mechanisms that enable industry to collaborate with the laboratories, reduce risk, and develop targeted approaches in advanced computing. The recommendations corresponding to this barrier follow:



• Make Contracting Vehicles Easier to Use: Industry is challenged by each laboratory’s varying approaches to contracting collaborative research and development. DOE would create standardized vehicles for engagement across all DOE activities (templates and IP language for CRADA, Work for Others [WFO], Agreements for Commercializing Technology [ACT], etc.) to improve the ease and effort with which companies can fund laboratories for technical support.

• Provide “Test Drive” for Advanced Computing: Frequently, industry is interested in exploring advanced computing but sees the cost of entry to be too high. Given a good first experience, they would be more willing to develop more ambitious projects with the laboratories. To enable this, DOE and the laboratories would offer mechanisms such as the INCITE program (http://www.doeleadershipcomputing.org/incite-program/) and hpc4energy incubator project (http://hpc4energy.org/incubator/) to facilitate access to HPC and lab expertise to conduct pilot projects, develop collaborations, and explore the value of advanced computing solutions.

• Engage Independent Software Vendors (ISVs): In general, industry prefers to use ISV software since these are easy-to-use tools and provide robust user support. Frequently, the computational methods used in such software are not state-of-the-art and could benefit from DOE research. DOE would incentivize and fund collaborations between the laboratories and industry to transition software tools into industrial ISV applications. As a first step, DOE would support a workshop to explore how to expedite integration of DOE computational technologies into industry software.

The final recommendation is associated with the structure of DOE programs. There currently exists a gap between DOE Office of Science ASCR and NNSA ASC programs and the applied energy programs that execute more industrially-facing activities in the industry sector. The following recommendations are proposed to build a link between those, possibly implemented through an office that coordinates all such activities on behalf of the applied offices:

• Joint DOE Office of Science - Applied Energy Computational Research: Create new applied mathematics and computing research programs, beyond traditional HPC programs, to do fundamental and applied research in emerging energy areas that require advances in control sciences, optimization, and engineering design. We recommend using Office of Science/ASCR program models using open solicitations. These programs would provide a foundation for the development of new R&D and HPC tools necessary to ensure the solution of DOE problems and to address missing expertise required to advance the discipline. This program would provide a transformative approach to the development of new tools of use to both DOE and industry .

• Create Forums for Collaboration: Create forums that enable laboratories, industry and academia to jointly focus on the solution of a specific design-optimization-control problem for engineered energy systems. These could be initiated as hands-on workshops of considerable length to enable cross fertilization of disciplines and the transfer of expertise. Workshops of this type would serve to introduce external researchers and users to DOE contacts and capabilities. These workshops would provide a foundation for the development of HPC “design-optimization-control” tools necessary to ensure broad industrial usage and provide feedback between DOE and industry on tool requirements and on tool readiness to be of practical use to industry.

http://www.doeleadershipcomputing.org/incite-program/

http://hpc4energy.org/incubator/



BREAKOUT CONCLUSIONS

The set of recommendations from the “Potential User” breakout serve as a set of measures to significantly enhance the connections and accelerate ongoing efforts to build collaborations between the DOE laboratories and industry. Industry has identified a number of barriers that inhibit adoption of technologies and access to advanced computing resources that, if properly harnessed, would serve to enhance competitiveness. The improvement of laboratory/industry understanding and of the contractual and programmatic mechanics to build those bridges will help accelerated innovations. Recent SBIR proposals focused on advanced computing used in energy and manufacturing serve to highlight the promise for enabling new advances. Results from INCITE, hpc4energy incubator, and similar programs are highlighting the promise of collaborations and serve as good first steps. Deeper and sustained interactions must be pursued if DOE is to convert those industry sectors who do not employ advanced computing to its full advantage into believers and practitioners.

SUMMARY OF FINDINGS AND RECOMMENDATIONS



Summary of Grand Challenge Findings: During the course of preparing for and holding the workshop, it became apparent that there were three distinct types of grand challenges. The distinctions are important because the actions needed to overcome one type of grand challenge will be different from the actions needed for the other types. We believe that understanding the different types of grand challenges may be one of the significant findings of the workshop.

• Technical Grand Challenges: These challenges concern the technical ability of advanced computing to address energy innovation needs. Issues include the following:

• The advanced computing capabilities developed by the DOE national laboratories offer tremendous opportunities to enhance U.S. energy innovation. However, these tools are currently inaccessible to many innovators because of the level of expertise needed to productively use them to address energy innovation problems.

• DOE-developed advanced computing technologies are not immediately suitable for a particular use by energy innovators. These capabilities must be tailored to a particular use. Also, most importantly, the capabilities must be validated to ensure that results can be trusted.

• As advanced computing transitions from peta-scale to exa-scale computers, a number of challenges must be considered. These include new programming models and the ability to adapt existing modeling and simulation software. This transition will undoubtedly impact the ability of energy innovators to take advantage of the power of the new computing architectures.

• Structural Grand Challenges: These challenges pertain to the transfer and use of DOE advanced computing resources (hardware and software) for energy innovation in commercial environments. Issues include the following:

• Energy innovators and their supporting advanced computing technology vendors (software and hardware) are interested in learning what is being done at the national laboratories. However, they have found it difficult to understand what is happening at the DOE laboratories and how they might participate in the research and development activities.

• The working relationship between the Independent Software Vendors (ISVs) and the national laboratories needs to be improved. This is an important relationship because ISVs offer a means to adapt and maintain DOE-developed advanced computing capabilities for targeted commercial uses by energy innovators.

• The ability of DOE-developed advanced computing to assist energy innovators has been limited by the process and pace of completing technology transfer agreements. Companies are finding that the technology transfer process takes too long and that there is too much uncertainty in the resulting agreement. Also, companies are finding that each laboratory and DOE Operations Office has its own unique technology transfer process.

• Incentive Grand Challenges: Even if technically useful advanced computing capabilities exist and the means of leveraging those capabilities are in place, private companies and the laboratories need to have proper incentives to make investments needed to implement the use of advanced computing for energy innovation. Issues include the following:



• The workshop found that in many cases, energy innovation companies were interested in using advanced computing. However, given the large initial investment required, they were unwilling to adopt unproven technology and wanted short-term access to DOE capabilities to determine the value of advanced computing for their own applications.

• Companies generally do not have the “in-house” expertise to take advantage of DOE national laboratory-developed advanced computing capabilities and are unwilling to make investments in hiring that expertise without a firm understanding of the return on the investment.

• U.S. energy innovators need a better understanding of the benefits and value of advanced computing technologies.

• DOE and its national laboratories need a better understanding of the opportunities to help energy innovators through advanced computing.

Summary of Workshop Recommendations: During the breakout sessions there was a great deal of discussion about what should be done to address the identified grand challenges. Numerous suggestions from the workshop have been consolidated into four major recommendations:

1. Improve the usability and availability of DOE-developed advanced computing solutions.

The DOE, largely through its laboratories, develops advanced computing modeling, simulation, and analytics software for its various energy, science, and security programs. More often than not, this software is developed for expert users in support of research activities. This software must be “hardened” for commercial use by non-experts. In particular, greater attention must be paid to usability and extensibility, as well as to incorporation of the software in existing innovation workflow processes. Industry participants also noted that it was too difficult to learn about relevant computing capabilities, including available software and hardware. Here the DOE should

• simplify access to computing resources at the laboratories, including allowance of in-kind contributions to meet full cost recovery requirements

• create a few primary points-of-contact (web portal, email, and telephone) to facilitate identification by industry of relevant advanced computing capabilities and resources within the DOE and the DOE laboratories

• engage relevant and qualified ISVs to partner with the laboratories to mature and harden their software (see the next recommendation).

2. Engage the Independent Software Vendor community to promote energy innovation.

The DOE should design and implement an outreach program to relevant and qualified ISVs. As heard during the workshop, ISVs figure prominently in most companies’ use of software. Few companies have the wherewithal to develop and maintain their own modeling, simulation, and analytics software. Moreover, the laboratories are ill-equipped (financially or culturally) to fill this role. ISVs help bridge the research capabilities of the laboratories and the commercial needs of companies. For example, they harden, adapt, and customize laboratory-developed software for use by industry, including companies in the energy sector. Elements of this outreach program might include

• regular forums that bring together industry and laboratory personnel engaged in the development and use of advanced computing solutions for energy innovation

• incentives to engage ISVs in the hardening and deployment of DOE-developed advanced computing capabilities



• incentives to encourage ISVs to modify their licensing model to facilitate greater use of parallel processing power via mid-range computing resources

• representation of the ISV community in the proposed Advanced Computing subcommittee of the Secretary of Energy Advisory Board (see next recommendation).

3. Implement policies that will facilitate adoption of advanced computing for energy innovation.

Industry participants at this workshop, as well as all three industry-laboratory workshops held in 2012, cited numerous bureaucratic and policy barriers to working with the national laboratories and DOE. It is essential that DOE review and revise policies related to technology transfer, access to computing resources, and engagement of laboratory expertise. For example:

• Develop a standard, simplified software licensing agreement.

• Either direct fund technology transfer activities or increase (e.g., double) the amount of overhead funding that laboratories can devote to technology transfer activities, and encourage the increased flexibility to be used for maturation and hardening of laboratory-developed advanced computing software.

• Work with ISVs to develop a software licensing business model that will encourage the use of parallel processing power.

• Establish an Advanced Computing subcommittee of the Secretary of Energy Advisory Board (SEAB) to guide DOE advanced computing policy.

4. Establish an Advanced Computing for Energy (ACE) program within the Department of Energy.

In our view, the best way to implement the above recommendations to speed energy innovation through advanced computing is to create a program focused on this important outcome. This program would do for energy innovation what the Advanced Simulation and Computing (ASC) program did for stockpile stewardship and what the Scientific Discovery through Advanced Computing (SciDAC) program did for scientific discovery. ACE would complement ASC and SciDAC by focusing on the commercial “hardening” and dissemination of modeling, simulation, and analytics software—and access to mid-range computing resources—to demonstrably speed the testing, evaluation, and deployment of new energy products and services to the marketplace. Key elements of this program might include




• aforementioned primary web portal into DOE advanced computing capabilities

• aforementioned ISV engagement program.

CONCLUSIONS


CONCLUSIONS

THE CENTRAL QUESTION

This report opened with a single, central question: What is holding the DOE back from exploiting its advanced computing investments for energy innovation? This question stems from the realization by the Secretary of Energy and the many attendees at previous workshops that DOE advanced computing has the opportunity to play an important role in changing the U.S. energy future. So the question remains: What is holding us back?

GETTING SOME ANSWERS

The workshop on Grand Challenges of Advanced Computing for Energy Innovation assembled a very diverse group of energy innovators and advanced computing experts who participated in some very lively discussions to define the challenges. An important result of these discussions was not only to identify challenges, but to identify the types of challenges. As was learned during the workshop, identifying the types of challenges is critical to understanding “who” needs to be responsible for confronting them.

RECOMMENDING SOME ACTIONS

An important aspect of this workshop was that the discussions went beyond defining challenges. The workshop participants were confronted with the need to provide recommendations for solutions to those challenges. They were asked to create suggestions that were

• actionable (preferably doable by the DOE)

• quantifiable (if possible)

• bounded by a completion date

• accompanied by a description of the expected impact.

The workshop participants did an excellent job and provided a very thorough set of recommendations that provide the DOE with a number of workable actions. These will be difficult to implement but if done will definitely address the challenges that impede the Department’s course toward achieving the vision articulated by Secretary Chu.

NEED FOR A SINGLE POINT OF RESPONSIBILITY

Perhaps one of the most important and most challenging recommendations was provided by the workshop chairs. They said the following:

In our view, the best way to implement the above recommendations to speed energy innovation through advanced computing is to create a program focused on this important outcome. This program would do for energy innovation what the Advanced Simulation and Computing (ASC) program did for stockpile stewardship and what the Scientific Discovery through Advanced Computing (SciDAC) program did for scientific discovery. ACE would complement ASC and SciDAC by focusing on the commercial “hardening”

CONCLUSIONS


and dissemination of modeling, simulation, and analytics software—and on providing access to mid-range computing resources—to demonstrably speed the testing, evaluation, and deployment of new energy products and services to the marketplace. Key elements of this program might include




• a primary web portal into DOE advanced computing capabilities

• an ISV engagement program.

The chairs recognize that without a single point of responsibility within the Department, many of the recommendations provided by the workshop would not succeed. As the Chairs had learned through experiences with ASC and ASCR, progress in changing the way advanced computing was used for nuclear weapons and science required a focused program. Therefore, they recommend doing something similar for the DOE applied energy programs.

IMMEDIATE NEXT STEPS

The DOE is fortunate in that there are already some current activities that can provide “testbeds” to learn how to overcome the grand challenges identified in this report. These current activities include the following:

• Combustion Research Facility (CRF) at Sandia National Laboratories

• High Performance Computing Innovation Center (HPCIC) at Lawrence Livermore National Laboratory

• Consortium for Advanced Simulation of Light Water Reactors (CASL) led by Oak Ridge National Laboratory.

These three activities are at different stages. The CRF has existed for decades, and the HPCIC is relatively new. CASL is only a bit older, but because of its Energy Innovation Hub nature, it has been required to quickly deal with many of the challenges identified in the workshop.

Since these three examples are dealing with questions about how advanced computing technologies can be made useful to energy innovators (technical challenges), they also are required to deal with the technology transfer issues (structural challenges) and the need to attract industry users (incentive challenges). So, as an immediate next step, these three examples should be studied closely to understand what is working well and what is not. These “lessons learned” should be shared and incorporated into any future Department-wide Advanced Computing for Energy program.

CONCLUSIONS


WORKING TOWARD THE FUTURE

The final question in the Forbes May 8th, 2012 interview with Secretary Chu was, “How is the Department of Energy playing a role in this emerging space?” He replied:

In every way possible. We have been the leader in the development of supercomputers in the United States for decades. We also help to develop the software to work with these machines, developing new algorithms to increase the calculation ability and make them easier to work with. The DOE is really playing a critical role by maintaining large user facilities and employing a core of computer scientists and applied mathematicians who help industry take full advantage of our supercomputing capabilities.

However, as the workshop participants learned, the Secretary’s answer in the interview falls into the category of being necessary but not sufficient. He made this clear at the workshop and encouraged the participants to really understand what “In every way possible” really means and to provide recommendations to put those words into action. To paraphrase the title of the article, there is no questioning that “DOE Thinks Supercomputing Will Change Our Energy Future.” As a result of this workshop we now have a better understanding of what challenges lie ahead and what actions we must take to achieve that vision.

CONCLUSIONS


APPENDIX A: WORKSHOP PARTICIPANTS

Grand Challenges of Advanced Computing for Energy Innovation Appendix A-1


Name Affiliation Email Steve Ashby Pacific Northwest National Laboratory [email protected] Ray Bair Argonne National Laboratory [email protected] Charles Barbour Sandia National Laboratories [email protected] Philippe Bardet George Washington University [email protected] Michael Bernstein [email protected] Gil Bindewald Department of Energy, OE [email protected] David Brown Lawrence Berkeley National Laboratory [email protected] John Burns Virginia Polytechnic Institute and State

University [email protected]

John Cary Tech-X Corporation [email protected] George Chiu IBM Corporation [email protected] Dana Christensen National Renewable Energy Laboratory [email protected] Steven Chu Department of Energy [email protected] Phil Colella Lawrence Berkeley National Laboratory [email protected] Trevor Cook Department of Energy, NE [email protected] Dave Corson Altair [email protected] Dona Crawford Lawrence Livermore National Laboratory [email protected] Mike Crowley National Renewable Energy Laboratory [email protected] Tom Cully Dassault Systèmes [email protected] Dave Danielson Department of Energy, EERE [email protected] Michael Derby Department of Energy, EERE [email protected] Lori Diachin Lawrence Livermore National Laboratory [email protected] Lee Ann Dudney Pacific Northwest National Laboratory [email protected] Kevin Duffy Caterpillar, Inc. [email protected] Wayne Eckerle Cummins, Inc [email protected] Fort Felker National Renewable Energy Laboratory [email protected] Noah Goldstein Lawrence Livermore National Laboratory [email protected] Dennis Goo Intel Corporation [email protected] Steve Gravante Navistar [email protected] Robert Graybill Nimbis Services, Inc. [email protected] John Grosh Lawrence Livermore National Laboratory [email protected] Benjy Grover Lawrence Livermore National Laboratory [email protected] Mark Guiton Cray, Inc. [email protected] Ahmad Haidari ANSYS [email protected] Thomas Halsey Exxon Mobil Corporation [email protected] Steve Hammond National Renewable Energy Laboratory [email protected] Dan Hitchcock Department of Energy, SC [email protected] Patricia Hoffman Department of Energy, OE [email protected] Henry Huang Pacific Northwest National Laboratory [email protected] Barbara Hutchings ANSYS [email protected]


Appendix A-2 Grand Challenges of Advanced Computing for Energy Innovation

Name Affiliation Email Carl Imhoff Pacific Northwest National Laboratory [email protected] Dennis Jadin Navistar [email protected] Kirk Jordan IBM Corporation [email protected] Mahesh Kailasam Dassault Systèmes [email protected] Moe Khaleel Pacific Northwest National Laboratory [email protected] Walt Kirchner Argonne National Laboratory [email protected] Mike Knotek Renewable And Sustainable Energy Institute

(RASEI), University of Colorado Boulder [email protected]

Douglas Kothe Oak Ridge National Laboratory [email protected] Scott Kruger Tech-X Corporation [email protected] John Kuzan Exxon Mobil Corporation [email protected] Bob LaBarre United Technologies Research Center [email protected] Sandy Landsberg Department of Energy, SC [email protected] Stephan Lany National Renewable Energy Laboratory [email protected] Alex Larzelere Department of Energy, NE [email protected] Steven Lee Department of Energy, SC [email protected] Gary Leonard General Electric [email protected] Terrence Liao Total [email protected] Rob Lisle Newport News Shipbuilding [email protected] Eugene Litvinov ISO New England [email protected] Pete Lyons Department of Energy, NE [email protected] John Magerlein IBM Corporation [email protected] John Mandrekas Department of Energy, SC [email protected] Justin Marble Department of Energy, EM [email protected] Gary Mays Oak Ridge National Laboratory [email protected] Chuck McConnell Department of Energy, FE [email protected] Colin McCormick Department of Energy [email protected] Michael McQuade United Technologies Research Center [email protected] David Miller National Energy Technology Laboratory [email protected] Satish Narayanan United Technologies Research Center [email protected] Jeff Nichols Oak Ridge National Laboratory [email protected] Mark O'Riley IBM Corporation [email protected] Martin Ossowski North Dakota State University [email protected] Shing Pan Altair Engineering, Inc. [email protected] Jim Peltz Department of Energy, NE [email protected] Art Pontau Sandia National Laboratories [email protected] Rob Pratt Pacific Northwest National Laboratory [email protected] Padma Raghavan Pennsylvania State University [email protected] Sumit Ray Westinghouse Electric Corporation [email protected] Nick Reynolds Accelrys, Inc. [email protected] Mike Robinson Department of Energy, EERE [email protected] Bruce Robinson Los Alamos National Laboratory [email protected]


Grand Challenges of Advanced Computing for Energy Innovation Appendix A-3

Name Affiliation Email Doug Rotman Lawrence Livermore National Laboratory [email protected] Tariq Samad Honeywell International, Inc. [email protected] Ramanan Sankaran Oak Ridge National Laboratory [email protected] Gene Schultz Boeing Company [email protected] Arie Shoshani Lawrence Berkeley National Laboratory [email protected] Alex Simpson General Electric [email protected] Gurpreet Singh Department of Energy, EERE [email protected] Steve Smith Lawrence Livermore National Laboratory [email protected] Ned Stetson Department of Energy, EERE [email protected] Fred Streitz Lawrence Livermore National Laboratory [email protected] Ray Stults National Renewable Energy Laboratory [email protected] David Sun Alstom [email protected] Suzy Tichenor Oak Ridge National Laboratory [email protected] Dave Turek IBM Corporation [email protected] Paul Turinsky North Carolina State University [email protected] John Turner Oak Ridge National Laboratory [email protected] Timothy Valentine Oak Ridge National Laboratory [email protected] Paul Van Slooten United Technologies Research Center [email protected] Tim Wagner United Technologies Research Center [email protected] Gil Weigand Oak Ridge National Laboratory [email protected] Jack Wells Oak Ridge National Laboratory [email protected] Chap Wong Chevron Corporation [email protected]


Appendix A-4 Grand Challenges of Advanced Computing for Energy Innovation

APPENDIX B: WORKSHOP AGENDA

Grand Challenges of Advanced Computing for Energy Innovation Appendix B-1



Appendix B-2 Grand Challenges of Advanced Computing for Energy Innovation


Grand Challenges of Advanced Computing for Energy Innovation Appendix B-3


Appendix B-4 Grand Challenges of Advanced Computing for Energy Innovation

APPENDIX C: ACRONYMS AND ABBREVIATIONS

Grand Challenges of Advanced Computing for Energy Innovation Appendix C-1


API application programming interface ASC Advanced Simulation and Computing ASCI Advanced Scientific Computing Initiative ASCR Advanced Scientific Computing Research CAE Computer Aided Engineering CASL Consortium for Advanced Simulation of Light Water Reactors CFD computational fluid dynamics CRADA Cooperative Research and Development Agreement CRF Combustion Research Facility DOE U.S. Department of Energy EFRC Energy Frontier Research Center EMS energy management system HPC High Performance Computing HPCIC High Performance Computing Innovation Center INCITE Innovative and Novel Computational Impact on Theory and Experiment IP intellectual property ISV Independent Software Vendor MCA Massive Contingency Analysis NDEMC National Digital Engineering and Manufacturing Consortium NERSC National Energy Research Scientific Computing Center NNSA National Nuclear Security Administration OEMs Original Equipment Manufacturers R&D research and development ROI return on investment SBIR Small Business Innovation Research SciDAC Scientific Discovery through Advanced Computing SEAB Secretary of Energy Advisory Board STTR Small Business Technology Transfer


Appendix C-2 Grand Challenges of Advanced Computing for Energy Innovation

grand challenges of advanced computing for energy innovation€¦ · laboratory-developed advanced...

Documents