VERIFI code optimization yields three-fold increase in engine simulation speed

For engine designers in the digital age, time is money. And that time is measured in computer cycles.

Researchers at the U.S. Department of Energy's Argonne National Laboratory are partnering with Convergent Science, Inc. (CSI), to speed up a key piece of modeling and simulation software to ensure those cycles are used as effectively as possible, reducing product development time and resulting in better engines and savings for consumers. The scale of the speed gains were recently demonstrated when researchers ran the largest engine simulation to date on more than 4,000 computer cores.

The research is part of Argonne's Virtual Engine Research Institute and Fuels Initiative (VERIFI), which is working with CSI to optimize the company's CONVERGE code, a CFD software program widely used in industry to conduct modeling and simulation for engine design. While the effort has been ongoing for more than two years, it has recently moved into a code optimization phase that is showing dramatic gains.

"Our latest round of optimization has yielded a three-fold increase in speed, which correlates directly into faster design of better engines," said Janardhan Kodavasal, a postdoctoral appointee who led the optimization work along with with Marta Garcia Martinez, an assistant computational scientist, and Kevin Harms, a senior software developer at the Argonne Leadership Computing Facility. "The unique High Performance Computing resources we have available at Argonne allowed us to make great progress in a short amount of time."

Engine designers use modeling and simulation software to test new designs and tweak existing plans in a virtual space, drastically reducing time to market. When the project first started, simulations ran on systems with 50 cores. VERIFI quickly scaled up those numbers to 1,000 cores, and recently conducted an engine simulation on 4,096 cores.

The VERIFI work is focused on a key aspect of engine design – the extraordinarily complicated fluid dynamics and combustion characteristics that are at the heart of all internal combustion engines. Using high performance computing and X-ray radiography data from Argonne's Advance Photon Source, VERIFI was able to gain unprecedented insight into the performance of fuel injectors in engines. Once that modeling was completed, it was incorporated into CONVERGE and is currently being used by industry partners in engine design.

The latest phase involved optimizing the CONVERGE code for greater efficiency. One of the key breakthroughs came in the area of load balancing. The varying levels of complexity in the chemistry of ignition meant that some cores weren't actively engaged in computation, while other cores handling more complex parts of the simulation were working overtime. By balancing the computational load evenly over all the cores, great gains in efficiency were achieved.

Another important development came when the team tweaked CONVERGE to use parallel read/write processes, which allow simultaneous file writing by processors, rather than having to wait for one action to complete before taking another. This resulted in a speed up of more than 100 times in writing large data files generated by the software.

In the end, the advances mean that engine designers can try out more designs in shorter times, yielding more efficient, reliable engines with less cost.

"Working with the people at Argonne allowed us to implement high-performance computing improvements in our software much faster than we could do alone. The Mira computer system at the ALCF is a powerful tool, but it was Argonne's experienced staff that allowed us to really maximize the computing resources available to us and make great progress in a very short amount of time," said Keith Richards, co-founder and vice president of Convergent Science.

VERIFI and CSI will be presenting the findings of the work in a forthcoming ASME paper.