Training AI to win a dogfight

Artificial intelligence has defeated chess grandmasters, Go champions, professional poker players, and, now, world-class human experts in the online strategy games Dota 2 and StarCraft II. No AI currently exists, however, that can outduel a human strapped into a fighter jet in a high-speed, high-G dogfight. As modern warfare evolves to incorporate more human-machine teaming, DARPA seeks to automate air-to-air combat, enabling reaction times at machine speeds and freeing pilots to concentrate on the larger air battle.

Turning aerial dogfighting over to AI is less about dogfighting, which should be rare in the future, and more about giving pilots the confidence that AI and automation can handle a high-end fight. As soon as new human fighter pilots learn to take-off, navigate, and land, they are taught aerial combat maneuvers. Contrary to popular belief, new fighter pilots learn to dogfight because it represents a crucible where pilot performance and trust can be refined. To accelerate the transformation of pilots from aircraft operators to mission battle commanders—who can entrust dynamic air combat tasks to unmanned, semi-autonomous airborne assets from the cockpit—the AI must first prove it can handle the basics.

To pursue this vision, DARPA created the Air Combat Evolution (ACE) program. ACE aims to increase warfighter trust in autonomous combat technology by using human-machine collaborative dogfighting as its initial challenge scenario. DARPA will hold a Proposers Day for interested researchers on May 17, 2019, in Arlington, Virginia.

"Being able to trust autonomy is critical as we move toward a future of warfare involving manned platforms fighting alongside unmanned systems," said Air Force Lt. Col. Dan Javorsek (Ph.D.), ACE program manager in DARPA's Strategic Technology Office (STO). "We envision a future in which AI handles the split-second maneuvering during within-visual-range dogfights, keeping pilots safer and more effective as they orchestrate large numbers of unmanned systems into a web of overwhelming combat effects."

ACE is one of several STO programs designed to enable DARPA's "mosaic warfare" vision. Mosaic warfare shifts warfighting concepts away from a primary emphasis on highly capable manned systems—with their high costs and lengthy development timelines—to a mix of manned and less-expensive unmanned systems that can be rapidly developed, fielded, and upgraded with the latest technology to address changing threats. Linking together manned aircraft with significantly cheaper unmanned systems creates a "mosaic" where the individual "pieces" can easily be recomposed to create different effects or quickly replaced if destroyed, resulting in a more resilient warfighting capability.

The ACE program will train AI in the rules of aerial dogfighting similar to how new fighter pilots are taught, starting with basic fighter maneuvers in simple, one-on-one scenarios. While highly nonlinear in behavior, dogfights have a clearly defined objective, measureable outcome, and the inherent physical limitations of aircraft dynamics, making them a good test case for advanced tactical automation. Like human pilot combat training, the AI performance expansion will be closely monitored by fighter instructor pilots in the autonomous aircraft, which will help co-evolve tactics with the technology. These subject matter experts will play a key role throughout the program.

"Only after human pilots are confident that the AI algorithms are trustworthy in handling bounded, transparent and predictable behaviors will the aerial engagement scenarios increase in difficulty and realism," Javorsek said. "Following virtual testing, we plan to demonstrate the dogfighting algorithms on sub-scale aircraft leading ultimately to live, full-scale manned-unmanned team dogfighting with operationally representative aircraft."

DARPA seeks a broad spectrum of potential proposers for each area of study, including small companies and academics with little previous experience with the Defense Department. To that end, before Phase 1 of the program begins, DARPA will sponsor a stand-alone, limited-scope effort focused on the first technical area: automating individual tactical behavior for one-on-one dogfights. Called the "AlphaDogfight Trials," this initial solicitation will be issued by AFWERX, an Air Force innovation catalyst with the mission of finding novel solutions to Air Force challenges at startup speed. The AFWERX trials will pit AI dogfighting algorithms against each other in a tournament-style competition.

"Through the AFWERX trials, we intend to tap the top algorithm developers in the air combat simulation and gaming communities," Javorsek said. "We want them to help lay the foundational AI elements for dogfights, on which we can build as the program progresses."

AFWERX will announce the trials in the near future on its website: www.afwerx.af.mil/ .