Problem

The research addresses the challenge of developing an AI-assisted pilot training module for advanced jet trainer aircraft to enable precise execution of aerobatic maneuvers. The goal is to create a system that can effectively train future pilots by simulating a multitude of aircraft maneuvers using reinforcement learning agents, thereby enhancing pilot skill acquisition. Previous methods, particularly supervised learning approaches, often suffered from stability issues in aerobatic maneuver replication, and much of the existing data was not from expert pilots.

Method

The authors developed two distinct reinforcement learning (RL) approaches utilizing Soft Actor-Critic (SAC) models and hyper-parameter optimization. Both methods use classical state variables such as roll, gamma, yaw, speed errors, control surface angles, attitude, speed, and altitude. The first method uses real flight records from an experienced pilot as input, with roll, gamma, yaw, and speed targets derived from these records. An appropriate reward function then guides the AI to replicate the pilot's actions. This approach requires precise matching of pilot examples to expectations. The second method is employed when desired pilot input data is unavailable. It involves constructing and inputting a mathematical model of the intended maneuver, specifying target yaw, roll, pitch degrees, and speed at each time step, rather than simulating flight data. The reward function, central to both approaches, is a weighted average of eight error components. These include asymptotic errors (roll, gamma, yaw angle errors, and Mach errors) that reward the aircraft for approaching target states, and command errors (aileron, elevator, rudder, and throttle command errors) that penalize rapid control changes to prevent oscillations. Initial conditions for training episodes were randomized for yaw angle to ensure generality across different directions and explored variable altitudes. Observation space includes internal aircraft states, angular rates, control surface states, errors, a time scaling factor, and remaining maneuver steps.

Results

The developed AI models successfully learned and executed various aerobatic maneuvers (barrel roll, loop, Immelmann turn) on both F16 and Hurjet aircraft, demonstrating performance comparable to professional pilots. The models were shown to be capable of learning any physically acceptable maneuver. The research highlighted the generality of their pipeline, the effective use of real pilot data, improved maneuver stability over supervised learning methods, and the possibility of maneuver scalability through time-scaling. Artificially created trajectories resulted in smoother maneuvers due to their noise-free nature, while real pilot data allowed for a closer match to human-like performance. The models also exhibited robustness, being able to execute maneuvers across varying initial yaw angles and, in some cases, different altitudes without further training. It was also found that the quality of results heavily depends on extensive hyper-parameter optimization, though adjusting reward function weights can also yield capable models. Longer maneuvers required more training time and were prone to errors, suggesting the need for concise maneuvers or sequential execution of trained models for complex tasks.

Implications

This research provides a significant step towards developing AI-assisted pilot training modules for advanced jet trainers, demonstrating the applicability of sophisticated reinforcement learning methodologies in aerospace education. The ability of basic maneuvers to construct more complex ones implies a pathway to covering the entire flight envelope through a set of trained elemental maneuvers. Future work plans to further explore this by obtaining a basic maneuver set using reinforcement learning and demonstrating its capability to cover a comprehensive range of flight operations, ultimately enhancing pilot training and potentially improving flight safety and efficiency.