Researchers Develop AI System to Predict and Control Extreme Turbulence in UAVs

FALCON, a groundbreaking AI strategy, enables UAVs to sense and respond to turbulence in real-time, making air travel safer and more efficient.

​​​​​​​A Complex airflow structures in urban environments. B The wing has 9 sensors to measure the airflow (8 equally spaced pressure taps and 1 pitot tube) and is mounted on a one-dimensional load cell to measure the lift. Trailing-edge flaps change orientation to manipulate the aerodynamic forces. C Experiment setup to create irregular turbulent wake of a bluff body under high wind speeds. D Smoke visualization of the turbulent wake of a cylinder at a smaller Reynolds number. This image is obtained at the Caltech Real Weather Wind Tunnel system at a significantly lower flow speed than the experiments conducted in this work for visualization purposes. The actual flow conditions used in our studies were too turbulent to have clear smoke visualization. E Under a uniform flow U_∞, symmetric airfoils do not have any vertical aerodynamic forces on them when they are aligned with the airflow. However, altering the position of a trailing edge flap on the airfoil can modify the lift coefficient C_L, yielding an upward or downward aerodynamic lift force. F Outline of FALCON, a model-based reinforcement learning framework that allows effective modeling and control of the aerodynamic forces due to turbulent flow dynamics and achieves state-of-the-art disturbance rejection performance.

In nature, flying animals sense coming changes in their surroundings, including the onset of sudden turbulence, and quickly adjust to stay safe. Engineers who design aircraft would like to give their vehicles the same ability to predict incoming disturbances and respond appropriately. Indeed, disasters such as the fatal Singapore Airlines flight this past May, in which more than 100 passengers were injured after the plane encountered severe turbulence, could have been avoided if aircraft had such automatic sensing and prediction capabilities combined with mechanisms to stabilize the vehicle.

Now, a team of researchers from Caltech's Center for Autonomous Systems and Technologies (CAST) and Nvidia has taken an important step toward such capabilities. In a new paper in the journal npj Robotics, the team describes a control strategy they have developed for unmanned aerial vehicles, or UAVs, called FALCON (Fourier Adaptive Learning and CONtrol). FALCON is a model-based reinforcement learning strategy, meaning it builds a model of the environment—in this case, turbulent wind conditions—as opposed to model-free methods that simply respond to stimuli without understanding the underlying dynamics. The strategy uses reinforcement learning, a form of artificial intelligence, to adaptively learn how turbulent wind can change over time. Then, it uses that knowledge to control a UAV based on what it is experiencing in real-time.

"Spontaneous turbulence has major consequences for everything from civilian flights to drones. With climate change, extreme weather events that cause this type of turbulence are on the rise," says Mory Gharib (PhD '83), the Hans W. Liepmann Professor of Aeronautics and Medical Engineering, the Booth-Kresa Leadership Chair of CAST, and an author of the new paper. "Extreme turbulence also arises at the interface between two different shear flows—;for example, when high-speed winds meet stagnation around a tall building. Therefore, UAVs in urban settings need to be able to compensate for such sudden changes. FALCON’s ability to learn and model turbulence in real-time gives these vehicles a deeper understanding of turbulent flows, enabling smarter and quicker adjustments."

FALCON is not the first UAV control strategy to use reinforcement learning. However, previous strategies have not tried to learn the underlying model that truly represents how turbulent winds work. Instead, they have all been model-free methods. Such methods focus on maximizing a reward function that cannot be used to tackle different settings, such as different wind conditions or vehicle configurations, without retraining because they focus on just one environment.

"That's not so good in the physical world, where we know that situations can change drastically and quickly," says Anima Anandkumar, the Bren Professor of Computing and Mathematical Sciences at Caltech and an author of the new paper. "In contrast, FALCON’s model-based approach learns the underlying turbulence model, enabling the AI to predict how wind will change in the future and take proactive action accordingly."

"Advancements in fundamental AI will change the face of the aviation industry, enhancing safety, efficiency, and performance across a range of platforms, including passenger planes, UAVs, and carrier aircraft. These innovations promise to make air travel and operations smarter, safer, and more streamlined," says Kamyar Azizzadenesheli, a co-author from Nvidia.

As the FALCON acronym says, the strategy is based on Fourier methods, meaning that it relies on the use of sinusoids, or periodic waves, to represent signals—here, wind conditions. The waves provide a good approximation of standard wind motions, keeping needed computation to a minimum. FALCON leverages the fact that most turbulent energy is concentrated in low-frequency components, allowing the system to model and predict changes in turbulent flows efficiently. Within those waves, when extreme turbulence arises, the unsteadiness shows up as a noticeable change in frequency.

"If you can learn how to predict those frequencies, then our method can give you some prediction of what is headed your way," says Gharib, director of the Graduate Aerospace Laboratories at Caltech.

"By focusing on the low-frequency components that carry the most energy in turbulent flows, FALCON simplifies the task of turbulence modeling and control," says co-lead author Sahin Lale (PhD '23), now a senior staff research engineer at Neural Propulsion Systems, Inc., who completed the work while at Caltech. "Using this prior knowledge simplifies both the learning and control of turbulent dynamics, even with a limited amount of information."

To test the effectiveness of the FALCON strategy, the researchers created an extremely challenging test setup in the John W. Lucas Wind Tunnel at Caltech. They used a fully equipped airfoil wing system as their representative UAV, outfitting it with pressure sensors and control surfaces that could make adjustments online to things like the system's altitude and yaw. They then positioned a large cylinder with a moveable attachment in the wind tunnel. When wind flowed over the cylinder, it created random, significant fluctuations in the wind reaching the airfoil.

"Training a reinforcement learning algorithm in a physical turbulent environment presents all kinds of unique challenges," says Peter I. Renn (BS '19, PhD '23), co-lead author of the paper, who is now a quantitative strategist at Virtu Financial. "We couldn't rely on perfectly clean signals or simplified flow simulations, and everything had to be done in real-time."

After about nine minutes of learning, the FALCON-assisted system was able to stabilize itself in this extreme environment, which is significantly faster than traditional methods that require much more data and retraining.

"With each new observation, the program gets better because it has more information," says Anandkumar.

"The future really depends on how powerful the software gets in terms of needing less and less training," says Gharib. "Quick adaptation is going to be the challenge, and we are going to push, push, push."

Looking to the future, he adds that the researchers envision giving UAVs and even passenger planes the ability to share sensed and learned information about conditions with each other. Based on FALCON’s learned models, this plane-to-plane sharing of turbulence data could help keep aircraft safe by giving them advance warnings about incoming disturbances.

The paper, "FALCON: Fourier Adaptive Learning and Control for Disturbance Rejection Under Extreme Turbulence," was published online on September 24. Babak Hassibi, the Mose and Lillian S. Bohn Professor of Electrical Engineering and Computing and Mathematical Sciences at Caltech, is also an author of the new paper. The work was supported by CAST and by grants from the National Science Foundation and the U.S. Office of Naval Research.