Augmented or Enhanced Reality is changing the way we think about situational awareness in aviation. Designed to work alongside and assist experienced operators and pilots, as well as ground crews and operational managers, AR introduces immersive, real-time data overlays that can significantly enhance crew awareness and coordination, thereby improving their decision-making processes and overall safety.
In this article, we’ll look in more detail at how AR can assist crews, especially those in high-intensity situations such as helicopter pilots, and improve situational awareness. We’ll examine what situational awareness actually is within an aviation context, and what key features AR brings to the table. We’ll also discuss the innovations in Enhanced Reality, as developed by FlySight’s revolutionary OPENSIGHT system, and how it integrates into the modern cockpit to enhance safety and situational awareness in both civilian and military roles.
What is situational awareness in aviation?
Situational awareness may sound like a comparatively simplistic term, but when applied to aviation, it is vital to the safety of any aircraft and its crew. It is also mission-critical: without high levels of situational awareness, key information can be overlooked or missed, and the outcome of the mission can be compromised.
There are three levels of situational awareness in aviation:
Perception – This is the surrounding information available to the operator, including data such as the weather conditions, their location and the airspace around them, whether they are in an urban or rural environment and the potential hazards they may encounter (such as power lines and tall buildings in urban settings or mountainous terrain in a rural environment).
Comprehension – Understanding what this information means to the flight. For example, whether they should change course to avoid certain hazards, if they should fly around a storm cell or through it, and how the information can affect operational procedure and unit safety.
Projection – Basically, what might happen next. Projection involves anticipating how the information accrued can affect the outcome of the mission, whether there is a direct threat to the safety and well-being of the crew and aircraft, or if the data will have an impact on the mission’s outcome.
This process of perception, comprehension, and projection illustrates the complexity of situational awareness in aviation. Under ideal conditions, the human brain performs these tasks with relative ease. However, when conditions are less than perfect, the consequences of any lack of awareness at any stage of the process can seriously impact the safety of an aircraft. In situations like this, even the most experienced pilot and crew could do with a little extra help. That’s where Augmented Reality in helicopters comes in.
Augmented Reality for Safer Skies
Enhanced Reality is not designed to replace human pilots or operators. It’s designed to support highly trained and skilled operators, improving their situational awareness through the use of practical, advanced tools and software. The simple fact is that AR tools are being fully integrated into mission systems and used in training simulators to improve situational awareness and reduce the possibility of miscommunications.
Improved coordination between crew members leads to a safer and more efficient operation. With the high cost of keeping a helicopter in the sky, efficiency must be maximised for every minute a craft is airborne, without compromising the safety and welfare of the crew.
How AR helps crews
Augmented Reality in helicopters introduces multi-layered informational input into a mission, combining real-time data with libraries of information that can be instantly accessed via the Cloud or other technological advances. We’re not just talking about ‘maps’ here, but data that enhances the pilot and crew’s awareness of the world around them.
For example, real-time data visualisation in the mission display provides operators with a better understanding of the terrain, even if a major disaster, such as an earthquake or wildfire, has dramatically altered its visual appearance. This can assist in SAR operations as well as help the pilot to more easily orient their position.
Information can be delivered in a range of ways, from Heads-Up displays that project navigation and hazard data directly into the pilot’s field of view, to electronic moving maps (EMMs) that allow crews to visualise routes and potential obstacles from a top-down perspective.
AR in training and maintenance
AR also plays a role in the training of operators and crews. Simulated training is a constant in aircraft crews, enabling them to try new ways of carrying out operations without tying up expensive aircraft or putting the crew at risk. AR simulated training on the ground can be carried out whenever necessary, better preparing operators for emergencies and allowing them to familiarise themselves with the latest AR technology in a safe environment. This also keeps training costs down, as well as reducing the potential for errors when operating the system in a real-time situation.
For maintenance crews, the use of AR allows them to look ‘inside’ a system to gain a better understanding of its workings. For example, the wiring on a helicopter is incredibly complex. AR can help technicians and ground crew examine the ‘loom’ in a virtual simulation before applying that knowledge to the real system. This can help speed up maintenance, as well as make fault-finding easier and more accurate. That in turn translates to a better-maintained and safer aircraft that can be relied upon to perform at peak efficiency.
FlySight OPENSIGHT AR – Creating a more effective solution
The foundation of the AR or Enhanced Reality system is the collation of data through a variety of sensors and operational functions. Integrating this and visualising the information in a way that enhances the efficiency of the human operators and pilots is the key to AR systems. For several years, FlySight’s OPENSIGHT platform has been at the forefront of AR-enhanced technology, delivering turnkey solutions that are customisable depending on the needs and operating parameters of the end user.
OPENSIGHT isn’t a single system. It’s a collection of integrated systems that use the latest equipment and software to assess, collate and present real-time information in an easy-to-understand format. Complex algorithms can generate 3D maps of an area that, rather than being static (and therefore potentially outdated), provide a dynamic, ever-changing visual representation of the area. This revised data significantly enhances situational awareness among aviation crews, ensuring their safety and team effectiveness.
OPENSIGHT allows users to overlay data maps on top of one another, identify and capture a target (even in situations where multiple points would otherwise confuse the system), and introduce other elements that can help the crew gain a greater understanding of their environment and surroundings from every angle.
By integrating AR systems with advanced video processing systems, Enhanced Reality can be seamlessly integrated into both new and legacy systems. This means that operators are already familiar with the ‘hardware’, so the only additional training needed is familiarisation with the software packages.
The enhanced awareness of a landscape is possible even when conditions are less than ideal. For example, the OPENSIGHT Fog Suppression system dehazes visual images, providing pilots and crews with a significantly enhanced awareness of their environment, enabling them to see features and potential hazards with equal clarity.
Similarly, False Color – OPENSIGHT Enhanced Reality System – FlySight enhances the visibility of essential details without introducing unnecessary sensor information, keeping the clarity required to identify and capture a target on the ground.
Where next?
The future of Augmented or Enhanced Reality lies in the development of new and improved methods for enhancing interaction between technology and human operators. Augmented Reality in helicopters is already an essential tool in various applications, including search and rescue operations, surveillance, crowd control, target identification for law enforcement, military operations, fire control, and offshore rescue. As AR advances, it has a role to play in other aspects of aviation, including commercial flights, to enhance the operational environment for pilots, navigators, and crew.
Augmented Reality offers numerous safety benefits and operational advantages, ensuring its continued role in aviation. In addition to enhancing situational awareness in aviation, it can also be adapted to participate in various other aspects, including training and maintenance.
To learn more about FlySight’s OPENSIGHT Enhanced Reality System and its capabilities, browse our video library and brochures, or contact us today to discuss your needs with a member of our team.
