Airborne decision making and the role of AR software integration

24 Jan, 2023

When faced with multiple courses of action during an airborne operation, a set protocol helps narrow down the selection process quickly and easily. The reason that airborne decision-making (or ADM) is different from other operational considerations is that the consequences of ‘getting it wrong’ are potentially catastrophic. At the very least, they can lead to mission failure and wasted time and money. At worst, especially in situations such as search and rescue, it could cost someone their life. 

Airborne decision-making generally takes a four-step approach which streamlines the operational process and helps operators make a considered decision quickly and with the maximum amount of informational analysis. Following this protocol makes operators more likely to make the correct decision.  

A systematic approach to decision-making ensures the safety of the aircraft and the team onboard. It also enables coordinated responses on the ground and at central control points. In the case of airborne surveillance, this ensures that operations are carried out effectively, and that data transfer occurs seamlessly to maximise efficiency. 

Defining the problem 

The first aspect of airborne decision-making is to define the problem. For example, in a surveillance operation within an urban area, this could be identifying a target vehicle in a built-up area with multiple potential targets. The potential for ‘data overload’ could result in the surveillance operation being inefficient and, ultimately, ineffective.  

Another element of problem definition is how it could impact the operation of the flight itself, including identifying potential hazards to safe flight. Again, in an urban environment, this could include areas where elevated hazards such as power lines could impact the operation’s success and the aircraft’s safety.  

Considering the options 

At its most basic level, the 5P plan is most commonly used to prepare for an operation and will allow the crew and ground support teams to consider the problems and potential solutions before take-off. The 5Ps – Plan, Plane, Pilot, Passengers and Programming – work synergistically to create a pre-flight working plan that considers potential problems such as terrain considerations and excessive data gathering.  

Operators can fine-tune their plan at this juncture, especially using the final P: Programming. Using software such as OpenSight’s innovative Automatic Target Recognition takes much of the hard work away from operators. It provides an ATR system that enables instant automatic real-time detection, classification and identification. Utilising Deep Learning protocol, it can be fully integrated into existing operations and exploits the benefits of AI processing for exceptionally fast and accurate data acquisition.  

Additional Programming tools such as Augmented Reality can give operators vital real-time information in 3D overlays, defining potentially hazardous areas and positioning the aircraft in its exact geospatial position. This assists pilots in planning not only the most effective and efficient route but the safest too. 

Implementation 

Operations systems such as OpenSight have a significant advantage because they are easily integrated into existing ground-level or onboard set-ups. They are also intuitive, allowing operators to quickly familiarise themselves with the processes and to adapt and course-correct mid-operation if necessary. The implementation of a progressive and augmented reality system such as OpenSight provides a multitude of options for operations that are quickly and easily implemented.  

A system incorporating ATR can eliminate one final yet crucial factor – operator fatigue. By analysing the data and removing any extraneous data automatically, it allows the operators to focus more on the crucial mission-critical elements, reducing the chances of operator error through fatigue or informational overload. 

Post-mission analysis 

The final aspect of ADM is reviewing the outcome or post-mission analysis. Because systems such as OpenSight use Deep Learning techniques, the mission can be revised, and future operations parameters can be redefined using the data gathered in real-time. This makes post-mission reviews more beneficial to future operations. The 5P system can be streamlined, including pre-mission checks and hazard identification protocols. Risk evaluation can be factored into future decision-making processing.  

With the use of AR software, airborne decision-making can be refined to a much higher degree, ensuring the safety of crews, the safety of the aircraft, and a much more efficient use of airborne units in all types of operations. 

OPENSIGHT Augmented Reality System

Enabling automatic target detection classification and identification

Related articles