Touchscreens allow the user to provide direct, context-sensitive interaction. In the cockpit, touchscreens will reduce workload and optimize pilot's workflow.
Operationally, touchscreens also offer potential benefits in support of reduced crew operations, rapid aircraft start-up/take-off and information sharing between the flight deck and company's operation center.
It is well known that touchscreens will be very useful. The increased surface area afforded by entire touchscreen cockpits allows for broader and shallower menu control structures. Currently, many procedures depend on sequenced checks and actions on different panels distributed across the cockpit, particularly in relation to emergency procedures. The integration of these procedures into one area has the potential to reduce pilot workload and enhance situational awareness. Similarly, the use of touchscreens will promote better consistency across the cockpit. There are, however, potential issues and concerns that need researching. For example, arm fatigue and discomfort requires careful consideration. Touchscreens require direct interaction and therefore their positioning is limited by reach, which has consequences for the location of displays.
The need for feedback also requires careful consideration. The current use of physical buttons provides immediate feedback to the pilot. Haptic or audio feedback mechanisms are potential contenders to complement visual feedback. Furthermore, the effectiveness of touch-based interaction under conditions of turbulence or vibration is also a concern, as the motion of the screen and hand could lead to unintended interactions that have workload and safety implications. Ultimately, the resulting interface needs to be intuitive and unambiguous for the control and display of all aircraft functions, whilst providing the pilot with a comfortable interaction that can be used in all operating conditions.
Even with the introduction of touchscreens, a requirement remains for an indirect device as a back up method or to complement touchscreens in a multi-modal solution. Input devices that are being considered for use in future cockpits include:
Trackball: a ball held in a socket and rolled using the hand or fingers. They are advantageous in areas where there is limited surface space for device manipulation.
Rotary controller: this can be rotated, pushed down or moved up/down/left/right in order to control the movements and actions of an on-screen cursor. Rotary controllers have been shown to produce faster task performance than other indirect input devices.
Touchpad: A tactile surface which is capable of sensing the movement of a person’s fingers and translating this into actions of an on-screen cursor. Like trackballs, they require little space for installation and manipulation. However, touchpads can require more complex manipulations when compared with other input devices.
Direct Voice Input (DVI): this is likely to be the most flexible input application used in future cockpits. It has the potential to be used for a variety of tasks including radio tuning, navigation functions and checklist procedures. However, the usefulness of DVI is currently outweighed by a host of technical problems that need solving. These include adapting the vocabulary to be suitable for all accents, identifying individual speakers in multi-speaker environments and suppressing background noise.
Applications already exist that allow pre-recorded templates to be loaded to negate the problem of accents and the use of active input lines can resolve the issue of multi-speakers. DVI is a promising input technology but is only likely to be implemented cockpits, not within the ‘2020’ vision that current research programs are aiming towards.
3D audio uses natural audio processing capabilities through the positioning of audio signals in 3D space using appropriate hardware in order to emulate how audio is perceived in the natural world.
In the cockpit, this technology aims to improve situational awareness during fixed obstacle avoidance maneuvring for rotorcraft. It has the potential to optimize crew workload in busy environments by spatially separating multiple simultaneous voice channels.
The design for this technology needs to consider the comfort of the headsets and how best to track pilots’ head movements. This is because there is a small area in which this technology will work and it is dependent on the listener’s head position and orientation. The effectiveness of 3D audio is also dependent on the hearing capability of the pilot, however this issue could be overcome via individual headsets, amplification aids and thorough and regular hearing tests.
There are slightly different considerations in a fixed-wing environment where pilots are not routinely required to wear headsets and rarely maintain head position for extended periods, except in critical phases of flight.
Advanced displays: Eyes out, head up and conformal symbology
The use of eyes out displays and appropriate symbology is considered to be a key enabler for enhancing operations in degraded visual environments and enhancing situation awareness. Due to operational differences between the two aircraft types, fixed-wing are likely to use Head Up Display (HUD) solutions, whereas rotary-wing are likely to use tracked Head Mounted Displays (HMD).
The eyes out displays will include both primary flight information and relevant conformal symbology such as landing sites. They will also allow the pilot to look through the displays to see the outside world. Displays are aligned at infinity so that the pilot can view real world objects and be presented with information on the display without having to adjust eye focus.
Current HUD technology does exist but offers a very limited field of view and so for longer flights the pilot must either maintain an uncomfortable position for extended periods or deviate from the design eye reference point, which has the potential for missed information. It is intended that these display interfaces will implement an augmented reality approach to allow for the presentation of 3D information onto the interface
In rotary-wing operations conformal symbology will be used to provide virtual 3D on route, approach and landing references, as well as primary flight and status data. The use of HUD symbology for fixed-wing aircraft is intended to provide significant capability enhancement to All Conditions Operations for taxiing, take-off and approach/landing. Expected benefits of the advanced displays include enhanced situational awareness and increased safety with regards to airborne obstacles and navigation hazards. Human factors evaluation trials are underway to assess the various implications of using advanced displays including situation awareness, workload and general usability of the displays.
These technologies provide just a taste of the cockpit of the future. The ideal cockpit will be capable of supporting the ever-changing Air Traffic Management environment and operating within the demands of All Conditions Operations whilst providing a scalable solution that is physically and cognitively optimised for the pilot.