The Pilot’s Brain

April 7, 2016 | Author: Thea Feyereisen

Computers can perform many tasks at higher efficiency and reliability than pilots; however, there are areas where a pilot still dominates machines, e.g., novel problem solving. Because of our ability to perform critical functions well, pilots remain a central and critical part of our commercial and corporate aviation transport systems.


But pilots are only human, and have vulnerabilities that hamper their ability to perform well. Pilot error is still cited as the leading cause of all aircraft accidents. For over the past decade, Honeywell has been leading research efforts aimed at developing practical applications of Neurotechnology -- to try and better understand when humans may be more vulnerable to errors and to invoke automation when humans are vulnerable. Areas of Honeywell research include estimation of cognitive state, prediction of future cognitive state, design of computer interfaces tied to brain signals, and enhancement of the cognitive capabilities of individuals (or how do we increase the IQ of pilots?!!).

Estimating Cognitive State

Brain imaging data can help detect increases in mental demand even before physical behavior is revealed. If we can read the brain, it can give us an advanced warning of when a pilot might be entering a mental state where he/she isn’t capable of handling the situation safely (think fatigue, or extreme workload).

Honeywell has investigated the analysis of electroencephalography (EEG) sensor output for estimating cognitive state. In a program sponsored by DARPA, called Augmented Cognition, Honeywell was able to estimate the cognitive load of dismounted soldiers engaged in training simulations through real-time EEG analysis. This research program was followed by an internally funded program were we were able to objectively classify pilot workload levels via EEG. Would it surprise you if I say overconfidence in performance or underestimation of workload with a new display design can be quite common in the pilot population demographic? Because of the great variability in pilot response when they self-report usability and workload measures this novel workload classification tool shows great potential for use in new product research.

Predicting Cognitive State

BrainBesides providing an objective estimate of current mental demands to the pilot, Honeywell is interested in measures that allow us to predict the pilot’s future mental state, e.g., fatigue and loss of situation awareness. Both are commonly identified causes of pilot error in accidents. We have been engaged in identifying brain wave signatures associated with attention and onset of fatigue. If we are able to detect when a pilot is becoming fatigued, we can provide appropriate stimulation to reengage the pilot or alternatively offload some of the piloting duties to automation to improve aircraft safety.

As part of a NASA-funded Spatial Disorientation research program in cognitive state prediction, Honeywell is working to predict when a pilot may become disoriented. In this study we tap into the aircraft databus and look at aircraft performance measures (e.g., rate of turn, acceleration, pitch) and run that through an computational model of the human vestibular system (ear and brain connection) to predict potential scenarios where a pilot may become spatially disoriented. If we are able to make a prediction that the pilot is not paying attention or is potentially confused, we can engage an intelligent alerting system to reengage the pilot or in some cases have the aircraft’s automation take over temporary control.

The Brain as a Computer Interface

Probably the most famous application of a pilot using a brain control interface is in the 1982 movie Firefox where the pilot, Clint Eastwood, controls weapons on a fighter jet via brain neural control. While we are not quite at that level of science fiction meeting reality yet, we are making inroads. The human brain is very powerful in making split second perceptual judgments. When you think about how quickly the human body reacts to a 100mph baseball pitch or tennis serve, it is quite remarkable. It is only after the fact where the human can think and reflect about what he or she saw.

We leveraged the split-second perceptual capabilities of the human in a DARPA Nuerotechnology for Intelligence Analysts study where intelligence analysts were looking at satellite images for targets of interest. Instead of studying images deliberately by zooming, scrolling, and panning, the Honeywell team presented large images in bursts of 10-20 image chips per second. Honeywell researchers examined the analyst’s EEG signals to identify chips that grabbed their interest. We were able to detect a blip in the brain signals or an “aha moment” much more quickly and reliably than the human could realize or verbalize what was seen.

We are exploring applications for neurotechnology in the cockpit -- providing a hands free way to control or activate systems. And yes, we have “Gone Firefox” and successfully broken into the autopilot one of our test planes and flown it with neural control aka The Brain as a Computer Interface.

Test Aircraft Simulator in Redmond
 Honeywell Test Aircraft in Seattle, Washington Used for Neural Control  Honeywell’s B737 Simulator in Redmond, WA Used for Neural Control

Increasing Pilot IQ

OK, the latest Honeywell neural research challenge is almost too far out to believe, but it is real and it is happening and I learned about it from my friend that also happens to be Honeywell’s lead neural researcher, Dr. Santosh Mathan (same guy that led the project where we flew the test plane with the brain interface). Honeywell has been working on an IARPA Strengthening Human Adaptive Reasoning and Problem solving (SHARP) funded project, in partnership with Oxford, Harvard and Northeastern University, to investigate the potential to increase intelligence, using combination of cognitive training, and mild electrical stimulation to boost the activity of brain activity engaged by the training activity. A four-site international study with about 500 participants recently concluded where we demonstrated the ability to positively impact IQ through technology. The technology interventions used in the study included brain stimulation and – I kid you not -- a video game to engage and exercise brain areas that play a crucial role in intelligent behavior. The results were independently validated by an outside government-funded research team. The techniques were demonstrated as being safe, of short duration and a positive benefit to IQ. So yes, I am convinced there is a way to increase the pilot IQ through technology. 

Although some of these brain based technologies may seem stranger than fiction, this is not some Hollywood sci-fi movie -- this is real research happening at Honeywell today. There will always be a trade-off and debate between what humans can do well and what machines can do well – what role pilots should play vs. what role the automation should play. But I think the most exciting area is not the battle of man vs. machine, but what we can do better together – where 1 and 1 can be made into three and the whole is greater than the sum of its parts.

Thea Feyereisen

Thea Feyereisen

Thea Feyereisen is an Engineering Fellow in the Flight Safety Systems group of Honeywell’s Aerospace Advanced Technology organization. She is an innovation leader in the areas of synthetic vision, safety, navigation and situation awareness display design and cross-cultural user interface. She is the technical lead of Honeywell’s Synthetic Vision and Interactive Moving Map Display research projects and leads a cross-cultural flight deck program with Honeywell’s China Air Traffic Management research lab. Previously she has led programs for NASA-funded High Speed Research and Aviation Weather Information Network. Ms. Feyereisen represents Honeywell on the FAA Commercial Aviation Safety Team for Airplane State Awareness and is on the leadership team for the RTCA committee on Synthetic and Enhanced Vision. Thea is a pilot and flight instructor and prior to joining Honeywell 20 years ago, logged flight time as a bush pilot in Alaska. She is co-inventor on over 25 patents and has a Masters in Aeronautical Science Human Factors from Embry-Riddle Aeronautical University.

Contact Information


  • Stephon

    Great read!

  • Alex

    It is nice observation. I have opened a discussion towards it. I can share my view and update if we can discuss further. Thanks..