Applying AI in the real world is no simple task. While artificial intelligence has much to offer, it is the responsibility of engineers, computer scientists, and technologists to create AI systems that operate in symbiotic synchronicity with human beings. And yet, all too often, AI is seen as the silver-bullet solution to nearly every problem—from cutting manufacturing costs, to guaranteeing same-day delivery, to curing cancer. But in our rush to deliver on AI’s promise, we have made some very big—and very human—mistakes.
The most recent example of this, of course, is Boeing’s 737 Max 8. The two deadly crashes that occurred in October and March have been blamed on the planes’ flight systems, which initiated a dive soon after takeoff. While the exact cause is still under FAA investigation, one thing is clear: the pilots did not have enough information to properly and effectively alter the flight path of the plane.
As someone who has dedicated my life to the study of robotics, I believe the root of the problem is clear. In an effort to deliver a more advanced AI system, Boeing’s engineers made a catastrophic error: they took the human pilot out of the equation. As a result, when the plane’s autopilot system detected a problem, it did nothing to alert or inform the pilot; it simply pitched the nose of the plane downward. In that moment, the rational mind of the pilot was overloaded with decisions—all of which had to be made in a matter of seconds. What is happening? Does the system have more information than I do? Is it acting appropriately? Should I take action to override this "smart" system? Tragically, as the seconds ticked by, the planes continued on their steep trajectories. Boeing designed a system that ignored the human factor and effectively disempowered the pilot.
This all-in approach to AI that gives the system control over the user is a stark contrast to the “stick-shaker” approach that is so familiar to every pilot. Designed to warn the pilot of an imminent stall, the device causes the plane’s control yoke to shake noisily. That action alerts the pilot to the problem, prompting immediate action based on the pilot’s training, experience, and skill. It’s a great example of technology as an enabler of better decision-making.
The lesson to be learned is that it is a mistake to throw the human baby out with the bathwater when developing the AI systems of tomorrow. Common sense and reasoning may be the most powerful traits we humans possess. AI systems need to be developed in a way that puts that human intuition to work shoulder-to-shoulder with the robots. I believe companies that take the time to apply ergonomics and human factors to new AI systems—and do so with great care—will be most successful over the long term.
At the same time, I am well aware that a slower, more deliberate approach to AI development is easier said than done, especially in a competitive landscape in which the companies who are using AI to its greatest abilities are already winning the race. That wasn’t always the case. The companies that were first off the starting block had a distinct advantage. Because there was no competition to speak of, they had the luxury of time to develop AI systems that were meticulously designed to preserve human agency:
All too often, the “race to AI” is viewed as a drag race, with the winner defined as the first one to reach the finish line. What we’re learning today is that speed does not always equal success. Boeing is learning that lesson the hard way. So is Tesla. So is Watson Oncology. Perhaps as companies continue to witness the costly and sometimes tragic results of developing AI systems without a focus on human interaction, the way AI systems are developed will shift to put human beings back at the core.
I am hopeful that the pendulum is swinging back toward this more human-centric approach. In the business world, I’m hearing CEOs ask how their executives can become more fluent about the implications of AI to apply it more thoughtfully across their organizations. At the same time, more and more, universities are requiring engineering and computer science students to take courses in the humanities—history, psychology, human factors, and design. The hope is that by expanding the students’ world views, tomorrow’s developers of AI systems will understand the weight of the decisions they are making with every design they build. They will understand the human need for connection, the consequences of negotiations of power, and the importance of preserving human agency as we strive for a world that utilizes the power of AI to create a better, more efficient way of being. Perhaps the most important lesson AI has to teach us is this: in everything we do, humans matter. It’s that simple.
About lllah R. Nourbakhsh, PhD
A valuable member of the ROBO Global Strategic Advisory Board, Illah is a Professor of Robotics at Carnegie Mellon University, as well as the director of Carnegie Mellon’s CREATE Lab, which explores socially meaningful innovation and deployment of robotic technologies. He has served as Robotics Group lead at NASA/Ames Research Center, and he was a founder and chief scientist of Blue Pumpkin Software, Inc. His current research projects explore community-based robotics, including educational and social robotics and ways to use robotic technology to empower individuals and communities.
Illah is the CEO and Chairman of Airviz, Inc., a World Economic Forum Global Steward, a member of the Global Future Council on the Future of AI and Robotics, and a member of the IEEE Global Initiative for the Ethical Considerations in the Design of Autonomous Systems. He also serves on the Global Innovation Council of the Varkey Foundation and is a Senior Advisor to The Future Society, Harvard Kennedy School. He earned his BS, MA, and PhD degrees in computer science from Stanford University and is a Kavli Fellow of the National Academy of Sciences. His book Robot Futures is available on Amazon.
 “The History of Robotics in Surgical Specialties,” PubMed Central, 12/14/15