Jim Connett, on August 05, 2021, 12:00 PM
Human, Machine, or Software?
Disclaimer: The views and positions expressed below and are just one of many views that may exist within SYSTEMA as a company on this topic.
I have an odd but purposeful confession – I enjoy watching airplane crash investigation documentaries on YouTube. I know not from where this guilty pleasure came, but I’m constantly amazed (and often educated in the process) as investigators find that proverbial needle-in-a-haystack evidence pointing to the reason for the accident. Thankfully, not all accidents involve fatalities. More times than not, the heroic skills of the pilots allow a wounded bird to land safely. We celebrate these achievements and mourn the loss of life when this is not possible. We remember those lives lost and redeem their memories by implementing recommendations by the investigators following a crash. These findings help make air travel the safest form of human transportation in the world today.
I bring up this odd personal admission because it provides a good illustrative framework as we try to answer the multi-faceted question posed in the title – when a decision must be made, who should make the final decision…the human, the machine, or the software?
As we know, aircraft today are highly automated machines where advanced avionics now handle many of the monitoring functions of aviation. Increasingly, avionics make decisions based on this monitored data, and – to a certain degree – avionics have the power to alter the flight conditions. But what if the computer makes a decision based on bad data? Is the pilot able to recover (if needed) from this data-driven decision? Who is ultimately responsible when data-driven conditions lead to a decision detrimental to the end result?
Let’s think about your shop floor and the automation you’ve implemented (or would like to implement). Software (and by extension, automation) executes repeatable tasks without ceasing and with unparalleled precision. This is probably the reason why you implemented automation at specific points in the process. Human resources are scarce, so automation helps to free up the time and focus of employees to do more critical tasks and activities. All things being equal, this is a valid reason to implement automation. So…why don’t you automate the next step and the next step after that – in fact, why don’t you automate the entire process line? Let’s assume money is not a limiting factor. Why wouldn’t you automate your entire line? Automation helps pilots monitor the performance of the aircraft so that the pilot can concentrate on flying the plane. Why not automate the entire flight experience…from takeoff to touchdown? Empty the cockpit? Where are our fully autonomous planes?
Humans possess a unique and differentiating characteristic among all living creatures: humans can respond to the dynamics of a changing situation at not only logical but ethical and moral levels. We can quickly evaluate various ramifications of a decision before it is made. Humans can take inputs and make the best decision possible given the environment – which may not be the most logical decision given all the options. On the other hand, software depends on logical answers to logical questions. In the end, there is (or should be) only one logical response to a logical question. When the ethical and moral elements are not considered, the software-based answer may be incorrect – even though it’s logically correct.
On March 10, 2019, Ethiopian Airlines flight 302 took off from Bole International Airport in Addis Ababa, Ethiopia en route to Kenya. Mere minutes after takeoff, the Maneuvering Characteristic Augmentation System (MCAS) software on the Boeing 737 MAX-8 determined the climb was too steep. As it was programmed to do, the software-driven avionics pitched the nose of the plane down to arrest what appeared to be an extreme ascent. The pilots, recognizing this was the wrong action, pitched the plane upward when the MCAS aborted its decision. This automatic corrective action happened repeatedly. And, repeatedly, the pilots responded and attempted to correct the situation. Sadly, each MCAS decision consumed more altitude than the pilots were able to gain. The result was the loss of all 157 passengers and crew on board1.
In this extremely unfortunate event, the software detected a problem (determined by logic) and implemented a response that it determined to be the right answer. While the response was logical from the software’s perspective, the pilots had a different view – they responded to the MCAS system by applying their own logic based on their training and the surrounding environment (we should be climbing, not descending…the nose of the plane should be up, not down…flight school 101-type of stuff). In this fatal accident, logic won out over reason. The decision-making process of humanity took a backseat to the rigidity of a flowchart, and the results were horrific.
Both the massive multi-ton airplane and the ever-shrinking computer chip have benefited from the advancement of software automation over the last 20+ years. We can only imagine the growth we will see in the next 20 years. As we dream about what is possible, we encourage you to push the limits of software automation while balancing the need for human creativity and problem-solving in the decision-making process. We recognize that we must use software and machines to be competitive in this world economy. We must also recognize the fallibility of personifying software and embrace the capabilities humans possess to make the best decision possible given all the inputs. While it’s hard to imagine the need to make an ethical or moral decision in semiconductor manufacturing, the fruit of any decision based on logic and reason showcases the creativity and ingenuity of human thought.
Artificial intelligence continues to press the boundaries of software capabilities. A decade from today, we may need to rewrite or retract this blog post as we wave the white flag of surrender to a new victor. But for now, in 2021, humans possess what is required to make the best decision possible. While software and machines are increasingly capable and enhance our efforts, humans should retain the final say – whether we go right or left, or up or down.
This conclusion is, well, logical.
Agree or disagree? Leave your comments below.
1 https://en.wikipedia.org/wiki/Ethiopian_Airlines_Flight_302
August 05, 2021, 11:02 AM
AHA Udaipur
thank you for sharing useful article