Episode Summary: Dr. András Kornai wants to put emphasis on the real and near-term ethical considerations around AI. In this interview, Kornai peels off the Hollywood myth “layers” around consciousness and AI in order to spotlight the very real, present, and advancing algorithms. He explains how such algorithms, which are slowly taking over the financial, medical, and automotive industries, are increasingly relevant as machines start to govern and make more decisions in our everyday interactions.
Guest: Dr. András Kornai
Expertise: Mathematics and Linguistics
Recognition in Brief: Dr. András Kornai earned his PhD in Mathematics from Eötvös Loránd University, Budapest, and later his PhD in Linguistics from Stanford University. He has published a book titled (and written numerous papers on topics related to) Mathematical Linguistics; his research interests and publications also extend into the field of bounded artificial general intelligence (AGI).
Current Affiliations: Professor at Budapest Institute of Technology and Senior Scientific Advisor at the Computer and Automation Research Institute of the Hungarian Academy of Sciences
Concerns About Autonomous AI, Present Tense
When asked if he is concerned about the possibility of a future, more intelligent AI taking control or perhaps destroying humanity, Kornai replies with square logic – “I squarely fall in the middle of the concern camp.” His emphasis is on the well-known (but not always recognized by the public) fact that AI is not science fiction – it exists among us, in more places than might be realized, right this moment.
For example, High Frequency Trading (HFT) uses computer algorithms i.e. AI to quickly trade stocks. About 90% to 95% of all high frequency quotes are now done by computers, not humans. “The mere idea that we have given up on human decision-making capabilities in favor of better algorithms…this means we haven given up a huge area of human competence.” That number continues to climb, with human trading literally dying out; but the dealing of money is just one aspect of this new AI-driven reality.
We are also seeing significant upticks in algorithms making medical diagnoses. Anyone who follows this industry has certainly seen IBM’s debut of Watson into the medical research field. Kornai believes we can expect the same thing to happen in the legal industry, and the full entry of AI in autonomous vehicles might happen even faster.
At present, we humans still make the decision about crushing another human in the way of our car’s path or swerving and crushing our car instead, but these decisions will be turned over to algorithms in the nearer future. Concerning the generous talk around military uses of drones, “these are generally human-controlled now, but they don’t have to be”, says Kornai. There is intense research taking place into more autonomous drone vehicles. This is not the future, this is now.
Debunking Hollywood Myths
Kornai stresses that he would like to distinguish the robotic experience from AI, which is a pure algorithmic concept that runs on our workstations, mainframes, phones, etc. “The algorithms themselves are not capable of hurting people directly,” says Kornai, but there are a plethora of opportunities for indirect harm. AI is erroneous functioning, explains András; it works on a statistical basis, so there is a certain amount of error built into any AI system.
The idea of a terminator-like robot still lies in the realm of science fiction, primarily because of the state of robotics. “AI can be the best chess player…the gap is so huge its unbelievable,” says Kornai. But the world’s best ping pong player is very far from getting beaten by a ping-pong paying robot.”
AI, in terms of algorithms, is in many ways ahead of human abilities, but there still exists a huge gap in activities requiring physical manipulation, though he acknowledges the progress that has been made like entities such as Boston Dynamics in the past couple of decades.
Another part of the present Hollywood myth that Kornai seeks to debunk is “Skynet coming to life.” Instead, he believes the primary worry – and the one on which we should be focused – are the everyday, ubiquitous algorithms. We seem headed for a track on which the whole world financial system is dependent on AI, and an economy in which half of the global population is unemployed. “The economic forces being steered by algorithms makes productive investment less likely (in the future),” says András.
Developing Bounded and Safe AI
Alongside the roll out of these very real and disruptive AI technologies, Kornai is working on a “two-pronged project” that looks to hedge risks and help protect human interests. One is in the area of finite automata that have the capability to model human indecision. András believes this is an important next step for AI. In terms of general intelligence, human levels is the best we have today.
The ability to have doubts and be slightly indecisive is a hallmark of this intelligence. “This is not an accident..there is a mathematical foundation for supporting this approach to decision-making,” says Kornai. The concept of “utility function” is key to AI; Kornai says that an intelligent system cannot decide goals by a single function, this is an idealization of a more complex reality, and he is trying to integrate the more complex theory into AI
The other prong of his current project work deals with “bounded AI”. Kornai analogizes the incident of the Folsom prison riots in the 1970s. The prisoners wrote out their demands in a manifesto and bill of rights, including the right not to be tear gassed when moved into their cells. “Most ordinary people see this this as cruel and inhumane,” remarks András.
The important idea in this story is the moment when you start thinking about what makes this act by the prison guards ethically and morally wrong. Kornai is working on a formal mathematical theory that will program such explanations i.e. the “notion of empathy” into AI algorithms. Humans empathize with prisoners who are defenseless against tear gas, but usually don’t think twice to exterminate a colony of ants that has taken over a corner of the yard. András argues that there needs to be an algorithm that can decipher between these two types of contextual situations when machines being making ethically-based decisions.
