Episode summary: Unlike the field of self-driving cars, the fields of construction, mining, agriculture, and other classes of “heavy industry” involve a huge variety of equipment and use-cases that go beyond traveling from A to B. The heavy industry leaders of today are no farther behind automakers in their understanding that AI and automation will be essential for the future of their companies. In this episode, guest Dr. Sam Kherat discusses the applications of AI in heavy industry, including: What type of capabilities and functions are automate-able, and at what level. He also shines a light on how AI might affect the future of the industry within the next 2-3 years, and in what ways we can expect large equipment to become more autonomous.
We’d like to thank RE-WORK for introducing us to Dr. Kherat at their autonomous vehicles conference in San Francisco.
Expertise: Dr. Sam Kherat: Ph.D. in Aeronautics and Astronautics from Purdue University, West Lafayette, Indiana and MSEE from Bradley University, Peoria, Illinois. Dr. Kherat has expertise in the fields of robotics, autonomous excavation, and mining systems. He currently teaches robotics courses at the Mechanical Engineering Department at Bradley University in Peoria, Illinois
Brief recognition: Dr. Kherat was involved in the DARPA Grand Challenges and Urban Challenge. He was previously Adjunct Professor in the Electrical and Computer Engineering Department and member of the Robotics Institute at Carnegie Mellon University (CMU.)
Dr. Kherat’s publications include The IEEE Transactions on Automatic Control, ASME, the Journal of Mathematical Analysis and Applications, and Advanced Technology (C-TAT).
He has been a keynote or guest speaker at several conferences or symposia including SAE’s Driver Assist and Autonomous Vehicle Technologies in Pittsburgh, 2005 & 2011, NI week in Austin TX, 2008, IEEE’s Industrial Forum: in Anchorage, Alaska, May, 2010, AUVSI’s Unmanned Systems Interoperability Conference, San Diego, CA, October, 2010. He received very complimentary feedback from the Oregon Office of Energy and the WA State Dept. of Ecology, on his presentation to help with cleanup efforts at the Hanford site.
Current affiliations: Dr. Kherat is on the Technology Committee for the Robotics Technology Consortium.
Big Idea:
Autonomous equipment in heavy industry may be widely adopted well before most consumer vehicles are self-driving.
The key application for AI in heavy industry is related to how vehicles can perceive their surroundings and each other. For the time being vehicle automation within heavy industry is limited to the use of GPS and a fusion of sensors such as RADAR and LiDAR. Full automation is only currently in place for heavy vehicles that have to make repetitive maneuvers, such as hauling ore. However, in the future this is set to change and it may happen sooner in the heavy industry space than in the arena of autonomous personal vehicles, for a number of reasons:
First, while there is an erie strangeness to sitting in a compact car that drives itself down the highway, there is absolutely nothing strange or unusual about automating heavy machinery work. Manufacturing, mining, and agriculture have always eagerly automated tasks whenever possible, and there is little resistance to doing so now with the advent of AI. We’ve covered a number of applications of machine learning in robotics in previous articles, it’s worth noting that as industrial robots gain dexterity, their heavy equipment counterparts are likely to gain autonomy.
Second, the operation of heavy equipment requires specific, valuable skills (the operation of a crane or of an industrial earth-moving vehicle is not something taught in a weekend seminar). Some of these skills are hard to come by, and expensive. Corporations eager to get more work done at less cost are benefitted financially by automation – much more so that an individual person would be financially benefitted by having an autonomous car of their own.
Third, while the self-driving car field may be seen to have a safety “barrier” preventing adoption, the heavy industry world could be seen to have a safety “imperative.” Commercial vehicles on the highway must constantly handle interactions with other cars, pedestrians, crossing guards, signs, lights and more. For the most part, heavy equipment operates on private property (such as mines, construction sites, factories) that doesn’t require the same number of dangerous and unpredictable interactions. Similarly, heavy industry work such as mining is inherently dangerous, and autonomous systems would likely save lives.
These advances are bound to have a knock-effect in many industries including mining where automation will increase speed and safety, the factory where it will make moving products more efficient and in warehousing to facilitate storing and locating inventory.
Bringing Insight to Action: In order for us to see more rapid advances with AI in the heavy industry changes may need to be made in the way companies conduct their research and development. While currently, independent companies are using the same hardware and repeating the same tests, the development of a centralized data bank could save everyone time and money. It would mean that companies can dip into the consortium of information and use it to create algorithms to suit their individual applications (Dr. Kherat goes into detail on such a common data bank in the full interview audio, embedded above).
There will be much more creative freedom for autonomous developments in heavy industry as compared to commercial self-driving cars. Businesses involved in manufacturing, construction, agriculture, or other heavy industries may be benefitted by looking at the autonomous innovations of industry leaders in their sector – and better understanding how equipment autonomy (and it’s resulting efficiencies) may impact their operations and finances.
Interview Highlights from “AI in Heavy Industry” with Dr. Sam Kherat:
The following is a condensed version of the full audio interview, which is available in the above links on Emerj’s SoundCloud and iTunes.
(4.28) What are the aspects of machine learning and AI in heavy industry that are starting to drive business value?
Sam Kherat: In the heavy industry business…we have mobility, that’s the power train, the chassis, what drives the machine. The second component is the controls, which play a huge role in the stability of the machine, and making it go from point A to point B in a safe manner. The third component is tools. You have to have some way to quickly interchange your tools to continue doing the job. The fourth component is really behind in terms of development, and that it perception. That’s the ability of a machine to sense its surroundings…such as infrastructure. That’s what created this need to have some autonomous operations for the military because you want to be able to sense the distance and to sense when you have an intrusion coming in front of you and to be able to adequately swerve that. We’ve seen tremendous improvements in sensing technologies, whether it’s light-based like LiDAR also some improvements in RADAR systems. These systems complement each other…fusing has become a trend to enable a better sensing of the surroundings (for readers with a keen interest with sensor technology may want to listen to our machine vision interview with Cortica CEO Igal Raichelgauz).
(11.04) With an excavator or in a factory setting with some fork lift truck action…where do we see perception beginning to emerge meaningfully?
Sam Kherat: The industry is evolving so fast it’s amazing that just about a decade ago when you had a job, for example a construction site…where a foreman guides a machine operator, whether it’s a grader or a tractor, with some stakes that they dump on the ground so that the operator can see where to grade. Currently we have an automated system using GPS where the antennae are on eh blade itself and the whole thing is automated.
(12.18) So the machine can drive itself over there, know where it’s inserting the scoop and can pick the stuff up and know where to put it. Is this what’s happening?
Sam Kherat: Based up on a plan. Say you have a plan from somebody who wants to build a golf course…so the blade knows exactly where and how deep it needs to go to start pushing the dirt…The motion of the vehicle is controlled by the operator, but the operator has no control over its blade…Now with the future technologies that are being implemented we’ll see more automation. The machine is going to be able to perform all these tasks on its own and that is going to happen before we have autonomous vehicles.
(13.33) Yes, because we don’t have as many crashing into each other..?
You own your own construction site; you own your mine, so you can do what you want. Plus the social acceptance, When you really ask kids what they want to be when they grow up, how many tell you I want to be a miner, I want to be a construction worker? They would really rather do something else. It’s really not something that is appealing to a lot of high school students. So we want to take advantage of some of the skills that kids are growing up with, for example playing video games…give them a remote control and then the machine is away and they will do a good job.
(14.48) So we’re talking about in the future of AI in heavy industry and agriculture, because I’m sure that agriculture is big in heavy industry as well, to the best of my knowledge there are probably tractors that can drive perfect back and forth harvesting of a certain grain by itself, no people in it based on GPS. You’re talking about excavators doing something similar. The level of the blade and the adjustment of how earth is being moved is being done by a computer and sensors and a person is just controlling movement. Eventually the movement will be automated. And I’ve heard of dump trucks that have GPS and at the push of a button they can just perfectly align with where they have to be. Is this true?
Sam Kherat: Yes but it’s not just GPS, there has to be some sensing device on the machine itself because GPS gives you your location but you need some coordination. You mentioned the loading, so when you have a loader, putting whatever it is you need to put on a dump truck…then you have this coordination between the machines so they don’t bump into each other.
(16.19) I’d like to talk about these other kinds of sensors as well.
Sam Kherat: We’re really still at the infancy stage. We do have warning systems, just like you would have advanced cruise control on a car that slows down. We have an ability to tell the operator, “listen, you’re getting too close.” But there is no action. It’s not completely automated. The automation happens on the hauling systems where the truck goes from point A to point B. But it’s coming very quickly. The industry is actually an early adopter of these technologies… The haulage system now is completely autonomous, in the mine. Those big trucks hauling ore are completely autonomous, so they brake, they accelerate; they do the whole thing by themselves. Future work includes coordination with our machinery.
(21.42) When perception really kicks in….where will that really hit the ground running?
I think the issue is very similar to the on-road applications where perception is a stage of a threshold where we have a particular development that is being done by each company. And that is something that needs to be addressed, in order for us to have a better perception system. When you look at the perception systems we have, you can actually number the suppliers in that space because they are not really very many, so most of the companies, whether it’s on-road or off-road, use the exact same hardware. When any company is looking at using a particular sensor…they go through certain tests because they have to really make sure it does what it does. The problem, I see, is that everybody is doing exactly the same tests, so we’re really doing some redundant stuff just because we have this competitive mode. So now we spend a lot of money doing the exact same thing, plus we need to have the data that we have recorded and put that in the algorithm to make it work in the applications we’re using. So why don’t we think about having a center where everybody has a common understanding of what a piece of hardware can do for the entire industry?
Related Articles on the Applications of AI in Vehicles and Robotics:
At Emerj our goal is the serve business leaders with insights and facts about the applications and implications of artificial intelligence. “Robotics / Vehicles” is a sector of wide interest in the business community – as it effects essentially all industries. Stay in touch via our newsletter and social channels for more updates about developments in robotics and heavy industry, and explore some related articles below:
- Industrial Drone Use Cases – We recently compiled some of the most popular applications of drone technology to the fields of agriculture, construction, infrastructure management, and more
- When Will Autonomous Cars be Mainstream? – Our interview with Nexar Inc’s CEO Eran Shir covers the of the most important technical and legal concerns effecting self-driving cars
- Competition Rises for Industrial Robots – In this article we explore three specific use-cases for industrial robots, and well as some of the market forces pressing down on the robotics field at large
