A Conversation with Wyatt Newman on Harnessing the Power of Robotics

    Background in Robotics and Patent Experience

    Lisa Chai: We are here today with Wyatt Newman, who is one of our active members of ROBO Global strategic advisory board, and a leading researcher in the areas of robotics and computational intelligence. Hi, Wyatt.

    Prof. Wyatt Newman: Hi Lisa.

    Lisa Chai: Thanks for being here today.

    Prof. Wyatt Newman: Thanks. No, pleasure to be here. Thanks.

    Lisa Chai: Great. Wyatt, could you talk a little bit about your background as to what you're currently involved in? I know you're a professor. Could you talk a little bit about that?

    Prof. Wyatt Newman: Well, sure. I've been a professor for most of my career, over three decades at Case Western Reserve University. But prior to that also did industrial research for eight years with N.B Phillips, both here and abroad in the Netherlands. But I've also had placements abroad in Scotland, University of Edinburgh and in University of Hong Kong where I organized a DARPA robotics competition team.

    I've been in national laboratories, spent a year at Sandia National Labs in the intelligence systems robotics center. I've worked summers at NASA Glenn Research Center, spent a semester at Princeton and then neuroscience department in order to do some translational work from neuroscience to AI. So I've sort of sampled the field in different parts of the world and in different sectors, government, industrial, academic, and from here to China.

    Lisa Chai: Great. I know you have over 12 patents in over 130 technical publications. Can you kind of touch on sort of high level as to what's your patent portfolio look like?

    Prof. Wyatt Newman: Sure. Well, they're mostly robotics based. Electro mechanical devices. Current patent pending is on a road painting machine that involves a vehicle mounted with a large robot that can place markings at specified coordinates or vision based. Also patents related to rapid prototyping and specialty electro mechanical designs.


    About Prof. Wyatt Newman's Startup: RoadPrintz

    Lisa Chai: You're also a part-time CTO of a startup company when you're not teaching. Can you tell us a little bit about what the company does and how is the company using AI?

    Prof. Wyatt Newman: Yes. The company that I've recently co-founded is called RoadPrintz and its job is to take a vehicle mounted robot and be able to place markings on roads where they belong. That's currently done with a practice that's been unchanged over 100 years. It's a dangerous task to do and laborious and expensive, and it's ripe for robotic automation. So, that's what our system is intended to do. A single driver can take the vehicle out and the interface will inform the driver of here is a symbol that belongs to the following location. Part of it is driven by precision GPS.

    Part of it is driven by machine vision. If you're over-painting in existing marking, machine vision can help to position that marking, the operator can also be involved in modest edits in an intuitive way. So you can move things around on the screen and say, this is where that symbol belongs. Importantly, though, there is an organization of information. We don't have a mapping of where our road symbols are and where they belong. And then every time you put down fresh pavement, you have to go out with new surveying, new architectural plans, do new layout before you can put the symbols down.

    Instead, every time you paint a symbol with one of these localization enabled vehicles, it knows exactly what it painted and where and when, and post that into a cloud hosted repository so we can be building up a database of all of the markings on our streets. And I'm hopeful that will get turned around as well in terms of autonomous vehicles being able to access that database and say, I expect this type of marketing showing up at exactly these coordinates so that it will anticipate things like turns as well as be able to get precision localization off of what it can see or can only see partially.

    Fill in the blanks with the apiary knowledge that's already in the database. So I think the information that comes from this may ultimately be as valuable as the physical action of placing the markings.


    How Have You Seen Robotics Evolve Over the Past Few Decades?

    Lisa Chai: You've been at Case Western for many, many years. Do you see that your interest in the robotic science changing over time with your students?

    Prof. Wyatt Newman: Yes. I think the advent of the Robot Operating System has really transformed robotics. There were decades of essentially a robot crash. We went through a robot recession, and it was great to see the rebound. And I think in part, this was accomplished with new innovations in software engineering, which are really realized as part of the ecosystem of Robot Operating System.

    So that has helped lead to robot resurgence. We've seen a whole lot of robot startups enabled by dramatically decreasing the costs of software development. So many pieces that can be reused. So there's less uncertainty, there's less development time. You can immediately import excellent solutions from around the world. So that's been exciting.

    A second component that we're still only just seeing emerge is the combination of the new AI with robotics. Robots still are largely doing just what they're told, to very good precision and very good reliability. So excellent acceptance in industry. But I see excitement on the horizon as those robots become more intelligent. So that merging of deep learning with robots is really just starting.


    How Can We Increase the Adoption of Robotics?

    Lisa Chai: What can we do to increase the adoption of robots? Is it a matter of just having better sensors or are we looking for better training and labeling of data? What are we missing? What are the challenges that we're facing right now?

    Prof. Wyatt Newman: Well, a few things come to mind. Part of this will be market driven by... There's really a labor shortage. I know people selling equipment who say, "My customers aren't buying because they don't have people to operate it." So there is actually an employment shortage. Well, that's going to drive more robotics as well.

    As the robots become more competent that will open up more things that they can do. So they'll help create their own markets. Ease of use, I think, we've become convinced that ease of use is important. You could loosely say that Apple founded their business on ease of use and became the richest corporation in the world. So yes, ease of use matters. We shouldn't all have to be computer scientists in order to run robots. It's very important that it's self-evident, that you'd be able to say, "Yes, I know how to run this." You get in a car, you expect to know how to drive it. And we need to get there with robots. Here's a new robot, I expect to know how to use it.

    So, I think that those will help promote the use of robots. Also, issues of safety, when robots can do jobs that are now endangering people, or just in the advancement of a better lifestyle where we say, "This is a job that people shouldn't have to do." We'd rather have robots doing this dangerous or exhausting or boring work. If we can have them help us do that, so much the better.


     

    What Are the Biggest Use Cases of Robotics Today?

    Lisa Chai: Where do you see the biggest use cases of robots being applied today? Real world use cases. Are these robots truly intelligent? Can any of us use robots?

    Prof. Wyatt Newman: If we look back a bit in history, we would say the use case is well, the automotive industry is having the robots do spray painting and spot welding. There, that's robots. But more recently robots that are doing logistics, that are doing warehouse operations, customer fulfillment. That is now the dominant area for robotics.

    And I think that we will see in the not too distant future widespread use of autonomous vehicles, which are in fact robots, it's also a set of sensors and actuators, decision making. And that will greatly benefit from advances in artificial intelligence. Everybody's counting on deep learning, being able to interpret camera images to make good decisions. And we're seeing that happen. All of the manufacturers are introducing new capabilities, lane drift control, automatic parking, being able to observe and warn about vehicles approaching on the sides, from the rear.

    And recently, just in December TuSimple demonstrated a heavy truck going 80 miles with no people on board at all, completely autonomous. And particularly when we see the effects of the supply chain disruptions and shortage of truckers, as well as the highway deaths that we see and the impacts on the lives of long haul truckers, that I think will make a difference. And it's relatively low hanging fruit as far as the difficulty of autonomous driving. Highway driving is the simplest. In fact, the biggest danger is falling asleep. So I expect that to happen and once we're getting more comfortable with autonomous vehicles, I expect that'll be a real boost to the industry. I also expect that all that we learn from autonomous vehicles will come back again as advances in intelligence that will apply to other robotic applications.


    Who is Winning the Robotics Race?

    Lisa Chai: In terms of the enabling technologies who's winning the robotic race. Is it the US, China, Japan, or Europe? Could you share some thoughts there?

    Prof. Wyatt Newman: Sure. It's a tight race and the results aren't in. China has committed tremendous investment for a sustained period as a key technology, both in robotics and in AI. So we'll expect some implications of that. Japan was early in the game in robotics, and they also upped their game in terms of high performance computing. They have two of the three largest robot companies and they were a survivor in terms of certainly their expertise in continuous improvement. Their machines are remarkably precise and dependable. That makes a huge difference in their applications in industry.

    Europe, of course, is in the game. ABB robotics is an example of a top three competitor. The US largely lost out in the original race in industrial robots, but has continued to be a source of startups. So the technology engine and Silicon Valley continues to generate new novel startup companies. So I think that's still known as the best innovation. So we'll see what wins in the long run. Dump lots of money into it, or steady and careful, or move fast and break things.


    Is There a Sector of Robotics That You Are Most Bullish On?

    Prof. Wyatt Newman: The combination of the resurgence of robotics and the resurgence of AI will be extremely powerful. So if you look for companies that are doing both, more intelligent robots, I think will be a big deal. There is already a toolkit and set of foundations in AI that's being used widely. Natural language recognition, interpretation of images, interpretation of written text, facial recognition, these are all capabilities that can be extended to new areas. There should be, for example, automatic review of all radiology, cytology, pathology. Those are cases where humans can make mistakes just out of loss of attention, sort of like driving. And so the AI systems won't lose their attention. They won't make that sort of mistake. So in combining the newer capabilities of robots with greater intelligence, I think we'll see continued fueling of robot revolution. Really in the early days I think that what robots could do was over-promised, over-hyped. People expected them to be intelligent and they weren't. But I think that we're on the threshold of getting truly intelligent robots. So that's what I would look for, combination of the AI and the robotics.


    What Are the Computer Vision Companies You're Impressed By?

    Lisa Chai: Computer vision, also known as mission vision technology, it's one of the areas that we really care about, especially within the ROBO Global Strategies. Can you talk about some of the companies that you are really impressed with that has really strong demonstrations and capabilities around mission vision technology?

    Prof. Wyatt Newman: Sure. Well, one of the ROBO companies is Cognex, they're very early in the game in machine vision and they've kept up their pace in terms of being able to innovate it's... They have systems that are easy to install in the factory, you don't need a whole lot of expertise, and so you can get machine vision solutions in place relatively quickly, relatively cheaply. So I think that's a good example.

    Around the corner I expect more integration with deep learning so that you would be able to train your vision system by examples, and it would be more accommodating of variations of lighting, of different dimensions of smudge marks, or even the hard bin picking problem where everything is jumbled together, need to identify individual components inside there. I think that machine vision is getting better. Again, some of this will benefit from working autonomous vehicles and it will go the other way as well. Smarter machine vision systems will help make better autonomous vehicles.


    What's the Next Step for Computing Power?

    Prof. Wyatt Newman: Quantum computing will be the next leap. All right. Moore's law has kind of flattened out, so not so much a law anymore, but I think that quantum computing has the possibility of making a huge jump and that'll have its impacts on AI as well.


    When Will We See Self-Driving Cars?

    Lisa Chai: When do you think we'll have the autonomous cars and delivery drones in more of a kind of a mass market?

    Prof. Wyatt Newman: One could answer that question by saying it's already here, in degrees. So I gave the two simple example. There are earlier examples where a driver still sits behind the wheel, but it drives a truck. And there are relatively high-end consumer vehicles that do the same thing. You have to show that you still have attention, but you can take your hands off the wheel. So we're getting there incrementally. And at some point we'll say, "Oh, I guess they're autonomous now." But they're already becoming autonomous piece by piece.

    So like I say, the low-hanging fruit is going to be driving on highways when we can put on our cruise control and say, "You drive yourself here while I take a nap." That'll make a big difference. And it'll make a big difference in the quality of life. We lose, I think, 30,000 to 40,000 people a year in the US to automotive deaths from accidents, and millions who are hurt and need medical attention from it. And a lot of that is from either boredom or distraction, or falling asleep at the wheel.

    I think we're already at the point where autonomous vehicles have a better driving record on average than Americans. So at some point we'll say, "Yeah, we're we're there." It won't be like a switch. It'll be shades of gray until we have a majority of people saying, "Well, I guess it really is autonomous."


    Wyatt_Newman_Background (1)

    About Wyatt Newman, PhD

    Wyatt Newman is a leading researcher in the areas of mechatronics, robotics and computational intelligence, in which he has 12 patents and over 130 technical publications. After earning degrees from Harvard College, MIT and Columbia University, he was named an NSF Young Investigator in robotics, and subsequently was named a Herbold Fellow, a Tau Beta Pi “distinguished engineer,” a Woody-Flowers FIRST-robotics mentor awardee, and a CWRU awardee for teaching and for leadership. In 2007, he led “Team Case” in the DARPA Urban Challenge, involving autonomous vehicles operating among live and robotic traffic.

    In addition to visiting appointments at Sandia Labs, NASA, and Princeton, Wyatt has held international appointments at Philips (Eindhoven, The Netherlands), as a Distinguished Visiting Fellow at U. Edinburgh, and The Hung Hing Ying Distinguished Visiting Professor at U. of Hong Kong. Prof.  He also led HKU’s team in the DARPA Robotics Challenge, involving humanoid robots for disaster response.

    Wyatt is an active member of the ROBO Global Strategic Advisory Board.

    Back to All Entries