Hitachi collaborates with customers, partners, and educational institutions worldwide to create innovative solutions to issues facing society and contribute to a better future for all. In this article, we invited our partners, Huei Peng, Professor of Mechanical Engineering at the University of Michigan and Director of Mcity, and John Nunneley, Senior Vice President, Design Engineering at Hitachi Automotive Systems Americas, Inc, to join Harsha Badarinarayan, Senior Director and Leader of the Detroit Research Center and Subrata Kundu, Manager, Automotive Products Research Lab, Hitachi America R&D, in a discussion on how autonomous vehicles drive changes in society.
Photography by Dr. Kota Irie, Manager of Advanced Development Center for AD/ADAS Technologies, Hitachi Automotive Systems Americas.
What is the state of autonomous driving?
Harsha：Autonomous driving and advanced driver-assistance systems (ADAS) are hot topics. Automakers, suppliers and thousands of startups have been working on autonomous driving technologies for decades. The Society of Automotive Engineers ranks the levels of driving automation from level zero (no automation) to level five (full automation). Many expected fully autonomous vehicles to be on the road by now. However, the highest commercial level of automation is still level three, where automation is in effect only in certain, limited conditions.
John, from your perspective, what level of automation are customers seeking in the U.S.?
John：We see customer demand for low-cost level-two systems such as steering, braking and acceleration support. These features use either just a camera, or a camera and radar. Higher level-two or lower level-three features are on premium vehicles. The question is, will customers pay for higher-level features on standard vehicles?
Harsha：Dr. Peng, what is the public’s general level of acceptance for these ADAS features?
Huei：Technology for driving assistance systems has matured. Auto emergency braking is widely available now. There's no question that these systems are providing safety benefits. But drivers start to think that they may not need to pay full attention, which is dangerous.
People see the benefits of ADAS, but it’s a challenge for people to understand what the system is capable of. For instance, the driver of a level-two car could mistakenly think that the car is fully autonomous. It takes time for customers to understand this new technology, and this is the major barrier in the near term for ADAS. More training time may be needed for dealerships and customers.
How R&D is addressing the challenges
Harsha：Subrata, what kind of innovation is being created by Hitachi R&D in this area?
Subrata：At Hitachi R&D, we’ve been working on autonomous driving software and hardware to address complex issues. One of our solutions is a stereo camera that detects obstacles, road surfaces, traffic signs and so on. The system can then control braking, acceleration and steering accordingly to provide better comfort and safer, stress-free driving. In addition, we are working to minimize energy usage, lower emissions, and create seamless traffic networks and other innovations.
Since there are many different scopes in the autonomous driving area, we work directly with our customers to understand the complex and unknown issues and then develop a solution together that can provide the safety and comfort that autonomous driving requires.
Harsha：Moving to higher levels of autonomous driving will pose challenges for both technology and society. How can society understand and start adopting level-four or higher vehicles in the coming years?
Huei：Challenges experienced by level-four vehicles are similar to those at levels one and two. You must communicate to your customers what your level four is. What are the limits? Is your vehicle speed-limited or weather-limited? Is it geofenced? Is it able to handle a left turn where the view is partially blocked?
People are sometimes afraid of new technology, and that's natural. Deploying technology in a careful, limited way helps reduce some of the concern. It's up to us to deploy technology carefully and judiciously so that people won’t fear it.
Harsha：John, what are some of the main technical challenges that need to be addressed before reliable systems for autonomous driving are ready for commercial deployment?
John：Level three, conditional automation, is confusing for drivers. You're basically telling them the car will drive itself, that they don't have to pay attention until we need them to, and then they have to quickly pay attention. That's difficult. I can see legal issues and other problems arising in level three. At level four, the vehicle has to still be able to safely get itself off the road, stop, and give the driver plenty of time to decide whether to take over and drive.
Since no single sensor can handle every situation, we keep adding more sensors — LiDAR (light detection and ranging), cameras, radar, finite impulse response (FIR) and more. This enormously increases the amount of data that must be processed. We have to collect the data, process it, decide which data to trust depending on the environmental conditions, and then plan the path and actuate the vehicle based on these decisions. This requires server-level processing power. So not only is the sensor suite commercially prohibitive, the amount of computational power necessary makes it unrealistic for individuals to purchase one of these vehicles.
We've got to simplify the processing and the software. We've got to decide quickly which sensors we use in which conditions so that we don't have to process all this data. One of the main steps required is reducing the complexity and costs.
Harsha：Right. We've got to think about functional safety and redundancy. With a lane departure warning, you just get the warning and life goes on. But if you're in a level-three or higher vehicle and encounter an issue and the driver's not paying attention, it creates a difficult situation. Any thoughts?
John：Clearly you need redundancy and backup systems, which are expensive. That's why you're seeing the first adoption on small, controlled fleets that are owned by large companies, not by individuals.
Harsha：From a design point of view, are there any techniques or new approaches that R&D can take to simplify not only the design but also the process?
Subrata：Definitely. R&D can focus on finding the optimum combination of technologies that will work in all types of conditions. We can use multiple LiDAR units or cameras, but we need to consider the cost of the vehicle. And, as John mentioned, the enormous amount of data processing required by the electronic control unit for real-time applications results in high power usage. This also leads to heat generation, which ultimately reduces vehicle efficiency — R&D is also focusing on those issues.
And we must absolutely ensure that the system works perfectly in all conditions before we deliver it to the customer. It's difficult to ensure this by an actual road test. We need an effective co-simulation platform so that we can evaluate vehicle-level system performance and verify the real-time performance of a combination of sensors in the electronic control unit to help ensure the safety of the autonomous driving systems.
Connectivity, data sharing and privacy
Harsha：Dr. Peng, how do you think connectivity plays a role in enhancing ADAS features?
Huei：Connectivity provides additional useful information for both near-real-time and long-horizon problems. For example, a vehicle that is not connected will need to rely on in-vehicle technology such as sensors, cameras, radar and LiDAR, which won’t help if you have an occluded left turn or intersection.
At Mcity, we provide a research facility and testing ground for connected and automated vehicles. We work with dozens of partners from the automotive industry. If you are not sure your vehicle can handle unprotected left turns or a roundabout, you can bring it to Mcity to test it. You can control the exact test conditions, and you can repeat that all day long. Although we were created to focus on mobility, we intend to be part of the development of smart cities, and that means connectivity.
With connectivity, a roadside camera or LiDAR can see when there are moving objects approaching the vehicle and broadcast that information to the vehicle. That kind of communication will not be widespread quickly, but it could be employed at challenging intersections. This approach uses infrastructure intelligence to help the vehicle’s onboard intelligence. When connectivity is in place, instead of relying on onboard intelligence and sensors, you can use information from a high-definition map or roadside sensors to get information about road curvatures, ice or potholes.
Harsha：Connectivity and data sharing bring regulation. Dr. Peng and Subrata, can you briefly talk about the regulations here in the U.S. and elsewhere?
Subrata：Rules and regulations for using connected data vary by region. If transportation authorities can use information about the individual's driving history or destination, they can more easily provide traffic data and reduce congestion. The first wide application of connected service will probably be in Middle Eastern countries and China. North American countries will be later because of data privacy restrictions.
Huei：Nobody wants to be tracked from home to work. If cameras are everywhere, at every intersection, that's a problem. But if you only capture data in selected intersections, that's less of a concern. Still, it's important to be transparent and judicious about placing cameras in intersections.
Harsha：From a business point of view, John, how is connectivity going to benefit us in developing new business models or adding value for the customers? Why should customers pay for this kind of service?
John：Sharing data from a set of vehicles to the cloud and then broadcasting it to every vehicle minimizes the processing power required in each vehicle. You have to collect some data in the autonomous vehicle, but things like road and weather conditions are a little more static and don't need to be collected in real time by every vehicle. That reduces the onboard processing necessary, reducing the complexity.
Benefits of electrification: Electric vehicles and autonomous driving
Harsha：Connected vehicles are often powered by electricity, which is another hot topic. John, from talking to your customers, where do you think we stand now with electric vehicles (EVs)?
John：EVs are greener than internal combustion vehicles because they have lower emissions. However, if it takes a long time to make up for the extra cost of an EV in gas savings, customers will be less likely to purchase one. Some people will purchase EVs and take the financial hit because they're doing something good for the planet. But that's not the majority.
Some countries, such as China, are implementing strict regulations to force the move to EVs. That's necessary because the cost of batteries is significant. The volume must go up before prices can come down, and the only way to increase volume is to subsidize it through government programs.
Huei：I agree with John. We cannot afford to keep this a financial decision by individuals. Governments can help. For example, California allows EVs access to HOV carpool lanes, helping EV drivers avoid being stuck in rush-hour traffic. Norway offers free EV parking in some downtown areas.
Subrata：It’s not just the electric vehicle, but the connected autonomous electric vehicle that will change the future. And changes are coming soon for connected EVs. At Hitachi R&D, we’ve demonstrated how to use connected data to improve their efficiency. We’re conducting extensive research on materials and manufacturing process to further reduce the cost of electric motors and inverters.
Autonomous vehicles: The next 10 years
Harsha：Mcity is one of the premier institutes for connected autonomous vehicles. Dr. Peng, how can Mcity bring together people to solve societal challenges?
Huei：The name “Mcity” reflects our focus on smart cities. We’re building on what we're really good at, which is connected automotive technology, but also data. Mcity was one of the first facilities to use an early prototype of 5G. We have an edge computer with 24 GPU — a lot of computation power. We want to build on top of mobility and move to a better-living laboratory concept. Leaving an impact on people's lives is our objective.
Harsha：When and how do you see the implementation of autonomous vehicles?
Huei：I think that the current trend — level one, level two — will continue. Level-one automated vehicles will eventually have 99% precision. Level four will start in a shared managed fleet with a commercial application. Today's LiDAR, radar, cameras and GPUs are still expensive, so it makes sense to use them on vehicles that are used 12 hours per day, not two hours per day.
Harsha：John, from a business point of view, where do you think we are headed in the next 10 years?
John：I think we’ll see a quick acceleration in the low to high level-two applications. The high level-two applications now in premium vehicles will move down to non-luxury vehicles. We can get to level-four capabilities within the 10-year horizon, but that will be limited to shared fleets and geofenced, fixed-route applications.
Another growth area is long-haul trucking, where a truck that's now driven eight hours a day can be extended to drive 20 to 24 hours a day. This will reduce shipping times and costs. Also, in urban or densely populated areas, private ownership of vehicles will decrease.
Harsha：Subrata, what are some of the challenges we’ll face in design and technology over this 10-year horizon?
Subrata：I'm optimistic that we will be able to solve the existing problems. At Hitachi R&D, we have been working on high-resolution sensors that use connected data. We’re looking at how to effectively manage connected data for highly reliable autonomous driving, and we’re optimizing the digital network architecture to support real-time applications.
Since safety is our first priority, we have to consider using redundant safety controllers until we are sure that the AI-based controller works perfectly in all conditions. The world of the future will be a system of systems, and connected autonomous vehicles will be an integral part of that society. Those vehicles will significantly contribute to safe and comfortable mobility and reduce congestion and emissions. By using Hitachi's social innovation co-creation approach, we can collaborate with our customers to deliver significant contributions to society.
John：Manufacturing and automotive companies know about specifications and regulatory requirements. IT companies know connectivity and software development. Hitachi has both because we're both an IT company and an automotive company. We combine both of these worlds in one company.
Harsha：Working with partners such as Mcity will help us showcase these accomplishments.
Huei：Yes, we look forward to working together with Hitachi and others to continue developing smart city concepts well into the future.
Hitachi collaborates with customers, partners, and educational institutions worldwide to create innovative solutions to issues facing society and contribute to a better future for all. These conversations provide an opportunity to hear from experts and thought leaders about exciting advances that are happening today as well as sharing how Hitachi and others are approaching economic, societal, and environmental changes for a brighter future.
Profiles
(As at the time of publication)
Huei Peng, Ph.D
Director, Mcity
Roger L. McCarthy Professor of Mechanical Engineering,
University of Michigan Engineering
Huei Peng received his Ph.D. in Mechanical Engineering from the University of California, Berkeley in 1992. He is director of Mcity, the University of Michigan’s public-private partnership devoted to advancing the development of connected and automated vehicles.
Peng is also the Roger L. McCarthy Professor of Mechanical Engineering at U-M. Peng’s research interests include adaptive control and optimal control, with emphasis on their applications to vehicular and transportation systems.
In the last twenty years, Peng has worked on vehicle automation, vehicle dynamics, design and assessment of active safety systems, and human model development—with a special focus on understanding how they err. He has served as the principal investigator or co-principal investigator of more than fifty research projects, with a total funding of more than $30 million. He has more than 240 technical publications, including 100 in referred journals and four books.
Peng is an SAE Fellow and an ASME Fellow. He is a ChangJiang Scholar at the Tsinghua University of China.
John Nunneley
Senior Vice President, Design Engineering
General Manager, Farmington Hills, MI Office
Hitachi Automotive Systems Americas, Inc.
www.hitachi-automotive.us
John Nunneley is the senior vice president of design engineering and the general manager of Hitachi Automotive Systems Americas, Inc. in Farmington Hills, Michigan. John holds a BSEE from Michigan Technological University with more than 30 years of experience in engineering. He worked in the design engineering group for Hitachi Automotive Systems in Michigan from 1995 to 2011 on projects ranging from electronic throttle body to engine control units to HEV inverters. John worked for Hitachi Automotive Systems in Japan from 2012 – 2016 as the Deputy General Manager of the Brake System Design Group and the Electronic Control Unit Design Group. He returned to the United States in July 2016 to take his current position managing Hitachi Automotive Systems Americas’ full product portfolio including engine management, electric powertrain, drive control, and car safety and information systems.
Subrata Kundu, Ph.D.
Manager, Automotive Products Research Laboratory,
R&D Division,
Hitachi America, Ltd.
Subrata Kundu is a manager of the Research & Development Division at Hitachi America, Ltd. in Farmington Hills, Michigan. He leads a research team focused on advanced driver assistance systems (ADAS), autonomous driving (AD) and connected services for future smart cities. Prior to working at Hitachi, Dr. Kundu worked as a postdoctoral fellow at the Toyota Technological Institute and as a project assistant professor at the University of Tokyo in Japan. He holds a B.Sc. degree in mechanical engineering from Rajshahi University of Engineering and Technology in Bangladesh and M.E. and Ph.D. degrees in robotics and intelligent systems from Saga University in Japan. His research interests include soft computing, biomechatronics, robotics and intelligent systems.
Harsha Badarinarayan, Ph.D.
Senior Director and Leader of the Detroit Research Center,
R&D Division,
Hitachi America, Ltd.
Harsha Badarinarayan is the Senior Director and Leader of Detroit Research Center for Hitachi America Ltd./R&D Division. He has been with Hitachi for over 17 years. In his current role, Harsha is involved in strategic planning for automotive business expansion in the U.S., specifically related to the Connected Cars and Autonomous Mobility. His work focuses on developing automated driving technology and identifying potential go-to-market (GTM) solutions for automotive-related use cases. Harsha holds 14 U.S. patents and has authored or co-authored over 30 technical publications.
