AI Online

Ai INNOVATION, SINCE 1895

Capturing, analyzing and storing terabytes of vehicle-generated data

Fast-paced growth in the generation and capture of vehicle and driver data, including unstructured data via sensors lidar/radar, object data and storage platforms is creating opportunities for new business models in the automotive industry.

Automotive Industries (AI) asked Manuvir Das, Senior Vice President and General Manager, Unstructured Data Storage at Dell EMC to tell us more about the trend.  

Das: There are several areas where data is shaping the auto industry today and setting the stage for some emerging business models:

  • ADAS – automotive drive assisted systems – things like lane departure systems, automated braking, intelligent speed adaptation and the like
  • Connected cars – cars are getting more intelligent, with more customized functionalities just like smartphones did only a few years ago. Some cars are directly connected to 4G data networks, and much of the conversation around emerging business models is a direct result of this trend.
  • Then you have the holy grail of connected cars – the fully autonomous vehicle, which requires an on-board system that cannot only process incoming data in real-time, but also makes informed decisions. Getting to that point, however, requires the collection of millions of miles’ worth of data in order “train the machine”.

AI: Where are the opportunities?

Das: Connected cars will become more personalized and will support applications, and provide useful data to drivers via digital assistants such as Siri or Alexa. There are some exciting possibilities for automotive manufacturers – provided they are approved by the regulatory bodies. Should manufacturers sell their vehicles’ data to advertisers? Of course. Just think about all of the data that could be captured and stored for not only your car’s performance, but also your personal preferences. Manufacturers could connect your car’s navigation and GPS info with localized services at your destination. If you’re going to a football game, your car might be able to offer suggestions for nearby parking options. Could we see on-board advertising in the future? Absolutely.

Down the road, how much would you pay for a service that trains your self-driving car to follow the most effective route? If people are willing to pay for high occupancy traffic lanes that bypass traffic, how much would they pay for a service that consistently shaves 20 minutes off their commute? Quite a bit I imagine.

AI: How does data storage play a big role in the future of the auto industry?

Das: Data will be the ultimate differentiator between the services manufacturers come up with. In the connected car example, it will be important to deliver the digital experiences users crave. Where it will have the biggest impact though, will be with fully autonomous vehicles. It will require a massive data set – we’re talking in the exabytes – to train the autonomous vehicles before they become a reality.

AI then asked Varun Chhabra, Senior Director of Product Marketing, Dell EMC, what are some of the reasons for auto manufacturers to capture, store and analyze big data.

Chhabra: We’ve seen in other industries that data is becoming the single greatest asset these businesses own. Automotive will be no different. For example, as they develop assisted driving capabilities, manufacturers will have to do a lot of testing and validation, and that means storing a lot of data. Dell EMC is working with one international auto manufacturer that has a fleet of 100 test vehicles around Europe, and that fleet has an incredible number of sensors, 4K cameras, lidar, radar, and more. Those inputs translate to about 30 TB per day per car. You multiply all the different OEMs out there and the amount of data being created, and it’s mind-boggling. It all has to be stored.

Manufacturers have to be able to search through this data and tag it as it is collected. If you’re building an automated braking system, you need to identify when another car stopped quickly in front of your vehicle, for example. Manufacturers have to be able to go in and tag various incidents across a variety of data in a variety of formats.  Your storage infrastructure has to centrally store data from all of the different sensors and sources, consolidating them into a massive data lake and then enable manufacturers to access and analyze that data.

AI: What are the challenges in building up the data storage infrastructure?

Chhabra: The very first challenge for manufacturers is data collection. With both ADAS and connected cars, manufacturers need to store a massive volume of data in a wide variety of disparate formats. You need to outfit enough vehicles with the various sensors to collect the data, and then you need a way to get that data back to your main data center. The second challenge is the ability to conduct analytics on that data, which often differs in format. Video data, for example, is very different from lidar data.  

The fact this data is unstructured furthers the challenge as it needs to be sorted and tagged with metadata for it to be useful in training the system. Having a centralized storage platform allows manufacturers to analyze extremely large files that are rapidly proliferating.

Finally, given the incredible amount of data, and the fact it’s being collected in widely disparate forms, it can be easy to overwhelm networks. The amount of time to transfer high data volumes to the cloud, for example, can be massive. You might have a 100Gbps network inside your data center, but your connection to the public cloud is going to be a fraction of that. This leads to a tradeoff that must be made: either prioritize speed or reduce the dataset, a losing proposition. Dell EMC provides a single cost-effective data lake where massive amounts of data of all types can be combined and processed using advanced analyzing capabilities.

AI: What are the challenges facing hybrid cloud and data protection solutions?

Chhabra: One of the main things to consider around distributed or hybrid architectures with a public cloud is data privacy requirements and data sovereignty laws. Particularly with multinational automotive manufacturers, you need to consider how that data can be used, and how (if it all) it can be accessed by or sold to third parties.

Regulatory requirements are another factor to consider, regarding accidents, fleet recalls and the like. Can you put a legal hold on the data in the event of such an event? How is a manufacturer’s data stored, and for how long? You need that data to be protected. Finally, there is the issue of cost-effective storage. Remember most public cloud solutions charge access and egress fees meaning when you do analysis or send any data out of the public cloud you’re paying for it.

So, to recap, you have this massive incredibly valuable data and you want to protect it, but there’s probably going to have to be a distributed approach to it. Some data will remain at the edge, what is relevant maybe an isolated incident that is unusual gets pushed back to the core (your data center), and maybe the metadata is stored in the cloud. Hybrid is absolutely the way this approach will go.

AI: How will managing and monetizing the vast amount of data become a game changer in the automotive industry?

Chhabra: Given the importance of this data in delivering the digital experiences and new services for customers it will be critical for companies to have established a data platform that can keep up with all this data growth. It’s all about scale, the ability to store and manage different types of data for a long time, and to manage the data without complexity or silos, all without breaking the bank. By having the right solution in place, a manufacturer can collect and store all the data it will need – both the data which already has value, and the data which has yet to show its true value. Sometimes this is the most difficult choice with data – what to keep.

Previous posts

Next posts

Thu. March 28th, 2024

Share this post