Episode 789 09.20.22
Henry Bzeih, CTO/CSO, automotive & transportation, Microsoft joins Peggy to talk about the metaverse and how it is impacting the automotive and mobility industries. He shares the five areas that are impacted by the metaverse in the entire vehicle lifecycle and discusses how mobility-as-a-service will require a certain level of data estate.
Below is an excerpt from the interview. To listen to the conversation from The Peggy Smedley Show, click here or go to https://peggysmedleyshow.com/ to access the entire show.
Peggy Smedley: Henry, I was thinking about it. When you and I were talking, I think it’s almost a year that you were on the show previously. Almost to the day.
Henry Bzeih: Almost to the day and what a coincidence, right?
Smedley: Since then, so much has happened, the world is changing. It’s moving fast in automotive, mobility, transportation, and there’s no one better to give us an update and to go full circle at this point.
Bzeih: It is incredible, Peggy, what has happened in the last 12 months from the world perspective, from a global perspective. The good news, hey, we’re almost on the way out of this pandemic, it looks like, and things are starting to pick up again. But going full circle into the biggest topic of them all, in my opinion, is the topic of the metaverse. We’re hearing about it constantly from various industry sources, not just from an automotive perspective, but as an overall, overarching, very exciting new way to virtualization as we take ourselves from this physical model to this virtual model. And a lot of it really has been driven through the continuous innovation that’s happening in our space. We’re attacking new frontiers and a lot of it is built around this tectonic shift with social, mobile, and cloud. And we are potentially at the cusp of a new disruption, which is this metaverse, which is going to shift us to new paradigms centered at transforming our physical world into deeper and more personal live digital experiences.
Smedley: When I hear the word metaverse and I think everyone may agree, they always get a little confused on what that means. And so, when you think of a driving experience, are you thinking of connected cars or are you thinking of mobility, the cloud or what any of this means, how does the metaverse actually come into play? How do we perceive that? Because the last thing we want to think of is somebody having HoloLens and driving down the road and going, “Are they really driving?” Is it a complete experience or is it all of the above? And we all have to think the world is changing. There’s going to be all of that coupled together.
Bzeih: No, absolutely. To begin, there are definitely many definitions for the metaverse. What exactly is the metaverse? There was interesting research that was done around trying to define it. And BCG was one of the research outlets that basically said metaverse, in their opinion, is an intersection of three distinct technologies. When you start with essentially your XR or what we call the AR, VR, MR. Mixed reality, virtual reality, augmented reality with things like HoloLens, as you mentioned, Meta Quest and Apple products, and you have this meta world or M-world platforms, and there you have areas or platforms like Mesh, Meta Horizon Worlds, and NVIDIA Omniverse. And there’s that third dimension, which is Web3, which is probably the most controversial in terms of the NFT, virtual currency, fiat currency, and so forth. So, their definition is that this is the intersection of these three areas.
In my opinion, it’s really what you are trying to accomplish in terms of serving the end customer. How are you providing the best seamless experience? Going back to your point around the automotive aspect of it, there are so many opportunities there from product development to the manufacturing piece, to the sales and marketing piece, and even the entire vehicle life cycle post sale. And we can apply some of these technologies in a way to serve the end consumer, and that’s the whole goal. How do we make things more seamless, more customer intimate? And that’s the goal that really, we’re trying to accomplish. So, in summary, we don’t want to define it. Many have tried to define it. It’s what you could take out of these tools and try to create with these tools the best experience for your customers.
Smedley: It’s interesting to hear you say that because I think in the automotive space, we have to look at the benefits to the products that we’re trying to do, what we’re trying to describe. And I think we get lost in what the automotive industry has become. Is it a manufacturer, a maker of cars? Is it a mobility company today? Is it a dealership? So, I think if I were to ask you the biggest benefits of that, it’s multifaceted.… Are there some benefits bigger than others? I know you just described. The three areas. But there’s lots. Am I correct?
Bzeih: Great, great question. If you look in the entire cycle or vehicle life cycle from an automotive perspective, I talked about product development and you could talk about product, engineering, simulation, and what have you. There’s the manufacturing supply chain aspect with process engineering, there’s sales and marketing and after sales. I see these two areas that are B to C, business to consumer facing, as the biggest gravity of realizing the potential of this technology. So, think of the whole experience that we have today. The clunky, I should say, experience that we have today. When we go in to either research a vehicle, test drive, go to a dealership and deal with the sales experience there. This whole way of doing business that’s been around for a long time and while it’s great, I think there’s an opportunity to take it to the next level. So, think of digital showrooms, think of virtual test drives, think of virtual reality supported sales.
And so, the dealership can turn into this life experience center. It’s not called a dealership anymore. It’s a life experience center in which people can come in and take one of the virtual reality or augmented reality headsets and try different colors on the vehicle all virtually, take a test drive without having to leave the showroom and even go deeper to understand some of the technologies that are within that vehicle, for example, right now, especially with electrification and autonomous. And so, to me, that takes that experiential level of the customer to a whole new level.
I think this is to surprise and delight the consumer, so I see that as a great opportunity. And I see also post that point, the after sales experience in which … You think of when people are in these vehicles, they’re so complex and some of the features are very complex. So, think of the ways that we can virtualize essentially the training manual or how things work in a very seamless way, in a very virtual way, to make customers fall even more intimately in love with their car. And so those are the areas I see as having the biggest potential out of this technology.
Smedley: So, you’ve opened up Pandora’s box for me. It has been my biggest complaint about dealerships. So, two things I want to address. No 1, Ford just made the announcement, no more haggling. When you get an EV, they’ve said this. And so, it might be as people go in there and they want to buy something, they have it pre-decided in their mind what they want, and they can go and discuss it and say here’s what I want. So that opens that great relationship, that friendly experience….
But No. 2, which I think has been always my concern, is we’ve added so much connectivity in the car that people buy it and then they’re on the road and they’re distracted and how many lives are lost? And it’s not necessarily the technology, but most people don’t know how to use the technology. And now you’ve opened an opportunity to say I’m not going to go back to the dealer and ask them how to use it, but they don’t know how and they’re stumbling around in the vehicle. So now you’ve said, wow, this metaverse experience is going to allow you to learn something maybe you didn’t know before. And I think that’s what I’m hearing you say.
Bzeih: You nailed it. And a lot of family and friends come to me and say some of these features … They know I’m an automotive guy. I’ve been there a long time. And they always consult me or give me feedback, and some of these systems are extremely complex. Something as simple as adaptive cruise control. For someone who’s just started driving, they’re like, how does this really work? And there’s a lot of intimidation factors that go along with that. And I don’t think a PDF or even a paper copy of how this works is really going to help you. At the end of the day, even if you take that simple demographic approach of saying these folks know how to use the digital means to learn about stuff. Why don’t we create similar experiences for them so that they can know about this feature and capability? A lot of folks turn off these features because they either don’t know why they’re there or how they operate.
So, there’s a big disconnect, in my opinion, in which this virtualization and ability to do augmented reality, to explain how things work and make things simpler visually will go a long way in creating that intimacy and falling in love with the brand as hey, they are thinking about me. They are thinking about my personal needs beyond just a transactional experience of leasing, buying, subscribing to a vehicle.
Smedley: I love your vision of the future on that and some other things that we can talk about, but I love the idea of having an avatar in there. The ability to walk someone through it and say, “Create your own personal avatar, your car. It’s your personal car.” We’ve talked about this for years in this connected car world, but now you can actually do it in the metaverse and experience and have this education that can go on and the dealer can create that intimacy with the buyer. I think that was never done, and I think that’s so exciting because we’ve talked about it at the university level, but now the dealers can do this and say, “Let me walk you through.” And that whole experience just changes a safer road, changes a greater experience and the buying experience and that brand loyalty you just described, I imagine, would be so enhanced from this.
Bzeih: Totally. And just to be more precise and not paint a picture or the peanut butter approach, if you will, when we talk about the end customer and the in-car experiences, we want to be very specific when we talk about how we support that and the technologies like augmented reality. So that’s specific. We talk about the dealership and earlier as I mentioned, the whole life experience or life in which somebody can come in and essentially try out pretty much any color, any model, get more intimate information. You could utilize the headset aspect. When we talk about the dealership from a service perspective, think of mesh where you can bring in productivity and also augmented reality for a technician who’s servicing the electric vehicle, a very complex vehicle or a-
Smedley: You’re describing the digital twin, then.
Bzeih: Exactly. And there, the engineering folks or the technical folks at the OEM can work hand in hand with the technician at the dealership to solve a given problem because that’s one of the biggest things right now with vehicles becoming more and more complex. This level of interaction, this level of communication and collaboration between the auto maker and its dealer body becomes extremely important because these vehicles are extremely complex, and they will continue to get more complex with autonomous, as higher levels of autonomy come upon us. Are we going to just fly engineers to the dealers every time there is a problem? No. So we need to find an extremely immersive experience in which we can overlay on top of the battery pack the circuit diagram in an augmented way. And in the meantime, you have the technician talking with you through Teams or other mechanisms to say, “Hey, let’s try that, sir. Let’s try that.” So anyway, I want to give you some more concrete, precise examples of how we can apply some of these technologies across the board.
Smedley: We’re going even further. If we apply the digital twin with AI and machine learning all the way in there, we can actually see things that … As we have this aging out….So you can take it even a step further as we look to electric vehicles and all of these things that we’re talking about that you’ve just described makes for things that are happening in the cloud that enable this mobility experience to be different at all levels of producing a vehicle, of how we’re making a cleaner environment, a more sustainable one in different ways is how I hear you describe that as well. We’re thinking our ways of dealing with our partners is going to change and that’s what makes it even more immersive as you just described.
Bzeih: Totally. And think of the notion of what we call a software defined vehicle. This is where the digital twin of that hardware abstraction of that vehicle and a cloud becomes really important. So, it’s decoupling the hardware from the software. Think of it as the smartphone on wheels. And just like today, you can update the firmware of a smartphone. You could add apps, you could delete apps. It’s a center of your life digitally. The vehicle can become that.
And this is where the digital twin of the vehicle becomes an extremely immersive experience first for the customer. Second, for the OEM and for the partners as well that can come in and create a constant Realtime connection between this thing we call the vehicle and the world around it. And it’s not this insular wall garden approach to design, build, and ship. And any time you need to get back into that chain, you have to go to the dealer, make appointments. Things are extremely difficult to do, they’re tightly coupled, and so upgrades and things like that. So, there’s so much that can be done. That additional twin would also enable this whole notion of what we call the software defined vehicle.
Smedley: Maybe we could take a step back for those who are sitting there and always say every time we come up with these great ideas, there’s always these hurdles. There’re standards, there’s regulations. Do we have to really look at all this? Before we run, we have to crawl. Is that where we’re at? What stage? We always say there’s different stages in all of this. Where would you say we are in this call to action that we’re discussing right now?
Bzeih: It’s a great, great, great question. And this is where the metaverse has its challenges. First of all, it’s a nascent frontier and it’s really missing some of that standardization. And again, many are trying to define it. So, to me, that’s really one call to action to create, especially when we talk about Web3 and what can be done there. And in terms of the meta world, what can be done there? There’s got to be some level of standardization taking place where the industries have to come together to define some common set of standards on that. Even on a software defined vehicle, there’s been an attempt in the industry to come up with the open-source approach. So, Microsoft is part of the Eclipse Foundation, and we are trying to be the enabler, per se, to create or come use an open-source standard that automakers or other tier ones, in using to capture this digital twin, if you will, from the vehicle into the cloud.
And so, standards drive quite a bit of adoption. But at the same time, there’s still the tendency by some to kind of say, I want to go at it alone. We have many examples of that, and the industry has been really good at this. If you think about the CAN network, the CAN bus, the good old CAN bus, which is pretty much the network standard in vehicles even though we’re evolving now to ethernet and others. So, there’s been quite a bit of standard adoption that happens around automakers or industry coming together to adopt. But also, there’s been challenges. Some feel like this is an advantage and they want to do things by themselves. And so, it is a little bit of a two-sided approach, if you will.
Smedley: You wouldn’t have a lot of the success if you didn’t have some of these hurdles, I think in some ways. I think you need to have the hurdles. We all know in the auto industry, if we didn’t have that CAN bus (protocol), so to speak, and that network, we wouldn’t have some of the accomplishments that the industry has had. When we look now at the metaverse and we look at these three buckets that you described earlier in our conversation, are we going to be able to overcome them in a decade? Is it going to be shorter than a decade or is it still too early to really decide how we’re going to be able to do the things we’re talking about?
Bzeih: Another great question, Peggy. So, let’s peel back the onion kind of in its three layers for these three technologies. And I have an opinion. So, when we talk about augmented reality, virtual reality, mixed reality, and we’re talking about, as I say, a headset, a large physical device that is sitting on your head, you have to question the scale. You have to question the scale not just from the awkwardness of having something like this, but also the price point because these things are not cheap. They’re expensive. So, to me, it’s for us to realize that first bucket. We have to scale, and we have this guy here, right?
Smedley: Mobile device. We use our mobile devices to communicate with anything.
Bzeih: How do we scale to get to that level of X reality through devices we own or devices we can afford? I mean, to me, that is kind of a barrier. That’s a challenge that somehow, we need to figure out. And I believe over time, we will. We will because technology cannot bet against it. It will definitely scale and it’s Moore’s Law and many other things. I’m a believer that we will get there. On the meta world, those to me are achievable today. We’re talking about Microsoft Mesh or Meta Horizon Worlds and Nvidia Omniverse as examples. There’re about 300 to 500 million users in this space. So, there is some gravity there. The other challenge is the Web3, and this is where we have, I think, a challenge as well from an adoption perspective. Standardization around that would be something that needs to really take place for wide adoption and positions on Web3 differ.
Some corporations have taken a stance against it. Some have embraced it. And it’s really kind of an open season here on Web3 and what needs to happen there. So that’s an opportunity there. So, I believe that’s what’s going to drive adoption and scale is coming up with some standards or framework of how we operate, and we have got a long way to go. If you ask me about the next 10 years, I believe this is going to evolve in a much more seamless way and I feel we can make progress in all three areas to get there.
Smedley: You talked a little bit about those drivers. Are there specific drivers that will do that? Then that Web3 that we’re kind of looking at, are there other things … When we sit down every day and we’re kind of struggling as an in-specific industry, trying to figure out as OEMs (original-equipment manufacturers) where we come into play because we’re in a tough economy right now. We have to figure out how the supply chain comes together because some partners aren’t going to make it? We have to help each other grow from a tier one, tier two, tier three. And it’s a different environment than it used to be. Is all of this going to play a role when we think of the metaverse because we have to figure out how to do it? And in order to accomplish it in the meantime, some companies will not play, new players will emerge it. It’s a rocky road again. And we just got out of COVID-19 as you mentioned.
Bzeih: Maybe I’m excited to hear about this one, which is supply chain using blockchain in supply chain, right? Supply chain’s been a big topic for us for the last two years. One of the things that highlighted is how vulnerable it is. And how can we make it more seamless? How can we make it more transparent? How can we make it more efficient? There’s a lot of bureaucracy in supply chain. And so, one technology that can be applied here is the blockchain aspect, which brings in that transparency, brings in that, if you will, trust factor that can really erase a lot of these checks and balances that need to be in the system that somehow affects supply chain as well from an efficiency perspective. So that’s one area in which we could really help. And you started to see some of that adoption taking place there.
Ultimately, here’s the issue that, I feel also, we struggle with. You have to think about acknowledgement of digital asset ownership and the whole digital currency. There’re some countries who have taken a stance against it. So, from a global perspective, we’re talking about scale. It’s just been, like I said earlier, not a ubiquitous way of transaction or creating an economy. So, it’s going to differ. There is the issue of rules of behavior that we need to be contending with. If you can do anything in the metaverse, what can you do in the metaverse? Can you do malicious things? Is that acceptable? And so, there’s some really key elements that need to be solved to be successful. But talking about now, today, I would definitely talk about the customer experience aspect. We’re seeing some things we’re doing with companies like TouchCast on the whole experience of folks experiencing virtually configuration of a vehicle, for example. That’s low hanging fruit that’s available today. I’ll talk about supply chain, which is near and dear to everyone’s heart, as blockchain coming in and really helping take some of these boundaries and these walls and help make things efficient.
Smedley: So, we’ve got configuration of the vehicle. We’ve got to worry about the bad guys. We’ve talked about NFT. Is there anything that we’ve left out in this conversation? I think looking at it right now is how mobility in the metaverse for OEMs and partnerships and supply chains right now that we know. It’s fast. It’s evolving. It’s something we’re excited about yet there’s a lot of challenges. But there’s a lot of opportunities and I think it opens the door for a lot of new startups to work with a lot of existing well entrenched companies, is what I would see. And I would imagine that’s what you think as well.
Bzeih: Totally. And so, to me, the best way to describe a very big challenge that we have today regardless of the metaverse, but so critical to the metaverse, is data sharing and the whole notion of having some sort of a data collaborative around feeding this virtualization in a way in which multiple entities can drive a metaverse experience, a digital twin experience. I think we talk about mobility. Mobility without data sharing is no mobility in my opinion. Without having that level of collaboration is difficult. And you know what’s so interesting? It’s one of those things that is so obvious, yet it is so difficult to do. Entities sharing data with other entities, there’s a cultural mindset that’s always a challenge. It gets in the way in terms of people that are concerned about trust, concerned about security, concerned about … Hey, I’ve collected this data, it’s worth something to me. It’s a monumental challenge that any industry is dealing with.
And I think our industry’s dealing with it tremendously and we will never realize the whole full potential of mobility unless we reach that point in which we can share with each other and enable each other and create for the customer that end experience because mobility being multimodal, having different inputs from different areas of the journey, and without having this data being readily available from collective inputs, it’s hard to really come up with that best solution. So that’s a challenge that I throw out there as a call to action. How do we find ways to collaborate on data models? How do we find ways to enable data sharing in a way that is trustworthy, secure, but also beneficial, win-win for everyone? And also, we have to think about data sharing does not mean giving away the privacy principles or anything like that. There are ways that you could still share data in an anonymized way or a privacy protective way to make sure that we’re also respectful of that dimension as well.
Smedley: Talk about mobility as a service then.
Bzeih: Yes, mobility as a service to be fully realized requires that level of data estate, data gravity. Without it, it’s a one-dimensional mobility as a service. So, we can just say, “All right. If we have a robot taxi service that is picking somebody up and dropping them off by that entity and it’s a vehicle doing that, is that mobility as a service?” Yes, it fits the definition of that. But think about something better.
Someone going from their home in the suburbs to the city and along the way, they’re using a train, a rail or public transportation or a bus and a robot taxi and possibly an e-bike and going along the way and doing some other things. So, think of that journey being a seamless journey in which there are different models, but with different entities and with one experience for that customer, one transaction that can benefit everyone along the way, but at the same time, give that traveler, if you will, that mobility traveler, that seamless, frictionless experience along the multiple modals in the journey. So, to me, that takes it to the next level. And there’s one level, which is take from point A to point B, and there is another level in which it gets really complex, and how do we make it seamless there?
Smedley: And so now talking on what we’ve been saying about getting us there from looking at the metaverse, looking at our supply chain, again, there’s a lot of moving parts about the ability to get us there. Are we working on all these really great ideas right now? I mean, I know we’re working on it, but the ability to say we want to get there, and we have to get there sooner than later because the customer is excited about these opportunities.
Bzeih: This is our north star. We believe in the metaverse as a technology, as a technology frontier. We believe in the data estate and having the data gravity as the fundamental path to mobility as a service. And both areas are nascent in terms of achievement, right? We’re not going to declare a win yet. We know that’s the north star. We know that’s what we need to do, but we are on a journey, and we are at the beginning of that journey. And we have some small wins touching on the data side, but also on the metaverse side. I think that topic or that technology frontier, Microsoft is paving the way in many ways with our HoloLens products, our Mesh products. And we’re not talking about something that is motherhood and apple pie, it’s application.
Bzeih: I would encourage everyone to go to microsoft.com/industry/automotive.