Peggy and Junade Ali, consultant, Haystack Analytics, talk about road traffic safety and the potential burnout of software engineers. He shares his journey with road traffic sensor networks and behavioral interventions for drivers—and how he built an engineering team.
Below is an excerpt from the interview. To hear the entire interview on The Peggy Smedley Show, visit www.peggysmedleyshow.com, and select 08/4/2021 from the archives.
Peggy Smedley: When we think about road traffic safety, a lot of people don’t give enough attention to it as I do. It has been something that has been on my mind, and foremost on my mind for years. And that’s why I’m really excited to have you on the show because you’ve done a lot of research in this area, and I want to know from someone like you, how do you go from being a road traffic engineer , to now on engineer productivity. That’s what intrigued me the most with what you’ve been doing.
Junade Ali: Yeah, absolutely. So, I know you’ve definitely spoken to a lot of people who are quite notable in the road traffic entry into the IoT space before. My journey kind of started there and then originally the thesis of my master’s was really focused around road traffic sensor networks. I worked in these organizations which were building the road traffic sensor networks and the actual behavioral interventions for drivers to improve road traffic safety. So, you know, a few examples of this are if someone is, for example, driving on a motorway in the U.K., there’s a sensor system which counts people on and off the roads. They’ll make interventions based on how people are driving. A lot of these technologies are very invisible, but as a result of this these interventions, the U.K.’s road network has become one of the safest in the world, and this has been powered by a lot of these technologies which are invisible to people, and what I was able to pick up during my time in the road traffic engineering space is that, you know, a lot of these technologies which have a high impact on people, which can make life or death decisions for people, they often function best when they are invisible to people, and they’re behind the scenes and people don’t know that they’re there and kind of the complexity is hidden away, so, that was something I really took forward into my career and that I spent about four and-a-half years running a team, which was kind of a mix of both software engineers and research scientists in the internet infrastructure space. So, I saw both that world there when I was able to build an engineering team that I was really, really proud of and was able to do a lot of high impact work in cybersecurity, internet infrastructure, that type of thing. And then kind of going forward.
I then decided what I wanted to do was I wanted to take these productivity learnings and these engineering learnings and seek to apply them to the broadest field possible. So I decided to go into being a contractor myself and starting to build up my own consultancy, but also working alongside Haystack Analytics, I’ve been able to leverage that to bring these learnings on engineering productivity to a wide audience across the industry.
Smedley: What’s interesting to me is, road safety, we could learn a lot from what happens in the U.K. to the United States, because we clearly haven’t mastered that, it’s a disaster here in the United States, but what I’m most intrigued about now is you talked about working with all these high-impact engineers, who, and now we’ve seen a year, of all of us, over a year, you know, COVID has impacted us all, and I’m kind of curious now, you did this study, you know, what COVID’s done, you’ve talked about working with these amazing engineers, and I’m curious, what the study now, why you’ve decided to study engineers, I mean, is the pressure, the study that you did, you know, how did this impact them now? I’m very intrigued by this.
Ali: So, I think over the last 18 months, the world has really changed, and I think that has put a lot of different requirements on engineers across the industry, across all different industries really, and across the world, and I was really interested to understand the impact of what has happened on software engineers, because a lot of these things have had to change due to the way the world now works, that burden has often fallen on software engineers, and there hasn’t really been much done to really understand what’s going on there. So that was something I really wanted to drill into.
In order to do the study, I was lucky to be able to partner with a strong research agency known as Survation. They’ve done a lot of work in the opinion polling space, and actually, a lot of the advancements in terms of how you sample people and how you do these types of studies, so just as an idea of their track record, they correctly predicted the 2015 and 2017 U.K. general elections, which were quite surprising results. They correctly predicted the outcome of the last two referendums in the U.K., they use a lot of different and interesting tactics, so to reduce self-selection bias, they use panels instead of allowing just anyone to take part and they also randomize their options.
Smedley: So, not to interrupt you here, but let’s peel back the onion here a little bit. So, if your research, this research found 83% of software engineers are burned out, what is that going to indicate? We also recognize that this is a problem with a lot of people working during COVID. It’s not just software engineers, we’re finding emotional issues with individuals just in all, across all, as you indicated, industries. But what concerns me most is the reliability of what we’re going to get on the solutions that they’re working on. So, if we’ve got burnout, not only do they not want to maybe do the job, but what does that say about the products they’ve been working on? The underlying security, the underlying base, what they’re doing, what is that going to say overall?
Ali: Yeah, so I think that’s a really interesting point we talk about when, in aviation, or in other forms of road traffic systems, we talk about the behavior of the operator, how we don’t want people distracted. Well, as we move into this world where the software is even more complicated and it has to react to really challenging situations, we want the software in those circumstances to be able to behave reliably in those circumstances.
So, obviously we really don’t want our developers to be burnt out, and to kind of just to peel back what we looked into in this study—there’s kind of two dimensions here. There’s one dimension which is yes, 83% of software engineers are burnt out, but kind of the next layer down, we also understand that 81% said this has gotten worse due to COVID. And the leading driving factor behind this has been high workload, you know, the increase in workload due to the pandemic. And then there are other things around inefficient process and unclear goals and targets. But the absolute key driving factor, behind why people say their feelings of burnout have gotten worse during the pandemic is because of increased workload, you know, 47% of those who are burnt out reported that to us.
Smedley: So, here’s the bigger question: is it changing? Because companies now are still trying to decide whether they bring their software engineers, their employees, back to the office. We hear about this Delta variant that people are going to still stay home. That’s not going to solve the problem. There are bigger issues at stake here based on this research, wouldn’t you agree?
Ali: Absolutely, yeah. I think there’s some really critical things which are going on behind the scenes, and I think when we also look into things, whether it’s just remote working or things like that, you know remote working actually fell quite low in our list of reasons. The real things which were of substance were really around high workload, inefficient process, unclear goals and targets.
Smedley: So, you’re describing bad management, you’re describing bad management. I didn’t mean to interrupt again, but that’s what you just described. If that’s what the research said, then managers are bad managers. That’s what these software engineers are saying. It’s bad management. Or bad managers.
Ali: So, I think a big driving factor behind this has been, there’s been a lot of learnings over the past few decades which we haven’t really incorporated into our workplaces. One of the things I often talk a lot about in my consultancy work is this idea of cycle time, you know how quickly is a business able to react to change in circumstances. How quickly are they able to take something from an idea in someone’s head into like, a production ready environment where it’s deployed in users and users are seeing this and really that ends up being a measure of process. You need to see truth to that way, as a measure of process rather than saying, oh we can just work people harder, and I think you really need visibility into that end-to-end process. Where does the software start being built and where does it end up? And that entire lifecycle, when you look at it, there are clear blockers which kind of come out, and that’s the end-to-end process which managers need to work on and improve in order to get more successful business as well as having their staff being less burnt out.
Smedley: Is that part of the challenge? Because as I hear you talk, it’s really intriguing to me, because I think CEOs demand certain things. They have shareholders, they have investors that they have to drive for certain goals. But, trying to get to development is tricky because it requires a series of processes, and sometimes those processes get hindered by the way things change–market demands change. So it’s a tug-and-pull scenario and as a software engineer, someone who understands that, are we being pulled at different ends of the spectrum and that makes the development process kind of tricky?
Ali: Yeah, so I think there’s this mix of understanding almost. The businesses we think of which have been really moving forward in terms of their commercial goals and their technical goals, you know, the Googles and Netflixes of the world, all of these companies have engineer productivity teams, because they understand the importance of process, the importance of being able to move quickly as requirements change. And the impact of COVID has been a lot of businesses have had to change the way they operate in a really, really short timeframe, and a lot of them don’t have this capability of being able to understand what these processes look like for software engineers, so I think it’s absolutely the case that a lot of people have learned this the hard way, a lot of the companies which have been historically really good at reacting to changing and demands, really agile, those are the ones who have been able to benefit because they have these teams already in place to react, and the critical thing I think as well, looking at some of the prior research, is a lot of this ultimately comes down to psychological safety. You know, how free do people within the companies feel they are to raise the alarm to things which are kind of going wrong, able to raise their hands and say I have a new suggestion here as to how we can improve things. And, you know, psychological safety ends up being something that is really core in order to drive better productivity, reduce burnout, and be able to experiment faster.
Smedley: Am I hearing you correctly that the pandemic has just widened the gap between these very large behemoths, these tech companies who have the teams to actually be able to create better engineering leaders, who can develop better processing, engineering processing teams, than the smaller companies who are going to be left behind because they’re not able to do some of that. That’s what I just heard some of the research is revealing.
Ali: Yes, and one of the critical things we found is about in the current marketplace, as to where we’ve ended up after these waves of COVID is that about 59% of teams are able to deliver their product features in less than three days. So that gives a benchmark as to where businesses should be thinking about. Are they able to get something from being an idea to getting it tested in the real world within three days without having their developers burnt out? And that I think, you know, if companies aren’t able to do that, then I think they’re really falling behind. There’s only about 4% of companies who are reporting how it takes over a month for them to deliver product functionality. But at the same time, we do see these concerns about reliability there as well, we do see that there is 83% of developers overall feel concerned about the software reliability at their workplace and there’s 20% who report being concerned to a great extent about software reliability, and that I think, also raises concerns about, you know, critical national infrastructure and things like that. So I think it’s important to have that dimension of being able to look and to see, you know, are our processes resilient enough to deliver stuff not only fast, but reliably as well, and I think those two dimensions are things which businesses really need to think about, and that’s where I think a lot of competition exists nowadays.