Mehran Sahami, a Stanford professor and former Google employee, wants to see a reset from the technology industry. For the past few decades, the world’s technologists (many of whom become its corporate executives and venture capitalists) have been taught to prioritize optimization and efficiency without thinking a whole lot about ethics. The result has been stunning corporate success but significant costs to society. Sahami argues that regulation can certainly help right the balance. But he also believes that tech company leaders and employees can shift their mindsets and practices to ensure they’re serving the greater good, not just themselves. He’s the coauthor, along with Rob Reich and Jeremy Weinstein, of System Error: Where Big Tech Went Wrong and How We Can Reboot.
ALISON BEARD: Welcome to the HBR IdeaCast from Harvard Business Review. I’m Alison Beard.
In the past decade, the big five tech companies, Facebook, Amazon, Apple, Microsoft, and Google/Alphabet, have extended their reach and revenues in amazing ways. They’ve brought us lots of useful products and services, and they dominate various segments of their industry. They’re also great businesses. In 2020, they collectively earned income of nearly $200 billion.
But these tech giants and their leaders are also facing a lot of criticism for the negative impact they have on society, for the misinformation and vitriol spread online, for invading our privacy, for quashing competition, and for avoiding taxes in a way that allows them to pile up cash while a lot of the people whose personal data they profit from are struggling.
Is it possible to keep the good that big tech and all the smaller companies in the industry have created while getting rid of the bad?
Our guest today has some ideas. Mehran Sahami is a professor at Stanford and a former Google employee. Along with Stanford colleagues, Rob Reich and Jeremy Weinstein, he’s the author of System Error: Where Big Tech Went Wrong and How We Can Reboot. Mehran, thanks so much for speaking with me.
MEHRAN SAHAMI: Thanks for having me. It’s a pleasure to be here.
ALISON BEARD: So the first question is pretty obvious, where exactly did big tech go wrong? Facebook used to be a connector of people, now it’s a killer of democracy. Google was a search engine, now it’s a privacy invader. Amazon, a shopping platform that’s busting unions and small businesses. So how did we get here?
MEHRAN SAHAMI: That’s a great question, and part of the way we have to think about how we got here is the mindset of the people who’ve created these products. Now, if you think of the technology mindset, it’s oftentimes around quantitative metrics and around optimization. We like to refer to it as the optimization mindset. The idea is setting particular metrics that you want in your business that you want to try to optimize at scale. And so if you think about something like Facebook, what they want to do is they want to create connection, but how do they actually measure that? What they have is a proxy for connection, something like how often people engage on the site, how much time they spend there, how many pieces of content they click on. But clicking on something isn’t really connection. It’s a proxy for it. And if you take that at scale and you try to optimize it, what happens is you actually get externalities that are not what you wanted.
So you promote pieces of content that people are more likely to click on. Those might be pieces of content that are actually more titillating or more click-baity than, say, truthful content. And so, as a result, there might be a greater amplification of misinformation than truthful information because what it’s doing is maximizing that metric. And you can see this across a number of sites. So, for example, for YouTube, they might want to maximize the amount of time we spend watching videos because they equate the fact that we’re spending our time watching those videos with the fact that we’re happy. But, in fact, you can see the flaw there. Just because we’re watching videos doesn’t mean we’re happy and it ignores other values we might care about. And when we maximize one value like screen time because we’re equating it for happiness, we’re giving up other values that we might actually care about and that are important to society.
ALISON BEARD: And so the issue is that these companies are made up of engineers who are taught to optimize and be efficient. I would argue that people in the financial industry are taught the same thing. They then become the executives leading these companies, the VCs funding these companies, and so there’s no one waving the flag for other kinds of values?
MEHRAN SAHAMI: Well, they are well-meaning people by and large. I don’t think they have negative intent, at least the vast majority of them. But the problem is that most things in life involve a value trade off, and when you’re optimizing and you’re picking one of those values, the other ones are getting left behind. And that’s part of the issue is, how do you actually take a broader look at some of these criteria that they’re optimizing but also think about the fact that the criteria by themselves are just a poor proxy for what we actually care about?
ALISON BEARD: I would argue that lots of industries and companies have this problem. They’re making value trade-offs. They’re doing both good and bad things for society. So why is big tech different? Why are we so focused on big tech?
MEHRAN SAHAMI: Because at this point in time we’re seeing the externalities from big tech on display in full force. So we’re seeing the notion of connection turning into rampant misinformation online. We’re also seeing the platforms take the market power that they have and turn it into political power so that they can continue to maintain the same free regulatory structure that they’ve been under for the past 30 years. And so what we’ve lost in that process are guardrails that bring back the values we might care about as a society as opposed to the values that might be important to the company.
ALISON BEARD: Yeah. So Azeem Azhar, who hosts an HBR podcast and also has a new book out on some of these issues, he argues that our institutions just haven’t been able to keep up with the exponential growth of these companies. So in a way, governments and we the consumers have let all of it happen. Is that fair?
MEHRAN SAHAMI: I think it’s fair from the standpoint of first thinking that the government has basically given big tech a pass. The regulatory structure in the 1990s through things like the Communications Decency Act was set up to give companies pretty broad reign in terms of the way they did business in the United States. You can see that pretty clearly if you were to contrast, for example, the United States and the European Union with the kind of protections they have around data. Right now one of the choices that we give to people in the United States is in the free market we say, “If you don’t like these applications, well, you can just disengage.” There’s the Delete Facebook Movement. You don’t have to use these apps. So what’s wrong with that? You just have this choice. And the analogy I liken that to is consider driving on the road.
The CDC estimates that about one million people are killed annually on the road. So should our choice just be whether you drive or you don’t? If that’s the only choice we have, then we’ve lost a lot of values because you can see that there’s real value in being able to drive despite the fact that it’s dangerous. In the same sense, there’s real value to using these tech platforms despite the fact that they may take our data, they may try to get us to engage more. So what did we do in the case of roads? We didn’t just tell people, “Drive carefully. Good luck.” We created a whole set of safety regulations. There’s stoplights, there’s lanes on roads, there’s speed limits, and so there’s a whole system that makes driving more safe while at the same time we still count on individuals to drive safely.
That’s the kind of regulation that we would call for, for big tech, certain guardrails that prevent certain kinds of practices like being able to have free reign over someone’s data. If we get the values we care about, we can get a safer information super highway while at the same time promoting innovation.
ALISON BEARD: Yeah. But you also in the book advocate for self reform. What are some of the ways that the industry can course correct itself?
MEHRAN SAHAMI: So we look at four main areas. One of them is algorithmic decision-making where we think about the fact that more and more significant decisions in our life, things like whether or not we get credit, whether or not we get a mortgage, who we date, those kinds of things are now determined more by algorithms. And so, one of the things we talk about is the fact that these algorithms, one, may be biased because they are just a reflection of the data that gets put into them and the data that gets put into them is a result generally of previous historical human decision-making that oftentimes reflects bias. But we also talk about, what are the processes by which this can be improved? So you could audit algorithms to see what kinds of bias there might be in outcomes. You could create algorithms that provide for an explanation for why they came up with the results or decisions that they did.
You can also look at things like the distributional effects more broadly to see, is their disparate impact from the decision-making in these algorithms? And you can understand the data that goes into machine learning to think about things like, what is the historical context or embedded social factors that actually influence this data? Some of the other areas we dive into, for example, are data privacy, and we talk about, who has ownership of data? Where’s the transparency where someone can understand what data is being collected about them? How can data be made more portable across sites? We also talk about artificial intelligence, what that will mean in the longer term in terms of potential job displacement, what that means for economic policies to replace support, for example, for the social safety net that we get through employment taxes right now.
Well, if you don’t have employment taxes because someone’s job is replaced by a robot, how do you make up for that shortfall? How do you think about redistributing educational opportunities so that the workforce that is displaced, and those numbers look pretty significant, actually is able to re-engage in a meaningful way in the labor force so you don’t get large-scale unemployment? And, finally, we talk about the power of platforms in terms of what that means at a larger level, the barriers to competitors, thinking about merger and acquisition activity, and the fact that there’s likely going to be greater scrutiny of that as we see now new regulatory structures coming into place as the government takes a more critical look at the platform power and monopolistic power of these large tech players.
ALISON BEARD: And in the book, you also talk bigger picture about getting these engineers to be more in touch with human and societal values. There’s a line where you say you’d like to see a shift toward asking which problems are worth solving and whether some important ones can’t be reduced to a computational solution. So tell me a little bit more about that idea of values led technology.
MEHRAN SAHAMI: Right. So one of the ways you can think about it is, at a very basic level, what are the metrics that someone is trying to optimize in their business? And you can see simple examples of this in everyday life where the choices that actually get made with respect to a particular technology and the result you get from that has a pretty big impact.
ALISON BEARD: Right. Do we need more food delivery apps versus climate change solutions?
MEHRAN SAHAMI: Exactly. And since you mentioned climate change, a simple example of that is when you book your flight somewhere, sometimes you get these very strange flights, you’re flying from San Diego to Seattle and it routes you through Chicago. Why is it doing that? And it’s doing that because the metric it’s probably trying to optimize is the price that you’re paying. It’s not trying to do something like optimize for climate impact because measuring that is actually hard, whereas measuring for price is easy. And so you can see these cases where we just choose what’s easy versus what’s meaningful pushes us in directions that may get us further away from the values we actually care about.
ALISON BEARD: In that case though, you’re pushing for what the companies believe consumers care about and have proven to care about.
MEHRAN SAHAMI: That’s true, but at the same time, there are certain things that consumers care about that they don’t get a choice now. And that’s part of what we’re pushing for is, if we think about what are ways in which consumers don’t have choices? So, for example, takes something like privacy or data portability. If I want to be able to move from one platform to another platform because I like the policies of another platform better, I don’t have that choice right now. And that’s one of the things we can think that regulation can get us to.
ALISON BEARD: You are an educator and you’re working with the tech leaders of the future. When you’re in the classroom with students at Stanford, what do you see that worries you and what do you see that gives you hope for the future of the industry?
MEHRAN SAHAMI: Some of the things that worry me are sometimes the myopic view of what is success. And that’s a broader question, but sometimes success is just equated with things like making a lot of money. And there isn’t a lot of thought given to what are the externalities that are generated by a business? What does it mean for distributional impact among different people so that when you solve a problem for a particular sliver of the population, there’s another part of the population that’s getting ignored? And if you continue to put your emphasis on, say, the affluent portion of the population, because they’re going to pay for particular services or products that you deliver, it means that the non-affluent portion of the population gets ignored by the march of technology. And so that just further exacerbates inequality. But the place I really have hope is I think that students are paying more attention to these issues. They’re more aware of distributional impacts in society, they’re more aware of the inequality that exists, and they have more of an inclination to want to do something about