Screaming in the Cloud
aws-section-divider
Audio Icon
Hey, Apple, Move Over: We’re Solving Email Forever
Episode Summary
David Heinemeier Hansson might not need an introduction, but here’s one just in case: DHH is the co-founder and CTO at Basecamp, the popular SaaS project management tool, and the co-founder and CTO of Hey, a new way of thinking about email. He’s also the creator of Ruby on Rails, the popular open source web development framework. If that weren’t enough, he’s also a best-selling author and a champion racecar driver. And now, he can tell everyone he’s been a guest on Screaming in the Cloud.

Join Corey and David as they discuss why DHH decided to pick a fight with Apple (or rather, why Apple decided to pick a fight with him), how Hey originally intended to compete against Gmail but quickly realized they had to beat another boss before they could even begin, how a theoretical $40 million in legal fees deterred DHH from deciding to sue Apple, how every company is a tech company whether they know it or not, how Apple made DHH and his co-founder Jason Fried rethink whether they wanted to continue making software, what the internet was like back when Basecamp was released and how DHH hopes we can get back there at some point, what happens when big tech companies eat software, and more.
Episode Show Notes and Transcript
About David https://dhh.dk
Links:
Transcript

Announcer: Hello, and welcome to Screaming in the Cloud with your host, Cloud Economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of Cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.


Corey: This episode is sponsored by ExtraHop. ExtraHop provides threat detection and response for the Enterprise (not the starship). On-prem security doesn’t translate well to cloud or multi-cloud environments, and that’s not even counting IoT. ExtraHop automatically discovers everything inside the perimeter, including your cloud workloads and IoT devices, detects these threats up to 35 percent faster, and helps you act immediately. Ask for a free trial of detection and response for AWS today at extrahop.com/trial.

Corey: If your mean time to WTF for a security alert is more than a minute, it's time to look at Lacework. Lacework will help you get your security act together for everything from compliance service configurations to container app relationships, all without the need for PhDs in AWS to write the rules. If you're building a secure business on AWS with compliance requirements, you don't really have time to choose between antivirus or firewall companies to help you secure your stack. That's why Lacework is built from the ground up for the Cloud: low effort, high visibility and detection. To learn more, visit lacework.com.


Corey: Welcome to Screaming in the Cloud. I’m Corey Quinn. I’m joined today by a guest who probably needs no introduction, but is going to get one anyway. He’s the CTO of a company called Basecamp. Recently launched Hey, a new way of thinking about email. 

He’s the creator of Ruby on Rails, is a Le Mans class-winning race car driver, and of course enjoys one of my personal favorite hobbies, kicking trillion-dollar companies when they deserve it. David, thank you for taking the time to speak with me.


David: Thank you for having me on the show.

Corey: So, you’ve been up to a lot lately. Basecamp is one of those things that has become a staple of online work; your books, Rework and Remote, have been seminal as far as shaping the way people think about remote work. And your most recent one is doing very well as well, which is called It Doesn’t Have to Be Crazy at Work. But you’ve also made the headlines for the past year more or less picking a fight with Apple about their 30% charge on everything that gets sold through their app store, which, for anyone who’s building anything that they expect people to use, isn’t really an optional thing anymore.


David: It’s not, which is actually why I’d flip that statement and say, I didn’t pick a fight; Apple picked a fight. Particularly with us; particularly with the launch of Hey, our new email service. Because I’ve been interested in this as sort of a distant observer, annoyed participant in the software community just, like, “Hey, this doesn’t seem quite right.” But it never felt personal because it wasn’t personal. We got on with our business more or less under the unwritten rules that Apple and Google had set out and did not enter into any direct confrontations on this issue. 

All that changed last summer when we launched Hey and Apple showed up with the baseball bats and said, “Ayy, what a nice business you got. Give us 30% or we’re going to smash your kneecaps and burn down your store.” And our response was, “No. You’re not going to do any of those things, and we are going to kick and scream until you back down.” Which was then two weeks of terrible kicking and screaming from my side—from our side, and finally Apple backing down at least on that part of the question, whether we were going to hand over 30% of our revenues. 

And after that, I got, um, what’s the word? Radicalized. [laugh]. Or at least I got my attention focused on this, not just being, like, one of many, many issues I care about to one of the top five issues I care about, that I follow intensely and that I’m advocating on all the ways I possibly can to have something happen about. And as it were—today, there is a bill in the Arizona House of Representatives that actually goes up for a vote in a few hours, I think, on the question of whether Apple should be able to force app developers to use their exorbitantly priced payment processing system and whether they can retaliate against developers if they don’t. 

So, it’s very much on my mind, it’s very much I think, also in the time right now. This topic has just exploded in the last year. We’ve gone from essentially antitrust being something that weird, wonky policy people were talking about it in other rooms than the ones that I were in to, this is an absolutely mainstream story that’s being followed by outlets left and right, and that consumers are starting to care about because they’re starting to learn what’s actually going on and realizing, “Wait a minute. That doesn’t sound right.” So, that’s one of my—I was going to say big beefs. 

It is one of the big issues that I care about intensely, and one of the things that we care about intensely as a company, particularly now that we’re in the market of selling email services to consumers. The funny thing about Hey was that when we launched in the summer of last year, we legitimately thought that the main big tech company we were going to go up against was Google, that Gmail is this gorilla in email. They have more than 55% of all emails in the US, bigger than anyone else, and we thought, like, that’s the battle, right? We’re going to go up against Gmail, we’re going to give an alternative to Gmail, and then we realized, oh, before we can fight that boss, we’ve got to fight this other boss—Apple—to even be allowed in the marketplace. That’s kind of bananas.

Corey: You have to fight the person manning the toll booth in order to get to the castle. Yeah.

David: Yes.

Corey: You’re also somewhat uniquely positioned in that you’re one of the only people who can pick this fight effectively, or at least respond to the fight being picked, in that, sure, you wind up upsetting some small indie app developer, well, they’re going to rant like a loon to 500 followers, and it goes nowhere. You go on the other end of the spectrum, and a lot of companies that are VC-backed are going to be basically muzzled by their investors or by their customers of, “Whoa, whoa, whoa, whoa, whoa. We don’t want to rock the boat. We might want to sell Apple things, so let’s not do that.” And for all that you’ve done and all that you have built a reputation for, you are known as someone who it is not wise to cross, I think is probably the best way to frame that, because you have opinions and you’re not shy about sharing them. And frankly, from where I sit, it’s a wonderful thing.

David: It is indeed also a rare thing that most companies don’t end up in this weird Goldilocks part of the spectrum where, A) we’re not big enough that we have quarterly earnings we have to defend to Wall Street or other investors, and we’re also not small enough that Apple pulling this shit last summer would have, like, wiped us out overnight. So, we’ve been in business for 20 years; we’ve been profitable for 20 years, and if you’re profitable for 20 years, like, that accumulates to give you a buffer, to give you options, and to give you courage. And all of those things really do combine to just leading to this notion of, “What’s the good of having [BLEEP] you money if you never say [BLEEP] you to things that deserve a [BLEEP] you?” And Apple’s monopoly abuses absolutely deserve a [BLEEP] you, so I almost feel like it’s my moral duty and obligation, given the fact that we have the privilege to stand up and speak in a way that most other developers—on both ends of the spectrum: the very rich and the not so rich—can’t do because of who we are. So, if we’re not going to speak up, no one is. 

Which is, of course, exactly what’s been happening on the monopoly issue, in particular with Apple and their app store, is that tons of developers have been abused in these ways and they’ve just suffered in silence because no one was interested in getting Apple on their bad side because do you know what? I mean, if you get an Apple’s shitlist, you may not even be able to publish any software at all going forward. Like when Epic, which was perhaps the only company that—in the US, at least—has dared to go head-on with Apple. They tried to destroy their 3D engine.

Corey: Yeah, just pull Unity from the App Store, or threaten to do that to everyone who’s using it.

David: Yeah.


Corey: And if that’s not an antitrust issue, I really struggle to identify what it is.

David: And that’s exactly why people don’t do it. I mean, I think Epic is showing exactly why even huge companies would go, “Whoa, whoa, whoa. We can’t speak up against Apple because they have so many leverage points. They’re a conglomerate and they’re in so many different areas. And they probably have a thumbscrew somewhere they can apply to our business where it’s going to really hurt.” 

And that was the thing that was just phenomenal about Epic was that Tim Sweeney and Mark Rein just went, “You know what? We’re going to do it anyway.” This doesn’t actually make business sense. It doesn’t make business sense to go up against that, but let’s just put that out there. It didn’t even make business sense for us. 

The odds were so long, and the catastrophe if you get it wrong, would normally do—going up against Apple are so severe that I would absolutely not recommend that anyone else try to do what we’ve done. It’s one of those big stickers, “Don’t try this at home, folks.” And that comes apart, again, from this sense of do you know what? If we’re going to get wiped out, Basecamp is going to get destroyed because we stood up for our principles and what we believe it was right, I can be okay with that because I’ve ended up and we’ve ended up in this position of having had such a wonderful run. And I literally thought about that going into the fight with Apple, just thought, “Do you know what? This might be the end.” 

I mean, not perhaps in the literal sense that the company is going to shut down tomorrow, but one of the things we looked into was, we’re going to sue Apple. And then we talked to a bunch of lawyers, and they told us, “Yeah, it’s going to be, like, $25 to $40 million. And it’s going to be about ten years for you to get any sort of permanent relief because it has to go through two rounds of appeals. And so do you want to spend”—

Corey: Bring money and patience.

David: [laugh]. Exactly.

Corey: And a lot more of either than you think you’re going to need.

David: And I mean, we stood on the edge of that, talked to several law firms, almost pulled the trigger on that. And thankfully, we didn’t do it. I’m happy that someone like Epic, who actually employs an entire legal department at their company, could go forth with the fight, but that just shows how lopsided the fight is and why Apple acts the way that they do. Because they know they have this kind of power. They know they have this kind of leverage, even against governments. 

I talked about this Arizona bill; before that, there was a North Dakota bill that was very similar, which was essentially, again, let developers use their own payment processing and don’t retaliate against them if they do—

Corey: I was sad to see that get shut down.

David: It did. And what was the justification? Why did the senators not want to vote for this? One senator was quoted from the floor saying not, “I don’t believe this is a good bill,” or, “I believe Apple is amazing.” No, no, no. “We should not”—essentially, I’m paraphrasing here—“Upset big companies because they’re very big and we could get sued.” I mean, holy shit. 

When you have [laugh] entire states that are afraid of pissing off Apple because what they might be able to do with them in litigation, you know you’re really in trouble on the antitrust side. So, that just gives me even more fuel for my personal fire that carries this forward and for me to care about it and continue to kick and scream about it because it just seems like this is a really dark turn. Not just for the technology industry, right? Because technology industry is not, like, its own little thing anymore. Technology industries is society. The biggest companies in the world—

Corey: Every company is a tech company, whether they know it or not.

David: Exactly. And this is the slogan, the positive slogan for a while in tech was, “Software’s eating the world.” Right? People were cheering that on. “Woo-hoo. Yeah. Software’s eating the world,” but yeah, who’s eating software? Monopolists. So, once monopolists have eaten the software, they’ve eaten the world. Do we want the world eaten by, like, five companies? Uh, I don’t.

Corey: No. Absolutely not. From my perspective, if I wanted to have to bend the knee to giant companies and do business with people I can’t stand, and keep my mouth shut, and keep on the straight and narrow, there’s a word for that: it’s called having a job. If I’m going to deal with all of that, I may as well go get a job at one of those giant companies where I’ll be unhappy and have to keep my mouth shut all the time, but at least then I don’t have to worry about all the other things that come with running a business. I can let that stuff go. 

So, if I’m going to be in this position, where I’m not necessarily accountable in the same way that I am to one person being in a good mood for me to continue to put food on the table, I feel like it’s almost incumbent upon me to speak up and basically speak truth to power. Although someday I hope to do it nearly halfway as effectively as you have.

David: That’s exactly it. Actually, that thought crossed both Jason and my mind independently. We talked about it later, during the heyday of the Hey launch, with Apple breathing down our neck. It’s like, “Do I even then want to make software anymore?” If I have to ask Apple for permission to exist if I have to hand over 30% of the business to Apple, do I need this? Like, I’m 41. Could I retire? Could I do something else? Could I become a farmer? Could I just move somewhere? Both of us thought—seriously—about you know what, maybe it’s over. 

Maybe we lived through the glory days of software distribution where all you needed was the internet, and you didn’t have to ask anyone for permission—more or less—and you could just do your own thing. If anything, this whole experience just reminds me of just what a unique and simultaneously amazing place the internet at large is. It is such an aberration in the history of software in terms of who controls the platforms and who dictates what you can and can’t do. And we lived through the best parts of that—perhaps. I mean, I’m not willing to give up hope. 

I hope that we’re going to see a resurgence of that, that big tech is actually countered—both legislatively and in the marketplace—and we will see sort of a new dawn of this same kind of freedom that we enjoyed with the internet where we could throw up a website basecamphq.com; 2004; here’s the credit card field if you want to use our software. It’s between you and I. That’s it. No middlemen, no middle tiers, no one telling us whether we can be on the internet or not be on the internet.

Corey: No 600 companies that are included on your website, watching everything I do on there under a magnifying glass, then tracking me across half the internet. You know, all these old-fashioned, legacy ideas.

David: Exactly. And that’s—I’ve become ever more militant about bringing back. I lived through web 1.0. I started with the internet back in 95, and I’ve seen that whole cycle. And it wasn’t all great. Let’s first be fair about that. Nostalgia has a way of erasing everything that was—

Corey: Oh, yes. Those rose-colored glasses get ever rosier with time.

David: Exactly. So, it was not great, but there was a lot of it that really was great. And the ethics and the possibilities really were different, and in many cases, superior. Better. Could we bring some of that back? Why do we have to leave that to the past? I don’t believe that at all. 

So, this is one of the things by I care so much about privacy, why we’ve focused so much about privacy in Hey.com, why we focused so much about weird things like speed, or not using too much JavaScript, or just having really light pages, why we are oppose spy pixels, where we are oppose tracking of all kind is, I remember what it used to be, and there’s no reason it can’t be more like that today. Web 1.0 had so many things right that we’ve lost over the years. The sort of containment of the internet that has happened with big platforms has caused us amnesia; we’ve forgotten how it could be. And we’re here to bring that back.

Corey: When you launched Hey, you’ve mentioned previously that this was done on top of cloud. Basecamp is, somewhat famously, primarily hosted in an on-premises environment. What triggered that decision point?

David: Some historical factors. One historical factor was that about, let’s see, four or five years ago, I caught the cloud bug. I caught the argument, which is how so many of these things spread—or the meme, if you will. No one is producing their own power anymore. It’s not like, to run a business you have to construct your own power company and run that in your backyard. 

Why would you want to run your own infrastructure? It is like power, it should just be something you buy from the power company. And I thought, “Yeah. Yeah, yeah, yeah. Totally. Makes sense. Let’s not do physical hardware anymore. Let’s just go cloud. That’s great.” And then, I mean, things happen, right? The awareness of big tech. The awareness that cloud isn’t some magic thing. It’s just other people’s computers that you rent. And a lot of the promises of cloud, they don’t actually pan out, or didn’t pan out for a lot of companies, including ours. Is cloud simpler? No, not really.


Corey: I don’t think anyone has ever accused it of that.

David: It was part of the pitch in the beginning. And I understand there's a certain context in which it is, but when you run at our scale in something like, Hey or whatever, it absolutely is not. We run both things, we run the bulk of Basecamp on-prem and we run Hey on-cloud. And on-cloud is not simpler in a total perspective. There are aspects of it that are simpler, but by and large, not simple. 

It’s just other people’s computers, and you’re renting those computers. Then that comes to other things such as economics. Do you know what? If you’re renting other people’s computers by the hour, you’re going to pay a per-hour rate to rent those computers. Is that rate—

Corey: For the rest of your life because they never automatically turn off. Or as in the case of a data center, get carried it off at some point by the raccoons because you failed to maintain the data center in any reasonable way.

David: Exactly. So, that makes sense for certain things, especially speculative ventures where you’re like, I don’t want to plop down, I don’t know, hundreds of thousands of dollars buying all sorts of equipment. What if my idea doesn’t work? Then I end up with hundreds of thousands of dollars in computers that I don’t know what to do with. That is a waste of money. 

And I totally agree. The cloud does that well. It does well, the, “Let’s just see if there’s anything here.” But then the moment that there is something there, the economics totally change. And they’re totally shit, particularly for companies like us where we have a long horizon. We’ve been in business for 20 years. For me to depreciate a piece of hardware over 3, 5, 7, 10 years—I think we actually do have some hardware that’s 10 years old. Not a lot, but some—just completely changes the economics. 

Of course it’s going to be monumentally cheaper, in a lot of instances, to buy your own stuff and keep it for five years. Particularly when you come out to the edges of what cloud is good at or designed for. For us, that’s big databases. You can get some really big databases now, on AWS, I’m sure you’re aware. But they are phenomenally expensive. 

Some of the calculations we have are, like—for some of the largest databases that we’ve been looking at, the cost to rent them on a per-month basis is something like, in three months we would have bought it. Sometimes even less than that. And you just go, “Wait a minute. That really doesn’t add up.” And then that doesn’t even include the fact that you don’t actually have access to, sort of, the tall end of the spectrum. 

And for us, we’re trying to keep our operations as simple as possible, and that includes distributing your computing the least amount possible. And for databases, it means let’s not take on the burden of premature sharding, premature federation, this that and the other thing. Which means we run some pretty big, beefy databases. We’re trying really hard to pretend we just have one database and we can keep using that forever, and that has kind of been the truth for almost 20 years with Basecamp. As early back as 2007, I thought, “All right, we’re going to hit the wall here. We’re going to run up again—we’ve got a shard.” Right? 

I wrote it up at the time, I think there’s a blog post on this being from 2008, about how we starved off the sharding thing because we bought a slightly bigger machine. And then things just kept bailing us out, so to speak. Then SSDs came, then all of a sudden you could have a terabyte of RAM, then all these other things changed to the point that for Basecamp, it’s a completely non-issue. Like, technology got bigger, faster, better, quicker than our growth rate. So, that’s not a problem. On Hey, it's a little different. [laugh]. Email is surprisingly resource-intensive. And—

Corey: Oh, yes. And mentally overhead-intensive, too. Is sort of the entire reason you built the product in the first place.


David: Exactly. It’s just a massive amount of data, in a way that the data we accumulate and accrue for Basecamp just isn’t. So, we’re facing some of these issues again and we’re facing those cost constraints again, including the physical constraints of what kind of hardware is available, and now we’re also additionally facing the constraint—as we started to talk about—is big tech. Who do you rent these computers from? Are these companies—

Corey: Which of your favorite three big tech companies are you going to go with? Or are you going to pick something sketchy and dangerous on some level?

David: Exactly.


Corey: And there are things in between. For example, DigitalOcean and Linode are both not unreasonable options, and they, in many cases, are a lot more sensible. But at some point, you’re picking which devil you’re crawling into bed with. And I don’t know that there’s necessarily a way around that in many contexts.

David: And that was my conclusion for the longest time. Do you know what? Just in part, hold your nose, and that’s what it is, right? We’re going to use—and we’ve used both Google’s and Amazon’s cloud offerings. We actually used to use Google a whole lot, and then we had some traumatic, terrible, no good, very bad experiences with that, that really just burned us severely, and vowed we’ll never run anything on Google again in that capacity. 

And we moved all our stuff off, and we moved it to Amazon. But it’s not exactly like Amazon is wonderful in the scheme of big tech. So, the conclusion I’ve reached is that, why do we even have to have a devil in our bed? I understand there’s some advantages, and maybe we could take those advantages with us. Kubernetes, which is taking a hell of a lot of a beating as a super complex, convoluted beast, and not unfairly so, but at a certain scale, that complexity is not a showstopper. 

We have an ops department today of—what—nine people? Eight people. Nine people, full-time people who work on this stuff. That’s a thing we could do. That’s a thing we could do: we could run our own Kubernetes setup. And we can also run our own servers; we’ve done that for 20 years. 

Can we compare those two things or merge those two things such that we get a cloud-like experience, but on our own hardware, without being in the bed with some devil? Yeah. I actually think we could. And that’s the trajectory that we’re currently on. I would like to see less devil-based cloud computing and more learning and running our own clouds. 

And I think that other companies that are in this mid-size range, may very well come to the same conclusions. It’s really interesting. It’s almost like the who can afford to speak up thing. Again, you got to be in the Goldilocks phase. When you’re first starting out, you don’t have full-time ops people, you don’t know if this idea is even viable; cloud is a really interesting, cheap way to validate your idea. 

And then perhaps when you get to, sort of, all the way to the other end of the spectrum and you’re a billion-dollar company, cost just doesn’t matter. Like, it doesn’t matter whether it cost you $1 billion or $2 billion to run this stuff. There’s just other concerns that matter to you. Well, we’re in the middle phase where cost absolutely matters. We’re spending my money; Jason’s money. 

And I care about how we spend it, and could we spend it on other things? And then we also have the capacities in-house to do some of these things. So, it’s interesting. We are in that middle phase where we’re trying to figure out what is the path forward? For example, do we want to host our own files again? I was just talking with Troy, our head of ops, today about this question. Do you know what? S3 really makes it easy to—

Corey: It’s dirt cheap, at least compared to a lot of the other services you can pick.

David: Yes.

Corey: The durability guarantees are phenomenal. It’s getting increasingly intelligent, so you can start staging things from an economic point of view. Heck, look at Glacier Deep Archive: it winds up costing roughly $1,000 a month to store a petabyte of data, which gets us into the realm of why delete things ever again because you might theoretically need it, and in the context of business, $1,000 a month is peanuts. And that leads to some weird behaviors.

David: It does. And first, I’d say S3 is the one service from the whole cloud thing where, like, yeah, actually we can’t do it cheaper. Even if I’m willing to go on a ten-year depreciation thing, the setups that we’ve at least had so far and the benchmarking we’ve done, we can’t beat the pricing. Which is completely different than Hey, the beefiest database server, what does that cost? It’s not even remotely competitive on a pricing basis if you’re willing to buy hardware and depreciate it over the years of service, but storage absolutely is. 

But that’s run into this other thing where the storage is no longer physical. It’s not even an inconvenience. It’s a line item on an invoice. That’s it. But you know what? That’s, again, a mirage. This storage isn’t in the cloud; it’s just on other people’s computers. And those computers need power, they need operations, they need cooling, they need all the things that go into having computers that are turned on all the time. And that creates waste. And do you know what—particularly with email where we’ve just seen our data usages growing phenomenally fast. I mean, orders of magnitude faster than anything we’ve ever done in the past because email just takes up a lot and people get a lot of email.


Corey: Yeah. Please go ahead and send me two megabytes of images embedded in the email for a marketing email that I, in the best case, will glance at once. And then where’s it going to go? Oh, it’s going to sit in my inbox for the rest of my life. In fact, I’m sure that my great-great-grandchildren will pass down one of their ancestor’s old email inboxes and go through it. Who are we kidding?


David: Exactly. This was exactly the experience I first got when I exported all my email from Gmail. So, I’ve been using Gmail for like, 12, 14 years? Whatever. I’ve used Gmail since it started until, what is that, like, a year and a half ago when we switched over to Hey. 


And I exported all that and I got that whole archive, and I think it was like 40 gigabytes. And then I loaded it up in my local mail app—it’s just an inbox file. They’re quite easy actually to export and then use locally—and I started looking through what was in it. And I was like, “What am I, [BLEEP] packrat horder? Why am I saving old newsletters from 2009?” If any of this stuff was physical, I’d see the insanity of just saving this stuff forever.


Corey: Oh, my God. I’m the digital equivalent of a hoarder? What’s going on?


David: Exactly. And that was exactly what I was thinking. I was just like, “Oh, man, this doesn’t make any se—like, what is going on here?” Those 40 gigabytes combined with the other 2 billion users who each have their own, if not 40 gigabyte, but then many gigabytes, like, what is it cost the world in terms of energy, in terms of waste and pollution, to keep all that storage live, searchable, whatever? That is not insignificant. 


I saw one estimate saying that it costs the equivalent of 1.9 billion trees a year in terms of just spam. And this is not spam. The 40 gigabytes I had saved, there’s no spam in that. That was just email that was sent to me, and at the time I accepted in my inbox, and then I just didn’t delete it, and I kept it. This didn’t make sense. 


So, this brings us back to the whole Hey thing and us essentially having solved the storage problem to a place where it’s invisible, and I’m not at all sure that’s good. In some regards, I kind of wished that storage was painful again. It’s just that then I would at least think about, like, does this even make sense? The same report that quoted the 1.5—or 9 billion trees, said something about, like, “On a bunch of studies-blah, blah, blah—90% of all data that we keep is never accessed again, after”—I think 30 days or something like that. That just seems like a tremendous, horrendous, terrible, no good use of space.


Corey: Some people need dozens of tools to visualize their stack. But not you. You’ve got New Relic Navigator.All your hosts, services, containers, and anything else you can monitor are in one dense hex view, so you can check system-wide health at a glance.And you can sort by platform, service, app, or any other tag, which makes it easy to navigate yourentire system and see what needs your attention.Go to https://newrelic.com and get started for free


Corey: And there are occasions where you need that sort of thing. For example, one of the things I did about 10 years ago is I was the first ops person that Expensify and the pattern that we identified pretty quickly is that people would upload receipts and very often never look at the receipt again. But God help them if someday they needed one of those receipts and didn’t have it. That is not this. This is, “Oh, wow. If I never get that important marketing message”—that, again, is festooned with trackers and the rest—“I will be a happier person.” 


But the world has gone a different direction. I quick check online, because I haven’t looked into this space in a while, I didn’t realize this. They sell 16 terabyte hard drives now for 300 bucks. And it’s, “Oh, well, yeah, my email archive is growing, but that’s okay because drives are getting bigger faster than I’m collecting digital crap.” But it’s an arms race. And it becomes this enormous untrackable mess at some point where I can’t even begin to contextualize how much data I might have hanging around on my drive. And it’s all for no purpose.


David: Totally. And then if you look at it at an individual scale, like, “Oh, how much data do I use? Is that actually a big deal?” Maybe not. But we see it at a much larger scale. We have tens of thousands of paying customers for Hey, and already at that level, it’s real. And it’s not just another hard drive. It’s multiple computers. We just added three new nodes to our Elasticsearch setup. 


Because that’s the other thing; an email that comes in, it’s not stored once. It’s stored, like, 500 different times. It’s got to be stored once for backup, once for distribution, once for resilience, once for search, ba-da-da-da-da-da-da-da, right? This data just multiplies in a way that when you then add it all up, it’s actually real machines. And for what?
 
If it’s serving a purpose, if it’s, like, oh, it’s giving you an archive for these invoices, and you get audited in, like, five years, and you really need to prove it, where, fine. Totally. Good. Very nice. In fact, just having all that stuff around is really helpful for that. But the myth that Gmail, in particular, is responsible for sort of putting out there’s it’s like, “No, no, no. There’s no cost. Store whatever, for however long, and everything is fine.” And then you go back and think, “Wow, man. Wow, that’s such a generous offer. Why would they do that?” 


And you think, “What is Gmail in the business of doing?” Of training ever more sophisticated AI models to figuring out how they can organize and strip you of all your personal data, build ad profiles on users so that they can more efficiently sell you a pair of [BLEEP] sneakers. Okay. Well, I mean, maybe in that perspective it’s not that great that I’m giving Gmail in particular, like, 12 years of history of who I am, what I buy—


Corey: I really want to show Google of all companies, exactly how I evolve as a person—


David: [laugh]. Exactly.


Corey: —over time. Sure that won’t come back and bite me in any meaningful way.

David: Right. And I mean, that’s the thing. When you think about it an abstract terms, you go, “Yeah, weeell, I mean, ehh… what are they going to do, right?” And then you learn that Gmail literally does what Expensify did—except just, like, under the cover of darkness, almost—that they go in, they look through all your purchasing receipts, they mine that for data, and then they use it to build a profile of you, of what you bought, when, and so on and so forth. And you go, like, “Okay, that’s pretty [BLEEP] creepy.”


Corey: Oh, they had a page, ‘slash purchases’ I think it was, that they unveiled—like, they were super proud of this—that showed that of every receipt that had ever interpreted through your Gmail, and they talk about this like, “Look at this. We can show you all the stuff that you bought.”


David: Right.


Corey: And the response was, “You’re [BLEEP]-ing what?” And that became a big kerfuffle. And now they don’t talk about it very much, but it’s still there because of course, it’s still there. Why would they deprecate something horrible when they can deprecate something useful and beloved? But that’s the Google ethos in some respects, as the big Google company that it is.


David: Yes, once, quote-unquote, ‘normal people’ who are not, sort of, tech folks or marketing people follow all this stuff in detail, they learn about some of these things, they go, like, “They’re doing what?” This was what we just encountered with spy pixels. So, Hey tracks, shames, and names when someone sends you an email that has spy pixels in it, those little one-by-one pixels that you can’t see that sends data back about whether you opened the mail, how often you opened the email, where you were when you opened the email, what time, what device you used, all the personal information that’s sent back that techies in large regard—like, they’ve known this, right? They’ve know—this thing that happens. You explain this to normal people, and they go, “Wait, what? I opened the email and they can see that?”


Corey: “You know what about me?” Yeah. Oh, yeah.


David: Exactly. They’re like, “This is n—no. No, no, no, no, no, no, no, no, no. This has got to stop.”


Corey: And as someone who sends a newsletter, it turns out it is incredibly challenging to find an ESP—or email service provider—that even thought about letting you disable this. It’s creepy. And it’s just become so accepted that when you ask, “Oh, can I disable that?” The person you’re talking to at these companies looks at you like you’re simple. It’s, “Why on earth would you want to do that?” “Oh, I don’t know, I have this ancient belief, like you know, respecting my readers.” What a concept. It’s awful.


David: It really is. And I think this is where some of the practices of the industry, it just seeps into the assumptions, it seeps into the ideology that, like, the more data you can extract, the better. Full stop. The more data you can store on someone, the better. Full stop. 


Not thinking about, do you—does that person want their data stored in this way? It’s not even part of the conversation. And I think that’s one of the things I’m happy for us to do Hey, not just because I wanted to, quote-unquote, ‘solve email forever.’ Since I’ve been an email user forever, and I’m sort of selfish in that way, and I build software for myself, [laugh] and the majority of features I’ve worked on in Hey are things I wanted email to do forever. But also because it allows us to have this conversation and start these conversations. 


And spy pixels is a great example because I’ve been talking about this for a while now, and people just go, “Well, I mean, it kind of doesn’t matter. We’ve been doing this forever.” Yeah, but if the people you’re doing it to are just learning about it now, and they’re realizing they don’t want this done, perhaps it’s time for you to, like, stop doing that. And I hope that we can grow Hey to an extent that it becomes sort of meaningful enough that it can help shift the conversation. Which is, of course, already happening.


Corey: There has to be a shift.

David: Yes.

Corey: If I’m the only newsletter out there that tells sponsors, “Oh, we have no idea about any metrics of this,” they’re going to go elsewhere. There has to be something of a groundswell approach. The philosophy I’ve always held that seems to serve me reasonably well is, anything that I do, I should be able to take any random representative user and take them through everything that I’m doing, and they should come away and look at it and say, “Yeah, that makes perfect sense.” Take this podcast, for example. I have numbers of people who download this—because I have server logs for how many people have downloaded the file—and I can roughly geo-correlate that by continent. And that’s good enough. 


Every once in a blue moon, I will have a sponsor or prospective sponsor ask if, “Oh, can we embed a tracking pixel in the RSS feed?” And our answer is a very polite form of, “No, you the [BLEEP] may not.” If you need that in order to sell your product and figure out how to move your wares and grow your company, maybe you aren’t that good of a company, to be very direct.

David: Yes. This is one of the arguments I keep hearing for targeted advertising in general. There’s like, “Oh, there’s all these businesses that couldn’t exist unless they could totally pinpoint target”—do you know what, maybe they just shouldn’t then. Right? First of all, this is completely ahistorical. 


Targeted advertising is, in the grand, long scheme of the history of commerce, a blink of an eye. It’s existed for what, 15 years at the very, very most. It’s been super prevalent for maybe 10-plus years, a little more than that. Businesses sold their stuff before there was targeted advertisement. In fact, many businesses still sell stuff.

Corey: A billboard that you look at doesn’t look at you. I hope. Although I’m sure that there’s some VC-backed startup trying to change that.


David: Yeah. I mean, all sorts of Bluetooth tracking and all sorts of things going on. But yes, exactly. There are all these other forms of advertisement. And the reason that people feel like they kind of have to do this is in part because so much of the conversation and attention has been captured by these platforms—whether that’s Facebook, or Instagram, or Twitter—or anything else, that they have such a huge slice of the attention now, that take all that attention, be able to slice it down into, “Oh, this person is between 31 and 32 years old and just got engaged to someone who’s pregnant.” We should show them ads for prams, right? 


You go, “Okay. Like, that’s a relevant ad, but actually, I didn’t even tell my parents that we’re pregnant yet. Why do you know this already?” And then you realize, oh, actually, Google knows a whole lot of things based on your search history, and they use that data, and they’re paired with the receipts that you have in your Gmail account. And all of a sudden, yeah, they know that the couple is pregnant before they’ve told anyone else. 

And you go, like, “Actually, no. That’s not a world that I’m all that interested in living in. I’d like to have some privacy. I’d like to sort of control who knows that.” And then the argument often goes, “No, no, no. It’s just targeted advertisement. I mean, we’re not selling the information. What the hell are you talking about?”

Corey: Well, how else are we supposed to get in front of people—

David: Right. [laugh].


Corey: —who are interested in cloud computing? Oh, I don’t know maybe find publications that cater directly to that audience and put two and two together. Doesn’t require a whole lot of tracking to realize that if that’s the subject of a media publication, the people who are in the audience for that probably care about the thing.

David: Bingo.
Corey: It’s such a strange thing.


David: Well, it’s such a profitable thing, I think. That’s the problem.

Corey: Ugh.

David: The problem is that, like, exactly the scenario you described, where sort of niche—or not even that niche topic areas whether it’s a newspaper, or it’s a magazine, or it’s a blog, or it’s a podcast, or whatever, they would have a real advantage. They’d be like, “Okay, I mean. Talk about cloud on this podcast.” That’d be a good place to advertise your cloud-monitoring wares, or bug trackers, or whatever, whatever, right? You just, you don’t need that anymore now that you have targeted advertising. 


You can just as well show that ad against the most vile, disgusting content on the internet because the content no longer matters, only the attention does because we can pinpoint who’s looking at that. And then all of a sudden, all quality content loses most of its appeal and becomes way harder to make a business producing that kind of content. Because all content is now equal. All eyeball time spent is equal when you can slice and dice with targeted advertisement, exactly who should see what. And that’s just led to a collapse of local newspapers, all sorts of media institutions are completely scrambling. While we have a handful of organizations—primarily for this case, Google and Facebook—just surging into trillion-dollar market caps, posting ever higher results capturing ever more of online advertising. And you go, like—


Corey: And you need double-digit growth—


David: Exactly.


Corey: —year-over-year at all times. And at some point, you’re looking at this and they’re smacking into population limits, and the only way to continue that growth trajectory is to leverage what you’ve already built and expand into horrifying new markets that no one wants someone with your level of access in. Or else you’re looking at doing horrifying things that fundamentally betray user and customer trust.


David: Yes. This is the trap. Sounds so benevolent, but I’ll use it because I think more stringent terms for it might turn off your listeners. But this is the trap of capitalism. The trap of having to post growth year over year over a year. 

And once you get to a certain size, particularly when you’re a trillion-dollar company, if you just have to post 5, 10 percent growth every year, you have to find $100 billion worth of new business or value in new business every year. And then the year after that, you have to find out another a hundred—and now ten—billion dollars. This becomes very hard to keep up on the ethical side of the road. Sooner or later you’re going to start realizing, “Shit we’ve run out of all the ethical, good for society ways of making money. Now, we got to just start what’s beneath that.” 

And that goes for, I think, both Apple as we’ve seen with their pivot to services once they ran out of people on earth to sell a new $1,000 iPhone to every year, they started realizing, “Oh, shit. I mean, how are we going to get this up? Oh, services. If we just take all the people who bought an iPhone and we make a bunch more money on them.” And then, of course, to spin it all the way back to the start, “Hey, if we could just take 30% of everyone else’s business, oh, that’d be great. We don’t even have to do anything. We don’t even have to come up with any new”—

Corey: Amazon has expanded to advertising to the point now where I go to amazon.com and search for anything, the results are dogshit. They are unusably bad, where it’s not at all clear which ones are sponsored, which ones aren’t. Oh, ‘Amazon’s Choice?’ Turns out that algorithm-powered. Reviews are getting gamed constantly. I no longer have trust that whatever it is I’m going to buy is actually going to be any good. 

I’ve had a few bad counterfeit experiences by getting things, and sure they’re quick to do refunds and all, but you feel taken advantage of. You feel like they have abused your trust. And that’s a very hard thing to get back. I am increasingly buying a lot of things not on Amazon. I was an Amazon household for years because it was so convenient and so easy. And now I’m looking at this going, “I don’t want to give these bastards my money.”

David: Right there with you. So, we used to be, like, “Oh, man. Prime is the best thing and you get things so quickly.” In the past, I think it’s been almost two years now, we essentially just stopped using Amazon. And part of that is powered by where I don’t like what Amazon is doing. I think the product, the service—

Corey: Oh, it’ll be there in two hours or—basically if it’s not there with an hour or two, they’re going to be extremely apologetic. Yeah. Okay, cool. I love convenience. Who doesn’t? But at the cost of what?

David: Right.

Corey: Horse-whipping someone who is a contractor, not an employee?

David: Exactly. Yes. Yes.

Corey: You can’t treat employees that way.

David: Right.

Corey: It’s awful. This is not good for society.

David: It really isn’t. It’s not only not good for society, but even the product experience of using the service is getting worse. As you say, you can get counterfeit wares, you can’t trust reviews anymore, you can’t even trust the listings because they’ve been bought. The same thing with Google. Google is realizing that to keep juicing their returns on search, they have to make the search worse, they have to cram the search full of confusing elements of what’s an ad and what’s a not ad, that they take over entire areas of search, “Oh, travel, we’re not going to bother with this linking to the internet business anymore. We’re just going to show our own shit. And that’s how we capture an entire market.” 

All the products are just getting worse because there are these imperatives to keep posting growth. And you think, like, “You know what? We’re literally making things worse. Isn’t the future supposed to be better?” And I’m currently looking at the extrapolation of these companies keeping up the growth rates they currently have, and then think about where are they in 10 years? Things are pretty [BLEEP] bad now. Where are they in 10 years?

Tha—tha—no. The compound growth requirements needed to keep up, they’re going to take them down a very dark path unless we start putting up some legislative guard rails and saying, like, do you know what? You just can’t do that. You can’t just buy all your competitors and then kill them because then they might be a threat. You can’t just bully and abuse all your smaller vendors, or developers, or whatever business partners you have on your platform. 

That’s just no good. You can’t just both own all the platforms and operate in all the platforms, self-prefacing all your services at the same time. And this is how so many of these conversations lead back to, we have a monopoly problem. We have a big tech company problem, and we should do something about that. And if you had asked me about a year ago, two years ago, “Is that going to happen?” I’d say, “Phuh, I haven’t seen any containment at all on big tech companies since Microsoft was sued by the DOJ in, what, 2000.” It’s literally been 20 years since there’s been any antitrust activity. 

And then boom. Right? Boom. In the past year, there’s just been a huge amount of investigations, and inquiries, and hearings on multiple continents, and multiple states, and from multiple parties. And you just go, “Oh. Actually, okay, yeah. Maybe things could change.” I’m a little less gloomy than I was, like, two years ago. 

And I think perhaps it is possible for democracy and societies to wrestle back control. Because we’ve faced these issues before. Big tech is not exactly the first set of monopolies to come along in the history of, let’s just say, the U.S. Battled with telecoms, battled with railroads, battled with Standard Oil, battled with the banking trusts. There’s been a bunch of big fights on this. So, it’s in the DNA of the U.S. In particular, to eventually go, all right, we’re going to do something about it. It’s way too late; we should [laugh] have handled this 10 years ago, it would have been way easier if we’d done it then. But okay, now, enough is enough. You’re not going to push around anymore. Here we come.


Corey: Ugh. As much as I want to keep going. I feel like, at some point, I’m already going to get letters from friends and listeners who are at Amazon, and frankly, please send those letters. I love debating these things with folks. But I feel like at this point, our time is really drawing to a close. Any parting thoughts, concepts, things you wish the world saw differently than it does?


David: On particularly this notion of capitalism’s issues with compounding and never-ending growth, I just read a wonderful book by Jason Hickel called Less is More that I would seriously recommend to anyone where this is sort of uncomfortable territory, or maybe you work at a big tech company like this, and you’re like, “Well, yeah, maybe there’s some sort of abstract ideas, or… I don’t know.” Read the book. It’s great, and it discusses both what we talked about with the ecological challenges we’re facing with this never-ending growth, with never-ending, all the storage that we talked about. And then the issues with capitalism, and particularly as it relates to big tech companies. He doesn’t really go into monopolies, but many of the lessons and critiques apply directly. 

And I think it would really help someone open up their eyes or blinders of ideology just as I’ve opened up my eyes and blinders of ideology. I didn’t exactly come from, I don’t know, a Red Camp somewhere. I’m a graduate of the Copenhagen Business School. I got indoctrinated very well in [laugh] traditional tools, tactics, and techniques for how to juice growth, and got just a fundamental worldview that was very in line with The Economist and never—unending growth and how the GDP, as long as it keeps going up, the world keeps getting better. And it’s taken me a long time to deprogram from all that, but I still have all it with me. 

So, I think if I can go through that journey, and I can open my eyes a bit and realize in which ways things aren’t always wonderful, other people certainly can, too. So, that book is just a great starting point. I’ve just been reading it now. Huge fan of Jason Hickel’s work in general. So, there goes my recommendation. You could even buy it on Amazon.


Corey: We will, of course, include a link to this in the show notes. It is not an affiliate link because, frankly, I don’t feel the need to start doing those games. But it’s—yeah, there’s a lot to the idea of thinking critically about the things we use and the places we work, and if that makes us uncomfortable, well, that’s where growth comes from. You’re not exactly a barefoot hippie in the park saying these things. You understand what it takes to run a business because you’ve done it multiple times now. 


You have a perspective that I think the world desperately needs to hear. And of course, we will include your full-on bio and links to your books and whatnot in our show notes as well. Thank you so much for taking the time to speak with me today. It is a rare privilege to get to pick your brain directly like this.


David: It was my pleasure. All my favorite topics, stirred up good. It’s a—that’s a good time. Thanks for having me on.

Corey: Thank you, David Heinemeier Hansson—or DHH—CTO and founder of Basecamp and Hey, creator of Ruby on Rails, et cetera, et cetera, et cetera. Read the show notes.


I’m Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you’ve enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you’ve hated this podcast, please leave a five-star review on your podcast platform of choice and an insulting comment that is packed jam full of every spyware tracker you can shove into the thing.


Announcer: This has been this week’s episode of Screaming in the Cloud. You can also find more Corey at screaminginthecloud.com, or wherever fine snark is sold.


This has been a HumblePod production. Stay humble.
View Full TranscriptHide Full Transcript