Allowing Aspiration to Lead with Tom Totenberg

• LaunchDarkly: https://launchdarkly.com
• Heidi Waterhouse Twitter: https://twitter.com/wiredferret

Transcript

Announcer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.

Corey: Couchbase Capella Database-as-a-Service is flexible, full-featured and fully managed with built in access via key-value, SQL, and full-text search. Flexible JSON documents aligned to your applications and workloads. Build faster with blazing fast in-memory performance and automated replication and scaling while reducing cost. Capella has the best price performance of any fully managed document database. Visit couchbase.com/screaminginthecloud to try Capella today for free and be up and running in three minutes with no credit card required. Couchbase Capella: make your data sing.

Corey: This episode is sponsored by our friends at Revelo. Revelo is the Spanish word of the day, and its spelled R-E-V-E-L-O. It means “I reveal.” Now, have you tried to hire an engineer lately? I assure you it is significantly harder than it sounds. One of the things that Revelo has recognized is something I’ve been talking about for a while, specifically that while talent is evenly distributed, opportunity is absolutely not. They’re exposing a new talent pool to, basically, those of us without a presence in Latin America via their platform. It’s the largest tech talent marketplace in Latin America with over a million engineers in their network, which includes—but isn’t limited to—talent in Mexico, Costa Rica, Brazil, and Argentina. Now, not only do they wind up spreading all of their talent on English ability, as well as you know, their engineering skills, but they go significantly beyond that. Some of the folks on their platform are hands down the most talented engineers that I’ve ever spoken to. Let’s also not forget that Latin America has high time zone overlap with what we have here in the United States, so you can hire full-time remote engineers who share most of the workday as your team. It’s an end-to-end talent service, so you can find and hire engineers in Central and South America without having to worry about, frankly, the colossal pain of cross-border payroll and benefits and compliance because Revelo handles all of it. If you’re hiring engineers, check out revelo.io/screaming to get 20% off your first three months. That’s R-E-V-E-L-O dot I-O slash screaming.

Corey: Welcome to Screaming in the Cloud. I’m Corey Quinn. Today’s promoted episode is brought to us by our friends at LaunchDarkly. And it’s always interesting when there’s a promoted guest episode because they generally tend to send someone who has a story to tell in different ways.

Sometimes they send me customers of theirs. Other times they send me executives. And for this episode, they have sent me Tom Totenberg, who’s a senior solutions engineer at LaunchDarkly. Tom, thank you for drawing the short straw. It’s appreciated.

Tom: [laugh]. Anytime. Thank you so much for having me, Corey.

Corey: So, you’re a senior solutions engineer, which in many different companies is interpreted differently, but one of the recurring themes tends to pop up is often that is a different way of saying sales engineer because if you say sales, everyone hisses and recoils when you enter the conversation. Is that your experience or do you see your role radically differently?

Tom: Well, I used to be one of those people who did recoil when I heard the word sales. I was raised in a family where you didn’t talk about finances, you know? That’s considered to be faux pas, and when you hear the word sales, you immediately think of a car lot. But what I came to realize is that, especially when we talk about cloud software or any sort of community where you start to run into the same people at conferences over and over and over again, turns out the good salespeople are the ones who actually try to form relationships and try to solve problems. And I realized that oh, I like to work with those people. It’s pretty exciting. It’s nice to be aspirational about what people can do and bring in the technical chops to see if you can actually make it happen. So, that’s where I fit in.

Corey: The way that I’ve always approached it has been rather different. Because before I got into tech, I worked in sales a bunch of times and coming up from the—I guess, clawing your way up doing telesales was a polite way of describing—back in the days before there were strong regulations against it, calling people at dinner to sell them credit cards. And what’s worse is I was surprisingly effective at it for a kid who, like, you grew up in a family where we didn’t talk about money. And it’s easy to judge an industry by its worst examples. Another one of these would be recruiting, for example.

When everyone talks about how terrible third-party recruiters are because they’re referring to the ridiculous spray-and-pray model of just blasting out emails to everything that hold still long enough that meets a keyword. And yeah, I’ve also met some recruiters that are transformative as far as the conversations you have with them go. But some of that with sales. It’s, “Oh, well, you can’t be any fun to talk to because I had a really bad experience buying a used car once and my credit was in the toilet.”

Tom: Yeah, exactly. And you know, I have a similar experience with recruiters coming to LaunchDarkly. So, not even talking about the product; I was a skeptic, I was happy where I was, but then as I started talking to more and more people here, I’m assuming you’ve read the book Accelerate; you probably had a hand in influencing part of it.

Corey: I can neither confirm nor deny because stealing glory is something I only do very intentionally.

Tom: Oh okay, excellent. Well, I will intentionally let you have some of that glory for you then. But as I was reading that book, it reminded me again of part of why I joined LaunchDarkly. I was a skeptic, and they convinced me through everyone that I talked to just what a nice place it is, and the great culture, it’s safe to fail, it’s safe to try stuff and build stuff. And then if it fails, that’s okay. This is the place where that can happen, and we want to be able to continue to grow and try something new.

That’s again, getting back to the solutions engineer, sales engineer part of it, how can we effectively convey this message and teach people about what it is that we do—LaunchDarkly or not—in a way that makes them excited to see the possibilities of it? So yeah, it’s really great when you get to work with those type of people, and it absolutely shouldn’t be influenced by the worst of them. Sometimes you need to find the right ones to give you a chance and get in the door to start having those conversations so you can make good decisions on your own, not just try to buy whatever someone’s—whatever their initiative is or whatever their priority is, right?

Corey: Once upon a time when I first discovered LaunchDarkly, it was pretty easy to describe what you folks did. Feature flags. For longtime listeners of the show, and I mean very longtime listeners of the show, your colleague Heidi Waterhouse was guest number one. So, I’ve been talking to you folks about a variety of different things in a variety of different ways. But yeah, “LaunchDarkly. Oh, you do feature flags.”

And over time that message has changed somewhat into something I have a little bit of difficulty to be perfectly honest with you in pinning down. At the moment we’re recording this, if I pull up launchdarkly.com, it says, “Fundamentally change how you deliver software. Innovate faster, deploy fearlessly, and make each release a masterpiece.”

And I look at the last release I pushed out, which wound up basically fixing a couple of typos there, and it’s like, “Well, shit. Is it going to make me sign my work because I’m kind of embarrassed by a lot of it.” So, it’s aspirational, I get it, but it also somehow [occludes 00:05:32] a little bit of meaning. What is it you’d say it is you do here.

Tom: Oh, Office Space. Wonderful. Good reference. And also, to take about 30 seconds back, Heidi Waterhouse, what a wonderful human. wiredferret on Twitter. Please, please go look her up. She’s got just always such wonderful things to say. So—

Corey: If you don’t like Heidi Waterhouse, it is a near certainty it is because you’ve not yet met her. She’s wonderful.

Tom: Exactly. Yes, she is. So, what is it we’d say we do here? Well, when people think about feature flags—or at this point now, ‘feature management,’ which is a broader scope—that’s the term that we’re using now, it’s really talking about that last bit of software delivery, the last mile, the last leg, whatever your—you know, when you’re pushing the button, and it’s going to production. So, you know, a feature flag, if you ask someone five or ten years ago, they might say, oh, it’s a fancy if statement controlled by a config file or controlled by a database.

But with a sort of modern architecture, with global delivery, instant response time or fraction of a second response time, it’s a lot more fundamental than that. That’s why the word fundamental is there: Because it comes down to psychological safety. It comes down to feeling good about your life every day. So, whether it is that you’re fixing a couple typos, or if you’re radically changing some backend functionality, and trying out some new sort of search algorithm, a new API route that you’re not sure if it’s going to work at scale, honestly, you shouldn’t have to stay up at night, you shouldn’t have to think about deploying on a weekend because you should be able to deploy half-baked code to production safely, you should be able to do all of that. And that’s honestly what we’re all about.

Now, there’s some extra elements to it: Feedback loops, experimentation, metrics to make sure that your releases are doing well and doing what you anticipated that they would do, but really, that’s what it comes down to is just feeling good about your work and making sure that if there is a fire, it’s a small fire, and the entire audience isn’t going to get part of the splash zone, right? We’re making it just a little safer. Does that answer your question? Is that what you’re getting at? Or am I still just speaking in the lingo?

Corey: That gets it a lot closer. One of the breakthrough moments—of course I picked it up from one of Heidi’s talks—is feature flag seems like a front end developer thing, yadda, yadda, yadda. And she said historically, yeah, in some ways, in some cases, that’s how it started. But think about it this way. Think about separating out configuration from your deploy process. And what would that mean? What would that entail?

And I look at my current things that I have put out there, and there is no staging environment, my feature branches main, and what would that change? In my case, basically nothing. But that’s okay. Because I’m an irresponsible lunatic who should not be allowed near anything expensive, which is why I’m better at stateless things because I know better than to take my aura near things like databases.

Tom: Yeah. So, I don’t know how old you are Corey. But back—

Corey: I’m in my mid-30s, which—

Tom: Hey—

Corey: —enrages my spouse who’s slightly older. Because I’m turning 40 in July, but it’s like, during the pandemic, as it has for many of us, the middle has expanded.

Tom: There you go. Right. Exa—[laugh] exactly. Can neither confirm nor deny. You can only see me from about the mid-torso up, so, you know, you’re not going to see whether I’ve expanded.

But when we were in school doing group projects, we didn’t have Google Docs. We couldn’t see what other people were working on. You’d say, “Hey, we’ve got to write this paper. Corey, you take the first section, I’ll take the second section, and we’ll go and write and we’ll try to squish it back together afterward.” And it’s always a huge pain in the ass, right? It’s terrible. Nobody likes group projects.

And so the old method of Gitflow, where we’re creating these feature branches and trying to squish them back later, and you work on that, and you work on this thing, and we can’t see what each other are doing, it all comes down to context switching. It is time away from work that you care about, time away from exciting or productive work that you actually get to see what you’re doing and put it into production, try it out. Nobody wants to deal with all the extra administrative overhead. And so yeah, for you, when you’ve got your own trunk-based development—you know, it’s all just main—that’s okay. When we’re talking about teams of 40, 50, 100, 1000 suddenly becomes a really big deal if you were to start to split off and get away from trunk-based development because there’s so much extra work that goes into trying to squish all that work back together, right? So, nobody wants to do all the extra stuff that surrounds getting software out there.

Corey: It’s toil. It feels consistently like it is never standardized so you always have to wind up rolling your own CI/CD thing for whatever it is. And forget between jobs; between different repositories and building things out, it’s, “Oh, great. I get to reinvent the wheel some more.” It’s frustrating.

Tom: [laugh]. It’s either that or find somebody else’s wheel that they put together and see if you can figure out where all those spokes lead off to. “Is this secure? I don’t know.”

Corey: How much stuff do you have running in your personal stuff that has more or less been copied around for a decade or so? During the pandemic, I finally decided, all right, you know what I’m doing? That’s right, being productive. We should fix that. I’m going to go ahead and redo my shell config—my zshrc—from scratch because, you know, 15 years of technical debt later, a lot of the things I used to really need it to do don’t really apply anymore.

Let’s make it prettier, and let’s make it faster. And that was great and all, but just looking through it, it was almost like going back in time for weird shell aliases that I don’t need anymore. It’s, well, that was super handy when I ran a Ruby production environment, but I haven’t done that in seven years, and I haven’t been in this specific scenario that one existed for since 2011. So maybe, maybe I can turn that one off.

Tom: Yeah, maybe. Maybe we can get rid of that one. I mean, when’s the last time you ran npm install on something you were going to try out here and paid attention to the warnings that came up afterward? “Hey, this one’s deprecated. That one’s deprecated.” Well, let’s see if it works first, and then we’ll worry about that later.

Corey: Exactly. Security problems? Whatever. It’s a Lambda function. What do I care?

Tom: Yeah, it’s fine. [laugh]. Exactly. Yeah. So, a lot of this is hypothetical for someone in my position, too, because I didn’t ever get formal training as a software developer. I can copy and paste from Stack Overflow with the best of them and there’s all sorts of resources out there, but really the people that we’re talking to are the ones who actually live that day in, day out.

And so I try to step into their shoes and try to feel that pain. But it’s tough. Like, you have to be able to speak both languages and try to relate to people to see what are they actually running into, and is that something that we can help with? I don’t know.

Corey: The way that I tend to think about these things—and maybe it’s accurate, and maybe it’s not—it’s just, no one shows up hoping to do a terrible job at work today, but we are constrained by a whole bunch of things that are imposed upon us. In some of the more mature environments, some of that is processes there for damn good reasons. “Well, why can’t I just push everything I come up with to production?” “It’s because we’re a bank, genius. How about you think a little bit before you open your mouth?”

Other times, it’s because well, I have to go and fight with the CI/CD system, and I’m just going to go ahead and patch this one-line change into production. Better processes, better structure have made that a lot more… they’ve made it a lot easier to be able to do things the right way. But I would say we’re nowhere near its final form, yet. There’s so much yak-shaving that has to go into building out anything that it’s frustrating, on some level, just all of the stuff you have to do, just to get the scaffolding in place to write nonsense. I mean, back when they announced Lambda functions it was, “In the future, the only code you’ll write is business logic.”

Yeah, well, I use a crap-ton of Lambda here and it feels like most of the code I write is gluing all of the weird formats and interchanges together in different APIs. Not a lot of business logic in that; and awful lot of JSON finickiness.

Tom: Yeah, I’m with you. And especially at scale, I still have a hard time wrapping my mind around how all of that extra translation is possibly going to give the same sort of performance and same sort of long-term usability, as opposed to something that just natively speaks the same language end-to-end. So yeah, I agree, there’s still some evolution, some standardization that still needs to happen because otherwise we’re going to end up with a lot of cruft at various points in the code to, just like you said, translate and make sure we’re speaking the same language.

Getting back to process though, I spent a good chunk of my career working with companies that are, I would say, a little more conservative, and talking to things like automotive companies, or medical device manufacturers. Very security-conscious, compliant places. And so agile is a four-letter word for them, right, [laugh] where we’re going faster automatically means we’re being dangerous because what would the change control board say? And so there’s absolutely a mental shift that needs to happen on the business side. And developers are fighting this cultural battle, just to try to say, hey, it’s better if we can make small iterative changes, there is less risk if we can make small, more iterative changes, and convincing people who have never been exposed to software or know the ins and outs of what it takes to get something from my laptop to the cloud or production or you know, wherever, then that’s a battle that needs to be fought before you can even start thinking about the tooling. Living in the Midwest, there’s still a lot of people having that conversation.

Corey: So, you are clearly deep in the weeds of building and deploying things into production. You’re clearly deep into the world of explaining various solutions to different folks, and clearly you have the obvious background for this. You majored in music. Specifically, you got a master’s in it. So, other than the obvious parallel of you continue to sing for your supper, how do you get from there to here?

Tom: Luck and [laugh]. Natural curiosity. Corey, right now you are sitting on the desk that is also housing my PC gaming computer, right? I’ve been building computers just to play video games since I was a teenager. And that natural curiosity really came in handy because when I—like many people—realize that oh, no, the career choice that I made when I was 18 ended up being not the career choice that I wanted to pursue for the rest of my life, you have to be able to make a pivot, right, and start to apply some of the knowledge that you got towards some other industries.

So, like many folks who are now solutions engineers, there’s no degree for solutions engineering, you can’t go to school for it; everyone comes from somewhere else. And so in my case, that just happened to be music theory, which was all pedagogy and teaching and breaking down big complex pieces of music into one node at a time, doing analysis, figuring out what’s going on underneath the hood. And all of those are transferable skills that go over to software, right? You open up some giant wall of spaghetti code and you have to start following the path and breaking it down because every piece is easy one note at a time, every bit of code—in theory—is easy one line at a time, or one function at a time, one variable at a time. You can continue to break it down further and further, right?

So, it’s all just taking the transferable skills that you may not see how they get transferred, but then bringing them over to share your unique perspective, because of your background, to wherever it is you’re going. In my case, it was tech support, then training, and then solutions engineering.

Corey: There’s a lot to be said for blending different disciplines. I think that there was, uh, the naughts at least, and possibly into the teens, there was a bias for hiring people who look alike. And no, I’m not referring to the folks who are the white dudes you and I clearly present as but the people with a similar background of, “Oh, you went to these specific schools”—as long as they’re Stanford—“And you majored in a narrow list of things”—as long as they’re all computer science. And then you wind up going into the following type of role because this is the pedigree we expect and everything, soup to nuts, is aligned around that background and experience. Where you would find people who would be working in the industry for ten years, and they would bomb the interview because it turns out that most of us don’t spend our days implementing quicksort on whiteboards or doing other algorithmic-based problems.

We’re mostly pushing pixels around a screen hoping to make ourselves slightly happier than we were. Here we are. And that becomes a strange world; it becomes a really, really weird moment, and I don’t know what the answer is for fixing any of that.

Tom: Yeah, well, if you’re not already familiar with a quote, you should be, which is that—and I’m going to paraphrase here—but, “Diverse backgrounds lead to diversity in thought,” right? And that presents additional opportunities, additional angles to solve whatever problems you’re encountering. And so you’re right, you know, we shouldn’t be looking for people who have the specific background that we are looking for. How it’s described in Accelerate? Can you tell that I read it recently?

Which we should be looking for capabilities, right? Are you capable? Do you have the capacity to do the problem-solving, the logic? And of course, some education or experience to prove that, but are you the sort of person who will be able to tackle this challenge? It doesn’t matter, right, if you’ve handled that specific thing before because if you’ve handled that specific thing before, you’re probably going to implement it the same way, again, even if that’s not the appropriate solution, this time.

So, scrap that and say, let’s find the right people, let’s find people who can come up with creative solutions to the problems that we’re facing. Think about ways to approach it that haven’t been done before. Of course don’t throw out everything with the—you know, the bathwater out with a baby or whatever that is, but come in with some fresh perspectives and get it done.

Corey: I really wish that there was more of an acceptance for that. I think we’re getting there. I really do, but it takes time. And it does pay dividends. I mean, that’s something I want to talk to you about.

I love the sound of my own voice. I wouldn’t have two podcasts if I didn’t. The counterargument, though, is that there’s an awful lot of things that get, you know, challenging, especially when, unlike in a conference setting, it’s most people consider it rude to get up and walk out halfway through. When we’re talking and presenting information to people during a pandemic situation, well, that changes a lot. What do you do to retain people’s interest?

Tom: Sure. So, Covid really did a number on anyone who needs to present information or teach. I mean, just ask the millions of elementary, middle school, and high schoolers out there, even the college kids. Everyone who’s still getting their education suddenly had to switch to remote learning.

Same thing in the professional world. If you are doing trainings, if you’re doing implementation, if you’re doing demos, if you’re trying to convey information to a new audience, it is so easy to get distracted at the computer. I know this firsthand. I’m one of those people where if I’m sitting in an airport lobby and there’s a TV on my eyes are glued to that screen. That’s me. I have a hard time looking away.

And the same thing happens to anyone who’s on the receiving end of any sort of information sharing, right? You got Slack blowing you up, you’ve got email that’s pinging you, and that’s bound to be more interesting than whatever the person on the screen is saying. And so I felt that very acutely in my job. And there’s a couple of good strategies around it, right, which is, we need to be able to make things interactive. We shouldn’t be monologuing like I am doing to you right now, Corey.

We shouldn’t be [laugh] just going off on tangents that are completely irrelevant to whoever’s listening. And there’s ways to make it more interactive. I don’t know if you are familiar, or how much you’ve watched Twitch, but in my mind, the same sorts of techniques, the same sorts of interactivity that Twitch streamers are doing, we should absolutely be bringing that to the business world. If they can keep the attention of 12-year-olds for hours at a time, why can we not capture the attention of business professionals for an hour-long meeting, right? There’s all sorts of techniques and learnings that we can do there.

Corey: The problem I keep running into is, if you go stumbling down that pathway into the Twitch streaming model, I found it awkward the few experiments I’ve made with it because unless I have a whole presentation ready to go and I’m monologuing the whole time, the interactive part with the delay built in and a lot of ‘um’ and ‘ah’ and waiting and not really knowing how it’s going to play out and going seat of the pants, it gets a little challenging in some respects.

Tom: Yeah, that’s fair. Sometimes it can be challenging. It’s risky, but it’s also higher reward. Because if you are monologuing the entire time, who’s to say that halfway through the content that you are presenting is content that they want to actually hear, right? Obviously, we need to start from some sort of fundamental place and set the stage, say this is the agenda, but at some point, we need to get feedback—similar to software development—we need to know if the direction that we’re going is the direction they also want to go.

Otherwise, we start diverging at minute 10 and by minute 60, we have presented nothing at all that they actually want to see or want to learn about. So, it’s so critical to get that sort of feedback and be able to incorporate it in some way, right? Whether that way is something that you’re prepared to directly address. Or if it’s something that says, “Hey, we’re not on the same page. Let’s make sure this is actually a good use of time instead of [laugh] me pretending and listening to myself talk and not taking you into account.” That’s critical, right? And that is just as important, even if it feels worse in the moment.

Corey: This episode is sponsored in part by our friends at ChaosSearch. You could run Elasticsearch or Elastic Cloud—or OpenSearch as they’re calling it now—or a self-hosted ELK stack. But why? ChaosSearch gives you the same API you’ve come to know and tolerate, along with unlimited data retention and no data movement. Just throw your data into S3 and proceed from there as you would expect. This is great for IT operations folks, for app performance monitoring, cybersecurity. If you’re using Elasticsearch, consider not running Elasticsearch. They’re also available now in the AWS marketplace if you’d prefer not to go direct and have half of whatever you pay them count towards your EDB commitment. Discover what companies like Equifax, Armor Security, and Blackboard already have. To learn more, visit chaossearch.io and tell them I sent you just so you can see them facepalm, yet again.

Corey: From where I sit, one of the many, many, many problems confronting us is that there’s this belief that everyone is like we are. I think that’s something fundamental, where we all learn in different ways. I have never been, for example—this sounds heretical sitting here saying it, but why not—I’m not a big podcast person; I don’t listen to them very often, just because it’s such a different way of consuming information. I think there are strong accessibility reasons for there to be transcripts of podcasts. That’s why every 300-and-however-many-odd episodes that this one winds up being the sequence in, every single one of them has a transcript attached to it done by a human.

And there’s a reason for that. Not just the accessibility wins which are obvious, but the fact that I can absorb that information way more quickly if I need to review something, or consume that. And I assume other people are like me, they’re not. Other people prefer to listen to things than to read them, or to watch a video instead of listening, or to build something themselves, or to go through a formal curriculum in order to learn something. I mean, I’m sitting here with an eighth-grade education, myself. I take a different view to how I go about learning things.

And it works for me, but assuming that other people learn the same way that I do will be awesome for a small minority of people and disastrous for everyone else. So, maybe—just a thought here—we shouldn’t pattern society after what works for me.

Tom: Absolutely. There is a multiple intelligence theory out there, something they teach you when you’re going to be a teacher, which is that people learn in different ways. You don’t judge a fish by its ability to climb a tree. We all learn in different ways and getting back to what we were talking about presenting effectively, there needs to be multiple approaches to how those people can consume information. I know we’re not recording video, but for everyone listening to this, I am waving my hands all over the place because I am a highly visual learner, but you must be able to accept that other people are relying more on the auditory experience, other people need to be able to read that—like you said with the accessibility—or even get their hands on it and interact with it in some way.

Whether that is Ctrl-F-ing your way through the transcript—or Command-F I’m sorry, Mac users [laugh]; I am also on a Mac—but we need to make sure that the information is ready to be consumed in some way that allows people to be successful. It’s ridiculous to think that everyone is wired to be able to sit in front of a computer or in a little cubicle for eight hours a day, five days a week, and be able to retain concentration and productivity that entire time. Absolutely not. We should be recording everything, allowing people to come back and consume it in small chunks, consume it in different formats, consume it in the way that is most effective to them. And the onus for that is on the person presenting, it is not on the consumer.

Corey: I make it a point to make what I am doing accessible to the people I am trying to reach, not to me. And sometimes I’m slacking, for example, we’re not recording video today, so whenever it looks like I’m not paying attention to you and staring off to the side, like, oh, God, he’s boring. No. I have the same thing mirrored on both of my screens; I just prefer to look at the thing that is large and easy to read, rather than the teleprompter, which is a nine-inch screen that is about four feet in front of my face. It’s one of those easier for me type of things.

On video, it looks completely off, so I don’t do it, but I’m oh good, I get to take the luxury of not having to be presentable on camera in quite the same way. But when I’m doing a video scenario, I absolutely make it a point to not do that because it is off-putting to the people I’m trying to reach. In this case, I’m not trying to reach you; I already have. This is a promoted guest episode you’re trying to reach the audience, and I believe from what I can tell, you’re succeeding, so please keep at it.

Tom: Oh, you bet. Well, thank you. You know this already, but this is the very first podcast I’ve ever been a guest on. So, thank you also for making it such a welcoming place. For what it’s worth, I was not offended and didn’t think you weren’t listening. Obviously, we’re having a great time here.

But yeah, it’s something that especially in the software space, people need to be aware of because everyone’s job is—[laugh]. Whether you like it or not, here’s a controversial statement: Everyone’s job is sales. Are you selling your good ideas for your product, to your boss, to your product manager? Are you able to communicate with marketing to effectively say, “Hey, this is what, in tech support, I’m seeing. This is what people are coming to me with. This is what they care about.”

You are always selling your own performance to your boss, to your customers, to other departments where you work, to your spouse, to everybody you interact with. We’re all selling ourselves all the time. And all of that is really just communication. It’s really just making sure you’re able to meet people where they are and, effectively, bridge your point of view with theirs to make sure that we’re on the same page and, you know, we’re able to communicate well. That’s so especially important now that we’re all remote.

Corey: Just so you don’t think this is too friendly of a place, let’s go ahead and finish out the episode with a personal attack. Before you wound up working at LaunchDarkly. You were at Perforce. What’s up with that? I mean, that seems like an awfully big company to cater to its single customer, who is of course J. Paul Reed.

Tom: [laugh]. Yeah. Well, Perforce is a wonderful place. I have nothing but love for Perforce, but it is a very different landscape than LaunchDarkly, certainly. When I joined Perforce, I was supporting product called Helix ALM, which, they’re still headquartered—Perforce is headquartered here in Minneapolis. I just saw some Perforce folks last week. It truly is a great place, and it is the place that introduced me to so many DevOps concepts.

But that’s a fair statement. Perforce has been around for a while. It has grown by acquisition over the past several years, and they are putting together new offerings by mixing old offerings together in a way that satisfies more modern needs, things like virtual production, and game development, and trying to package this up in a way that you can then have a game development environment in a box, right? So, there’s a lot of things to be said for that, but it very much is a different landscape than a smaller cloud-native company. Which it’s its own learning curve, let me tell you, but truly, yeah, to your Perforce, there’s a lot more complexity to the products themselves because they’ve been around for a little bit longer.

Solid, solid products, but there’s a lot going on there. And it’s a lot harder to learn them right upfront. As opposed to something like LaunchDarkly, which seems simple on the surface and you can get started with some of the easy concepts in implementation in, like, an hour, but then as you start digging deeper, whoof, suddenly, there’s a lot more complexity hidden underneath the surface than just in terms of how this is set up, and some of those edge cases.

Corey: I have to say for the backstory, for those who are unfamiliar, is I live about four miles away from J. Paul Reed, who is a known entity in reliability engineering, in the DevOps space, has been for a long time. So, to meet him, of course I had to fly to Israel. And he was keynoting DevOpsDays Tel Aviv. And I had not encountered him before, and it was this is awesome, I loved his talk, it was fun.

And then I gave a talk a little while later called, “Terrible Ideas in Git.” And he’s sitting there just glaring at me, holding his water bottle that is a branded Perforce thing, and it’s like, “Do you work there?” He’s like, “No. I just love Perforce.” It’s like, “Congratulations. Having used it, I think you might be the only one.”

I kid. I kid. It was great and a lot of different things. It was not quite what I needed when I needed it to but that’s okay. It’s gotten better and everyone else is not me, as we’ve discussed; people have different use cases. And that started a very long-running joke that J. Paul Reed is the entirety of the Perforce customer base.

Tom: [laugh]. Yeah. And to your point, there’s definitely use cases—you’re talking about Perforce Version Control or Helix Core.

Corey: Back in those days, I don’t believe it was differentiated.

Tom: It was just called Perforce. Exactly right. But yeah, as Perforce has gotten bigger, now there’s different product lines; you name it. But yeah, some of those modern scalable problems, being able to handle giant binary files, being able to do automatic edge replication for globally distributed teams so that when your team in APAC comes online, they’re not having to spend the first two hours of their day just getting the most recent changes from the team in the Americas and Europe. Those are problems that Perforce is absolutely solving that are out there, but it’s not problems that everybody faces and you know, there’s just like everybody else, we’re navigating the landscape and trying to find out where the product actually fits and how it needs to evolve.

Corey: And I really do wish you well on it. I think there’s going to be an awful lot of—

Tom: Mm-hm.

Corey: —future stories where there is this integration. And you’d say, “Oh, well, what are you wishing me well for? I don’t work there anymore.” But yeah, but isn’t that kind of we’re talking about, on some level, of building out things that are easy, that are more streamlined, that are opinionated in the right ways, I suppose. And honestly, that’s the thing that I found so compelling about LaunchDarkly. I have a hard time imagining I would build anything for production use that didn’t feature it these days if I were, you know, better at computers?

Tom: Sure. Yeah. [laugh]. Well, we do have our opinions on how some things should work, right? Where the data is exposed because with any feature flagging system or feature management—LaunchDarkly included—you’ve got a set of rules, i.e. who should see this, where is it turned on? Where is it turned off? Who in your audience or user base should be able to see these features? That’s the rules engine side of it.

And on the other side, you’ve got the context to decide, well, you know, I’m Corey, I’m logging in, I’m in my mid-30s. And I know all this information about Corey, and those rules need to then be able to determine whether something should be on or off or which experience Corey gets. So, we are very opinionated over the architecture, right, and where that evaluation actually happens and how that data is exposed or where that’s exposed. Because those two halves need to meet and both halves have the potential to be extremely sensitive. If I’m targeting based off of a list of 10,000 of my premium users’ email addresses, I should not be exposing that list of 10,000 email addresses to a web browser or a mobile phone.

That’s highly insecure. And inefficient; that’s a large amount of text to send, over 10,000 email addresses. And so when we’re thinking about things like page load times, and people being able to push F12 to inspect the page, absolutely not, we shouldn’t be exposing that there. At the same time, it’s a scary prospect to say, “Hey, I’m going to send personal information about Corey over to some third-party service, some edge worker that’s going to decide whether Corey should see a feature or not.” So, there’s definitely architectural considerations of different use cases, but that’s something that we think through all the time and make sure is secure.

There’s a reason—I’m going to put on my sales engineer hat here—which is to say that there is a reason that the Center for Medicare and Medicaid Services is our sponsor for FedRAMP moderate certification, in process right now, expected to be completed mid-2022. I don’t know. But anybody who is unfamiliar with that, if you’ve ever had to go through high trust certification, you know, any of these compliances to make your regulators happy, you know that FedRAMP is so incredibly stringent. And that comes down to evaluating where are we exposing the data? Who gets to see that? Is security built in and innate into the architecture? Is that something that’s been thought through?

I have went so far afield from the original point that you made, but I agree, right? We’ve got to be opinionated about some things while still providing the freedom to use it in a way that is actually useful to you and [laugh] and we’re not, you know, putting up guardrails, that mean that you’ve got such a narrow set of use cases.

Corey: I’d like to hope—maybe I’m wrong on this—that it gets easier the more that we wind up doing these things because I don’t think that it necessarily has been easy enough for an awful lot of us.

Tom: When you say ‘it,’ what do you mean?

Corey: All of it. That’s the best part, I suppose the easy parts of working on computers, which I guess might be typing if you learn it early enough.

Tom: Sure. [laugh] yeah. Mario Teaches Typing, or Starcraft taught me how to type quickly. You can’t type slowly or else your expansion is going to get destroyed. No, so for someone who got their formal education in music or for someone with an eighth-grade education, I agree there needs to be resources out there.

And there are. Not every single StackOverflow post with a question that’s been asked has the response, “That’s a dumb question.” There are some out there. There’s definitely a community or a group of folks who think that there is a correct way to do things and that if you’re asking a question, that it’s a dumb question. It really isn’t. It’s getting back to the diverse backgrounds and diverse schools of thought that are coming in.

We don’t know where someone is coming from that led them to that question without the context, and so we need to continue providing resources to folks to make it easy to self-enable and continue abstracting away the machine code parts of it in friendlier and friendlier ways. I love that there are services like Squarespace out there now, right, that allow anybody to make a website. You don’t have to have a degree in computer science to spin something up and share it with the world on the web. We’re going to continue to see that type of abstraction, that type of on-ramp for folks, and I’m excited to be part of it.

Corey: I really look forward to it. I’m curious to see what happens next for you, especially as you continue—‘you’ being the corporate ‘you’ here; that’s like the understood ‘you’ are the royal ‘you.’ This is the corporate ‘you’—continue to refine the story of what it is LaunchDarkly does, where you start, where you stop, and how that winds up playing out.

Tom: Yeah, you bet. Well, in the meantime, I’m going to continue to play with things like GitHub Copilot, see how much I can autofill, and see which paths that takes me down?

Corey: Oh, I’ve been using it for a while. It’s great. Just tab-complete my entire life. It’s amazing.

Tom: Oh, yeah. Absolutely.

Corey: [unintelligible 00:36:08] other people’s secrets start working, great, that makes my AWS bill way lower when I use someone else’s keys. But that’s neither here nor there.

Tom: Yeah, exactly. That’s a next step of doing that npm install or, you know, bringing in somebody else’s [laugh] tools that they’ve already made. Yeah, just a couple weeks ago, I was playing around with it, and I typed in two lines: I imported the LaunchDarkly SDK and the configuration for the LaunchDarkly SDK, and then I just let it autofill, whatever it wanted. It came out with about 100 lines of something or other. [laugh]. And not all of it made sense, but hey, I saw where the thought process was. It was pretty cool to see.

Corey: I really want to thank you for spending as much time and energy as you have talking about how you see the world and where you folks are going. If people want to learn more. Where’s the best place to find you?

Tom: At launchdarkly.com. Of course, any other various different booths, DevOpsDays, we’re at re:Invent, we’re at QCon right now. We’re at all sorts of places, so come stop by, say hi, get a demo. Maybe we’ll talk.

Corey: Excellent. We will be tossing links to that into the [show notes 00:37:09]. Thanks so much for your time. I really appreciate it.

Tom: Corey, Thank you.

Corey: Tom Totenberg, senior solutions engineer at LaunchDarkly. I’m Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you’ve enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you’ve hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry and insulting comment, and then I’ll sing it to you.

Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

Announcer: This has been a HumblePod production. Stay humble.