Diving Duckbill First into the Depths of Data with Alex Rasmussen

• Twitter: https://twitter.com/alexras/
• Personal page: https://alexras.info
• Old Consulting website with blog: https://bitsondisk.com

Transcript

Announcer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.

Corey: The company 0x4447 builds products to increase standardization and security in AWS organizations. They do this with automated pipelines that use well-structured projects to create secure, easy-to-maintain and fail-tolerant solutions, one of which is their VPN product built on top of the popular OpenVPN project which has no license restrictions; you are only limited by the network card in the instance. To learn more visit: snark.cloud/deployandgo

Corey: Today’s episode is brought to you in part by our friends at MinIO the high-performance Kubernetes native object store that’s built for the multi-cloud, creating a consistent data storage layer for your public cloud instances, your private cloud instances, and even your edge instances, depending upon what the heck you’re defining those as, which depends probably on where you work. It’s getting that unified is one of the greatest challenges facing developers and architects today. It requires S3 compatibility, enterprise-grade security and resiliency, the speed to run any workload, and the footprint to run anywhere, and that’s exactly what MinIO offers. With superb read speeds in excess of 360 gigs and 100 megabyte binary that doesn’t eat all the data you’ve gotten on the system, it’s exactly what you’ve been looking for. Check it out today at min.io/download, and see for yourself. That’s min.io/download, and be sure to tell them that I sent you.

Corey: Welcome to Screaming in the Cloud. I’m Corey Quinn. I’m the chief cloud economist at The Duckbill Group, which people are generally aware of. Today, I’m joined by our most recent principal cloud economist, Alex Rasmussen. Alex, thank you for joining me today, it is a pleasure to talk to you, as if we aren’t talking to each other constantly, now that you work here.

Alex: Thanks, Corey. It’s great being here.

Corey: So, I followed a more, I’d say traditional path for a cloud economist, but given that I basically had to invent the job myself, the more common path because imagine that you start building a role from scratch and the people you wind up looking for initially look a lot like you. And that is grumpy sysadmin, historically, turned into something, kind of begrudgingly, that looks like an SRE, which I still maintain are the same thing, but it is imperative people not email me about that. Yes, I know, you work at Google. But instead, what I found during my tenure as a sysadmin, is that I was working with certain things an awful lot, like web servers, and other things almost never, like databases and data warehouses. Because if you screw up a web server, we all have a good laugh, the site’s down for a couple of minutes, life goes on, you have a shame trophy on your desk if that’s your corporate culture, things continue.

Mess up the data severely enough, and you don’t have a company anymore. So, I was always told to keep my aura away from the expensive spendy things that power a company. You are sort of the first of a cloud economist subtype that doesn’t resemble that. Before you worked here, you were effectively an independent consultant working on data engineering. Before that, you had a couple of jobs, but you had gotten a PhD in computer science, which means, first, you are probably one of the people in this world most qualified to pass some crappy job interview of solving a sorting algorithm on a whiteboard, but how did you get here from where you were?

Alex: Great question. So, I like to joke that I kind of went to school until somebody told me that I had to stop. And I took that and went and started—or didn’t start, but I was an early engineer at a startup and then was an executive at another early-stage one, and did a little bit of everything. And went freelance, did that for a couple of years, and worked with all kinds of different companies—vast majority of those being startups—helping them with data infrastructure problems. I’ve done a little bit of everything throughout my career.

I’ve been, you know, IC, manager, manager, manager, IT guy, everything in between. I think on the data side of things, it just sort of happened, to be honest with you, it kind of started with the stuff that I did for my dissertation and parlayed that into a job back when the big data wave was starting to kind of truly crest. And I’ve been working on data infrastructure, basically my entire career. So, it wasn’t necessarily something that was intentional. I’ve just been kind of taking the opportunity that makes the most sense for me it kind of every juncture. And my career path has been a little bit strange, both by academic and industrial standards. But I like where I’m at and I gained something really valuable from each of those experiences. So.

Corey: It’s been an interesting area of I won’t say weakness here, but it’s definitely been a bit of a challenge when we look at an AWS environment and even talking about a typical AWS customer without thinking of any of them in particular, I can already tell you a few things are likely to be true. For example, the number one most expensive line item in their bill is going to be EC2, and compute is the thing that powers it. Now, maybe that is they’re running a bunch of instances the old-fashioned way. Maybe they’re running Kubernetes but that’s how it shows up. There’s a lot of things that could be, and we look at what rounds that out.

Now, the next item down should almost certainly not be data transfer and if so we should have a conversation, but data in one form or another is very often going to be number two. And that can mean a bunch of different things, historically. It could mean, “Oh, you have a whole bunch of stuff in S3. Let’s talk about access patterns. Let’s talk about lifecycle policies. Let’s talk about making sure the really important stuff is backed up somewhere. Maybe you want to spend more on that particular aspect of it.”

If it’s on EBS volumes, that’s interesting and definitely worth looking into and trying to understand the context of what’s going on. Periodically we’ll see a whole bunch of additional charges that speak to some of that EC2 charge in the form of EMR, AWS’s Elastic MapReduce, which charges a per-hour instance charge, but also charges you for the instances that are running under the hood and under the EC2 line item. So, there’s a lot of data lifecycle stuff, there’s a lot of data ecosystem stories, that historically we’ve consulted out with experts in that particular space. And that’s great, but we were starting to have to drag those people in on more and more engagements as we saw them. And we realized that was really something we had to build out as a core competency for ourselves.

And we started out not intending to hire for someone with that specialty, but the more we talked to you, the more it became clear that this was a very real and very growing need that we and our customers have. How closely it is what you’re doing now as far as AWS bill analysis and data pattern deep-dive align with what you were doing as a freelance consultant in the space?

Alex: A lot more than you might expect. You know, I think that increasingly, what you’re seeing now is that a company’s core differentiator is its data, right, how much of it they have, what they do with it. And so, you know, to your point, I think when you look at any company’s cloud spend, it’s going to be pretty heavy on the data side in terms of, like, where have you put it? What are you doing to process it? Where is it going once it’s been processed? And then how is that—

Corey: And data transfer is a very important first word in that two-word sequence.

Alex: Oh, sure is. And so I think that, like, in a lot of ways, the way that a customer’s cloud architecture looks and the way that their bill looks kind of as a consequence of that is kind of a reification in a way of the way that the data flows from one place to another and what’s done with it at each step along the way. I think what complicates this is that companies that have been around for a little while have lived through this kind of very amorphous, kind of, polyglot way that we’re approaching data. You know, back when I was first getting started in the big data days, it was MapReduce, MapReduce, MapReduce, right? And we quickly [crosstalk 00:07:29]—

Corey: Oh, yes. The MapReduce white paper out of Google, a beautiful April Fool’s Day prank that the folks at Yahoo fell for hook, line, and sinker. They wrote Hadoop, and now we’re all stuck with that pattern. Great gag, they really should have clarified they were kidding. Here we are.

Alex: Exactly. So—

Corey: I mostly kid.

Alex: No, for sure. But I think especially when it comes to data, we tend to over-index on what the large companies do and then quickly realize that we’ve made a mistake and correct backwards, right? So, there was this big push toward MapReduce for everything until people realize that it was just a pain in the neck to operate and to build. And so then we moved into Spark, so kind of up-leveled a little bit. And then there was this kind of explosion of NoSQL and NewSQL databases that hit the market.

And MongoDB inexplicably won that war and now we’re kind of in this world where everything is cloud data warehouse, right? And now we’re trying to wrestle with, like, is it actually a good idea to put everything in one warehouse and have SQL be the lingua franca on top of it? But it’s all changing so rapidly. And when you come into a customer that’s been around for 10 or 15 years, and has, you know, been in the cloud for a substantial—

Corey: Yeah, one of those ancient customers. That is—

Alex: I know, right?

Corey: —basically old enough to almost get a driver’s license? Oh, yeah.

Alex: Right. It’s one of those things where it’s like, “Ah, yes, in startup years, you’re, like, a hundred years old,” right? But still, you know, I think you see this, kind of—I wouldn’t call it a graveyard of failed experiments, right, but it’s a collection of, like, “Well, we tried this, and it kind of worked and we’re keeping it around because the cost of moving this stuff around—the kind of data gravity, so to speak—is high enough that we’re not going to bother transitioning it over.” But then you get into this situation where you have to bend over backwards to integrate anything with anything else. And we’re still kind of in the early days of fixing that.

Corey: And the AWS bill pattern that we see all the time across the board of those experiments were not successful and do not need to exist, but there’s no context into that. The person that set them up left five years ago, the jobs are still running on time. What’s happening with them? Well, we could stop them and see who screams, but very often, that’s not the right answer either.

Alex: And I think there’s also something to note there, too, which is like, getting rid of data is very scary, right? I mean, if you resize a Kubernetes cluster from 15 nodes to 10, nobody’s going to look at you sideways. But if you go, “Hey, we’re just going to drop these tables.” The immediate reaction that you get, particularly from your data science team more often than not is, “Oh, God, what if we need that?” And so the conversation never really happens, and that causes this kind of snowball of data debt that persists in some cases for many, many years.

Corey: Yeah, in some cases, what I found has been successful on those big unknown questions is don’t delete the data, but restrict access to it for a few weeks and see what happens. Look into it a bit and make sure that it’s not like, “Oh, cool. We just did for a month, and now we don’t need that data. Let’s get rid of it.” And then another month goes by it’s like, “So, time to report quarterly earnings. Where’s the data?”

Oh, dear, that’s not going to go well, for anyone. And understanding what’s happening, the idea of cloning a petabyte of data so you can run an experiment on it. And okay, turns out the experiment wasn’t needed. Do we still need to keep all of that?

Alex: Yeah.

Corey: The underlying platform advancements have been helpful toward this as well, a petabyte of data now in Glacier Deep Archive cost the princely sum of a thousand bucks a month, which is pretty close to the idea of why would I ever delete data ever again? I can get it back within a day if I need it, so let’s just put it there instead.

Alex: Right. You know, funny story. When I was in graduate school, we were dealing with, you know, 100 terabyte datasets on the regular that we had to generate every time because we only had 200 terabytes of raw storage. [laugh]. And this was before cloud was yet mature enough that we could get the kind of performance numbers that we wanted off of it.

And we would end up having to delete the input data to make room for the output data. [laugh]. And thankfully, we don’t need to do that anymore. But there are a lot of, kind of, anti-patterns that arise from that too, right? If data is easy to keep around forever, it stays around forever.

And if it’s easy to, let’s say, run a SQL command against your Snowflake instance that scans 20 terabytes of data, you’re just going to do it, and the exposure of that to you is so minimal that you can end up causing a whole bunch of problems for yourself by the fact that you don’t have to deal with stuff at that low-level of abstraction anymore.

Corey: It’s always fun watching how this stuff manifests—because I’m dipping a toe into it from time to time—the easy, naive answer that we could give every customer but we don’t is, “Huh. So, you have a whole bunch of EMR stuff? Well, you know, if you migrate that into something else, you’ll save a whole bunch of money on that.” With no regard for the 500 jobs that run against that EMR cluster on a consistent basis that form is a key part of business process. “Yeah, if you could just do the entire flow of how data is operated with throughout your entire business that would be swell because you can save tens of thousands of dollars a month on that.” Yeah, how about we don’t suggest things that are just absolute buffoonery.

Alex: Well, and it’s like, you know, you hit on a good point. Like, one of my least favorite words in the English language is the word ‘just.’ And you know, I spent a few years as a freelance data consultant, and you know, a lot of what I would hear sometimes from customers is, “Well, why don’t we ‘just’ deprecate X?”

Corey: “Why don’t we just—” “I’m going to stop you there because there is no ‘just.’”

Alex: Exactly.

Corey: There’s always context that we cannot have as outsiders.

Alex: Precisely. Precisely. And digging into that really is—it’s the fun part of the job, but it’s also the hard part of the job.

Corey: Before we created The Duckbill Group, which was really when I took Mike Julian on as business partner and CEO and formed the entity, I had something in common with you; I was freelancing for a couple of years beforehand. Now, I know why I wound up deciding, all right, we’re going to turn this into a company, but what was it that I guess made you decide to, you know, freelancing is all well and good, but it’s time to get something that looks a lot more like a quote-unquote, “Traditional job.”

Alex: So, I think, on one level, I went freelance because I wasn’t exactly sure what I wanted to do next. And I knew what I was good at. I knew what I had a lot of experience at, and I thought, “Well, I can just go out and kind of find a bunch of people that are willing to hire me to do what I’m good at doing, and then maybe eventually I’ll find one of them that I like enough that I’ll go and work for them. Or maybe I’ll come up with some kind of a business model that I can repeat enough times that I don’t have to worry that I wake up tomorrow and all of my clients are gone and then I have to go live in a van down by the river.”

And I think when I heard about the opening at The Duckbill Group, I had been thinking for a little while about well, this has been going fine for a long time, but effectively what I’ve been doing is I’ve been you know, a staff-level data engineer for hire. And do I want to do something more than that, you know? Do I want to do something more comp—perhaps more sophisticated or more complex than that? And I rapidly came to the conclusion that in order to do that, I would have to have sales and marketing, and I would have to, you know, spend a lot of my time bringing in business. And that’s just not something that I have really any experience in or I’m any good at.

And, you know, I also recognize that, you know, I’m a relatively small fish in a relatively large pond, and if I wanted to get the kind of like, large scale people, the like the big, you know, Fortune 1000 company kind of customers, they may not pay attention to somebody like me. And so I think that ultimately, what I saw with The Duckbill Group was, number one, a group of people that were strongly aligned to the way that I wanted to keep doing this sort of work, right? Cultural alignment was really strong, good people, but also, you know, you folks have a thing that you figured out, and that puts you 10 to 15 steps ahead of where I was. And I was kind of staring down the barrel that, I’m like, am I going to have to take six months not doing client work so that I can figure out how to make this business sustain? And, you know, I think that ultimately, like, I just looked at it, and I said, this just makes sense to me, like, as a next step. And so here we all are.

Corey: This episode is sponsored by our friends at Oracle Cloud. Counting the pennies, but still dreaming of deploying apps instead of “Hello, World” demos? Allow me to introduce you to Oracle’s Always Free tier. It provides over 20 free services and infrastructure, networking, databases, observability, management, and security. And—let me be clear here—it’s actually free. There’s no surprise billing until you intentionally and proactively upgrade your account. This means you can provision a virtual machine instance or spin up an autonomous database that manages itself, all while gaining the networking, load balancing, and storage resources that somehow never quite make it into most free tiers needed to support the application that you want to build. With Always Free, you can do things like run small-scale applications or do proof-of-concept testing without spending a dime. You know that I always like to put asterisks next to the word free? This is actually free, no asterisk. Start now. Visit snark.cloud/oci-free that’s snark.cloud/oci-free.

Corey: It’s always fun seeing how people perceive what we’ve done from the outside. Like, “Oh, yeah, you just stumbled right onto the thing that works, and you’ve just been going, like, gangbusters ever since.” Then you come aboard, it’s like, “Here, look at this pile of things that didn’t pan out over here.” And it’s, you get to see how the sausage is made in a way that we talk about from time to time externally, but surprisingly, most of our marketing efforts aren’t really focused on, “And here’s this other time we screwed up as well.” And we’re honest about it, but it’s not sort of the thing that we promote as the core message of what we do and who we are.

A question I like to ask people during job interviews, and I definitely asked you this, and I’ll ask you now, which is going to probably throw some folks for a loop because who talks to their current employees like this? But what’s next for you? When it comes time for you to leave the Duckbill Group, what do you want to do after this job?

Alex: That’s a great question. So, I mean, as we’ve mentioned before, you know, my career trajectory has been very weird and circuitous. And, you know, I would be lying to you if I said that I had absolute certainty about what the rest of that looks like. I’ve learned a few things about myself in the course of my career, such as it is. In my kind of warm, gooey center, I build stuff. Like, that is what gives me joy, it is what makes me excited to wake up in the morning.

I love looking at big, complicated things, breaking them down into pieces, and figuring out how to make the pieces work in a way that makes sense. And, you know, I’ve spent a long time in the data ecosystem. I don’t know, necessarily, if that’s something that I’m going to do forever. I’m not necessarily pigeonholing myself into that part of the space just yet, but as long as I get to kind of wake up in the morning, and say, “I’m going to go and build things and it’s not going to actively make the world any worse,” I’m happy with that. And so that’s really—you know, might go back to freelancing, might go and join another group, another company, big small, who knows. I’m kind of leaving that up to the winds of destiny, so to speak.

Corey: One thing that I have found incredi—sorry. Let me just address that first. Like that—

Alex: Sure.

Corey: —is the right way to think about it. My belief has always been that you don’t necessarily have, like, the ten-year plan, or the five-year plan or whatever it is because that’s where you’re going to go so much as it gives you direction and forces you to keep moving so you don’t wind up sitting in the same place for five years with one year of experience repeated five times. It helps you remember the bigger picture. Because I’ve always despised this fiction that we see in job interviews where average tenure in our industry is 18 to 36 months, give or take, but somehow during the interviews, we all talk like this is now your forever job, and after 25 years, you’ll retire. And yeah, let’s be a little more realistic than that.

My question is always what is next and how can we align in a way that helps you get to what’s coming? That’s the purpose behind the question, and that’s—the only way to make that not just a drippingly insincere question is to mean it and to continue to focus on it from time to time of, great. What are you learning what’s next? Now, at the time of this recording, you’ve been here, I believe three weeks if I’m not mistaken?

Alex: I’ve—this is week two for me at time of recording.

Corey: Excellent. Yes, my grasp of time is sort of hazy at the best of times. I have a—I do a lot of things.

Alex: For sure.

Corey: But yeah, it has been an eye-opening experience for me, not because, “Oh, wow, we have an employee.” Yeah, we’ve done that a few times before. But rather because of your background, you are asking different questions than we typically get during onboarding. I had a blog post go out recently—or will be by the time this airs—about a question that you asked about, “Wow, onboarding into our internal account structure for AWS is way more polished than I’ve ever seen it before. Is that something you built in-house? What is that?”

And great. Oh, terrific, I’d forgotten that this is kind of a novel thing. No. What we’re using is AWS’s SSO offering, which is such a well-built, polished product that I can only assume that it’s under NDA because Amazonians don’t talk about it ever. But it’s great.

It has a couple of annoyances, but beyond that, it’s something that I’m a big fan of, but I’d forgotten how transformative that is, compared to the usual approach of all right, here’s your username, here’s a password you’re going to have to change, here are your IAM credentials to store on disk forever. It’s the ability to look at what we’re doing through the eyes of someone who is clearly deep into the technical weeds, but not as exposed to all of the minutiae of the 300-some-odd AWS services is really a refreshing thing for all of us, just because it helps us realize what it’s like to see some of this stuff for the first time, as well as gives me content ideas because if it’s new to you, I promise you are not the only person who’s seeing it that way. And if you don’t really understand something well enough to explain it, I would argue you don’t really understand the thing, so it forces me to get more awareness around exactly how different facets work. It’s been an absolutely fantastic experience so far, from my perspective.

Alex: Thank you. Right back at you. I mean, spending so many years working with startups, my kind of level of expected sophistication is, “I’m going to write your password on the back of a napkin. I have fifteen other things to do. Go figure it out.” And so you know, it’s always nice to see—particularly players like AWS that are such 800-pound gorillas—going in and trying to uplevel that experience in a way that feels like—because I mean, like, look, AWS could keep us with the, “Here’s a CSV with your username and password. Good luck, have fun.” And you know, they would still make—

Corey: And they’re going to have to because so much automation is built around that—

Alex: Oh yeah—

Corey: In so many places.

Alex: —so much.

Corey: It’s always net-additive, they never turn anything off, which is increasingly an operational burden.

Alex: Yeah, absolutely. Absolutely. But yeah, it’s nice to see them up-level this in a way that feels like they’re paying attention to their customers’ pain. And that’s always nice to see.

Corey: So, we met a few years ago—in the before times—at a mixer that we wound up throwing—slash meetup. It was in Southern California for some AWS event or another. You’ve been aware of who we are and what we do for a while now, so I’m very curious to know—and the joy of having these conversations is that I don’t actually know what the answer is going to be, so this may never see the light of day if it goes to weird—

Alex: [laugh].

Corey: —in the wrong direction, but—no I’m kidding. What has been, I guess, the biggest points of dissonance or surprises based upon your perception of who we are and what we do externally, versus joining and seeing how the sausage is made?

Alex: You know, I think the first thing is—um, well, how to put this. I think that a lot of what I was expecting, given how much work you all do and how big—well, ‘you all;’ we do—and how big the list of clients is and how it gets bigger every day, I was expecting this to be, like, this very hyper put together, like, every little detail has been figured out kind of engagement where I would have to figure out how you all do this. And coming in and realizing that a lot of it is just having a lot of in-depth knowledge born from experience of a bunch of stuff inside of this ecosystem, and then the rest of it is kind of free jazz, is kind of encouraging. Because as someone that was you know, as a freelancer, right, who do you see, right? You see people who have big public presences or people who are giant firms, right?

On the GCP side, SADA Systems is a great example. They’re another local company for me here in Los Angeles, and—

Corey: Oh, yes. [unintelligible 00:24:48] Miles has been a recurring guest on the show.

Alex: Yeah. And he’s great. And, like, they have this enormous company that’s got, like, all these different specializations and they’re basically kind of like the middleman for GCP on a lot of things. And, like, you see that, and then you kind of see the individual people that are like, “Yeah, you know, I’m not really going to tell you that I only have two clients and that if both of them go away, I’m screwed, but, like, I only have two clients, and if both of them go away, I’m screwed.” And so, you know, I think honestly seeing that, like, what you’ve built so far and what I hope to help you continue to build is, you know, you’ve got just enough structure around the thing so that it makes sense, and the rest of it, you’re kind of admitting that no plan ever survives contact with the client, right, and that everybody’s going to be different than that everybody’s problems are going to be different.

And that you can’t just go in and say, “Here’s a dashboard, here’s a calculator, have fun, give me my money,” right? Because that feels like—in optimization spaces of any kind, be that cloud, or data or whatever, there’s this, kind of, push toward, how do I automate myself out of a job, and the realization that you can’t for something like this, and that ultimately, like, you’re just going to have to go with what you know, is something that I kind of had a suspicion was the case, but this really made it clear to me that, like, oh, this is actually a reasonable way of going about this.

Corey: We thought otherwise at one point. We thought that this was something could be easily addressed their software. We launched our DuckTools SaaS platform in beta and two months later, did the—our incredible journey has come to an end, and took it off of a public offering. Because it doesn’t lend itself to solving these problems in software in any reasonable way. I am ever more convinced over time that the idea of being able to solve cloud cost optimization with software at VC-scale is a red herring.

And yeah, it just isn’t going to work because it’s one size fits some. Our customers are, by definition, exceptional in many respects, and understanding the context behind why things are the way that they are mean that we can only go so far with process because then it becomes a let’s have a conversation and let’s be human. Otherwise, we try to overly codify the process, and congratulations, we just now look like really crappy software, but expensive because it’s all people doing it. It doesn’t work that way. We have tools internally that help smooth over a lot of those edges, but by and large, people who are capable of performing at especially at the principal level for a cloud economics role, inherently are going to find themselves stifled by too much process because they need to have the freedom to dig into the areas that are relevant to the customer.

It’s why we can’t recraft all of our statements of work in ways that tend to shy away from explicitly defined deliverables. Because we deliver an outcome, but it’s going to depend entirely, in most cases, up on what we discover along the way. Maybe a full-on report isn’t the best way of presenting the data in the way that we see it. Maybe it’s a small proof of concept script or something like that. Maybe it’s, I don’t know, an interpretive dance in front of the company’s board.

Alex: [laugh]. Right.

Corey: I’m open to exploring opportunities. But it comes down to what is right for the customer. There’s a reason we only ever charge a fixed fee for these things, and it’s because at that point, great, we’re giving you the advice that we’d implement ourselves. We have no partnerships with any vendor in the space just to avoid bias or the perception of same. It’s important that we are the authoritative source around these things.

Honestly, the thing that surprised me the most about all this is how true to that vision we’ve stayed as we’ve as we flushed out what works, what doesn’t. And we can distantly fail to go out of business every month. I am ecstatic about that. I expected this to wind up cratering into a mountain four months after I went freelance. Not yet.

Alex: Well, I mean, I think there’s another aspect of this too, right? Because I’ve spent a lot of my career working inside of venture capital-backed companies. And there’s a lot of positive things to be said about having ready access to that kind of cash, but it does something to your business the second you take it. And I’ve been in a couple of situations where, like, once you actually have that big bucket of money, the incentive is grow, right? Hire more people get more customers, go, go, go, go, go.

And sometimes what you’ll find is that you’ll spend the time and the money on an initiative and it’s clearly not working. And you just kind of have to keep doubling down because now you’ve got customers that are using this thing and now you have to maintain it, and before you know it, you’ve got this albatross hanging around your neck. And like one of the things that I really respect about the way that Duckbill Group is is handling this by not taking outside cash is, like, it frees you up to make these kinds of bets, and then two months later say, “Well, that didn’t work,” and try something else. And you know, that’s very difficult to do once you have to go and convince someone with, you know, money flowing out of their ears, that that’s the right thing to do.

Corey: We have to be intentional about what we’re doing. One of the benefits of bringing you aboard is that one, it does improve our capacity for handling more engagements at the same time, but it also improves the quality of the engagements that we are delivering. Instead of basically doing a round-robin assignment policy we can—

Alex: Right.

Corey: —we consult with each other; we talk about specific areas in which we have specific expertise. You get dragged into a lot of data portions of existing engagements, and the rest of us get pulled into other areas in which you might not be as strong. For example, “What are all of these ridiculous services? I can’t make heads or tails have the ridiculous naming side of it.” Surprise, that’s not a you problem.

It comes down to being able to work collaboratively and let each other shine in a way that doesn’t mean we load people up with work. We’re very strict about having a 40-hour or less work week, just because we’re not rushing for an exit. We want to enjoy our time working, we want to enjoy what we’re doing, and then we want to go home and don’t think about work until it’s time to come back and think about these things. Like, it’s a lifestyle company, but that lifestyle doesn’t need to be run, run, run, run, run all the time, and it doesn’t need to be something that people barely tolerate.

Alex: Yeah. And I think that, you know, especially coming from being an army of one in a lot of engagements, it is really refreshing to be able to—see because, you know, I’m fortunate enough, I have friends in the industry that I can go and say like, “I have no idea how to make heads or tails of X.” And you know, I can get help that way, but ultimately, like, the only other outlet that I have here is the customer and they’re not bringing me in if they have those answers readily to hand. And so being able to bounce stuff off of other people inside of an organization like this has been really refreshing.

Corey: One of the things I’ve appreciated about your tenure here so far is the questions that you ask are pitched at the perfect level, by which I mean, it is never something you could answer with a three-second visit to Google, but it’s also not something that you’ve spent three days spinning your wheels on trying to understand. You do a bit of digging; it’s a little unclear, especially since there are multiple paths to go down, and then you flag it for clarification. And there’s really so much to be said for that. Really, when we’re looking for markers of seniority in the interview process, it’s admitting you don’t know something, but then also talking about how you would go about getting the answer. And it’s—because no one has all this stuff in their head. I spend a disturbing amount of time looking at search engines and trying to reformulate queries and to get answers that make sense.

I don’t have the entirety of AWS shoved into my head. Yet. I’m sure there’s something at re:Invent that’s going to be scary and horrifying that will claim to do it and basically have a poor user interface, but all right. When that comes, we’ll reevaluate then because this industry is always changing.

Alex: For sure. For sure. And I think it’s, it’s worth pointing out that, like, one of the things that having done this for a long time gives you is this kind of scaffolding in your head that you can hang things over. We’re like, you don’t need to have every single AWS service memorized, but if you’ve got that scaffold in your head going, “Oh, like, this thing sounds like it hangs over this part of the mental scaffold, and I’ve seen other things that do that, so I wonder if it does this and this and this,” right? And that’s a lot of it, honestly.

Because especially, like, when I was solely in the data space, there’s a new data wareho—or a new, like, data catalog system coming out every other week. You know, there are a thousand different things that claim to do MLOps, right? And whenever, like, someone comes to me and says, “Do you have experience with such and such?” And the answer was usually, “Well if you hum a few bars, I can fake it.” And, you know, that tends to help a great deal.

Corey: Yeah. “No, but I’ll find out and get back to you,” the right answer. Making it up and being wrong is the best way to get rejected from an environment. That’s not just consulting; that’s employment, too. If 95% of the time, you give the right answer, but that one time and 20 you’re going to just make it up, well, I have to validate the other 19 because I never know when someone’s faking it or not. There’s that level of earned trust that’s important.

Alex: Well, yeah. And you’re being brought in to be the expert in the room. That doesn’t necessarily mean that you are the all-seeing, all-knowing oracle of knowledge but, like, if you say a thing, people are just going to believe you. And so, you know, it’s beholden on you—

Corey: If not, we have a different problem.

Alex: Well, yeah, exactly. Hopefully, right? But yeah, I mean, it’s beholden on you to be honest with your customer at a certain point, I think.

Corey: I really want to thank you for taking the time out of your day to got with me about this. And I would love to have you back on in a couple of months once you’re fully up to speed and spinning at the proper RPMs and see what’s happened then. I—

Alex: Thank you. I’d—

Corey: —really appreciate—

Alex: —love to.

Corey: —your time where’s the best place for people to learn more about you if they haven’t heard your name before?

Alex: Well, let’s see. I am @alexras on Twitter, A-L-E-X-R-A-S. My personal website is alexras.info.

I’ve done some writing on data stuff, including a pretty big collection of blog posts on the data side of the AWS ecosystem that are still on my consulting page, bitsondisk.com. Other than that—I mean, yeah, Twitter is probably the best place to find me, so if you want to talk more about any weird, nerd data stuff, then please feel free to reach out there.

Corey: And links to that will, of course, be in the [show notes 00:35:57]. Thanks again for your time. I really appreciate it.

Alex: Thank you. It’s been a pleasure.

Corey: Alex Rasmussen, principal cloud economist here at The Duckbill Group. I am Corey Quinn, cloud economist to the stars, and this is Screaming in the Cloud. If you’ve enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you’ve hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry, insulting comment that you then submit to three other podcast platforms just to make sure you have a backup copy of that particular piece of data.

Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

Announcer: This has been a HumblePod production. Stay humble.