Join Corey and Adithya as they discuss what it’s like to build a startup that operates in a heavily regulated industry on serverless architecture, Adithya’s experience as being hired as the first developer at Branch Insurance, how serverless helps remove all sorts of tasks from the average developer’s plate and reduces the barrier of entry, how going serverless locks you into the AWS ecosystem at least for the near term and why that’s not necessarily a bad thing, why the Branch Insurance team uses CloudFormation instead of Terraform, and more.
- Branch Insurance Main: https://ourbranch.com/
- Twitter: https://twitter.com/TheTallpants
- GitHub: https://github.com/tallpants
TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Cloud Economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.
Corey: This episode is brought to you by Trend Micro Cloud One™. A security services platform for organizations building in the Cloud. I know you're thinking that that's a mouthful because it is, but what's easier to say? “I'm glad we have Trend Micro Cloud One™, a security services platform for organizations building in the Cloud,” or, “Hey, bad news. It's going to be a few more weeks. I kind of forgot about that security thing.” I thought so. Trend Micro Cloud One™ is an automated, flexible all-in-one solution that protects your workflows and containers with cloud-native security. Identify and resolve security issues earlier in the pipeline, and access your cloud environments sooner, with full visibility, so you can get back to what you do best, which is generally building great applications. Discover Trend Micro Cloud One™ a security services platform for organizations building in the Cloud. Whew. At trendmicro.com/screaming.
Corey: Normally, I like to snark about the various sponsors that sponsor these episodes, but I'm faced with a bit of a challenge because this episode is sponsored in part by A Cloud Guru. They're the company that's sort of famous for teaching the world to cloud, and it's very, very hard to come up with anything meaningfully insulting about them. So, I'm not really going to try. They've recently improved their platform significantly, and it brings both the benefits of A Cloud Guru that we all know and love as well as the recently acquired Linux Academy together. That means that there's now an effective, hands-on, and comprehensive skills development platform for AWS, Azure, Google Cloud, and beyond. Yes, ‘and beyond’ is doing a lot of heavy lifting right there in that sentence. They have a bunch of new courses and labs that are available. For my purposes, they have a terrific learn by doing experience that you absolutely want to take a look at and they also have business offerings as well under ACG for Business. Check them out. Visit acloudguru.com to learn more. Tell them Corey sent you and wait for them to instinctively flinch. That's acloudguru.com.
Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this week by Adithya Reddy, the first developer hire at Branch Insurance. Adithya, welcome to the show.
Adithya: Hi, Corey, nice to meet you.
Corey: Well, always a pleasure. So, you're working somewhere fascinating, specifically, Branch Insurance, which despite all assurances to the contrary, I refuse to accept is not what developers need to get for when they really screw up a merge. So, is that roughly what you do, or is there a bit more to it than that?
Adithya: I would love if that was actually what we did. I would personally be the highest paying customer for that. But no, we are an [00:03:12 insurance startup], and we sell bundled home and auto insurance in a bunch of US states, list growing. And what sets us apart from a technology sense is that we've been serverless from day one. We started out servers, we are still today entirely serverless, we have never had a instance running. And it's an interesting perspective, just technically, about how that works in a industry like insurance, where that’s definitely not what you expect.
Corey: It's not, in fact. When you hear about a company that is, “Oh, we are all serverless, never had an instance running from the very beginning.” You think, “Oh, so what do you do?” “We're Twitter for Pets, which is just like regular Twitter, only somehow eighty times less racist.” But you're a [00:04:01 insurance startup], which is synonymous, for most of us, with regulated industry, which means not—shall we put this—the most avant-garde technology selections, if for no other reason than explaining it to regulators and auditors becomes a more than full-time job.
I mean, I had this problem back when I was working in FinTech startups, where we'd have auditors rock in and ask, “Great, so where's the Active Directory credentials?” “Yeah, we don't really have one of those.” And they looked at us like we were out of our minds. For all I know, maybe we were, but what has your experience been when talking to other folks about, “Oh, we're in the [00:04:39 insurance space], but we're also full serverless.” Is there a disconnect there?
Adithya: Most people, when I say that we work in insurance. It's just like, “So, you write PL/SQL queries, and you have stored procedures, and the best you could possibly hope for is lots and lots of Java.” But it's that assumption, but it's far from the reality of what needs to be done, and—
Corey: Oh, let's be fair, I'm sure there are some [00:05:05 insurance companies] that are all about the .NET instead.
Adithya: Oh, definitely. There's a lot of diversity in the space with technology.
Corey: [laughs]. It's one flavor of big enterprise software versus a different flavor of big enterprise software. But again, as you mentioned, you're an [00:05:18 insurance startup], and the idea of approaching an industry like that, that is-frankly—so incredibly rife, or in need of disruption, with a—frankly—bleeding edge architecture, I’m sorry, I just sort of look at this from a perspective of you can't help but admire the audacity of that.
Adithya: Yeah, I mean, we've had our own challenges, but I would say overall, I am pretty happy that this is what we've chosen, and just the benefits that we've seen from this, I don't think we would be able to replicate what we've built so far if we were just using the architecture that you would read about on Twitter, which is, like, a giant Kubernetes cluster and lots of Java apps. And just for some perspective, we’re pretty small. I think when we launched, we were two engineers inside the company, plus three people helping us out from Uruguay. And right now, we've just added a bunch more people, but we're still about 10 software developers, and we've managed to launch a complete insurance product that you can buy from five states. Everything works and people, kind of, don’t—they're not able to reconcile insurance plus a tiny team that builds stuff faster. And I would say probably the biggest thing that's enabled that for us is just not having to worry about a lot of things that serverless takes care of for us.
Corey: Now, as you mentioned, you're 10 engineers, and you have an established track record. Your founder, Joe is a known entity on the stage constantly at Serverlessconf. The hardest part is getting him off the stage long enough to have a conversation with him. And now it's a known thing, but you were the first engineer hired at Branch, and what was that like walking in the door? Because my immediate knee jerk response to, “Hey, we're doing this very legacy, very controlled thing in a brand new way, architecturally,” it sounds like Hacker News broke loose somewhere and is now dictating architecture. Today, it makes an awful lot of sense, but grinding back the clock a few years when you first started, did it seem as nuts then as it sounds like?
Adithya: So, there were a lot of very odd things about how I ended up here. So, for some backstory, I was in my final year of college in 2018, and I was like, “Okay, I need to find something to do now or I'm going to be broke.” So, there was a thread on Twitter—I forgot who it was—and it just said, if you’re a new grad looking for a job, put your resume here, we’ll critique it for you. So, I put it up and I got a DM from Joe who said, “Yo, I'm looking for an intern, a front-end development intern, would you be interested?” So, the whole thing was very suspicious because that's not the kind of offer you get on Twitter DMs, and then as I found out more about it, it's like, “Oh, we're going to build an [00:08:10 insurance startup].” And everything was just very fishy about it. But I thought, I'm out of college now. If I'm going to do something stupid, now’s the time.
Corey: Yeah, ridiculous job beats no job.
Adithya: Yeah. So, I said, “Yeah, let's go for it.” And I was an intern at Branch for six months. And at that point, I mostly just [00:08:30 unintelligible] the front-end stuff. So, I would build the front-end UI, I did some React Native stuff for our mobile app, and I did some weird UI sketches in Balsamic, and all that convinced me was that I shouldn't be designing UI.
And it just grew into a proper company, and everything just sort of came together, and that wasn't really something that I expected. So, I mean, now I'm really happy that's how it's all come together, and we have a great company running right now. But it was just very strange to me that we're building an [00:09:02 insurance startup], and also when I joined it’s like, “Oh, we're all serverless.”
And the day I got there, I asked, “Okay, so I'm going to build your front-end now. Where's the API?” And he said, “Oh, there's no API. It's, like, GraphQL.” And I said, “Well, what's GraphQL?” That was a big rabbit hole. And then I said, “Okay, so where is the back end running?” And he’s like, “It's not running anywhere. It's serverless.” And that was another six-month journey for me to discover that all this stuff that I had learned in college was not as mandatory as they made it sound.
Corey: I had to go through that same transformation myself, only instead of college—because some of us never went, or at least never graduated—it was, “Well, just unlearn the last 15 years of your career. It's fine.” And that 15 years of the career was spent on things that are now largely irrelevant in a serverless world, but they were the entire core of what I did. So, for an awful lot of folks looking at this with suspicion and distrust, it's not even so much a problem with the architecture or the constraints itself, but rather than it sort of cuts at the core of our identity. For the longest time, I was a Unix/Linux server administrator, and now there are no servers for me to administrate. What does that make me? It's the evolve or die dinosaur type of paradigm. And I won't lie, it's scary.
Adithya: Oh, yeah, definitely. I mean, there's still a lot of [00:10:19 unintelligible] questions that I get from people who I talk to, and I tell them this is what our architecture looks like. And, “Well, where’s your DBA, where’s your on-call, where’s your DevOps person? Who manages the infrastructure?” And it's just… Amazon does it because they do it better than I could, so why would I want to take on that load for myself?
And a lot of people seem to have just gone in the opposite direction of I want to [00:10:43 unintelligible] everything, and to me now with the perspective that I have at Branch, that seems just a tremendous waste of time, and also just a tremendous amount of stress and responsibility you take on yourself. Why would I try to do what Amazon has been doing for a decade now, and they have people who are paid much more than I would be able to pay someone to do it, and they're going to do a better job than me? So, why would I just not leave that up to them, and I will do what I'm good at, and I will do what my company needs? So, the big focus, the big shift, for me, has been coming out of school as I'm an engineer, and I do tech stuff, and that's what I care about, and it's all about the technology. And it's changed to, “What does the business want?” and, “What do we want to build for our customers?” And if we can pay someone else to take care of this stuff for us, and we can focus on the product that we're building and what our customers want, then that's a net win for me.
Corey: It really gives you the opportunity to focus, on some level, on the things that differentiate your business, that lead to success. I spent untold weeks or months of my life in aggregate now, configuring and building load balancers, but I never worked at a company whose job was to build the load balancer, it was always to do something thing else, be it, I don't know, help with expense reports or help dogs tweet to one another. It was never aimed at, I’m working at a load balancer company. Like, yeah, if you work at F5, you probably should spend a fair bit of time building load balancers, but for the rest of the world that is no longer a differentiator or internal core competency that you need to have in most companies. There are always exceptions, and they are always going to be these expressions of those core competencies.
I'm a big believer in the idea of the T-shaped engineer, where you should be broad across a wide variety of different things and deep on one or two areas. I started off being deep on email systems because every company back then needed an email administrator, and I thought it was fascinating and the closest thing I'd ever seen to magic. Now I look at this through a lens of, huh, it was pretty clear in retrospect that fewer and fewer companies ran their own email servers, so this was not going to be a job for most companies. Instead of every company needing dedicated email people, there were a couple dozen and that was it. I wanted to go with something that had a bit more mass-market appeal. And one of the neat things about serverless is that it seems to me at least, that it's making every engineer, to some extent, step through that gateway.
Adithya: Oh, yeah, definitely. I mean, I think one of the things that won me over about this whole architecture is that I came out as a front-end developer, right. So, that's what I do: I want to write react apps. And what was interesting to me is that Joe was like, “That's fine. You don't have to learn this stuff.”
And it's just been… it kind of moves the barrier to entry quite a bit back. I wouldn't say it removes it entirely; there's still like—you know, you have to be an engineer. It's not a no-code thing, but it makes it so that you will, as a front-end developer suddenly have a lot more power, so to say, about what you can build without having to rely on I need the DBA, and I need someone who knows Java really well, and I need someone who can deploy things to clusters and knows about Linux. It's just removing a lot of those things has made it so that we've hired engineers who have just been purely front-end, and then in two months, they're writing CloudFormation and setting up SQS queues and reading messages off it. It's just been really, really refreshing in that sense.
Corey: One of the things that you're almost certainly going to hear from the Hacker News ‘well actually’ brigade is, “Ah, but by going with serverless, you have now locked yourself into AWS. If you were to want to port Branch Insurance over to GCP, or Azure, or to your burning dumpster fire of your own data center, you're going to have an awful lot of work to do that, so you've made a grievous error, and now you are beholden to whatever Amazon decides to do with you.” How do you feel about that?
Adithya: I mean, initially, I kind of bought into it. You find new young developers and put them on Hacker News, and just, they’re going to absorb everything, and the end result is not going to be pretty. Luckily, I had a lot of people who—yeah, no, just don't fall into that clique, let's say, of Hacker News. And the realization for me about lock-in has been, yes, you're not going to leave Amazon anytime soon, but then I have never seen a company who were like, “Yeah, we're going to leave Amazon for Google because Amazon has decided to screw us by raising their prices.”
And as far as I know, I don't think Amazon has ever raised prices on anybody. And that's always been a constant: how do we make things cheaper, and sometimes far cheaper. And also the lock-in, there’s this assumption built into that statement of, “If we don't use serverless, we are not tied to Amazon at all, and we can just leave at the drop of a hat,” which is not true. There's a lot of stuff set up, implicitly, maybe not documented very well, but there’s just a lot of stuff going on with the way you set up your clusters on Amazon, on the way you use EC2, or the way you set up your load balancers, or your network, or DNS, and you're not going to be able to port all of that to Google, or Azure tomorrow. So, we're just talking about degrees of how hard is it going to be to migrate, notwithstanding the fact that it's very, very unlikely that you will ever have to migrate.
And then also, it becomes… if you work and build apps with the mentality that I need to be able to switch this away to other providers at the drop of a hat, you're locking yourself up in saying that I will only use the lowest common denominator of the services. And you're just making life harder for yourself, so why would you make the 99 percent case much harder to solve for the 0.1 percent case of, “Oh my god, Amazon is screwing us and we need to leave and go to Azure right now.”
Corey: In what you might be forgiven for mistaking for a blast from the past, today I want to talk about New Relic. They seem to be a relatively legacy monitoring company, and I would have agreed with that assessment up until relatively recently. But they did something a little out there: they reworked everything. They went open source, they made it so you can monitor your whole stack in one place and, most notably from my perspective, they simplified their pricing into something that is much more affordable for almost everyone. There's even a free tier with one user and 100 gigs per month, totally free. Check it out at newrelic.com.
Corey: I agree with everything you just said, but by hearing someone who's actually built something on top of these technologies, it comes across as way more genuine than if I sit here on my pedestal of thought leadership, which is, turns out, not a real thing, and then go and tell everyone that, “Oh, this is what people should or should not do.” Hearing people who have made actual business decisions around this is one of the most authentic ways to make the point. I keep looking for people to take the other side of it, “Oh, no, we actually had to build something in a way that was portable to other providers.” But you never see them taking advantage of that capability. It always, for whatever reason, comes down to a baseline model of, “Well, we built this optionality in, and kept everything agnostic, and then we feel better about running it all in AWS,” or GCP, or Azure, wherever they happen to be running.
But it's an optionality that people never take advantage of. So, my position is, go all-in on a provider—I don't care which one—and see what happens. If you have to move later, at least you've survived long enough to get to that point of having to make the pivot, rather than spending all of your time working on these agnostic shim layers, rather than building out features that will move your business, and get to the next milestone.
Adithya: I mean, I agree with that. I think one thing that people don't realize is that you might think you're building an agnostic application, but I can almost guarantee that at 99 percent of cases if you have to move your cloud provider, you're going to find out it wasn't really as agnostic as you thought it was.
Corey: That's one of the things that I find so… I guess ridiculously strange. As far as these people talking about, “Well, what if? What if everything changes, and we have to pivot massively? Well, what if AWS shuts down?” Okay, let me back up for a second. If AWS shuts down, are any of us going to keep our existing jobs or, far likelier, are we going to instead pivot to helping all of these companies with big problems and bigger pocketbooks migrate off as fast as they can? That is such an unrealistic outcome, that I can't quite fathom. It comes down to where I want to spend my time. I don't really think that hedging the risk of this Cloud thing being just a giant fad is the best use of my time anymore.
Adithya: That's true. It also just comes down to the probabilities. What is the probability that you are going to run out of money trying to build everything yourself, versus the probability that AWS is just going to decide that, “You know what? Screw you, we're going to raise all our prices.” And it’s just—I feel like a lot of people just want to tinker with cool stuff: they want to tinker with load balancers and play around with Linux, and that's fine, but it's the fact that I'm going to justify this with a business reason that I really don't get. That's very strange to me.
Adithya: So, I think there's a couple of reasons for it. The primary reason that we started off with CloudFormation is that we have a really excellent partner in Stackery, who support automation really well, and they give us a lot of help with our tooling. And a lot of times it's been, we need this feature, and we ask Stackery like, “Hey, how do we do this?” And they say, “Oh, no, we can't do that yet, but thank you for pointing that out. We will build it for you.” And we have it two weeks later.
So, that was the reason we started with CloudFormation. And then, as time has gone on, we realized that it's a lot simpler to do things with CloudFormation—if you're primarily being serverless—on AWS, just because of SAM and the way that AWS is supporting it. And they're doing a lot of work by creating these specific serverless resources of an AWS::Serverless::Function. So, having that kind of higher-level abstraction over the base CloudFormation, which encapsulates all these common patterns of here's one resource you can declare that will create IAM roles for you, and also our CLI is going to package up your code and upload it, and then set up the function running.
And you have things like an HTTP API resource, which would be very, very hard to set up manually, but because of this kind of thing where Amazon has seen the patterns that are generally prevalent with serverless applications and made it much easier to use and they seem to keep iterating on it, that's been a big plus for us. And also just having the state of your application be managed by Amazon itself, like the rollbacks, and updates. And Terraform’s kind of coming there now with their Terraform cloud offering, but that's how we started, and now it’s… maybe Terraform can do some of what it does, but I would much rather bet on CloudFormation in the sense. Of course, if you're doing this on not AWS, you probably might have a different answer, and you should do your own research and come to your own conclusions.
Corey: One of my favorite things is watching people, I guess, incorrect each other on oh, well, you should only ever use one or the other, or some other system because of this one weird edge case, that it turns out hasn't been true in three or four years. But as a counterpoint, I have problems personally with JSON. Specifically, it doesn't parse as well as YAML because I have a Python background and whitespace messing things up for me has really been my entire career. So, back originally, CloudFormation only supported JSON. And then one day, bam, YAML showed up. And this was awesome. And this was great.
And one of the problems I saw for the longest time was that all of the examples, all of the blog posts, all of the stuff people had written, it didn't really matter because all these examples in this giant history with CloudFormation had been written in JSON. So, it took time for there to start to be public examples of what the YAML version of it would look like. And I'm wondering, first, if you've encountered that, and secondly, if that seems like it's a bit of a carry-forward into other areas where a new feature comes out, for example, but all the old examples don't demonstrate how that feature works.
Adithya: I mean, I've seen my fair share of—I Google, how do I do this? And then it's a bunch of JSON. And I personally think the JSON representation of CloudFormation is pretty hard to read. There's no comments, there's no lots of flex spacing, you can't really differentiate things for the reader. So, luckily, if I just copy this JSON and paste it into a JSON to YAML converter, it does have to work.
It's not going to be accurate, but it's useful enough that I can write my own YAML now, looking at that. Or you could just write your own YAML looking at the JSON. But I think in general, this is something that most developers have to pick up as a skill of how do I differentiate old information, and how do I not fall into the trap of reading old blog posts, or old books and thinking, yeah, this is how it is now, just because this field changes quite a lot. And this is just the general skill that you need because a lot of stuff on the internet when it comes to anything related to software is going to be outdated, and you need to figure out what resources that you need to use, and also trust the authoritative source of Amazon's documentation first, and only then go around looking for supplemental information, let's say. And also be very suspect, look at the date of the article before you’ll start to read it, and don't take anything you read as gospel because it's quite possible that the thought leader whose blog post you're reading also might have been wrong about something.
Corey: Oh, my stars, yes. One of the hardest challenges for me when I'm putting together my newsletter every week is I see a lot of things come across the wire in terms of blog posts that touch on what the community is doing. I have to vet these because some of them are not just poorly written but actively harmful. I saw one somewhat recently that’s, “Oh, you can save money by moving all of your S3 long term stuff into Glacier Deep Archive.” Well, yes, you'll save some money, but there are some serious constraints around that, around access times, how it's built in certain ways, and it didn't even hint at some of those. If you blindly go ahead and click through and, “Sure that sounds awesome.” It's not going to go well for you. You want to make sure that the thing you're applying is relevant, and it takes your constraints into consideration. In some cases, it feels like people write things that don't even take their own constraints into consideration. It's, more or less, Hacker News fanfiction.
Adithya: Oh, yeah. I mean, you see this a lot in Hacker News specifically. You go look at a post about, “Oh, look at this cool new thing I built.” And you look at the comments and it's all, “Oh well, doesn't this have this issue? And doesn't this have this bug?” “Oh, this was fixed in 2006.”
But it's just, you read it once, and now you just—I mean, it's a general thing in software, where you read it once and you just assume, yeah, this is a constant. This is not going to change, but stuff changes all the time, and you always have to approach everything you read in this industry with a lot of skepticism about, is this still up to date? Does this person know what they're talking about? Even if this person does know what they're talking about, have they made a mistake with this? Do your own research.
And also, it applies to you, too. Know that your own research isn't infallible, and you're going to make mistakes, and when someone else points that out to you—in a blog post, let's say—don't go to the Hacker News comments for that blog post and point out the person writing it is a really stupid idiot and they don't know what they're talking about.
Corey: Oh, absolutely. People don't know how to, in many cases, ask questions or answer them. I tend to assume good faith, and I find that I am pleasantly surprised way more often than I am negatively surprised when I do that. I assume that people are asking a question that they've tried the basic things, I assume that they're asking in good faith, not trying to prove a point, and I assume that when someone says, I'm looking to do X with Y technology, that the correct answer is not, “Why technology sucks.” There has to be a reason that they're asking this question. Helping, in some cases, clarify and refine the question is a better path than assuming they're a moron and lighting into them. I wish more people took that perspective if I'm being perfectly honest and grouchy about it.
Adithya: [laughs]. I couldn't agree more.
Corey: So, if people want to hear more about what you have to say, where can they find you?
Adithya: You can find me on Twitter. You can follow me on GitHub. I should start a blog sometime soon, but I just never get around to it.
Corey: The story of our lives. You are @TheTallpants on Twitter. And we'll put a link to that in the [00:28:49 show notes], of course. So, Adithya, thank you so much for taking the time to speak with me today. I really appreciate it.
Adithya: Great. Thank you. Thank you so much. Thanks for having me.
Corey: Of course. Thank you for taking the time. Adithya Reddy, first engineer hired at Branch Insurance. I am Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on Apple Podcasts, whereas if you've hated this podcast, please leave a five-star review on Apple Podcasts and file your complaint in both JSON and YAML.
Announcer: This has been this week’s episode of Screaming in the Cloud. You can also find more Corey at ScreamingintheCloud.com, or wherever fine snark is sold.
This has been a HumblePod production. Stay humble.