Periodically I get someone excited to talk to me about their new AWS bill optimization tool. This is the shorthand “cheat sheet” that addresses how I think about these things.
It looks like you’re building a SaaS tool to manage AWS cost optimization. I don’t think it will work. Here’s why I don’t think it will work:
- Customers will not put up with it
- If you’re building something hard to use, more complex or less fully-featured than the AWS billing console itself, or require the customer to know as much as you do about the space, it won’t work. It’s a nuanced and difficult area, and (crappy billing console to the contrary) incredibly hard to get right for all customer profiles.
- Customers care less about the big bill than they do about the fact that it can vary wildly from month to month; their pain is in predicting it rather than lowering it
- Customer employees in some cases embezzle more money in office supplies than they spend on the AWS bill. Payroll dwarfs the infra bill almost everywhere; the problem is not that the numbers are big, it’s that they’re unpredictable. Restating earnings because of the cloud bill is a terrifying concept to CFOs, and it’s very easy to mishear that as “the bill’s too high.”
- AWS will not put up with it
- AWS exhibits a cultural distaste for “percentage of bill” models, for even slight implications that the cloud is too expensive, and for me in general. That said, if you’re building a product that integrates tightly with their bill, you absolutely need and want them on your side. “We’ll save you money by migrating workloads transparently to other cloud providers” will irritate them–apart and aside from the fact that this almost never works in the first place.
- Your tool can’t tell the difference between idle EC2 instances and a hot DR site
- To an API, “this instance is idle because it’s not doing anything” and “this instance is idle because it will have roughly three seconds of warning before it has to take production traffic” look identical. If you suggest turning off the DR site in order to save money, the customer will suggest you get the hell out of their office.
- Buying Reserved instances or Savings Plans for a cluster that’s getting turned off in two weeks will get you hit with a belt by your customer
- Reserved Instance purchases are inherently scary. You’re committing to use some level of resources in a given region for a fixed period of time. If your tool lacks situational awareness of temporary workloads and other business constraints, you’re going to be asking AWS for a refund on the reserved instance purchase on behalf of the customer. Should they say no, you’re eating that cost. Did I mention you want AWS on your side?
- The police will not put up with it
- “We’ll make money by mining cryptocurrency with your idle cycles” sounds great in your dorm room, but when a division of a financial institution rolls your offering out and their security team sees it, you’ll be amazed by how quickly law enforcement, compliance, legal, and AWS all come down on you. Yes, I’ve seen several companies propose this – apparently with a straight face. I suppose it’s a good thing to hold in my back pocket for those rare engagements wherein I want my buyer to go to prison at the end.
- Customers instinctively recoil at paying you a percentage of their bill
- People begrudgingly have come to accept a variable cost every month with regard to their cloud spend. If you want to charge them a variable derived from that fluctuating cost, their finance folks will hate you. It works mathematically, but not psychologically. You also get to spend time in the weeds of “you want us to pay you a percentage of what we spend for support / CloudFront / 3 exabytes of data in Glacier Deep Archive that won’t ever change again?”
- Charging customers a percentage of savings means you need to be able to audit their books and bill them accurately, a process that is doomed from the start
- It’s tempting to charge a percentage of what you save the customer. This falls apart in a few ways. First, you’re tied to customer implementation cycles that can stretch to years. Second, you’re now saying you need to be in a position to audit their cloud bills for a period of time that may well extend past your product’s engagement. Third, the contract terms get incredibly complex (“We turned that service off three months in, so you don’t get to claim it anymore!”). Fourth, customers will be very angry at some of your findings (“We will absolutely not pay you $8 million for telling us to buy that list of reserved instances, I don’t care WHAT we signed.”)
- You built a dashboard that requires three years of AWS operations experience to understand–and display it to accountants
- It turns out that accounting’s collective understanding of the AWS bill trends less towards a nuanced understanding of cloud products, and more towards “Amazon is an online bookstore.” And this is fine! You’re not going to train accountants to understand engineering constructs in any reasonable period of time–particularly as a third party vendor.
- You’re displaying time-value-of-money calculations to engineers
- Accounting is a nuanced and complex field. If you’re expecting a team of engineers to understand TVM, amortization tables, or whether treating a subset of the bill as CapEx makes sense (it absolutely does not), you are attempting to ice skate uphill.
- You’re charging a fixed fee to customers that’s too much for a tiny startup, yet makes you look like a joke to a Fortune 500 customer
- If you charge my Twitter for Pets side project $5,000 a month to optimize my AWS bill, I will laugh you out of the room–that’s more than the bill itself. If you attempt to charge a Fortune 100 company $5,000 a month to optimize their AWS bill, they will laugh you out of the room–that’s a terrific indicator that you’re not serious, or that you’re not cognizant of the scale of their operations.
- Your tool requires three full days of training to use effectively
- This may astonish you, but people have full time jobs that don’t distill down to “working with your product.” It gets worse–talk to any existing company in this space (or someone on the AWS billing team if you can find one with a tendency to overshare when drinking) and see just how many users at a customer site log in to look at the bill–and how often they do it.
- You’re asking customers to decide on 15 variables to buy reserved instances, then click a button to spend $4 million
- Nobody is at all certain what the right RI purchase is to make, and the sheer number of variables that go into it mean the discussion becomes nuanced. If you abstract all of that away, you’re asking a customer to take your recommendation on blind faith and agree to spend enough money to launch five startups through your product. Oh, and you don’t want to know what happens if you get any of the variables wrong…
- There are fifteen companies like you and you aren’t differentiated
- “Ah, none of the existing tools are like us because we–” if you don’t have a tremendous second half of that sentence that fits in a tweet, give up now. The space is crowded already, and people are tired of hearing about yet another tool that does the same things everyone else talks about. That said, if your second half of the sentence is “…have a marketing budget that rivals the GDP of a small to mid-sized country” I withdraw the criticism entirely and direct your attention to my sponsorship offerings!
- There’s even a slight chance that the cost savings things your tool does could take down services
- Everyone cares about the bill, right up until a cost-cutting measure takes the site down. That’s the kind of narrative that gets people fired on the spot, and companies sued. You do have business insurance, right?
- One feature release from AWS Billing and you don’t have a business plan anymore
- If the entire value proposition you have to offer is “we have a gorgeous interface to the billing system,” then first–you’re right. The AWS billing system is indeed terrible. That said, to fix the UI takes a feature team one day pushing it out, possibly without even a blog post–and then you’re competing against something that’s both native to the platform, as well as free. You do you, but I’d not want to go to sleep at night with that sword of Damocles hanging over my head, however unlikely AWS getting better at user interfaces is.
- You’re focusing on an esoteric part of the problem
- So you built a system that schedules developer instances / arbitrages the RI secondary market / drives Spot fleet adoption. Great! There are a lot of niche ways to save money, but when your solution gets rolled out and saves a customer $10K a month when they’re bleeding more than that per day due to poor RI management, it’s not sustainable. More directly, when someone is brought in to fix the problem systemically (HELLO!) and highlights the things you missed, your customer will not be delighted.
- Machine Learning® doesn’t mean what you think it does.
- “We use AI/ML to–” just stop, please. I’m not a VC, I’m not trying to raise a funding round, so let’s call it what it is: math. You can have all kinds of predictive analytics around customer behavior, but it won’t catch someone spinning up a big cluster to do a single import job, it won’t understand that a service is set to be deprecated, and it will fail to grasp context. Think about Siri, Alexa, Cortana, or any other robot person that lives in your phone. If you ask it to do something and it gets it wrong, you feel foolish. You aren’t going to go back and ask a second time. That feeling / outcome intensifies in the finance operations space. After a few recommendations that miss the mark, people don’t trust your recommendations anymore, and they stop caring what you have to say.
- Lambda bills don’t matter.
- Every time I see a Lambda bill in the thousands, I see EC2 bills in the hundreds of thousands or millions. Lambda is fascinating, and there are a lot of opportunities to display cost information, to optimize spend, to chart the flow of capital through your organization–and a complete lack of customers who have this problem at any meaningful level of scale. I’ve tried some of these systems myself. They “helpfully” tell me that this month’s Lambda bill is going to be 12¢, and it’s trending down 2¢ from last month.
- You want too much trust from your customers
- So after the deal is signed, you expect customers to fork over a set of credentials that have view access into all of their data, their full cloud spend financials, the architecture of their next generation unannounced service, and be happy about it? Worse, you want them to install a CloudFormation stack they likely don’t understand that spins up a bunch of Lambda functions, IAM roles, and the like? Don’t get me wrong, some customers will happily sign over the keys to the kingdom–but without very specific, very carefully bounded answers to questions from more operationally mature companies, you’re likely to be laughed out of the room.
- You want too much work from your customers
- Nobody has “save money on the bill” or “understand their cost attribution story” as a top level priority. The AWS bill is generally something that crops up as unplanned work. If you require your customers to apply tags globally, spend three days in training to learn your offering, or redeploy all of their existing applications, then their adoption will remain low and they’re unlikely to remain your customer for very long – provided their initial test gets off the ground in the first place.
So, if you can solve all of those problems with a single tool? Great, please come and talk to me. I should like to know about it.
