--- title: "Why Your CPU-Based Utilisation Metric is Absolute Nonsense" id: "14184" type: "post" slug: "why-your-cpu-based-utilisation-metric-is-absolute-nonsense" published_at: "2023-09-13T14:30:00+00:00" modified_at: "2023-09-12T22:24:35+00:00" url: "https://www.lastweekinaws.com/blog/why-your-cpu-based-utilisation-metric-is-absolute-nonsense/" markdown_url: "https://www.lastweekinaws.com/blog/why-your-cpu-based-utilisation-metric-is-absolute-nonsense.md" excerpt: "Picture this: You’re in your swivel chair, feet propped up on your standing desk because you are a glorious acrobat, and you’re looking over your company’s Amazon EC2 fleet utilization report. You’re captivated by the custom colorful dashboard, carefully tuned..." taxonomy_category: - "Uncategorized" --- Picture this: You’re in your swivel chair, feet propped up on your standing desk because you are a glorious acrobat, and you’re looking over your company’s Amazon EC2 fleet utilization report. You’re captivated by the custom colorful dashboard, carefully tuned to a 1st-grade reading level. You see the overall number in its soft, non-threatening font, and you say to yourself, “We’re operating at 70% capacity — we’re golden!” That metric is usually nothing more than a feel-good security blanket that doesn’t give you better insight into the efficiency of your spend. Why? Because the number at the top of your report is simply your CPU utilization, which is being passed off as a standalone metric for fleet utilization. It’s the cloud’s equivalent of ordering a pizza based on the box’s size, without giving a second thought to what’s actually inside. What’s more disturbing is the number of cost optimization vendors who similarly adhere to that metric without context; it’s in danger of becoming a de facto “best practice” that will lead you down the primrose path. ## The limits of CPU utilization metrics CPU utilization tells you, perhaps obviously, how much of your CPU resources are being used at any given time. Any cloud provider can easily query, “How busy is the compute on this instance?” In a theoretical world where disk, RAM, burst capacity, network throughput, and latency are all irrelevant, then — yes! OK! terrific! — the question of utilization would strictly come down to how many CPU cores can you throw at the problem, and then using CPU as a proxy for utilization is great. If that’s you, stop reading now, go [buy some Last Week in AWS mugs](https://store.lastweekinaws.com/) or something, and carry on with your charmed existence. If it isn’t you, keep reading. Huh, we just lost a couple HPC folks and several of the more naive analyst firms, but the rest of you are all still here. Imagine that … For the rest of us, the problem is that CPU utilization is a single data point that doesn’t tell much of a story. It’s impossible to say at first glance whether your CPU numbers are worrying or an indicator that all is well, regardless of what the actual numbers are. High CPU usage could mean: - your applications are working efficiently, or - they’re straining under the load, desperately crying out for relief. Low CPU usage could mean: - your instances are idling, wasting precious cloud dollars, - your applications are well-optimized and aren’t CPU bound, or - you need idle capacity to burst into when a bunch of your users all show up at once. The CPU utilization metric blissfully ignores other critical aspects of your instances’ operation, such as network activity, disk I/O, and memory usage. A high CPU usage with low network activity could signal a performance bottleneck that leads to a data-starved instance, or it could be an application that barely needs to talk to other things on the internet. A low CPU usage with high memory utilization could mean your application is inefficiently coded, or that it’s a database that lives in RAM for latency purposes. ## The risks of relying on CPU metrics This reductionism of cloud instance health to CPU utilization stems from its ease of access. It’s readily available, easy to measure, and undeniably simplistic to interpret. Cloud providers can grab it via API, slap it onto a pretty graph, and voila, they’ve got themselves a utilization report. And the resulting CPU metric seems to level the playing field to reason about workloads that are remarkably diverse, making it easier to benchmark yourself against other companies ([which you should not do](https://www.duckbillgroup.com/blog/why-benchmarks-miss-the-mark-for-aws-spend/) ). But easy access doesn’t equal quality insight. Take a look at the fact that a c7g.large instance in EC2 is about [6% more expensive](https://aws.amazon.com/ec2/pricing/on-demand/) than a c6g.large instance. Amazon points out that the price/performance of that instance means you get improved price/performance, but that assumes an awful lot of things about your workload. If you need a cluster of 10 nodes to chew on a problem because that’s how your application works, then your cluster just got 6% more expensive if you upgrade to the latest generation — without a clear upside benefit that accrues to you. ## How to actually determine your fleet utilization A nuanced approach, taking into account a bouquet of metrics including network I/O, disk read/write speeds, and memory usage alongside CPU utilization, provides a holistic picture of your cloud instance fleet. Those metrics require a lot more insight into the environment and, in the case of memory, an agent running on the actual instances themselves. Cloud providers *could* deliver these kinds of nuanced reports, but the effort required from them is likely too high. So, next time you’re in your swivel chair, resist the temptation to rely solely on the CPU utilization column. Dive deeper, venture beyond, and ask probing questions. In so doing, uncover the true health of your server fleets. Because in the world of cloud economics, ignorance isn’t bliss; it’s just expensive. by Corey Quinn Corey is the Chief Cloud Economist at Duckbill, where he specializes in helping companies improve their AWS bills by making them smaller and less horrifying. He also hosts the "Screaming in the Cloud" and "AWS Morning Brief" podcasts; and curates "Last Week in AWS," a weekly newsletter summarizing the latest in AWS news, blogs, and tools, sprinkled with snark and thoughtful analysis in roughly equal measure. ## More Posts from Corey [Back to the Blog](https://www.lastweekinaws.com/blog/) [https://www.lastweekinaws.com/blog/cloudwatch-omni-infinidash-review/](https://www.lastweekinaws.com/blog/cloudwatch-omni-infinidash-review/) ### [Infinidash: AWS’s New Observability Play Has Great Bones and a Terrible Bedside Manner](https://www.lastweekinaws.com/blog/cloudwatch-omni-infinidash-review/) [By Corey Quinn](https://www.lastweekinaws.com/blog/author/cquinn/) Infinidash: AWS’s New Observability Play Has Great Bones and a Terrible Bedside Manner AWS has just released something fascinating in the form of CloudWatch Omni. I want to be upfront before I dive further: “CloudWatch Omni” is a wretched name that I will not be using, so for the rest of this article and the […] [Read More about Infinidash: AWS’s New Observability Play Has Great Bones and a Terrible Bedside Manner](https://www.lastweekinaws.com/blog/cloudwatch-omni-infinidash-review/) [https://www.lastweekinaws.com/blog/reading-observability-tools-thats-a-robots-job/](https://www.lastweekinaws.com/blog/reading-observability-tools-thats-a-robots-job/) ### [Reading Observability Tools? That’s a Robot’s Job](https://www.lastweekinaws.com/blog/reading-observability-tools-thats-a-robots-job/) [By Corey Quinn](https://www.lastweekinaws.com/blog/author/cquinn/) Last Thursday Honeycomb was kind enough to hand me the closing keynote slot at O11yCon, a conference whose entire premise is that observability matters and the people in the room are the ones who would know. I used that slot to argue that the primary reader of your telemetry is no longer a human, and […] [Read More about Reading Observability Tools? That’s a Robot’s Job](https://www.lastweekinaws.com/blog/reading-observability-tools-thats-a-robots-job/) [https://www.lastweekinaws.com/blog/s3-is-not-a-filesystem-but-now-theres-one-in-front-of-it/](https://www.lastweekinaws.com/blog/s3-is-not-a-filesystem-but-now-theres-one-in-front-of-it/) ### [S3 Is Not a Filesystem (But Now There’s One In Front of It)](https://www.lastweekinaws.com/blog/s3-is-not-a-filesystem-but-now-theres-one-in-front-of-it/) [By Corey Quinn](https://www.lastweekinaws.com/blog/author/cquinn/) I’ve been saying “S3 is not a filesystem” for over a decade. I’ve said it on stages, in newsletters, on podcasts, and directly to the faces of large company employees who were too polite to tell me to shut up before they went back to their FUSE monstrosities. It was one of those reliable truths […] [Read More about S3 Is Not a Filesystem (But Now There’s One In Front of It)](https://www.lastweekinaws.com/blog/s3-is-not-a-filesystem-but-now-theres-one-in-front-of-it/) ## Get the newsletter! Stay up to date on the latest AWS news, opinions, and tools, all lovingly sprinkled with a bit of snark. "*" indicates required fields