Good Morning!
If you’re around Seattle tomorrow (Tuesday) night, I’m going to be at Old Stove Brewing Ship Canal for a couple of hours starting around 6:15. Come by, say hi, let me buy you a drink while you tell me **exactly** what my problem is.
And if you’re in SF next week, despair not; I’m doing very similarly with my friend and yours, Philip Kiely at BaseTen. Hit the signup so I know you’re coming.
Things I Found on the Internet
GoDaddy reverted to its nature (“being shitty to everyone it touches”) by putting up a “competitive analysis” the likes of which you’d tend to only see in recent Garter Magic Quadrants®. They published blatant misinformation about actually decent registrar Porkbun, who took umbrage. Why am I mentioning it here? Because this is the first time I can recall that my filter for “Setting the Record Straight” returned something that wasn’t Amazon being defensive about something.
A customer getting breached via AWS key isn’t news, but I do like how this particular breach was detected via a spike in data egress charges. The bill remains the best AWS observability tool that money (oh so much money) can buy.
I built an AI-powered singing coffee pot; here’s a five-minute video of me unveiling it to the team. I think I want to do a full conference talk about this (the engineering was non-trivial and spanned a bunch of domains, including a number of AWS AI services); hit reply and nominate a conference you think it’d fit at.
What AWS Has For Us This Time
Amazon EC2 introduces application status checks
EC2 has discovered the concept of a health check, only two short decades after AWS’s own load balancers did the same thing. I don’t want to hear anything else about “innovation” this week, please.
AWS IAM Identity Center supports one-click multi-Region option for new organization instances
Great, the thing I spent the last week implementing this the hard way that launched a few weeks ago now gets a one-click option. I am less than amused by the timeline here.
Amazon S3 adds additional policy details to access denied error messages
Okay, you get exact ARNs for denials, which is a HUGE win for diagnosis. Only… it’s just for explicit deny. “Nothing permitted it” apparently still results in the usual opaque message intended to send you haring off after side-quests.
AWS Secrets Manager adds managed external secrets support for Jenkins and SonarQube
I’m sorry, Jenkins support? What century is this?
Burst to Region: Overflow AWS Outposts workloads to Amazon EC2
Excellent news: you can now handle excess traffic on the device you’re paying for to avoid the AWS region by sending that extra traffic back to the AWS region. This is just “extra steps” where each one costs a king’s ransom.
Designing for failure: Building resilient systems on AWS
-
Well this is a pile of mistakes stacked atop one another; see how many you can spot in your own infrastructure. The customer was running Cassandra (mistake) in three instances on the same physical node (mistake) in the SFO region (expensive mistake) and went down. There was no Partition Placement Group, which many of you will be learning about for the first time in this sentence. There was no CloudWatch alarm on EC2 status checks, so detection became inherently reactive, and so there was a 90 minute window before they got to root cause. Glad the story got told, sad it got to this point.
Amazon Quick for Microsoft 365: Agentic AI where you work
I can’t wait to see Amazon Quick starting a slapfight with Microsoft Copilot over who’s the best kid on the elementary school playground. Meanwhile the grown-ups are using… not these tools.
Introducing the next-generation AWS VPN Client with CLI support and admin controls
At long last the VPN client gets a CLI. Given that this charges per connection-hour, I wonder if easily scriptable disconnections will reduce revenue by more than “agents can now connect easily” will increase it by.
If you haven’t switched to DNS cert validation yet, by god are you about to. Note, this isn’t AWS, but rather the industry group around certificate management pushing this change. It’s a good one, but it might hurt for some of you.
How AWS IAM role manager rethinks the starting point for IAM roles
I think I like this. One caution, though: if this becomes normalized, there’s gonna be a whole generation of accounts that accumulate AWS-created roles that nobody scoped. Thus, the blast radius during the inevitable next credential incident is going to go back to these AWS-maintained templates, and where exactly that lands in the Shared Responsibility Model is relatively untrod territory. Bring popcorn!
How to authenticate customers during chat with Amazon Connect Customer
This got yanked, most likely because someone didn’t realize that Connect Customer (god, what a name) does this natively via OIDC customer authentication.
How WeatherBug reduced storage costs by 80% using Amazon S3 Storage Lens and Kiro CLI
The kind of company who didn’t realize that 80% of their storage bill was waste is exactly the same kind of company who thinks Kiro is a good coding assistant offering compared to other vendor options, so this tracks. I do get it though; changing storage patterns can break things, sometimes permanently.
A customer who cannot meaningfully describe their problem might just be the most accurate simulation of a real-world customer that AWS has ever built.
Missing input validation in the OpenSearch Security Analytics plugin (y’know, the thing you deployed specifically to notice bad input) shows up as a bulletin filed under 079, and then, apparently because once wasn’t enough, the same CVE again at 081, which is either a numbering hiccup or the most honest admission yet that nobody reads these except me. Wedged between them is a pair of memory-safety bugs in Base64 decoding in the C++ SDK, proving that even a text encoding from 1987 can get you not one but two CVE IDs. AWS shipped that fix, but you’re not done yeet, because SURPRISE, you now suddenly get to own the part where every statically linked binary in your fleet gets rebuilt and revalidated. All of this was of course announced via RSS, the delivery mechanism for a company that will happily sell you an alerting product.
… and that’s what happened Last Week in AWS.
