Good Morning!
I have three announcements!
-
The Last Week in AWS Charity T-Shirt fundraiser closes tonight Pacific time; it’s now or never. All proceeds benefit 826 National; you know you want to see AWS Marketing write a copy of this newsletter if we hit our $40K fundraising goal…
-
re:Quinnvent draws ever-closer. If you want to get your message to roughly 100K people, consider sponsoring.
-
By the time you read this, I’ll be gone on my first vacation in two years. Please try not to burn down the internet while I’m out.
From the Community
Blackboard Inc. is a world leader in education technology, committed to supporting learners throughout their lifelong journey. Traditionally, Blackboard’s Product, SRE, DevOps and Support teams depended on a combination of custom-managed ELK stacks and managed Elasticsearch service offerings for centralized log management. But growing daily log volumes and variable spikes in log volumes were causing pain. Unpredictable spikes would cause the ELK stack to go down, making it unusable at times while management and data storage costs grew. Enter ChaosSearch. To see how ChaosSearch helped Blackboard solve this plethora of Elasticsearch challenges, check out the full Blackboard case study here. And when you’re ready to talk to ChaosSearch yourself – tell them that I, the magnanimous Corey Quinn, sent you! Sponsored
Multicloud failover is almost always a terrible idea is a great statement, made all the greater by the fact that it’s coming from Lydia Leong, VP and Distinguished Analyst at Gartner. But you know better. You’re going to disagree. So meet me halfway on this: get multi-region failover working first (should be easy! It’s the same services!), then come talk to me. You’re in for a delightful surprise.
Someone’s put together a re:Invent MaaS (Meetup as a Service); if you’re going to be attending, the idea is you throw your contact info and interests into this Google Form to be matched with a coffee buddy. It’s a shame and also inevitable that this needs to be done by the community instead of AWS itself.
On the lighter side, I tweeted in September about a time I installed the “Cloud to Butt” browser extension. It rewrote an email I was replying to in Outlook Web Access, and freaked out a manager. I fessed up immediately and uninstalled the extension. My guidance was to not use that extension for precisely this reason. I see that advice went unheeded, and I’m apparently indirectly responsible for what happened next.
Last week, AWS briefly had a job posting for their Aurora team that opened with the glorious phrasing: “Are you interested in building hyper-scale database services in my butt? Do you want to revolutionize the way people manage vast volumes of data in my butt?” Then it closed with “Come, join us in reinventing database systems for my butt!”
Look, this is hilarious. It was a mistake, but a harmless one. People are talking more about this job posting than any Amazon employment initiative in recent memory that didn’t involve peeing in a bottle. It’s something that could be turned into something hilarious if AWS weren’t quite so uptight with their corporate messaging. We’re all human; mistakes happen. You can take things seriously but still have a sense of humor in the world of butt computing.
We have a guest post that continues our series; this time it’s What is File Storage? A Definition and Overview.
I revisited my old post from yesteryear, Why I Turned Down an AWS Job Offer.
Someone else did the math on just how expensive AWS data egress really is.
Someone made the cardinal mistake of taking AWS at its word when they said “free tier,” got a bit sloppy with their credentials, and were then presented straight-faced by AWS with a bill for $61,261.19. If AWS doesn’t fix this soon, I fear something truly tragic might happen. For god’s sake – you’re being made to look like fools by Oracle Cloud, whose free tier is uniformly excellent! I used to joke that “save money by choosing Oracle” was the funniest thing you’d read in a week. Well, we’re not laughing anymore.
Annoyed with the constant attacks against their hilariously lax security posture, Azure decided to demonstrate their versatility by also falling over globally for eight hours. Seriously, global VM provisioning was down for an entire workday.
Jobs
If you’ve got an interesting job for this newsletter’s eminently employable subscribers, get in touch!
The Duckbill Group (that’s me!) is hiring a Head of Consulting Services to join the team. We’re looking for someone skilled in managing and leading people, as well as in building and optimizing delivery processes. As a member of the leadership team in a nine-person company, you contributions will be instrumental to our continued growth and success. AWS expertise isn’t required, but it’s certainly a bonus. If you’re interested in a role that’s fully-remote, has big impact, and you want off the VC rollercoaster, come check us out.
Q: What is Amazon GuardDuty?
Amazon GuardDuty offers threat detection that enables you to continuously monitor and protect your AWS accounts, workloads, and data stored in Amazon S3. GuardDuty analyzes continuous streams of meta-data generated from your account and network activity found in AWS CloudTrail Events, Amazon VPC Flow Logs, and DNS Logs. It also uses integrated threat intelligence such as known malicious IP addresses, anomaly detection, and machine learning to identify threats more accurately. #### Q: What are the key benefits of Amazon GuardDuty?Amazon GuardDuty makes it easy for you to enable continuous monitoring of your AWS accounts, workloads, and data stored in Amazon S3. It operates completely independently from your resources so there is no risk of performance or availability impacts to your workloads. It’s fully managed with integrated threat intelligence, anomaly detection, and machine learning. Amazon GuardDuty delivers detailed and actionable alerts that are easy to integrate with existing event management and workflow systems. There are no upfront costs and you pay only for the events analyzed, with no additional software to deploy or subscriptions to threat intelligence feeds required.
AWS EKS is AWS’s managed Kubernetes service offering for those folks who don’t have the good sense to run ECS instead. They’re apparently setting out to build a new service within the EKS offering – because if there’s one thing AWS needs, it’s another way to run containers. This new service will tackle a complex and unique use case of some of their customers who are no doubt scowling at the heavy editorializing I’m doing here. This team makes a point of contributing back to the upstream EKS and open source K8S service community and are emphatic that they collaborate closely with other service teams at AWS to deliver this ground breaking new capability. The containers group is awesome – but after a pitch like that, whatever this new service is had better deliver, right?
Choice Cuts
If you’re anything like me, you’ve screwed up the database part of a deployment so severely you’ve been banned from ever touching anything that remotely sounds like “SQL” in at least three companies. We’ve (mostly) got code deployments solved for, but when it comes to databases we basically rely on “desperate hope” with a rollback plan of “keeping our resumes up to date.” It doesn’t have to be that way; meet Liquibase. Both an [open source project] and [a commercial offering], Liquibase lets you track, modify, and automate database schema changes across most any database with guardrails that ensure you’ll still have a company left after the change. No matter where your database lives, Liquibase can help you solve your database deployment issues. Check them out today. (Offer does not apply to Route 53.) Sponsored
Amazon EC2 Auto Scaling now supports describing Auto Scaling groups using tags – As opposed to the way that most engineers describe Auto Scaling groups: “badly, and gesticulating with their hands.”
Amazon ECS Anywhere now supports GPU-based workloads – So before this feature, a better name would have been “Amazon ECS Almost Anywhere?”
Amazon Fraud Detector launches new ML model for online transaction fraud detection – “Amazon Fraud Detector launches new online transaction model for online ML detection” also works equally well.
Amazon MemoryDB for Redis is now available in 11 additional AWS Regions – A badly conceived service with crappy implementation that shouldn’t exist in the first place is now available in a bunch more places.
Amazon QuickSight doubles SPICE capacity limit to 500m row – The SPICE must flow faster and more expensively.
Amazon SageMaker Projects now supports Image Building CI/CD templates – “Time to re-implement everything from the rest of the ecosystem within SageMaker so someone can get promoted!”
AWS Console Mobile Application adds support for Amazon Elastic Container Service – I’m just waiting for “Sign In with Apple” support. It’s gotta be better than anything AWS offers today for identity management…
AWS Outposts adds new CloudWatch dimension for capacity monitoring – From the publisher of “What Color is Your Parachute” and “Who Moved My Cheese” comes the new hit business book, “How Full is My AWS Outpost?”
AWS announces a price reduction of up to 56% for Amazon Fraud Detector machine learning fraud predictions – They of course had to do this because until they did, Amazon Fraud Detector kept Detecting itself and wouldn’t proceed further.
New Foundational Technical Review Process for Partner Hosted Solutions – This is a new process by which AWS reviews partner offerings to determine how they can best build a competing service with a bad name.
Introducing the new AWS Well-Architected Machine Learning Lens – The different “lenses” for the Well Architected Tool adjust the way that different criteria are evaluated. For the Machine Learning Lens, the “cost” pillar of the Well Architected Framework is reduced to simply “LOL.”
How Amazon Transportation Service enabled near-real-time event analytics at petabyte scale using AWS Glue with Apache Hudi – I hadn’t heard of the Amazon Transportation Service before and wondered how I’d take a trip on it. It turns out that it handles shipments, so the answer is “stuff myself into a cardboard box and get myself shipped.” The way I’d do that of course is to take a job at Amazon and become subject to their employee travel policy.
Optimize performance and reduce costs for network analytics with VPC Flow Logs in Apache Parquet format – This is a big win, since it’s basically the only way you can figure out just what the hell it is that’s passing through your Managed NAT Gateways. Seriously, consider using these.
Avoiding recursive invocation with Amazon S3 and AWS Lambda – “There are one or two corner cases where the ‘Lambda invokes itself’ pattern is desirable and many thousands in which it isn’t, so rather than making it something customers need to affirmatively enable, here’s a bunch of architectural work you should do to avoid that situation.”
Build an AI powered agent for Amazon Connect using AWS QnABot – I’m not the only person who misread this as “AWS QanonBot.” This may be the worst AWS naming of 2021.
How to build your containers for ARM and save with Graviton and Spot instances on Amazon ECS – This post is so sarcastically long that it’s like it’s designed to elicit the reaction “oh, screw that.” I presume it’s sponsored by Intel. I especially like the all-caps “ARM” in the headline in direct contravention of Arm’s expressed brand guidelines. Other people’s brand requirements remain something for which Amazon cares not a whit.
How we halved the publish size of modular AWS SDK for JavaScript clients – Holy crap, there’s an actual joke image in this blog post contrasting the relative weights of the sun, a neutron star, a black hole, and node_modules. Someone’s almost certainly getting PIPed for this, as having a sense of humor is apparently not an Amazonian leadership principle. That said, I adore this and wish we saw more of it.
Third-party Cookies are Going Away: What Should Retailers Do about It? – This is a serious problem for Amazon as they increasingly pivot to shoving ads into your face at every opportunity.
Secure content using CloudFront Functions – “Dynamically run code on every HTTP request” is somehow not a punchline, but a real thing that AWS advises for securing static content.
Control developer account costs with AWS CloudFormation and AWS Budgets – Sometimes I worry that AWS might fix their billing problems and put me in a situation of having to find something else to do. Then I read posts like this one and then I fret that they never will.
Tools
Observability is critical for managing and improving complex business-critical systems. With observability, any software engineering team can gain a deeper understanding of system performance, so you can perform ongoing maintenance and ship the features your customers need. Preview Honeycomb’s upcoming O’Reilly book to understand the value of observable systems and how to build an observability-driven development practice. Sponsored
Green Cost Explorer is a way of seeing how much of your cloud bill is spent on fossil fuels so you can react accordingly. I like this!
Lambda Power Tuner now supports ARM architecture Lambda functions as well, so you can see just how much money you’d not be saving by switching to them.
… and that’s what happened Last Week in AWS.
