Tomorrow Amazon reports its quarterly earnings. I’ve talked in some depth about AWS’s compensation model being heavily stock driven, and the market being the market that means a number of excellent AWS friends who have been absolutely killing it find their fortunes rising and falling based entirely upon how well Amazon’s Underpants Store division performs. I’m not one to keep my opinions to myself, and one of those opinions is that a $1.412T company can afford to pay decent wages to the folks that run an ever-larger swath of the internet.
Expressing that opinion in not-particularly subtle ways is of course a great excuse for me to try out the Cloud Development Kit, or CDK.
There’s an App For That
Calling Amazon a $1.412T company to three decimal places of precision is all well and good, but the stock market moves. I want my periodic references to that figure to at least be up to date. I could of course check Amazon’s market capitalization every time I make the reference, but that sounds a lot like undifferentiated heavy lifting. (So does wrestling with AWS services, but that’s strangely never a point that gets brought up in AWS’s keynotes.)
The SVP of Bad Decisions over at Yahoo has been steadily turning off services for years and eventually made their way over to the Yahoo Finance API as well. Fortunately some suspiciously named third party clone calling itself “Yahoo finance API” has shown up and works for my purposes.
Further, Apple has brought its “Shortcuts” app from iOS to the Mac as well. I was able to build a shortcut relatively quickly that could grab the result of the API call I was making and copy it to the system clipboard, but I quickly got lost in the weeds of how exactly Apple expects you to work with JSON in its constrained Shortcuts environment. After a few minutes I realized that I didn’t really want to pay for the privilege of using an artificially constrained programming environment that was specific to one company’s platform and kept laying obstacles before me, so I used AWS Lambda instead.
A dozen lines of Crappy Python later I had something that would query the suspicious and frightening finance API, then massage the result to return the string I wanted. Now all I had to do was deploy it and teach the Apple Shortcut to simply hit the URL of my API Gateway and copy the result to the clipboard without doing anything fancy. For novelty, I decided to deploy it into my AWS account via the CDK; as an added bonus this tracks almost perfectly with the cdkworkshop.com tutorial. What luck!
Travels With the CDK
The CDK supports JavaScript, TypeScript, Python, Java, C#, and Go. Within any of those languages, it gives you tools for provisioning and working with AWS resources in that language. Then you run cdk deploy and it renders down whatever you’ve written into standard CloudFormation and deploys it to your AWS account. I’m certain I’m missing some nuance here; I’m equally certain that I do not care, so please don’t email me.
The CDK Workshop’s Python section is missing a few things, most notably a warning asking if you’re sure you want to do this. The Python support has a lot more sharp edges than TypeScript or Go, as per my CDK friends on Twitter.
Because it’s Python, there’s the usual dependency management hassles to jump through. Then to install the CDK to use with Python you of course use npm, which is how one installs Node.js packages because computers are terrible. I use the asdf version manager system, about which the CDK whined mightily about my node version being too new; I had to back out and restart with something a bit longer in the tooth.
Just as an aside here: if you’re shipping a command line tool for people to use, particularly if that tool is not isolated to the JavaScript ecosystem, you really want to get away from the “use npm to install it” model. For those of us who aren’t proficient with JavaScript or its ecosystem, it’s a very finicky process that lends itself to feeling like we’re constantly either doing battle with the package manager or destroying our system’s installation of Node just to get a text renderer working. Ship an omnibus binary, never show the user a JavaScript stack trace unless explicitly asked to do so, and suddenly you’ll find a lot of folks on the DevOps side of the world will take what you’ve built a heck of a lot more seriously.
I also had to translate between the workshop’s assumption that I was using VS Code as my editor and the reality of “I’ve been using vim for fifteen years and I’m not likely to stop now.”
Once that was done, it largely “worked,” though I had some things to wrap my head around as I went.
What I Learned
The CDK is very particular about its directory structure. If you want to deploy a Lambda function (let alone a more complex application) via the CDK, you might think that your application would dictate the directory structure and layout for the project. You would be wrong; you’re in the CDK’s world now.
I also slowly realized that just because the Lambda I was writing was in Python did not mean that I had to manage the infrastructure via CDK with Python as well. I could have just as easily used the TypeScript or Go CDK expressions and kept the Python Lambda function exactly as it was. Well, in theory anyway; in practice I do not know those languages and thus instead felt like every sharp Python CDK edge was a reflection on my lack of technical acumen. You really, really don’t want to put users in that position if you can at all help it.
The Big Unanswered Question
Now that this is “working” I will likely not touch it again for years, if ever. That’s kinda the entire point of a “utility” Lambda function like this. If it breaks it’s not going to be the end of the world for me, and is likely going to be tied to something upstream shifting instead.
The problem I have though is that I have ~14 AWS accounts spread across three Organizations. Each of those accounts has ~20 different AWS regions from which to choose. If this thing breaks in a few years and I have to fix it, I’m not going to remember which AWS account it lives in (“Shitposting,” for future me), nor what region it lives within (“Ohio,” since it seemed a natural home for something that sucks).
AWS sees this application as just another CloudFormation stack, since the CDK’s magic happens client-side before interfacing with AWS via more traditional means. The trouble is, I’ll collect dozens or hundreds of things like this over time, and now I’ve got a somewhat serious library management problem. What is the current state of the art for managing a giant pile of one-offs like this in a way that lets me find them again when I need them? If you know the answer, please do reach out, because this problem is only going to get worse.
Takeaways
All in all, it was a fun way to spend a couple of hours. I “get” the CDK in ways I didn’t before kicking the tires on it myself. That said, I’m uncertain that it’s for me. The Python support remains touch-and-go, and I can’t shake the feeling that its primary user is intended to be folks who are significantly better at programming than I am – or at least who think about programming differently.
I’d be curious to know what you think I might be missing, or failing to properly appreciate about the offering. That’s a sincere offer; understanding how to work with things like this always an ongoing process. I found the promise compelling for a certain type of user, but the sharp edges I encountered more than a little bit off-putting.
After all, I expect a bit more polish from a $1.412T company.
