GitHub actions vs. the good old Makefile

Ghostbusters

The story so far: shifted to a mix of Debian and Windows 11 with wsl2 as development machines, reawakened bits of my brain that I haven't used for years.

Egon, this somehow reminds me of the time you tried to drill a hole in your head. Do you remember that?

So, I'm sitting here at Chungus the Proliant struggling to remember the magic incantation that deployed my .NET core docker container up to AWS. Some digging through my bash history eventually uncovered it and I was on my merry way again, although I found myself in vi, staring at a blank Makefile and not quite remembering why. It was a reflex action from years ago when I was working with a small crew at a bank. Any time we came across some system that needed building on a Sun or other Unix machine, or some set of commands that were hard to remember, it had been beaten into us by our mentors to fire up an editor and make a Makefile.

Years of working in IDE's on windows machines, where "make" was never a given tool in your arsenal had made me a bit rusty, but a quick google had me punch out this:

.PHONY: deploy

deploy:
        dotnet ecs deploy-service

You might well think, what is the point of that? You can't remember that command? The truth is, I only deploy the website about once a week and the command is pretty new to me since I had been right-clicking in Visual Studio using the AWS tools. Shifting to Visual Studio Code means you don't get any of those niceties (it's really a fancy editor, not an IDE). The Makefile is there not only to remind me what to run, but to allow a simple command make deploy that I can customise for each item that needs deployment. I have a slightly different one for the lambda functions:

.PHONY: deploy

deploy:
        dotnet lambda deploy-function

For me, the bits and pieces of the dotnet AWS tools kind of rhyme, but they are different enough that it's hard to remember which one does what. Makefiles are perfect for that, I don't have to remember, I just see the Makefile in that directory and either run make deploy or run the command manually if I need to do one of the pre-flite options. It works for me and now with wsl2 on Windows exactly the same commands do exactly the same thing.

Let's show this prehistoric bitch how we do things downtown.

Which leads me to a problem I was trying to find a good solution for. The website has a very bare-bones CMS that was hand-rolled specifically for my needs, it rather stupidly stored the HTML in the database, the images in an S3 bucket and used the FCKEditor as a tool for maintenance. It was actually very productive for creating pages (they are simple pages, no templates) but it was awful for maintaining them. I often found myself making a boneheaded mistake in the FCKEditor and having to drop back to the database to fix it. What I needed was to pull all the content down into a git repository, edit it locally, then push it back to an S3 bucket whenever I was happy with it. This entailed a bit of futzing around. The details are unimportant, but I wrote a small C# console program that loaded the entities from the database, created a json file that could be used to re-inflate them and wrote all the HTML into separate files ready to be imported into git. The [JsonIgnore] annotation is very handy for stripping out things you don't need, so the resulting file needed no editing at all. So far so good - the rewrite on the website itself was minimal (so everything comes out of the S3 bucket, not the database) and despite being a bit slower to load the page cache, it ran pretty well. Massive bonus: a content history I can make sense of rather than trying to dig through the audit tables I had triggered off the content tables. It's beautiful.

AWS supply a neat command line utility for taking your directory and syncing it to an S3 bucket which is absolutely perfect. I have a small python script to safely manipulate the page content json file, can edit the html locally using much better editors than the FCKEditor (no disrespect guys, for what it is, it's awesome). When it looks good, do a local integrity test and then push the changes into the production branch. I christened this two day crunch effort "SmackeyCMS" because why not.

Of course, I initially used a Makefile to sync things to S3 because it's another weird command I will never remember:

.PHONY: sync

sync:
	aws s3 sync SmartShepherd s3://smartshepherdbucketofun

You could get super fancy at this stage and have a staging bucket etc. but for local testing, the web server can be pointed to a local directory instead of S3 so it's a bit redundant at the moment. The sync process is super smart, it only updates the files that need updating and with an optional --delete flag it will also remove things that no longer exist in the source. This has about quadrupled my productivity on the content of the site so I need to get busy and fix everything, update the images and generally make it nicer. This is something I'll set an hour or so each day on.

Ray, when someone asks you if you're a god, you say "YES"!

But, shiny new things were afoot in the shape of GitHub actions. More specifically the fancy, fine work of somebody called Jake Jarvis and his S3 sync action. Setup should be fairly simple (in theory). You make a directory called ".github", make a subdirectory "./github/workflows" and add a Yaml file with your desired action in it.

I'm not a fan of Yaml. I'm not a fan of any language where the indentation in the file is part of the logic. It's just painfully stupid. Normal people have used braces or brackets or BEGIN/END since time eternal and you are left in no doubt about what the structure of the program is, whether the numpty who typed it in formatted it correctly or not. Yaml sucks.

I copied Mr Jarvis' script into my workflows, made the appropriate secrets in my account and pushed a change through to the production branch. Of course it crashed.

I grabbed a Yaml linter and the stupid file is fine. It's fine. I edit it, push it again, still nothing. Finally I remove the comments and all the white space and change everything to two spaces from whatever the hell got pasted in there and it finally worked. I swear it's more voodoo than science though. Did I mention I hate Yaml? Those who are unaware of history are doomed to repeat it and whoever the goober is who decided Yaml was a good idea can go suck an egg.

Ray, the sponges migrated a foot

The downside of GitHub actions: It has to be the most convoluted way ever invented to run a command line program. It starts a fat container, grabs your secrets, loads up the environment needed to run your command, does a clone of your directory into the container and then finally runs the command. The horrible side effect is that the super-neat feature of aws s3 sync that only updated the necessary files no longer works. It just dumps everything up there. The jury is out, but for all the good things you get through only running the action when your files are pushed to production, the sheer waste of what is going on hurts my brain. I'll leave it running for a while but I'm tempted to delete that stupid Yaml file and just stick to "make sync".

The FCKEditor: a postscript

I've used the FCKEditor before, at a job where we only had limited access to the production servers, needed a simple CMS and about the only reliable storage we had was the database. It always resulted in some fairly puzzled expressions from the users i.e. "Hey Dave, the F*ck editor?" or "Hey Dave we really edited the f*ck out of that didn't we". Hilarious. I am glad they changed the name.