Once again I did a full circle and got back to amd64 architecture, technically replacing everything I setup around 3 years ago. This time I will do it right. After Hetzner rolled out their new cloud offering, it seems like AMD on Hetzner is kinda dead end at this point (don’t quote me on this). So I ended up in a weird situation where I had some old dangling ARM k3s node, some new PoC Coolify ARM-based server and it was time to put it all back together - obviously by setting up a third new project.

I wasted a few last weekends setting up a self-hosted runner set on a Kubernetes cluster just to… that’s actually a good question - I’ve already forgotten why. So, I have a fairly easy CircleCI configuration for my Rails app. There are a few jobs running on different containers that prepare the dependencies cache, followed by some linters. Then, there are a few parallelized jobs that run tests, a fan-in job that collects test coverage results, and uploads them to Codecov. There’s also an optional step that builds a Docker image, pushes it to ECR, and optionally triggers ArgoCD deployments.

Almost a year ago, I wrote about how to deploy a simple MySQL leader/follower setup on Kubernetes. It’s been a while, and I have decided to look into ‘operator’ solutions, which should be fully compatible with native MySQL implementations (MariaDB was out of the question because of reasons). If you want something fully MySQL-compatible, you have two major options (I’m not going to mention operators, which seem to be abandonware at this point): Percona Server MySQL Operator - which went from technical preview to general availability fairly recently; it can run percona-server-mysql, which should be a drop-in MySQL 8.x replacement official MySQL-Operator from Oracle - where you just run official MySQL database (8.x/9.x) I tested MySQL-Operator on a small testing cluster and decided to move my small workload after a month of battle testing. Migration was supposed to be a quick 20-minute adventure.

Another click-baity headline, don’t go, but hear me out, it’s a good idea. Every now and then I try to look into “simpler” deployment options than my forgotten kubernetes cluster. I eventually abandoned nomad for k3s after running into constant pain from insufficient tooling. Then I forgot about that cluster for like 2 years and I’m waiting for an epic crash which would force me to redo the whole thing again.

Let’s say you’re going on a trip for few weeks and you prefer not to take your laptop with you. But you manage some small kubernetes cluster which is behind Cloudflare Zero Trust which can be only accessed: using Cloudflare WARP using google sign-in via kubelogin And you want to have some sort of emergency access while you’re away from your iPad. You can do it using Github Codespaces, but whole process is not well documented, let’s start with some references which were helpful when I glued all this hack together:

Rollback to July 2003. I’m something like 17 years old, and I play video games. I mostly play Quake 3. And mostly Quake 3 with a DeFRaG mod. We - and by we - I mean me with a group of random internet folks - just released something called “Polish Defrag MAPpack no.1” - 9 maps made by 9 different folks. I met those randos online on an internet forum located under q3arena.gry.wp.pl - a site which was moved around between domains since then, but that’s not the point.

You might be surprised by Rails’ behavior when rendering a partial via Turbo. There is no session context - thus form_tag won’t render the “authenticity token” hidden field - it will silently omit it. This behavior was changed around 2021, and you can check out this lengthy discussion about the problem.

Nameserver limits were exceeded, some nameservers have been omitted. Have you ever run into such errors on your k3s cluster? The problem was already mentioned in a few places: https://vadosware.io/post/sometimes-the-problem-is-dns-on-hetzner/ https://github.com/syself/cluster-api-provider-hetzner/issues/77 Some folks suggest modifying etc/resolved.conf, but as my node shipped with netplan (and I didn’t want to deal with that, I trust my hosting provider can handle networking), I decided to create a dedicated resolv config for kubelet needs.

I love Sentry since I discovered it many years ago. Back in the days, self-hosting it was really easy - a single Docker image which you would use for spinning up like 2-3 separate services, Postgres, Redis, a few lines of config, and you’re done. Nowadays (2024), self-hosting Sentry requires spinning up 50+ different services - and that is of course without any fancy HA setup. It’s still doable, especially with Kubernetes, but the learning curve is definitely more steep. Then again, the feature set of Sentry itself is much richer - it’s not about just catching errors anymore; you have full-fledged build-in performance monitoring, session recording, and tons of other observation-related goodies. One thing is kinda tricky is monitoring of your Sentry instance. Here is how I do it.

Things. You need things in your life. Up to a point where those things start to own you. Your space, your mind, your time. I like some of my things, some I simply need, and some, well, do not spark joy. I declutter mostly using three options: giving stuff away, selling and throwing away. The last is last resort option - I tend to lean toward the two first one.