It’s been awhile y’all
It's been a hot minute since I've documented some of my work, so I guess in keeping with making a main blog post, I'll make a devblog post today too.
cfn-mode / flycheck-cfn
https://gitlab.com/worr/cfn-mode/
I've been an emacs user for some time, and at my current job, I've been hurting for good support for cloudformation templates in my editor. I wrote this mode and flychecker to at least add some basic syntax highlighting and linter support. I'm currently in the process of getting them added to MELPA.
imdb-api
I made a bunch of changes fairly recently to imdb-api, most notably adding front-end support, migrating to Gitlab and migrating to ky after node-requests was deprecated. Normally I'd link patches, but there are too many since my last update. Here's the changelog: https://gitlab.com/worr/node-imdb-api/-/blob/master/CHANGELOG.md
fluent-bit
At work, we discovered an issue where our fluent-bits were sticky to the same instance of fluentd if we turned on keepalive and used a load-balancer.
To mitigate this, I ended up adding a new option to fluent-bit that will recycle keepalive connections after sending a number of messages, to cycle between backend instances periodically.
https://github.com/fluent/fluent-bit/commit/44190c2a1c4b939dc9ecb2908148d38c82a40831
https://github.com/fluent/fluent-bit-docs/commit/8d43b502123e366a1722a0051918ce7d78a8506b
fluentd-s3-plugin
Also at work, we found a case where the fluend plugin for s3 would spend forever trying to upload logs. By default, the naming scheme for the log chunks would be something like <time_slice>_<idx>. The time slice is the time when the log was uploaded, and the idx value is a monotonically increasing integer.
The problem, is that if you have mutliple threads uploading (or multiple fluentd's, or both), they have to check the presence of the filename to ensure that the formulated filename doesn't exist. Additionally, it doesn't track the last-used index, so when doing this check, fluentd will start at 1, check, increment, check again, increment again, etc. This obviously doesn't scale very well when you are outputing a ton of logs.
We fixed this my changing our file format to include a uuid and disabling the behavior to check for collisions.
However, since the defaults are dangerous, I've submitted this PR to try and make things less dangerous for new users (not accepted yet at the time of this writing).
https://github.com/fluent/fluent-plugin-s3/pull/355/files
This works by tracking the last used index in an atomic that's shared between threads. As outlined in the PR, it doesn't solve the whole problem, but it does make the defaults considerably safer.
logging-operator
Perhaps you've noticed a theme with my recent, work-driven contributions. :)
logging-operator is a kubernetes operator to automate adminitration of logging infrastructure in a kubernetes cluster. I've been contributing a bit to it lately, since we adopted it fairly early, and have needed to add a few features to make it work for us.
This first diff adds support not just for the configurable I added to fluent-bit that I mentioned earlier, but exposes all net parameters as configurables.
https://github.com/banzaicloud/logging-operator/commit/3c9e3938590209716918bc7cc197b43b09bb4361
There was a string conversion bug in how the operator would report on prometheus configuration.
https://github.com/banzaicloud/logging-operator/commit/86503b52150cf0dcf62d4b636eb247d0807101e7
We needed to configure where in s3 fluentd was uploading these logs
https://github.com/banzaicloud/logging-operator/commit/29fccfc2b8cee6c38c88fb34cf73a112eeb534de
We also needed way more support for configuring certain scheduling attributes in fluentd and fluentbit
https://github.com/banzaicloud/logging-operator/commit/45dffe5ebb38a3dbba4ecb217235f45c13f7856e
https://github.com/banzaicloud/logging-operator/commit/961fd788bb90f8f46d188a731aac0a916b30f933
https://github.com/banzaicloud/logging-operator/commit/0ec91f72831e1e63bd560224450454b33084553d
I also had to expose a number of these features in their helm charts
https://github.com/banzaicloud/logging-operator/commit/efc74711c5336063a6da72bf39239c57c81c7dff
https://github.com/banzaicloud/logging-operator/commit/f581da2e9daadae9b786362f69d379f8151ad918
https://github.com/banzaicloud/logging-operator/commit/4e74e36dfe7d63212b19401fe645a198734da1fd
wsh
Someone reached out to me privately to report several Solaris 11 compatibility bugs with wsh, my multi-host ssh tool.
Use -std=c11 flag with SunStudio: https://github.com/worr/wsh/commit/b11d2668ef6b85913d1901cfbfe6eb612be69bdc
Don't use __attribute__ with SunStudio, since none of the ones I used were supported: https://github.com/worr/wsh/commit/25ed3fc6fa36a1202e33c8fb36893d03cd5bce8c
Don't unconditionally compile in memset_s (Found because Solaris actually has a libc with this function): https://github.com/worr/wsh/commit/3876745a5cc4bce80d5e9fff0ab70b2dc429287f
This also led to a protobuf PR for better SunStudio support, which it looks like I need to follow up on.
https://github.com/protocolbuffers/protobuf/pull/7049/files
python-fido2
Last post, I mentioned I was working on getting my yubikey to work on OpenBSD. Part of that included adding support in ykman, which also required changes in python-fido2.
First, I added general OpenBSD support
https://github.com/Yubico/python-fido2/pull/82/files
This impl is arguably a bit brittle, since I essentially had to build the device probing for it in scratch from python, using the primitives from libusbhid to probe every uhid(4) device to see if it was a yubikey.
However, some time later, fido(4) was rolled into OpenBSD meaning that this code could be greatly simplified. I think someone reached out to me about this directly? I don't really remember, since it was awhile ago.
https://github.com/Yubico/python-fido2/pull/87/files
What a year
That's basically been the last year or so for me. Honestly, it's been a weird one, and I haven't been able to really do as much OSS as I've wanted to. A lot of it has been through work, which while nice, doesn't touch the types of projects that I want to be doing.
I am working on a gemini server on OpenBSD, which has been feeling quite rewarding, and I have other projects kicking around in my head that I'm going to be following up on.








