My journey running scheduled jobs using systemd has been going fine. But one challenge I am facing is that because systemd does things in the background — which is good and as intended — it can easily fail silently and I won’t know that jobs aren’t completing. Of course this matters more for some things than others. But as a general rule, I want to know when things aren’t working.

I initially attempted to set up desktop notifications for this purpose, but this quickly led to some bash hackery that I was not entirely comfortable with, so I backed off. Then I tried setting up a Teams notification, but working with Teams was irritating: systemd worked just fine, but Teams was incredibly hard to debug. The third (and final) approach I took was to set up notifications as Gitea issues. This meant using the Gitea API, which is mercifully much more straightforward than Teams. I am in Gitea a lot these days, as more of my workflows have been migrating there, so it is an obvious place for me to check notifications.

All of this has led me further into the details of systemd, and has shown that notifications are a harder problem than I had anticipated. Anyhow, my setup seems reasonably functional now, so I am happy with the situation.