David - Musings of an SRE

How I configured TrueNAS to send alerts to Telegram and Discord

Some days, I find myself feeling deeply disillusioned about the things I’ve been building, questioning whether I’ve just been deluding myself into thinking they were useful. Other days, it feels like the tools I’ve created truly serve a purpose and have real value.

Today was one of those days where its more of the latter.

Just the other day, I had a HDD die in my TrueNAS but only realize it when investigating why my server fans was suddenly spinning at full speed.

Since I only just started to use TrueNAS just a few months ago, I haven’t setup any alerting system to inform me of a disk’s impending death.

I needed a quick way to get notified.

Most systems/applications that deal closer to Infrastructure work tend to default to using emails as a notification tool (think Proxmox, Nagios).

Because of the complexity of email sending, if you haven’t configured your tool to use a properly configured SMTP server, it will tend to send out your notification emails using the local sendmail which will likely be disregarded by your email providers.

Relaye.io

This was one of the reasons why I’ve built https://relaye.io to help me manage such system notifications better. I wanted a tool that allowed me to configure notification once at the system/tool side, and then allowing me to change how I want to receive the notification (on slack, discord, email etc) without having to reconfigure the notification at the system/tool side.

TrueNAS Alert page

TrueNAS Alert page

When I checked out TrueNAS’s alert settings, they do not have a generic webhook as an option, but they seem to support Slack.

Relaye.io’s multiple inputs selection page

Relaye.io’s multiple inputs selection page

I quickly logged into relaye.io, created a new Webhook Input and passed the url into the Slack configuration

Adding a new slack webhook to TrueNAS

Adding a new slack webhook to TrueNAS

With that configured, I could now configure how I want to receive my notification on relaye.io end.

Viewing relaye.io’s outputs selection

Viewing relaye.io’s outputs selection

With relaye, I can create outputs to a bunch of integrations that I’ve support with many more planned in the coming months ahead.

As I already have an existing output setup, all I need to do is to have it configured to route the Input to an existing Output.

Adding a new route with the Telegram Output

Adding a new route with the Telegram Output

The neat thing is I can also route the same webhook from TrueNAS to multiple outputs at the same time so I can also receive the same notification via Discord if I add a new route to the same Input.

Adding a new route with the Discord output

Adding a new route with the Discord output

Now that I’ve wired the Input to a bunch of Outputs, I can start to send out a test alert on TrueNAS.

Oops

After hitting “Send Test Alert”, nothing seem to happen. I didn’t receive any message or notification on both Telegram and Discord.

I quickly went back into relaye.io to check my Input

Viewing webhook details on relaye.io

Viewing webhook details on relaye.io

Yikes! I forgot to change my template, so relaye wasn’t able to send out any message. By default relaye.io expects the webhook payload to contain a message key, but in this case, there isn’t such a key.

Luckily relaye.io has a way to view the actual TrueNAS webhook payload, allowing me to update the output according to the payload, with a fancy “preview” widget to boot!

Modifying templates with a live preview on relaye.io

Modifying templates with a live preview on relaye.io

With that out of the way, lets test the Alert again.

Viewing the discord notification

Viewing the discord notification

Viewing the telegram notification

Viewing the telegram notification

Nice.