Guide: Sending alerts to Nagios via email

It’s usually not ideal when monitoring a piece of infrastructure to rely on another piece of your infrastructure as no one likes cascading impact, however sometimes it can be the lesser (and simpler) of two evils. So this little guide is a how-to on sending service problems to Nagios via email.

I am going to assume that you are using postfix and have already configured emailing alerts to admins/device owners. It’s also important to note that there are some security implications here as you will essentially be able to set any service to any state and description via email. I’ll discuss ways of preventing bad things from happening as I go… but think about the implications it could have in your environment.

  1. Create a user to receive alerts on the Nagios box and ensure they are in the nagios group:
useradd alerts -g users -G nagios -m
  1. Ask your mail administrator to create an SMTP forwarder for alerts@my-nagios-box.com* that will redirect mail for that address to your Nagios server. Ensure that when that forwarder is setup that only addresses INTERNAL to your company can reach it, bonus points if you can restrict it only to the servers that you are going to be emailing alerts from.

I’ve left this part intentionally a little vague as you could be using anything from exchange to sendmail as your SMTP relay and there are plenty of existing tutorials out there for configuring this kind of stuff that would do a much better job than I would.

obviously my-nagios-box.com is the FQDN for your nagios server.

  1. Install procmail (yum, zypper, apt-get, etc)

  2. Edit /etc/postfix/main.cf to add the following:

inet_interfaces = all
mailbox_command = /usr/bin/procmail -a "$EXTENSION" DEFAULT=$HOME/Mail/ MAILDIR=$HOME/Mail
  1. Create the file /usr/local/nagios/libexec/eventhandlers/mailproc, give the file executable writes (chmod +x mailproc) and then put the following script in the file:
#!/usr/bin/perl
use strict;
use warnings;

my $strIn = "$ARGV[0]";
my $dtNow = `/bin/date +%s`;
chomp $dtNow;
my $strCommandFile = '/usr/local/nagios/var/rw/nagios.cmd';

my %hshStates = ( 'OK' => 0,
                'WARNING' => 1,
                'CRITICAL' => 2,
                'UNKNOWN' => 3 );

my @alert = split(/^/,$strIn);
$alert[0] =~ s/^s+|s+$//g; #STATE
$alert[1] =~ s/^s+|s+$//g; #HOST
$alert[2] =~ s/^s+|s+$//g; #SERVICE
$alert[3] =~ s/^s+|s+$//g; #DESCRIPTION

my $result = "PROCESS_SERVICE_CHECK_RESULT;$alert[1];$alert[2];$hshStates{$alert[0]};$alert[3]";

open(F, ">$strCommandFile") or die print "$!n";
print F "[$dtNow] $resultn";
close F;
  1. Under the new alerts users’ home directory create the file .procmailrc with the following information:
LOGFILE=$HOME/.procmailrc.log
MAILDIR=$HOME/Mail
VERBOSE=yes
PATH=/usr/bin
:0
* ^Subject:[    ]*/[^  ].*
| /usr/local/nagios/libexec/eventhandlers/mailproc "${MATCH}"

So now that we have all the bits what does this do and how does it work? When the Nagios box receives an email, poxtfix will look at the mailbox_command entry and forward that onto procmail for processing. Procmail using the config we created in the alerts user home directory will grab the subject field of the email and then pass that on to our Nagios injection script.

The Nagios injection script will split the subject up using the carrot symbol: ^ as a fieldĀ separatorĀ and pump it into the nagios command file to be processed by the Nagios daemon. The script expects the subject to be in the order of State^Host^Service name^Description text here.

It’s worth noting that the script was written under the assumption that you got bonus points during step two for appropriately restricting your SMTP forward. If you did not then you will probably want to add another field to the Nagios injection script that’s used as a kind of “sanity” password… kind of like an SNMP community string.

Hopefully this helps some one!

comments powered by Disqus