Installing, Configuring, and Connecting RSyslog and Sentry · English

Rsyslog and Sentry logos

I wanted to collect all OS logs (from all servers) in Sentry. Naturally, I’d have to use UDP to transmit the logs; fortunately, Sentry already has UDP support. After discussing it with Sasha Mironov, I decided to use RSyslog. A little googling turned up this Gist: Bridge to send rsyslog logs to sentry (which I then successfully forked and refactored). Now all that remained was to write the rules and get this chain running.

This article is mainly for FreeBSD, but as usual, the configs are the same—just replace “install from ports” with “install from packages.” I don’t think anything else will change significantly.

Starting the UDP Server for Sentry

emacs /etc/sentry.conf

Add the following:

SENTRY_UDP_HOST = '198.51.100.1'  # Внешний IP-адрес вашего сетевого интерфейса
SENTRY_UDP_PORT = 9091

Install the additional applications we need:

. /var/www/sentry/bin/activate
pip install raven eventlet

And check whether the UDP server works:

/var/www/sentry/bin/sentry --config=/etc/sentry.conf start udp

Now add the following lines to Supervisor:

[program:sentry_udp]
directory=/var/www/sentry
command=/var/www/sentry/bin/sentry --config=/etc/sentry.conf start udp
autostart=true
autorestart=true

Update the program list in Supervisor and start sentry_udp:

supervisorctl reread
supervisorctl start sentry_udp

Installing and Configuring RSyslog

Now log in to the server that will send the logs (in our setup, this is the FreeBSD client):

cd /usr/ports/sysutils/rsyslog6
make install clean

Now let’s configure RSyslog, using Nginx logs as an example:

emacs /usr/local/etc/rsyslog.conf


$ModLoad imfile

# Nginx
$InputFileName /var/log/nginx-error.log
$InputFileTag nginx-error:
$InputFileStateFile stat-nginx-error
$InputFileSeverity info

$InputRunFileMonitor

$ActionQueueType           LinkedList   # use asynchronous processing
$ActionQueueFileName       test         # set file name, also enables disk mode
$ActionResumeRetryCount    -1           # infinite retries on insert failure
$ActionQueueSaveOnShutdown on           # save in-memory data if rsyslog shuts down

# Transfer all of the logs to this UDP port
*.* @@127.0.0.1:10514;RSYSLOG_SyslogProtocol23Format

Here, “127.0.0.1” is the IP address of the local RSyslog->Sentry bridge, which I’ll discuss below, and “10514” is its UDP port.

Next, let’s stop Syslog and start RSyslog:

echo 'syslogd_enable="NO"' >> /etc/rc.conf
echo 'rsyslogd_enable="YES"' >> /etc/rc.conf
/etc/rc.d/syslogd stop
/usr/local/etc/rc.d/rsyslogd start

That’s it—we’re now sending logs over UDP to our bridge. All that’s left is to configure it and start it on this same server.

Starting the RSyslog->Sentry Bridge

Thanks to Patrick for saving both my time and yours by writing this bridge. Let’s install the dependencies the bridge needs:

pip install raven loggerglue

Download my fork of the bridge:

fetch -o /path/to/rsyslog_sentry.py https://gist.github.com/raw/4296753/ca09825373ace6cb815cda9a9cc921bd67b11bd6/rsyslog_sentry.py

Now open this file for editing and find the line <your dsn here>. Specify the DSN you got from Sentry (don’t forget to add the UDP port number you specified in SENTRY_UDP_PORT):

client = Client(dsn='udp://<PUBLIC_KEY>:<SECRET_KEY>@sentry.example.org:9091/2')

Of course, you can also transmit logs over http:
client = Client(dsn='http://<PUBLIC_KEY>:<SECRET_KEY>@sentry.example.org/2')

Then find the line where SyslogServer is started (it will listen locally on the UDP port that receives messages from RSyslog) and specify the desired IP address and UDP port:

s = SyslogServer(('127.0.0.1', 10514), SimpleHandler)

At this point, you can already run:

python /path/to/rsyslog_sentry.py

But it’s better to add it to Supervisor:

[program:rsyslog_sentry]
command=python /path/to/rsyslog_sentry.py
autostart=true
autorestart=true

Start it in Supervisor:

supervisorctl reread
supervisorctl start rsyslog_sentry

That’s it—the chain is configured. If you run into problems, drop me a line!

Starting the UDP Server for Sentry

Installing and Configuring RSyslog

Starting the RSyslog->Sentry Bridge

What to Read?