Configure a syslog server with rsyslog on Ubuntu
A syslog server represents a central log monitoring point on a network, to which all kinds of devices including Linux or Windows servers, routers, switches or any other hosts can send their logs over network. By setting up a syslog server, you can filter and consolidate logs from different hosts and devices into a single location, so that you can view and archive important log messages more easily.
In this tutorial, we cover how to configure a centralized syslog server using rsyslog on Linux. Before we go into the details, it is instructive to go over syslog standard first.
Basic of Syslog Standard
When logs are collected with syslog mechanism, three important things must be taken into consideration:
- Facility level: what type of processes to monitor
- Severity (priority) level: what type of log messages to collect
- Destination: where to send or record log messages
The facility levels define a way to categorize internal system processes. Some of the common standard facilities in Linux are:
- auth: messages related to authentication (login)
- cron: messages related to scheduled processes or applications
- daemon: messages related to daemons (internal servers)
- kernel: messages related to the kernel
- mail: messages related to internal mail servers
- syslog: messages related to the syslog daemon itself
- lpr: messages related to print servers
- local0 - local7: messages defined by user (local7 is usually used by Cisco and Windows servers)
The severity (priority) levels are standardized, and defined by using standard abbreviation and an assigned number with number 7 being the highest level of all. These levels are:
- emerg: Emergency - 0
- alert: Alerts - 1
- crit: Critical - 2
- err: Errors - 3
- warn: Warnings - 4
- notice: Notification - 5
- info: Information - 6
- debug: Debugging – 7
Finally, the destination statement enforces a syslog client to perform one of three following tasks: (1) save log messages on a local file, (2) route them to a remote syslog server over TCP/UDP, or (3) send them to stdout such as a console.
In rsyslog, syslog configuration is structured based on the following schema.
[facility-level].[severity-level] [destination]
Configure Rsyslog on Ubuntu
Step One: Initial System Requirements
To set up a Linux host as a central log server, we need to create a separate /var partition, and allocate a large enough disk size or create a LVM special volume group. That way, the syslog server will be able to sustain the exponential growth of collected logs over time.
Step Two: Enable Rsyslog Daemon
rsyslog daemon comes pre-installed on modern Linux distributions, but is not enabled by default. To enablersyslog daemon to receive external messages, edit its configuration file located in /etc/rsyslog.conf.
Once the file is opened for editing, search and uncomment the below two lines by removing the # sign from the beginning of lines.
$ModLoad imudp
$UDPServerRun 514
This will enable rsyslog daemon to receive log messages on UDP port 514. UDP is way faster than TCP, but does not provide reliability on data flow the same way as TCP does. If you need to reliable delivery, you can enable TCP by uncommenting the following lines.
$ModLoad imtcp
$InputTCPServerRun 514
Note that both TCP and UDP can be set on the server simultaneously to listen on TCP/UDP connections.
Step Three: Create a Template for Log Receipt
In the next step we need to create a template for remote messages, and tell rsyslog daemon how to record messages received from other client machines.
Open /etc/rsyslog.conf with a text editor, and append the following template before the GLOBAL DIRECTIVES block:
$template RemoteLogs,"/var/log/%HOSTNAME%/%PROGRAMNAME%.log" *
*.* ?RemoteLogs
& ~
This template needs a little explanation. The $template RemoteLogs directive ("RemoteLogs" string can be changed to any other descriptive name) forces rsyslog daemon to write log messages to separate local log files in /var/log/, where log file names are defined based on the hostname of the remote sending machine as well as the remote application that generated the logs. The second line ("*.* ?RemoteLogs") implies that we apply RemoteLogs template to all received logs.
The "& ~" sign represents a redirect rule, and is used to tell rsyslog daemon to stop processing log messages further, and not write them locally. If this redirection is not used, all the remote messages would be also written on local log files besides the log files described above, which means they would practically be written twice. Another consequence of using this rule is that the syslog server's own log messages would only be written to dedicated files named after machine's hostname.
If you want, you can direct log messages with a specific facility or severity level to this new template using the following schema.
[facility-level].[severity-level] ?RemoteLogs
For example:
Direct all internal authentication messages of all priority levels to RemoteLogs template:
authpriv.* ?RemoteLogs
Direct informational messages generated by all system processes, except mail, authentication and cron messages to RemoteLogs template:
*.info,mail.none,authpriv.none,cron.none ?RemoteLogs
If we want all received messages from remote clients written to a single file named after their IP address, you can use the following template. We assign a new name "IpTemplate" to this template.
$template IpTemplate,"/var/log/%FROMHOST-IP%.log" *.* ?IpTemplate & ~
After we have enabled rsyslog daemon and edited its configuration file, we need to restart the daemon.
$ sudo service rsyslog restart
We can verify that rsyslog daemon is functional by using netstat command.
$ sudo netstat -tulpn | grep rsyslog
The output may look like the following:
tcp 0 0 0.0.0.0:514 0.0.0.0:* LISTEN 3267/rsyslogd
tcp6 0 0 :::514 :::* LISTEN 3267/rsyslogd
udp 0 0 0.0.0.0:514 0.0.0.0:* 3267/rsyslogd
udp6 0 0 :::514 :::* 3267/rsyslogd
Step Four: Check Log File
If you receive remote log file, it should be located at /var/log.
