How to Setup Rsyslog Client to Send Logs to Rsyslog Server in CentOS 7

Log management is one of the most critical component in a network infrastructure. Logs messages are constantly generated by numerous system software, such as utilities, applications, daemons, services related to network, kernel, physical devices and so on.

Log files proves to be useful in case of troubleshooting Linux system issues, monitor the system and review a system security strength and problems.

Rsyslog is an Open Source logging program, which is the most popular logging mechanism in a huge number of Linux distributions. It’s also the default logging service in CentOS 7 or RHEL 7.

Rsyslog daemon in CentOS can be configured to run as a server in order collect log messages from multiple network devices. These devices act as clients and are configured to transmit their logs to a rsyslog server.

However, the Rsyslog service can be also configured and started in client mode. This setup instructs the rsyslog daemon to forward log messages to a remote Rsyslog server using the TCP or UDP transport protocols. Rsyslog service can also be configured to run as a client and as a server in the same time.

In this tutorial we’ll describe how to setup a CentOS/RHEL 7 Rsyslog daemon to send log messages to a remote Rsyslog server. This setup ensures that your machine disk space can be preserved for storing other data.

The place where almost all log files are written by default in CentOS is the /var system path. It’s also advisable to always create a separate partition for /var directory, which can be dynamically grown, in order to not exhaust the /(root) partition.

An Rsyslog client always sends the log messages in plain text, if not specified otherwise. You should not setup an Rsyslog client to transmit log messages over Internet or networks that are not under your complete control.

Requirements

1. CentOS 7.3 Installation Procedure
2. RHEL 7.3 Installation Procedure
3. Configure a Rsyslog Server in CentOS/RHEL 7

Step 1: Verify Rsyslog Installation

1. By default, the Rsyslog daemon is already installed and running in a CentOS 7 system. In order to verify if rsyslog service is present in the system, issue the following commands.

# rpm -q | grep rsyslog
# rsyslogd -v

2. If the Rsyslog package is not installed in CentOS, execute the below command to install the service.

# yum install rsyslog

Step 2: Configure Rsyslog Service as Client

3. In order to enforce the Rsyslog daemon installed on a CentOS 7 system to act as a log client and route all of locally generated log messages to a remote Rsyslog server, modify the rsyslog configuration file as follows:

First open the main configuration file for editing.

# vi /etc/rsyslog.conf

Then, append the below line at the end of the file as illustrated in the below excerpt.

*. *  @192.168.10.254:514

On the above line makes sure you replace the IP address of the FQDN of the remote rsyslog server accordingly. The above line instructs the Rsyslog daemon to send all log messages, regardless of the facility or severity, to the host with the IP 192.168.10.254 via 514/UDP port.

4. If the remote log server is configured to listen only on TCP connections or you want to use a reliable transport network protocol, such as TCP, add another @ character in front of the remote host as shown in the below example:

*. *  @@logs.domain.lan:514

The Linux rsyslog also allows has some special characters, such as = or !, which can be prefixed to priority levels to indicate “this priority only” for equal sign and “not this priority or higher than this”.

Some samples of Rsyslog priority level qualifiers in CentOS 7:

• kern.info = kernel logs with info priority and higher.
• kern.=info = only kernel messages with info priority.
• kern.info;kern.!err = only kernel messages with info, notice, and warning priorities.
• kern.debug;kern.!=warning = all kernel priorities except warning.
• kern.* = all kernel priorities messages.
• kern.none = don’t log any related kernel facility messages regardless of the priority.

For instance, assuming you want to send only a specific facility messages to a remote log server, such as all related mail messages regardless of the priority level, add the below line to rsyslog configuration file:

mail.* @192.168.10.254:514 

5. Finally, in order to apply the new configuration, Rsyslog service needs to be restarted in order for the daemon to pick-up the changes, by running the below command:

# systemctl restart rsyslog.service

6. If for some reasons Rsyslog daemon is not enabled during the boot time, issue the below command to enable the service system-wide:

# systemctl enable rsyslog.service

Step 3: Send Apache and Nginx Logs to a Remote Log Server

7. Apache HTTP server can be configured to send logs messages to a remote syslog server by adding the following line to its main configuration file as illustrated in the below example.

# vi /etc/httpd/conf/httpd.conf

On Apache main conf file add the below line.

CustomLog "| /bin/sh -c '/usr/bin/tee -a /var/log/httpd/httpd-access.log | /usr/bin/logger -thttpd -plocal1.notice'" combined

The line will enforce the HTTP daemon to write the log messages internally to the filesystem log file, but also process the messages further through a pipe to logger utility, which will send them to a distant syslog server, by marking them as coming from the local1 facility.

8. If you want to also direct Apache error log messages to a remote syslog server, add a new rule as the one presented in the above example, but make sure to replace the name of the httpd log file and the log file severity level to match error priority, as shown in the following sample:

ErrorLog "|/bin/sh -c '/usr/bin/tee -a /var/log/httpd/httpd-error.log | /usr/bin/logger -thttpd -plocal1.err'"

9. Once you’ve added the above lines, you need to restart Apache daemon to apply changes, by issuing the following command:

# systemctl restart httpd.service                 

10. As of version 1.7.1, Nginx web server has build-in capabilities in order to directly log its messages to a remote syslog server, by adding the following lines of code to an nginx configuration file.

error_log syslog:server=192.168.1.10:514,facility=local7,tag=nginx,severity=error;
access_log syslog:server=192.168.10.254:514,facility=local7,tag=nginx,severity=info main;

For an IPv6 server, use the following syntax format to enclose the IPv6 address.

access_log syslog:server=[7101:dc7::9]:514,facility=local7,tag=nginx,severity=info;

11. On the remote Rsyslog server you need to make the following change to rsyslog configuration file, in order to receive the logs send by Apache web server.

local1.* @Apache_IP_address:514

That’s all! You have successfully configured Rsyslog daemon to run in client mode and, also, you’ve instructed Apache HTTP server or Nginx to forward its log messages to a remote syslog server.

In case you system crashes, you should be able to investigate the problem by inspecting the log files content which are stored on the remote syslog server.