You might be a Sysadmin, developer, DBA or whatever, logs are like treasure boxes for anyone working in IT. And the best practice to keep logs in a central location together with local copy. Most of the logging programs have the ability to send logs to a remote logging server (as well as receive logs from remote machines); eg rsyslog, syslog-ng etc.
But, still there is a concern for sending server/application/database logs sending over tcp as plain text; yes indeed. But no need to worry as most of the logging programs will have simple mechanisms to implement TLS Tunnels for sending and receiving logs. In below demo, we will implement TLS tunnel to send logs from one machine (using syslog-ng) and receive the logs on another logging server (syslog-ng).
You will most likely need to enable Extra Packages for Enterprise Linux (EPEL)
# cd /etc/yum.repos.d/
# wget https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng321/repo/epel-7/czanik-syslog-ng321-epel-7.repo
# yum install syslog-ng
# systemctl enable syslog-ng
# systemctl start syslog-ng
Note : If you have rsyslog or other logging systems running, you need to stop that first and configure custom items (if any) in syslog-ng manually.
We will store the keys in /etc/syslog-ng/ssl
# cd /etc/syslog-ng
# mkdir ssl
# cd ssl
# openssl genrsa -des3 -out logserver.key 2048
# openssl req -new -key logserver.key -out logserver.csr
cp logserver.key logserver.key.org
openssl rsa -in logserver key.org -out logserver.key
openssl x509 -req -days 365 -in logserver.csr -signkey logserver.key -out logserver.crt
Edit /etc/syslog-ng/syslog-ng.conf
and add below section.
(You can either directly edit /etc/syslog-ng/syslog-ng.conf
or add separate configuration file under conf.d
for easy configuration file management. eg: /etc/syslog-ng/conf.d/mylog.conf
)
# Step 1 - TLS Source Listen
source source_514_tls {
tcp(port(514)
tls(
# SSL Certificates which we have created in previous steps
key_file("/etc/syslog-ng/ssl/logserver.key")
cert_file("/etc/syslog-ng/ssl/logserver.crt")
peer_verify(optional-untrusted)
)
flags(no-multi-line)
);
};
This can be another logging server (eg: Elastic, Splunk, ArcSight Connectors etc) or a local destination.
# Step 2 - Configure Destinations
# Local Destination on same server
destination dest_514_local { file("/var/log/messages_514"); };
# Destination on another server:1514
destination dest_1514 {
tcp("10.1.10.100" port (1514)
};
# Step 3 - Configure Source to Destination Logging
# Forward Logs from 514 to another server:1514
log { source(source_514_tls ); destination(dest_1514); };
# Log same logs received on 514 to locally.
log { source(source_514_tls ); destination(dest_514_local); };
Here client means any machine who send data to connector machine
You need to get the certificate from logging server and configure to send data over TLS.
Download logserver.crt and keep it under directory /etc/syslog-ng/ssl/ssl-for-client/ (or any other suitable location).
# ls -l /etc/syslog-ng/ssl/ssl-for-client/
total 4
-rw-r--r-- 1 root root 1257 Jun 27 09:52 logserver.crt
# openssl x509 -noout -hash -in /etc/syslog-ng/ssl/ssl-for-client/logserver.crt
acd0d3bb
Create a symbolic link to the certificate that uses the hash returned by the previous command, with an added .0 suffix.
# ln -s /etc/syslog-ng/ssl/ssl-for-client/logserver.crt /etc/syslog-ng/ssl/ssl-for-client/84d92a45.0
# ls -l /etc/syslog-ng/ssl/ssl-for-client/
total 4
lrwxrwxrwx 1 root root 47 Jun 27 09:54 acd0d3bb.0 -> /etc/syslog-ng/ssl/ssl-for-client/logserver.crt
-rw-r--r-- 1 root root 1257 Jun 27 09:52 logserver.crt
Please note the certificate path mentioned at client side in destination d_mesg_copy.
# Step 1 - Destination 10.1.10.200:514
destination d_mesg_copy {
tcp("10.1.10.200" port (514)
tls(
ca_dir("/etc/syslog-ng/ssl/ssl-for-client/")
)
);
};
# Step 2 - Send Copy of var-log-messages to d_mesg_copy
log { source(s_sys); filter(f_default); destination(d_mesg_copy); };
Make sure you have enabled and restarted syslog-ng daemons on both servers after config updates.
Disclaimer:
The views expressed and the content shared in all published articles on this website are solely those of the respective authors, and they do not necessarily reflect the views of the author’s employer or the techbeatly platform. We strive to ensure the accuracy and validity of the content published on our website. However, we cannot guarantee the absolute correctness or completeness of the information provided. It is the responsibility of the readers and users of this website to verify the accuracy and appropriateness of any information or opinions expressed within the articles. If you come across any content that you believe to be incorrect or invalid, please contact us immediately so that we can address the issue promptly.
Gineesh Madapparambath
Gineesh Madapparambath is the founder of techbeatly and he is the co-author of The Kubernetes Bible, Second Edition. and the author of 𝗔𝗻𝘀𝗶𝗯𝗹𝗲 𝗳𝗼𝗿 𝗥𝗲𝗮𝗹-𝗟𝗶𝗳𝗲 𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗶𝗼𝗻.
He has worked as a Systems Engineer, Automation Specialist, and content author. His primary focus is on Ansible Automation, Containerisation (OpenShift & Kubernetes), and Infrastructure as Code (Terraform).
(aka Gini Gangadharan - iamgini.com)
This site uses Akismet to reduce spam. Learn how your comment data is processed.2 Responses
Leave a Reply Cancel reply
Doesnt work. problem with key certificate.
Error setting up TLS session context; tls_error=’digital envelope routines:EVP_DecryptFinal_ex:bad decrypt’
This is something wrong with the certificate generation. You can try any methods to generate the certificate.
Also, share the output of history command so that we can see if anything wrong with command executed.
You may contact us via telegram: t.me/techbeatly