Log file management tramite syslog-ng Giacomo Lanzi

Log File Management with the Secure Online Desktop service

IT systems produce large quantities of log files, very useful tools for guaranteeing data security and application stability. However, in a complex ecosystem, the quantity of files and their location can become two insurmountable obstacles to overcome, in case it is necessary to consult the data efficiently. This is where log management systems come into play, which thanks to technologies such as Syslog-ng, are able to circumvent the problem. In the article, we see how a log file management solution can be a valuable investment.

What is a log file, what is it for

Any action that is performed on a machine or by it can be recorded in a log file. To understand what it is, let’s imagine that it is a ship’s logbook, in which every single event that happened on the boat is noted. In fact, the name derives from the nautical environment, in which the use of a logbook was common. This was nothing more than a diary in which navigation data were recorded at regular intervals: speed, wind strength and direction, water conditions and so on.

With the concept of recording useful information in a file that can be consulted later, the log file contains any changes, actions, states or modifications for security reasons. In case something goes wrong, it is easy to understand what happened by consulting a log file. This is especially true when we talk about servers and applications, data dissemination, IT security, etc.

Amount of log files

Some companies have up to a few dozen servers, others have hundreds, some thousands, and there are others that manage tens of thousands of servers. These systems produce a huge amount of data in the form of log files.

Complicating things is IT architecture. Very often machines are organized into subsystems, both for reasons of convenience and safety. In the unfortunate event that someone wants to consult the log files following an accident, we should despair. Which server holds the data we are interested in? Which subsystem is it in? These are not questions that can be answered simply, especially if you don’t know the source of the problem.

The management of the log files of a system (or Log Management) is essential in the collection of data, prevention and resolution of problems.

Secure Online Desktop Log Management

SOD offers a log management solution through Syslog-ng Premium Edition agents. These are in charge of the collection, transmission and storage of log files. Not only are they collected and centralized in a single virtual place, but the data are also normalized, ie “translated” into standardized formats so that they can be consulted and compared more easily.

Real-time normalization, reporting and classification

Thanks to normalization, it is possible to carry out cross-sectional full-text searches in a few seconds to all the log files collected. Complex operations are guaranteed by the possibility of using wildcards and Boolean operators. The analysis of the collected data is therefore very simplified, which allows the data to also be used to monitor the efficiency of the system, identify possible future problems and intervene before it is too late.

It is also possible to generate customized reports consisting of graphs and statistics with the aim of certifying compliance with standards and regulations such as PCI-DSS, ISO 27001, SOX and HIPAA.

One of the most interesting features of syslog-ng is the ability to automatically classify messages and sort them into classes. These can then be used to label the type of event described in the log. Examples of possible classes: user login, application crash, file transfer, etc.

Extraction and correlation of messages in log files

The classification of messages opens the door to a further functionality: the extraction and correlation of messages. Once each message contained in the log file has been normalized and classified according to your needs, it is possible to assign different tags, to add an additional filter level.

To give an example: once a user’s login messages have been collected, it is possible to label them as user_login, and then isolate them by extracting them and collecting them in a separate file to perform further processing on these messages.

Syslog-ng also makes it possible to correlate events in real time, to prevent data from a single event being scattered across multiple log files. For example, the access and exit data (log-in and log-out) are often recorded far from each other, even in different log files. Through correlation, the data of a single event can be collected and analyzed in isolation.

Automatic backup

The stored log messages and the configuration of the Log Management service can be periodically transferred to a remote server using the following protocols:

– Network File System protocol (NFS)
– Rsync over SSH
– Server Message Block protocol (SMB / CIFS)

Performance

The log file collection and management system with syslog-ng PE agents operates on over 50 platforms, including all Linux distributions and commercial versions of Unix and Windows. The service is able to manage huge quantities of messages, up to over 100,000 per second and over 70 GB of raw log files per hour, from 5000 different sources (servers, applications, etc.).

Ask us for more information about our Log Monitoring service to know specifically how it can be implemented in your systems and how it can help you.

[btnsx id=”2931″]

Useful links:

Log Management

Log Management features

New service | Log Management – High performance service for collecting logs

 

 

 

Share


RSS

More Articles…

Categories …

Tags

RSS Dark Reading:

RSS Full Disclosure

  • Backdoor.Win32.Nbdd.bgz / Remote Stack Buffer Overflow July 27, 2021
    Posted by malvuln on Jul 26Discovery / credits: Malvuln - malvuln.com (c) 2021 Original source: https://malvuln.com/advisory/6fab73bf104c6a9211b94f9559faa134.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Threat: Backdoor.Win32.Nbdd.bgz Vulnerability: Remote Stack Buffer Overflow Description: NetBot_Attacker VIP 5.9 on initial startup listens on port 8080 and on subsequent restarts port 80. Third-party attackers who can reach an infected system […]
  • Backdoor.Win32.Bifrose.acci / Local Stack Buffer Overflow July 27, 2021
    Posted by malvuln on Jul 26Discovery / credits: Malvuln - malvuln.com (c) 2021 Original source: https://malvuln.com/advisory/611dbff0d68df777c6d6881e00440143.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Threat: Backdoor.Win32.Bifrose.acci Vulnerability: Local Stack Buffer Overflow Description: Bifrost doesn't properly validate the IP address when importing Bifrost settings (.set) files. The IP address offset is located after a NULL byte which […]
  • Backdoor.Win32.PsyRat.b / Remote Denial of Service July 27, 2021
    Posted by malvuln on Jul 26Discovery / credits: Malvuln - malvuln.com (c) 2021 Original source: https://malvuln.com/advisory/5817183894cb513239f6aef28895130c_B.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Threat: Backdoor.Win32.PsyRat.b Vulnerability: Remote Denial of Service Description: The PsyRAT 1.02 malware listens by default on TCP port 9863. Third-party attackers who can reach infected systems can send a specially crafted command […]
  • Backdoor.Win32.PsyRat.b / Unauthenticated Remote Command Execution July 27, 2021
    Posted by malvuln on Jul 26Discovery / credits: Malvuln - malvuln.com (c) 2021 Original source: https://malvuln.com/advisory/5817183894cb513239f6aef28895130c.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Threat: Backdoor.Win32.PsyRat.b Vulnerability: Unauthenticated Remote Command Execution Description: The PsyRAT 1.02 malware listens by default on TCP port 9863, but can be changed when building backdoor servers. Third-party attackers who can reach...
  • Backdoor.Win32.Agent.cu / Unauthenticated Remote Command Execution July 27, 2021
    Posted by malvuln on Jul 26Discovery / credits: Malvuln - malvuln.com (c) 2021 Original source: https://malvuln.com/advisory/ce1963d3fd6a8e1383aac40a1f1c4107_C.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Threat: Backdoor.Win32.Agent.cu Vulnerability: Unauthenticated Remote Command Execution Description: The malware listens on TCP ports 10426, 56185. Third-party attackers who can reach infected systems can execute commands made available by the backdoor....
  • Backdoor.Win32.Agent.cu / Port Bounce Scan (MITM) July 27, 2021
    Posted by malvuln on Jul 26Discovery / credits: Malvuln - malvuln.com (c) 2021 Original source: https://malvuln.com/advisory/ce1963d3fd6a8e1383aac40a1f1c4107_B.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Threat: Backdoor.Win32.Agent.cu Vulnerability: Port Bounce Scan (MITM) Description: The malware listens on TCP ports 10426, 56185, its FTP component accepts any username/password credentials. Third-party attackers who successfully logon can abuse the...
  • Backdoor.Win32.Agent.cu / Authentication Bypass RCE July 27, 2021
    Posted by malvuln on Jul 26Discovery / credits: Malvuln - malvuln.com (c) 2021 Original source: https://malvuln.com/advisory/ce1963d3fd6a8e1383aac40a1f1c4107.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Threat: Backdoor.Win32.Agent.cu Vulnerability: Authentication Bypass RCE Description: The malware listens on TCP ports 10426, 56185. Third-party attackers who can reach infected systems can logon using any username/password combination. Intruders may then...
  • Backdoor.Win32.Mazben.me / Unauthenticated Open Proxy July 27, 2021
    Posted by malvuln on Jul 26Discovery / credits: Malvuln - malvuln.com (c) 2021 Original source: https://malvuln.com/advisory/6681d5e4b68abd21a14c704edf9e2ff5.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Threat: Backdoor.Win32.Mazben.me Vulnerability: Unauthenticated Open Proxy Description: The malware listens on random TCP ports like 3515, 7936, 3972. Third-party attackers who can connect to the infected system can relay requests from the […]
  • Backdoor.Win32.Hupigon.aaur / Unauthenticated Open Proxy July 27, 2021
    Posted by malvuln on Jul 26Discovery / credits: Malvuln - malvuln.com (c) 2021 Original source: https://malvuln.com/advisory/16d598c01f7b391986c8c19eded005b1.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Threat: Backdoor.Win32.Hupigon.aaur Vulnerability: Unauthenticated Open Proxy Description: The malware listens on TCP port 8080. Third-party attackers who can connect to the infected system can relay requests from the original connection to the...
  • ATLASSIAN - CVE-2020-36239 - Jira Data Center and Jira Service Management Data Center July 27, 2021
    Posted by Atlassian on Jul 26This email refers to the advisory found at https://confluence.atlassian.com/adminjiraserver/jira-data-center-and-jira-service-management-data-center-security-advisory-2021-07-21-1063571388.html CVE ID: * CVE-2020-36239 Products: Jira Data Center, Jira Core Data Center, Jira Software Data Center, and Jira Service Management Data Center. Affected Versions - Jira Data Center, Jira Core Data Center, and Jira Software Data Center: 6.3.0

Customers

Newsletter