Log File Management with the Secure Online Desktop service
IT systems produce large quantities of log files, very useful tools for guaranteeing data security and application stability. However, in a complex ecosystem, the quantity of files and their location can become two insurmountable obstacles to overcome, in case it is necessary to consult the data efficiently. This is where log management systems come into play, which thanks to technologies such as Syslog-ng, are able to circumvent the problem. In the article, we see how a log file management solution can be a valuable investment.
What is a log file, what is it for
Any action that is performed on a machine or by it can be recorded in a log file. To understand what it is, let’s imagine that it is a ship’s logbook, in which every single event that happened on the boat is noted. In fact, the name derives from the nautical environment, in which the use of a logbook was common. This was nothing more than a diary in which navigation data were recorded at regular intervals: speed, wind strength and direction, water conditions and so on.
With the concept of recording useful information in a file that can be consulted later, the log file contains any changes, actions, states or modifications for security reasons. In case something goes wrong, it is easy to understand what happened by consulting a log file. This is especially true when we talk about servers and applications, data dissemination, IT security, etc.
Amount of log files
Some companies have up to a few dozen servers, others have hundreds, some thousands, and there are others that manage tens of thousands of servers. These systems produce a huge amount of data in the form of log files.
Complicating things is IT architecture. Very often machines are organized into subsystems, both for reasons of convenience and safety. In the unfortunate event that someone wants to consult the log files following an accident, we should despair. Which server holds the data we are interested in? Which subsystem is it in? These are not questions that can be answered simply, especially if you don’t know the source of the problem.
The management of the log files of a system (or Log Management) is essential in the collection of data, prevention and resolution of problems.
Secure Online Desktop Log Management
SOD offers a log management solution through Syslog-ng Premium Edition agents. These are in charge of the collection, transmission and storage of log files. Not only are they collected and centralized in a single virtual place, but the data are also normalized, ie “translated” into standardized formats so that they can be consulted and compared more easily.
Real-time normalization, reporting and classification
Thanks to normalization, it is possible to carry out cross-sectional full-text searches in a few seconds to all the log files collected. Complex operations are guaranteed by the possibility of using wildcards and Boolean operators. The analysis of the collected data is therefore very simplified, which allows the data to also be used to monitor the efficiency of the system, identify possible future problems and intervene before it is too late.
It is also possible to generate customized reports consisting of graphs and statistics with the aim of certifying compliance with standards and regulations such as PCI-DSS, ISO 27001, SOX and HIPAA.
One of the most interesting features of syslog-ng is the ability to automatically classify messages and sort them into classes. These can then be used to label the type of event described in the log. Examples of possible classes: user login, application crash, file transfer, etc.
Extraction and correlation of messages in log files
The classification of messages opens the door to a further functionality: the extraction and correlation of messages. Once each message contained in the log file has been normalized and classified according to your needs, it is possible to assign different tags, to add an additional filter level.
To give an example: once a user’s login messages have been collected, it is possible to label them as user_login, and then isolate them by extracting them and collecting them in a separate file to perform further processing on these messages.
Syslog-ng also makes it possible to correlate events in real time, to prevent data from a single event being scattered across multiple log files. For example, the access and exit data (log-in and log-out) are often recorded far from each other, even in different log files. Through correlation, the data of a single event can be collected and analyzed in isolation.
The stored log messages and the configuration of the Log Management service can be periodically transferred to a remote server using the following protocols:
– Network File System protocol (NFS)
– Rsync over SSH
– Server Message Block protocol (SMB / CIFS)
The log file collection and management system with syslog-ng PE agents operates on over 50 platforms, including all Linux distributions and commercial versions of Unix and Windows. The service is able to manage huge quantities of messages, up to over 100,000 per second and over 70 GB of raw log files per hour, from 5000 different sources (servers, applications, etc.).
Ask us for more information about our Log Monitoring service to know specifically how it can be implemented in your systems and how it can help you.
- What is it for? Hadoop Security Data Lake (SDL)
- Secure Online Desktop achieves ISO 27001: the security certification for managed services
- SOCaaS and Active Defense Deception Webinar – Guide to the next cybersecurity online event
- Auditing IT della sicurezza: guida completa all’analisi proattiva di vulnerabilità e conformità
- CIS Controls and Vulnerability Assessment: practical guide to adopting best practices
- Kerberoasting: a threat to cybersecurity and how to mitigate it with Security Posture analysis
- Protect Your Business: Antivirus vs. SOC Service with EDR and Next Generation Antivirus (NGA)
- CSIRT and SOC: Differences between incident management and security monitoring
- Backup as a Service (17)
- Cloud Conference (3)
- Cloud CRM (1)
- Cloud Server/VPS (22)
- Conferenza Cloud (4)
- ICT Monitoring (5)
- Log Management (2)
- News (23)
- ownCloud (4)
- Privacy (7)
- Secure Online Desktop (14)
- Security (190)
- Web Hosting (15)
- Interpol Arrests Smuggler With New Biometric Screening Database December 1, 2023Interpol has upgraded its biometric background check tech. It'll help catch criminals, but will it protect sensitive, immutable data belonging to the innocent?
- The European Space Agency Explores Cybersecurity for Space Industry December 1, 2023An ESA cybersecurity expert explains how space-based data and services benefit from public investment in space programs.
- The Latest Delinea Secret Server Release Boosts Usability With New Features December 1, 2023
- Flow Security Launches GenAI DLP December 1, 2023
- North Korea APT Slapped With Cyber Sanctions After Satellite Launch December 1, 2023Sanctions on Kimsuky/APT43 focuses the world on disrupting DPRK regime's sprawling cybercrime operations, expert says.
- Critical 'LogoFAIL' Bugs Offer Secure Boot Bypass for Millions of PCs December 1, 2023Hundreds of consumer and enterprise-grade x86 and ARM models from various vendors, including Intel, Acer, and Lenovo, are potentially vulnerable to bootkits and takeover.
- Japan's Space Program at Risk After Microsoft Active Directory Breach December 1, 2023The agency, known as JAXA, has shut down parts of its network as it conducts an investigation to discover the scope and impact of the breach.
- Emirates CISOs Flag Rampant Cybersecurity Gaps December 1, 2023UAE security leaders warn that people, tech, and process gaps are exposing their organizations to cybercrime.
- Saudi Companies Outsource Cybersecurity Amid 'Serious' Incidents December 1, 2023Saudi companies are seeking extra help in droves, because of a lack of tools and personnel.
- The US Needs to Follow Germany's Attack-Detection Mandate December 1, 2023A more proactive approach to fighting cyberattacks for US companies and agencies is shaping up under the CISA's proposal to emphasize real-time attack detection and response.
- SEC Consult SA-20231123 :: Uninstall Key Caching in Fortra Digital Guardian Agent Uninstaller November 27, 2023Posted by SEC Consult Vulnerability Lab, Research via Fulldisclosure on Nov 27SEC Consult Vulnerability Lab Security Advisory < 20231123-0 > ======================================================================= title: Uninstall Key Caching product: Fortra Digital Guardian Agent Uninstaller (Data Loss Prevention) vulnerable version: Agent:
- SEC Consult SA-20231122 :: Multiple Vulnerabilities in m-privacy TightGate-Pro November 27, 2023Posted by SEC Consult Vulnerability Lab, Research via Fulldisclosure on Nov 27SEC Consult Vulnerability Lab Security Advisory < 20231122-0 > ======================================================================= title: Multiple Vulnerabilities product: m-privacy TightGate-Pro vulnerable version: Rolling Release, servers with the following package versions are vulnerable: tightgatevnc < 4.1.2~1 rsbac-policy-tgpro
- Senec Inverters Home V1, V2, V3 Home & Hybrid Use of Hard-coded Credentials - CVE-2023-39169 November 27, 2023Posted by Phos4Me via Fulldisclosure on Nov 27Sent through the Full Disclosure mailing list https://nmap.org/mailman/listinfo/fulldisclosure Web Archives & RSS: https://seclists.org/fulldisclosure/
- [SYSS-2023-019] SmartNode SN200 - Unauthenticated OS Command Injection November 27, 2023Posted by Maurizio Ruchay via Fulldisclosure on Nov 27Advisory ID: SYSS-2023-019 Product: SmartNode SN200 Analog Telephone Adapter (ATA) & VoIP Gateway Manufacturer: Patton LLC Affected Version(s):
- CVE-2023-46307 November 27, 2023Posted by Kevin on Nov 27running on the remote port specified during setup
- CVE-2023-46307 November 27, 2023Posted by Kevin on Nov 27While conducting a penetration test for a client, they were running an application called etc-browser which is a public GitHub project with a Docker container. While fuzzing the web server spun up with etcd-browser (which can run on any arbitrary port), the application had a Directory Traversal vulnerability that is […]
- Survey on usage of security advisories November 27, 2023Posted by Aurich, Janik on Nov 27Dear list members, we are looking for voluntary participants for our survey, which was developed in the context of a master thesis at the University of Erlangen-Nuremberg. The goal of the survey is to determine potential difficulties that may occur when dealing with security advisories. The focus of the […]
- [CVE-2023-46386, CVE-2023-46387, CVE-2023-46388, CVE-2023-46389] Multiple vulnerabilities in Loytec products (3) November 27, 2023Posted by Chizuru Toyama on Nov 27[+] CVE : CVE-2023-46386, CVE-2023-46387, CVE-2023-46388, CVE-2023-46389 [+] Title : Multiple vulnerabilities in Loytec L-INX Automation Servers [+] Vendor : LOYTEC electronics GmbH [+] Affected Product(s) : LINX-151, Firmware 7.2.4, LINX-212, firmware 6.2.4 [+] Affected Components : L-INX Automation Servers [+] Discovery Date :...
- [CVE-2023-46383, CVE-2023-46384, CVE-2023-46385] Multiple vulnerabilities in Loytec products (2) November 27, 2023Posted by Chizuru Toyama on Nov 27[+] CVE : CVE-2023-46383, CVE-2023-46384, CVE-2023-46385 [+] Title : Multiple vulnerabilities in Loytec LINX Configurator [+] Vendor : LOYTEC electronics GmbH [+] Affected Product(s) : LINX Configurator 7.4.10 [+] Affected Components : LINX Configurator [+] Discovery Date : 01-Sep-2021 [+] Publication date : 03-Nov-2023 [+]...
- Senec Inverters Home V1, V2, V3 Home & Hybrid Exposure of the Username to an Unauthorized Actor - CVE-2023-39168 November 12, 2023Posted by Phos4Me via Fulldisclosure on Nov 12Sent through the Full Disclosure mailing list https://nmap.org/mailman/listinfo/fulldisclosure Web Archives & RSS: https://seclists.org/fulldisclosure/
Estimated reading time: 6 minutes L'impatto crescente delle minacce informatiche, su sistemi operativi privati op… https://t.co/FimxTS4o9G
Estimated reading time: 6 minutes The growing impact of cyber threats, on private or corporate operating systems… https://t.co/y6G6RYA9n1
Tempo di lettura stimato: 6 minuti Today we are talking about the CTI update of our services. Data security is… https://t.co/YAZkn7iFqa
Estimated reading time: 6 minutes Il tema della sicurezza delle informazioni è di grande attualità in questo peri… https://t.co/tfve5Kzr09
Estimated reading time: 6 minutes The issue of information security is very topical in this historical period ch… https://t.co/TP8gvdRcrF