Hadoop Open Data Model: “open” data collection
Estimated reading time: 5 minutes
With the advent of big data platforms, IT security companies can now make guided decisions on how to protect their assets. By recording network traffic and network flows, it is possible to get an idea of the channels on which company information flows. To facilitate the integration of data between the various applications and to develop new analytical functionalities, we the Apache Open Data Model meets.
The common Open Data Model for networks, endpoints and users has several advantages. For example, easier integration between various security applications, but companies are also made it easier to share analytics in case new threats are detected.
Hadoop offers adequate tools to manage a Security Data Lake (SDL) and big data analysis. It can also detect events that are usually difficult to identify, such as lateral movement , data leaks, internal problems or stealth behavior in general. Thanks to the technologies behind the SDL it is possible to collect the data of the SIEM to be able to exploit them through SOCaaS since, being a free Open Data Model, the logs are stored in such a way that they can be used by anyone.
What is Hadoop Open Data Model
Apache Hadoop is free and open source software that helps companies gain insight into their network environments. The analysis of the collected data leads to the identification of potential security threats or any attacks that take place between the resources in the cloud.
While traditional Cyber Threat Intelligence tools help identify threats and attacks in general, an Open Data Model provides a tool that allow companies to detect suspicious connections using flow and packet analysis.
H adoop Open Data Model combines all security-related data (events, users, networks, etc.) into a single visual area that can be used to identify threats effectively. It is You can also use them to create new analytical models. In fact, an Open Data Model allows the sharing and reuse of threat detection models.
An Open Data Model also provides a common taxonomy to describe the security telemetry data used to detect threats. Using data structures and schemas in the Hadoop platform it is possible to collect, store and analyze security-related data.
Open Data Model Hadoop, the advantages for companies
- – Archive a copy of the data security telemetry
- – Leverage out-of-the-box analytics to detect threats targeting DNS, Flow and Proxy
- – Build custom analytics based on your needs
- – Allows third parties to interact with ‘Open Data Model
- – Share and reuse models of threat detection, algorithms, visualizations and analysis from the community Apache Spot .
- – Leverage security telemetry data to better detect threats
- – Using security logs
- – Obtain data from users , endpoints and network entities
- – Obtain threat intelligence data
Open Data Model: types of data collected
To provide a complete security picture and to effectively analyze cyber threat data, you need to collect and analyze all logs and alerts regarding security events and contextual data related to the entities you are dealing with referenced in these logs . The most common entities include the network, users and endpoints, but there are actually many more, such as files and certificates.
Due to the need to collect and analyze security alerts, logs and contextual data, the following types of data are included in the Open Data Model.
Security Event Alerts in Open Data Model
These are event logs from common data sources used to identify threats and better understand network flows. For example operating system logs, IPS logs, firewall logs, proxy logs, web and many more.
Network context data
These include network information that is accessible to anyone from the Whois directory, as well as resource databases and other similar data sources.
User context data
This type of data includes all information relating to the management of users and their identity. Also included are Active Directory, Centrify and other similar systems.
Endpoint context data
Includes all information about endpoint systems (server, router, switch). They can come from asset management systems, vulnerability scanners and detection systems.
Contextual threat data
This data contains contextual information on URLs, domains, websites, files and much more, always related to known threats.
Contextual data on vulnerabilities
This data includes information on vulnerabilities and vulnerability management systems.
Articles from the RoadMap
This is file context data, certificates, naming convention.
Name of attributes
A naming convention is required for an Open Data Model in order to represent attributes between the vendor’s products and technologies. The naming convention consists of prefixes (net, http, src, dst, etc) and common attribute names (ip4, usarname, etc).
It is still a good idea to use multiple prefixes in combination with one attribute.
We have seen what the Hadoop Open Data Model is and how it can be used thanks to its ability to filter traffic and highlight potential cyber attacks by listing suspicious flows, threats to users, threats to endpoints and major network threats.
If you have any doubts or would like further clarification, do not hesitate to contact us by pressing the button below, we will be happy to answer any question.
- The SOAR benefits: simplifying investigation and response
- Security Code Review: How the service works
- Integration of the automated response: the automations in SOCaaS
- Coordination between CTI and SOC: how to further raise the defenses
- New Cloud Server: redundant internet
- Quality certificate for the SOCaaS of SOD
- Managed Detection and Response: a new preventive approach
- CLUSIT: our collaboration for better services
- Backup as a Service (17)
- Cloud Conference (3)
- Cloud CRM (1)
- Cloud Server/VPS (22)
- Conferenza Cloud (4)
- ICT Monitoring (5)
- Log Management (2)
- News (21)
- ownCloud (4)
- Privacy (7)
- Secure Online Desktop (15)
- Security (170)
- Web Hosting (15)
- CyberDanube Security Research 20221124-0 | Authenticated Command Injection Hirschmann BAT-C2 November 29, 2022Posted by Thomas Weber on Nov 29CyberDanube Security Research 20221124-0 ------------------------------------------------------------------------------- title| Authenticated Command Injection product| Hirschmann (Belden) BAT-C2 vulnerable version| 22.214.171.124R8 fixed version| 09.13.01.00R04 CVE number| CVE-2022-40282 impact| High ...
- Exploiting an N-day vBulletin PHP Object Injection Vulnerability November 29, 2022Posted by Egidio Romano on Nov 29Hello list, Just wanted to share with you my latest blog post: http://karmainsecurity.com/exploiting-an-nday-vbulletin-php-object-injection Best regards, /EgiX
- Win32.Ransom.Conti / Crypto Logic Flaw November 29, 2022Posted by malvuln on Nov 29Discovery / credits: Malvuln (John Page aka hyp3rlinx) (c) 2022 Original source: https://malvuln.com/advisory/99e55ce93392068c970384ab24a0e13d.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Backup media: infosec.exchange/@malvuln Threat: Win32.Ransom.Conti Vulnerability: Crypto Logic Flaw Description: Conti ransomware FAILS to encrypt non PE files that have a ".exe" in the filename. Creating specially crafted file names...
- Ransomware Deception Tactics Part 1 November 29, 2022Posted by malvuln on Nov 29Did you know? some Ransomware like CONTI and others will FAIL to encrypt non PE files that have a ".exe" in the filename. Test.exe.docx Test.exe.pdf Conti MD5: 9eb9197cd58f4417a27621c4e1b25a71 ATOMSILO MD5: 5559e9f5e1645f8554ea020a29a5a3ee
- Trojan.Win32.DarkNeuron.gen / Named Pipe Null DACL November 29, 2022Posted by malvuln on Nov 29Discovery / credits: Malvuln (John Page aka hyp3rlinx) (c) 2022 Original source: https://malvuln.com/advisory/d891c9374ccb2a4cae2274170e8644d8.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Backup media: infosec.exchange/@malvuln Threat: Trojan.Win32.DarkNeuron.gen Vulnerability: Named Pipe Null DACL Family: DarkNeuron (Turla Group) Type: PE32 MD5: d891c9374ccb2a4cae2274170e8644d8 Vuln ID: MVID-2022-0661 Disclosure: 11/24/2022...
- Backdoor.Win32.Autocrat.b / Weak Hardcoded Credentials November 29, 2022Posted by malvuln on Nov 29Discovery / credits: Malvuln (John Page aka hyp3rlinx) (c) 2022 Original source: https://malvuln.com/advisory/4262a8b52b902aa2e6bf02a156d1b8d4.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Backup media: infosec.exchange/@malvuln Threat: Backdoor.Win32.Autocrat.b Vulnerability: Weak Hardcoded Credentials Description: The malware is packed with PeCompact, listens on TCP port 8536 and requires authentication. However, the password...
- Backdoor.Win32.Serman.a / Unauthenticated Open Proxy November 29, 2022Posted by malvuln on Nov 29Discovery / credits: Malvuln (John Page aka hyp3rlinx) (c) 2022 Original source: https://malvuln.com/advisory/f312e3a436995b86b205a1a37b1bf10f.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Backup media: infosec.exchange/@malvuln Threat: Backdoor.Win32.Serman.a Vulnerability: Unauthenticated Open Proxy Family: Serman Type: PE32 MD5: f312e3a436995b86b205a1a37b1bf10f Vuln ID: MVID-2022-0659 Disclosure: 11/22/2022 Description: The...
- Open-Xchange Security Advisory 2022-11-24 November 29, 2022Posted by Martin Heiland via Fulldisclosure on Nov 29Dear subscribers, we're sharing our latest advisory with you and like to thank everyone who contributed in finding and solving those vulnerabilities. Feel free to join our bug bounty programs for OX AppSuite, Dovecot and PowerDNS at HackerOne and soon at YesWeHack. Yours sincerely, Martin Heiland, Open-Xchange […]
- [CVE-2022-33942] Intel Data Center Manager Console <= 126.96.36.199749 ”UserMgmtHandler" Authentication Logic Error Leading to Authentication Bypass November 29, 2022Posted by Julien Ahrens (RCE Security) on Nov 29RCE Security Advisory https://www.rcesecurity.com 1. ADVISORY INFORMATION ======================= Product: Intel Data Center Manager Vendor URL: https://www.intel.com/content/www/us/en/developer/tools/data-center-manager-console/overview.html Type: Authentication Bypass by Spoofing [CWE-290] Date found: 2022-06-01 Date published: 2022-11-23 CVSSv3 Score: 10.0 (CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:H) CVE:...
- crashing potplayer again November 29, 2022Posted by houjingyi on Nov 29I disclosured a crash in potplayer last year : https://seclists.org/fulldisclosure/2021/Mar/76 And I found a new one this year, this time is a mid file. Again I contacted Korea Internet & Security Agency(first-team () krcert or kr), they shared report to the onwer of the potplayer, Kakao Corp as they said. […]
Estimated reading time: 6 minutes L'impatto crescente delle minacce informatiche, su sistemi operativi privati op… https://t.co/FimxTS4o9G
Estimated reading time: 6 minutes The growing impact of cyber threats, on private or corporate operating systems… https://t.co/y6G6RYA9n1
Tempo di lettura stimato: 6 minuti Today we are talking about the CTI update of our services. Data security is… https://t.co/YAZkn7iFqa
Estimated reading time: 6 minutes Il tema della sicurezza delle informazioni è di grande attualità in questo peri… https://t.co/tfve5Kzr09
Estimated reading time: 6 minutes The issue of information security is very topical in this historical period ch… https://t.co/TP8gvdRcrF