open data Giacomo Lanzi

Hadoop Open Data Model: “open” data collection

Estimated reading time: 5 minutes

With the advent of big data platforms, IT security companies can now make guided decisions on how to protect their assets. By recording network traffic and network flows, it is possible to get an idea of the channels on which company information flows. To facilitate the integration of data between the various applications and to develop new analytical functionalities, we the Apache Open Data Model meets.

The common Open Data Model for networks, endpoints and users has several advantages. For example, easier integration between various security applications, but companies are also made it easier to share analytics in case new threats are detected.

Hadoop offers adequate tools to manage a Security Data Lake (SDL) and big data analysis. It can also detect events that are usually difficult to identify, such as lateral movement , data leaks, internal problems or stealth behavior in general. Thanks to the technologies behind the SDL it is possible to collect the data of the SIEM to be able to exploit them through SOCaaS since, being a free Open Data Model, the logs are stored in such a way that they can be used by anyone.

open data model nodes

What is Hadoop Open Data Model

Apache Hadoop is free and open source software that helps companies gain insight into their network environments. The analysis of the collected data leads to the identification of potential security threats or any attacks that take place between the resources in the cloud.

While traditional Cyber Threat Intelligence tools help identify threats and attacks in general, an Open Data Model provides a tool that allow companies to detect suspicious connections using flow and packet analysis.

H adoop Open Data Model combines all security-related data (events, users, networks, etc.) into a single visual area that can be used to identify threats effectively. It is You can also use them to create new analytical models. In fact, an Open Data Model allows the sharing and reuse of threat detection models.

An Open Data Model also provides a common taxonomy to describe the security telemetry data used to detect threats. Using data structures and schemas in the Hadoop platform it is possible to collect, store and analyze security-related data.

Open Data Model Hadoop, the advantages for companies

  • Archive a copy of the data security telemetry
  • Leverage out-of-the-box analytics to detect threats targeting DNS, Flow and Proxy
  • Build custom analytics based on your needs
  • – Allows third parties to interact with ‘Open Data Model
  • Share and reuse models of threat detection, algorithms, visualizations and analysis from the community Apache Spot .
  • Leverage security telemetry data to better detect threats
  • Using security logs
  • Obtain data from users , endpoints and network entities
  • Obtain threat intelligence data

Open Data Model: types of data collected

To provide a complete security picture and to effectively analyze cyber threat data, you need to collect and analyze all logs and alerts regarding security events and contextual data related to the entities you are dealing with referenced in these logs . The most common entities include the network, users and endpoints, but there are actually many more, such as files and certificates.

Due to the need to collect and analyze security alerts, logs and contextual data, the following types of data are included in the Open Data Model.

Security Event Alerts in Open Data Model

These are event logs from common data sources used to identify threats and better understand network flows. For example operating system logs, IPS logs, firewall logs, proxy logs, web and many more.

Network context data

These include network information that is accessible to anyone from the Whois directory, as well as resource databases and other similar data sources.

User context data

This type of data includes all information relating to the management of users and their identity. Also included are Active Directory, Centrify and other similar systems.

Endpoint context data

Includes all information about endpoint systems (server, router, switch). They can come from asset management systems, vulnerability scanners and detection systems.

Contextual threat data

This data contains contextual information on URLs, domains, websites, files and much more, always related to known threats.

Contextual data on vulnerabilities

This data includes information on vulnerabilities and vulnerability management systems.

Articles from the RoadMap

This is file context data, certificates, naming convention.

open data model cover

Name of attributes

A naming convention is required for an Open Data Model in order to represent attributes between the vendor’s products and technologies. The naming convention consists of prefixes (net, http, src, dst, etc) and common attribute names (ip4, usarname, etc).

It is still a good idea to use multiple prefixes in combination with one attribute.

Conclusions

We have seen what the Hadoop Open Data Model is and how it can be used thanks to its ability to filter traffic and highlight potential cyber attacks by listing suspicious flows, threats to users, threats to endpoints and major network threats.

If you have any doubts or would like further clarification, do not hesitate to contact us by pressing the button below, we will be happy to answer any question.

Useful links:

Share


RSS

More Articles…

Categories …

Tags

RSS Dark Reading

RSS Full Disclosure

  • Win32.MarsStealer Web Panel / Unauthenticated Remote Data Deletion January 16, 2022
    Posted by malvuln on Jan 16Discovery / credits: Malvuln - malvuln.com (c) 2022 Original source: https://malvuln.com/advisory/8abb41f6e7010d70c90f65fd9a740faa_C.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Threat: Win32.MarsStealer Web Panel Vulnerability: Unauthenticated Remote Data Deletion Description: The Mars-Stealer web interface has a "Grab Rules" component area that lets a user specify which type of files to collect from […]
  • Win32.MarsStealer Web Panel / Unauthenticated Remote Persistent XSS January 16, 2022
    Posted by malvuln on Jan 16Discovery / credits: Malvuln - malvuln.com (c) 2022 Original source: https://malvuln.com/advisory/8abb41f6e7010d70c90f65fd9a740faa_B.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Threat: Win32.MarsStealer Web Panel Vulnerability: Unauthenticated Remote Persistent XSS Description: The Mars-Stealer web interface has a "Marker Rules" component area. Third-party attackers who can reach the Mars-Stealer server can send HTTP...
  • Win32.MarsStealer Web Panel / Unauthenticated Remote Information Disclosure January 16, 2022
    Posted by malvuln on Jan 16Discovery / credits: Malvuln - malvuln.com (c) 2022 Original source: https://malvuln.com/advisory/8abb41f6e7010d70c90f65fd9a740faa.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Threat: Win32.MarsStealer Web Panel Vulnerability: Unauthenticated Remote Information Disclosure Description: The malware web interface stores screen captures named "screenshot.jpg" in the panel directory, ZIP archived. Third-party attackers who...
  • Ab Stealer Web Panel / Unauthenticated Remote Persistent XSS January 16, 2022
    Posted by malvuln on Jan 16Discovery / credits: Malvuln - malvuln.com (c) 2022 Original source: https://malvuln.com/advisory/9e44c10307aa8194753896ecf8102167.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Threat: Ab Stealer Web Panel Vulnerability: Unauthenticated Remote Persistent XSS Description: The "Ab Stealer" web Panel By KingDomSc for "AbBuild v.1.0.exe" is used to browse victim information "Get All Victims Passwords, With...
  • SEC Consult SA-20220113-0 :: Cleartext Storage of Phone Password in Cisco IP Phones January 14, 2022
    Posted by SEC Consult Vulnerability Lab, Research on Jan 14SEC Consult Vulnerability Lab Security Advisory < 20220113-0 > ======================================================================= title: Cleartext Storage of Phone Password product: Cisco IP Phone Series 78x1, 88x5, 88x1, 7832, 8832, 8821 and 3905 vulnerable version: Firmware
  • 🐞 Call for Papers for Hardwear.io USA 2022 is OPEN! January 14, 2022
    Posted by Andrea Simonca on Jan 14Hello, We are happy to announce that the CFP for Hardwear.io USA 2022 is OPEN! If you have a groundbreaking embedded research or an awesome open-source tool you’d like to showcase before the global hardware security community, this is your chance. Send in your ideas on various hardware subjects, […]
  • APPLE-SA-2022-01-12-1 iOS 15.2.1 and iPadOS 15.2.1 January 12, 2022
    Posted by Apple Product Security via Fulldisclosure on Jan 12APPLE-SA-2022-01-12-1 iOS 15.2.1 and iPadOS 15.2.1 iOS 15.2.1 and iPadOS 15.2.1 addresses the following issues. Information about the security content is also available at https://support.apple.com/HT213043. HomeKit Available for: iPhone 6s and later, iPad Pro (all models), iPad Air 2 and later, iPad 5th generation and later, […]
  • Reprise License Manager 14.2 - Reflected Cross-Site Scripting January 12, 2022
    Posted by Gionathan Reale via Fulldisclosure on Jan 12# Product:  RLM 14.2 # Vendor:   Reprise Software # CVE ID:   CVE-2021-45422 # Vulnerability Title: Reflected Cross-Site Scripting # Severity: Medium # Author(s): Giulia Melotti Garibaldi # Date:     2022-01-11 # ############################################################# Introduction: An issue was discovered in Reprise License Manager 14.2, Reprise License Manager 14.2 is affected […]
  • [RT-SA-2021-009] Credential Disclosure in Web Interface of Crestron Device January 12, 2022
    Posted by RedTeam Pentesting GmbH on Jan 12Advisory: Credential Disclosure in Web Interface of Crestron Device When the administrative web interface of the Crestron HDMI switcher is accessed unauthenticated, user credentials are disclosed which are valid to authenticate to the web interface. Details ======= Product: Crestron HD-MD4X2-4K-E Affected Versions: 1.0.0.2159 Fixed Versions: - Vulnerability Type: […]
  • Backdoor.Win32.Controlit.10 / Unauthenticated Remote Command Execution January 11, 2022
    Posted by malvuln on Jan 11Discovery / credits: Malvuln - malvuln.com (c) 2022 Original source: https://malvuln.com/advisory/859aab793a42868343346163bd42f485.txt Contact: malvuln13 () gmail com Media: twitter.com/malvuln Threat: Backdoor.Win32.Controlit.10 Vulnerability: Unauthenticated Remote Command Execution Description: The malware listens on TCP port 3347. Third-party attackers who can reach an infected system can run any OS commands made available by the […]

Customers

Newsletter