Querying DShield from Cortex

Published: 2018-11-20
Last Updated: 2018-11-20 13:40:22 UTC
by Xavier Mertens (Version: 2)
2 comment(s)

Cortex is a tool part of the TheHive project[1]. As stated on the website, it is a "Powerful Observable Analysis Engine". Cortex can analyze observables like IP addresses, emails, hashes, filenames against a huge (and growing) list of online services. I like the naming convention used by Cortex. We have "observables" that can be switched later to an "IOC" later if they are really relevant for us. Keep in mind that an interesting IOC for you could be totally irrelevant in another environment.

What makes Cortex so powerful and convenient is the long list of "analysers" (that's how they call the plugins). Though those small pieces of code, you can, in one click, search for observables in many sources. Cortex is available through a web interface but its REST API makes it easy to interconnect with other tools to enrich the data. Two popular tools that can interact with Cortex are MISP[2] and TheHive[3]. From their web interface, I can easily enrich data using the following analyzers (they are enabled in my own instance of TheHive):

  • Abuse_Finder_2_0
  • CIRCLPassiveDNS_2_0
  • CIRCLPassiveSSL_2_0
  • Censys_1_0
  • Cymon_Check_IP_2_1
  • DShield_lookup_1_0
  • DomainTools_ReverseIP_2_0
  • DomainTools_ReverseNameServer_2_0
  • DomainTools_ReverseWhois_2_0
  • DomainTools_Risk_2_0
  • DomainTools_WhoisHistory_2_0
  • DomainTools_WhoisLookup_2_0
  • DomainTools_WhoisLookup_IP_2_0
  • EmlParser_1_0
  • FileInfo_3_0
  • Fortiguard_URLCategory_2_0
  • HybridAnalysis_GetReport_1_0
  • MISPWarningLists_1_0
  • MISP_2_0
  • MaxMind_GeoIP_3_0
  • Msg_Parser_2_0
  • OTXQuery_2_0
  • Onyphe_Forward_1_0
  • Onyphe_Geolocate_1_0
  • Onyphe_Ports_1_0
  • Onyphe_Reverse_1_0
  • Onyphe_Threats_1_0
  • PassiveTotal_Enrichment_2_0
  • PassiveTotal_Malware_2_0
  • PassiveTotal_Osint_2_0
  • PassiveTotal_Passive_Dns_2_0
  • PassiveTotal_Ssl_Certificate_Details_2_0
  • PassiveTotal_Ssl_Certificate_History_2_0
  • PassiveTotal_Unique_Resolutions_2_0
  • PassiveTotal_Whois_Details_2_0
  • Robtex_Forward_PDNS_Query_1_0
  • Robtex_IP_Query_1_0
  • Robtex_Reverse_PDNS_Query_1_0
  • Shodan_Host_1_0
  • Shodan_Search_1_0
  • URLhaus_1_0
  • VirusTotal_GetReport_3_0
  • VirusTotal_Scan_3_0
  • WOT_Lookup_1_0

Writing new analyzers is very simple, an API is provided and any language can be used (by most of them are written in Python). Some analyzers query open services, others query private services (you need an API) or commercial services (you need a subscription). As you can see, there is an analyzer called "DShield_lookup". That's my contribution to the project. From Cortex, MISP, TheHive, you can query our DShield database to get more information about an IP address:

When if you click on the DShield tag, you can more details:

The DShield analyzer has been added to the official repository by the developers a few weeks ago. Just deploy Cortex and enable it to benefit from our DShield database!

[1] https://thehive-project.org/#section_cortex
[2] http://misp-project.org/
[3] https://thehive-project.org/#section_thehive

Xavier Mertens (@xme)
Senior ISC Handler - Freelance Cyber Security Consultant
PGP Key

2 comment(s)

Comments

Whoop ! Thanks!

Dom
Thanks for the tool!

One thing that's buried in the documentation for Cortex is that as of now it only will work with very specific observable fields. So for example if you have a field named 'source_ip' rather than just 'ip' an analyzer will not work even if you modify the analyzer code. I spent way too much time trying to modify existing Cortex analyzers to get them to work only to find this:

dataTypeList
The list of TheHive datatypes supported by the analyzer. Currently TheHive accepts the following datatypes:

domain
file
filename
fqdn
hash
ip
mail
mail_subject
other
regexp
registry
uri_path
url
user-agent

( https://github.com/TheHive-Project/CortexDocs/blob/master/api/how-to-create-an-analyzer.md )

Diary Archives