The Security Information Exchange (SIE), from Farsight Security® Inc., is a highly scalable security information sharing platform. It can be thought of as “radar for the Internet”, a way for you to study what’s happening online. Farsight collects and redistributes more than 200,000 new raw observations per second from its global network of sensors. Farsight also applies unique proprietary methods to improve the usability of that data, sharing refined intelligence with SIE customers directly and via DNSDB, one of the world’s largest passive DNS databases.

SIE distributes a variety of types of data of use for the security professional, including:

SIE Batch is a new delivery method that gives you access to a RESTful API that can be used to download data as needed. It also has a web-based interface that can be used to define your data sets and download them. With SIE Batch you can select the data sets and time periods of interest to you, download that data and have it available for your analysis.

SIE Batch allows you to access data two ways:

SIE Batch gives you access to the most recent data distributed via the SIE system. How much data is available depends on the channel you’re pulling data from, but is typically the most recent 12-18 hours.

Accessing SIE Data Interactively via SIE Batch

The SIE Batch system requires a subscription to the SIE data. When you set up the subscription you will receive an API key which will give you access to the system. If you don’t have an active subscription, please contact Farsight Security Sales to arrange for a demo or to work with them on defining the channels you’ll want to subscribe to. The Farsight Security Sales Team can be reached at sales@farsightsecurity.com or give them a call at +1-650-489-7919.

Once you are logged in you will see the SIE Batch dashboard. SIE data is returned in one of two formats: Newline Delimited JSON (ND-JSON) and NMSG. ND-JSON formatted files have a suffix of .ndjson, while NMSG formatted files have a suffix of .nmsg.

Once you have the files downloaded, you can hand them off to whatever program you have created to evaluate and process the data in them.

Newline Delimited JSON (ND-JSON) formatted files

ND-JSON files are formatted text files. The specific fields within the data will vary by channel, but it will look something like this sample, which is from Channel 213, Newly Observed Domains:

{
  "time": "2020-01-13 17:53:00.097326040",
  "vname": "SIE",
  "mname": "newdomain",
  "source": "a1ba02cf",
  "message": {
    "domain": "clienttons.com.",
    "time_seen": "2020-01-13 16:16:04",
    "bailiwick": "ipv4-only.cname.clienttons.com.",
    "rrname": "jdkyqftipq6rwxq4s7ca-pw7etn-d8f0af301.ipv4-only.cname.clienttons.com.",
    "rrclass": "IN",
    "rrtype": "CNAME",
    "rdata": [
      "a248.b.akamai.net."
    ],
    "keys": [],
    "new_rr": []
  }
}

ND-JSON files can be viewed directly or used with any tool that supports the ND-JSON format.

NMSG Formatted files

NMSG files are a binary format, so they can’t be used directly. Farsight has released tools to read NMSG formatted content. The NMSG tool can be found on Github at https://github.com/farsightsec/nmsg. If you are using Debian, there are packages that can be installed via apt-get. See https://www.farsightsecurity.com/technical/SIE-user-guide/sie-debian/ for instructions.

To look at NMSG data, you run nmsgtool, which will format an NMSG file as readable test. If you were to view a file downloaded from Channel 221 (NSDomains) via the command nmsgtool -r <filename>, you will see something like this:

[70] [2020-01-13 17:46:47.996805233] [2:6 SIE dnsnx] [a1ba02cf] [] []
qname: 246.25.155.49.in-addr.arpa.
qclass: IN (1)
qtype: PTR (12)
response_ip: 194.146.106.106
soa_rrname: 49.in-addr.arpa.

SIE Batch API

The SIE-Batch API is described online at https://batch.sie-remote.net/apidoc/.

Pulling data from the SIE Batch API involves sending an HTTPS POST to the API end point. Use of the API is language independent, allowing access via any programming language including Python, Java, Perl and C.

Conclusion

SIE Batch is an additional access method to Farsight’s SIE data, making SIE data available to threat intelligence analysts, Firewall policy systems, Malware professionals and other security teams without the server overhead and processing complexity of a real-time feed. The data channels available through SIE allow these groups access to information to help identify and manage various threats on the Internet, including spam sources and URLs, Phishing attacks and malware. SIE data can help inform Intrusion detection system (IDS) and firewalls, allow or deny connections to email and data servers, and many other activities that happen within an organization on a daily basis.

If you are interested in learning more about SIE and SIE Batch, please contact Farsight Security Sales to arrange for a demo or to work with them on defining the channels you’ll want to subscribe to. The Farsight Security Sales Team can be reached at sales@farsightsecurity.com or give them a call at +1-650-489-7919.

Additional Information

About Farsight Security

Farsight Security, Inc. is the world’s largest provider of historical and real-time DNS intelligence solutions. We enable security teams to qualify, enrich and correlate all sources of threat data and ultimately save time when it is most critical - during an attack or investigation. Our solutions provide enterprise, government and security industry personnel and platforms with unmatched global visibility, context and response. Farsight Security is headquartered in San Mateo, California, USA. Learn more about how we can empower your threat platform and security team with Farsight Security passive DNS solutions at www.farsightsecurity.com or follow us on Twitter: @FarsightSecInc.