Security Information Exchange (SIE) NX Domains
Channel 221, the SIE NX Domains channel is a source of DNS intelligence for DNS queries where the responding name server returned a “Non-Existent Domain” or “NXDomain” Response Code (RCODE). The name server returned this RCODE because the domain or hostname “does not exist” and could not be resolved. This channel is derived from Channel 220 DNS Errors, but only includes NXDomain RCODEs. This enables customers that are only interested in this specific response code to monitor it with a reduced bandwidth requirement.
About Security Information Exchange (SIE)
The Security Information Exchange (SIE), from Farsight Security® Inc. (now a part of DomainTools), is a scalable and adaptable real-time data streaming and information sharing platform. SIE collects and provides access to more than 200,000 observations per-second of raw data from its global sensor network. Farsight also applies unique and proprietary methods for improving usability of the data, directly sharing the refined intelligence with SIE customers and DNSDB®, one of the world’s largest passive DNS (pDNS) databases.
The diverse set of data available from SIE includes the following and is relevant and useful for practitioners in various technology roles:
- Raw and processed passive DNS data
- Darknet/darkspace telescope data
- SPAM sources and URLs
- Phishing URLs and associated targeted brands
- Connection attempts from malware-infected systems (as seen by a sinkhole)
- Network traffic blocked by Intrusion Detection Systems (IDS) and firewall devices
Each unique set of data in SIE is known as a channel and the data acquired from a specific channel can be customized to meet the needs of each customer, enabling you to subscribe to and access only the channels needed to solve your problem. A channel in SIE may be the result from analyzing the data or a subset of data from other channels.
Why Passive DNS (pDNS)?
DNS is a critical component of Internet communication and almost all Internet transactions begin with a DNS query and response.
- Visiting a website? Your system uses DNS to resolve the IP address of the hostname for the website you are attempting to access
- Sending an email? Email uses DNS to resolve the IP address of the mail exchange server your message should be delivered to
DNS serves as early warning and detection solution for phishing, spam, malicious and suspicious behaviors, and other attacks. DNS intelligence is considered the only source of “ground truth” information for the Internet.
Passive DNS (pDNS) begins with raw DNS traffic that is observed and collected by passive DNS sensors and contributed to Farsight’s Security Information Exchange (SIE) by pDNS sensor operators.
Farsight Security’s mission is to make the Internet a safer place. We provide security solutions that empower customers with meaningful and relevant intelligence. This information provides customers with insights about the network configuration of a threat and the surrounding network on the Internet for improving the value and impact of threat intelligence and research.
The Security Information Exchange (SIE), from Farsight Security Inc., is designed with privacy in mind. The passive DNS (pDNS) sensors do not collect Personally Identifiable Information (PII) from client resolvers (also known as stub) by deliberately collecting between recursive resolvers and authoritative servers.
The data from SIE enables security professionals to accurately identify, map, and protect their networks from cybercriminal activity by providing global visibility. It provides immediate access to a real-time global sensor network without the need to develop or deploy your own data collection infrastructure.
About SIE NX Domains
Channel 221 provides insights for DNS queries that return an “NXDomain” Response Code (RCODE). Billions of DNS queries occur every day and most return usable answers. However, in some cases DNS returns an RCODE because the domain you are interested in “does not exist“. A lot of interesting information can be discovered and learned about DNS by investigating “NXDomain” DNS messages.
Use Cases for SIE NX Domains
Customers can discover and learn a lot of interesting information about DNS by investigating these errors. By analyzing this DNS intelligence, customers can solve operational, security, risk, and brand protection related issues.
- DNS Operational Monitoring: Customers can monitor their domains on this channel and be immediately notified if DNS for your domains or hostnames fail anywhere on the globe
- Domain Decommissioning: Customers can use NX Domains to detect NXDomain (RCODE=3) responses for known domains you are monitoring. Observing this RCODE would indicate the “end of use” for that domain and monitoring could then be disabled
- Brand Protection: Your company has brands and trademarks that it needs to protect from infringement and counterfeit. Customers can monitor Channel 221 NX Domains and be notified when someone starts investigating domains that would infringe on your intellectual property
- Domain Monetization: Some companies have created businesses that focus on mistyped domain names. They register commonly misspelled domains, create websites, and then serve ads when users accidentally visit the website. Farsight does not license data to companies that perform this type of nefarious behavior. However, NX Domains can inform and help customers identify their commonly misspelled domains and monitor them to look for these predatory practices
- Detection of Domain Generation Algorithms (DGAs): Criminals that build Botnets often use Domain Generation Algorithms (DGAs) to spread their command and control (C&C) servers across many domains. The purpose is to ensure that if a C&C server is taken down by authorities, the infected systems can fall back and reconnect to an alternative C&C server. The NX Domains channel enables analysts and researchers to monitor failed DNS requests to help them identify these hidden C&C networks
Detection of Domain Generation Algorithms (DGAs)
One significant use-case for Channel 221 NX Domains is identifying users infected with malware. Infected computers will attempt to contact command and control (C&C or C2) servers to receive instructions or commands on what it should do.
Criminals that create malware may use a Domain Generation Algorithm (DGA) to generate a large number of fallback domain names. This effort is an attempt to ensure the criminal does not lose control of the botnet when the primary C&C server is taken down by authorities. If the domain name of primary C&C server can not be resolved, the malware may attempt to resolve a fallback domain name created by the DGA and then connect to a secondary C&C server.
This behavior can generate an excessive amount of DNS resolution failures, which can be used to identify infected systems attempting to contact C&C servers.
DGAs created by malware authors are designed to prevent reverse-engineering. This makes is difficult for analysts and security practitioners to discover and identify C&C servers from DNS responses returning a non-zero Response Code (RCODE).
Additionally, malware has started to generate and test numerous domains to make it even more challenging to reverse engineer the DGA and track C&C servers. While that makes it more difficult to identify the C&C servers, the excessive amount of DNS erros makes it easier to identify the infected computers so they can be remediated.
See Domain Generation Algorithms for more detail on DGAs. To see current channel traffic volumes and service options for accessing it, please see the Security Information Exchange (SIE) Channel Guide.
Channel Information for SIE Channel 221
| Channel Name | NX Domains |
| Channel Number | 221 |
| Description | Passive DNS observations where the responding server returned the “NXDomain” error. |
| Schema | SIE:dnsnx |
Data Format for SIE Channel 221
The NX Domains channel data uses the SIE NMSG dnsnx DNS Query and Response resource record schema that observes and collects “NXDomain” data returned from a query. Note, Channel 221 only returns a subset of the fields to reduce bandwidth requirements. The data available from this channel contains NMSG SIE:dnsnx type messages that include the following fields:
The NMSG header includes the following fields:
| KEY | VALUE |
|---|---|
| time | Time when a “NXDomain” RCODE was observed in Channel 220. |
| vname | Vendor Name, SIE. |
| mname | Message type, dnsnx. |
| source | Identifies pDNS operator contributing DNS data, not the sensor. |
| message | Embedded JSON record describing the observed DNS Query and Response RR. |
The embedded NMSG message payload is JSON formatted and includes the following
fields:
| KEY | VALUE |
|---|---|
| qname | Name of the queried resource record. |
| qclass | Class of the query, most commonly IN. |
| qtype | Type of the query. |
| response_ip | IP address of the server responding to the query. |
| soa_rrname | Authoritative domain name returned in the start of authority (SOA) record, if any. |
| response_time_sec | Time in Unix epoch the response was received from server responding to the query. |
| response_time_nsec | Time in nanoseconds for the response to be received. |
Example Message from SIE Channel 221
Data acquired from Channel 221 NX Domains is returned in NMSG format. NMSG is an adaptable container format that allows for consistent or variable message types.
The nmsgtool program is a tool for acquiring a variety of different inputs,
like data streams from the network, capturing data from network interfaces,
reading data from files, or even standard input and making NMSG payloads
available to one or more outputs. The nmsgtool program can acquire data from
SIE Channel 221 and convert it to a ND-JSON (newline-delimited JSON) text format
for display or additional processing and analysis. nmsgtool is a program
written by Farsight and released as open source.
See the following pages for instructions on how to install software packages for a specific distribution.
- Security Information Exchange (SIE) on Debian
- Security Information Exchange (SIE) on CentOS 6 / RHEL 6
- Security Information Exchange (SIE) on FreeBSD
After data for Channel 221 has been acquired, written, and saved to a file, you
need to decode it to ND-JSON using nmsgtool. The [-r ch221_nxdomain.nmsg]
option tells nmsgtool to read binary NMSG data from a file, [-c 1] limits
the output to single NMSG payload, and [-J -] displays the record in ND-JSON
format to stdout, which is typically the screen.
$ nmsgtool -r ch221_nxdomain.nmsg -c 1 -J -
{"time":"2020-01-13 17:46:47.995494902","vname":"SIE","mname":"dnsnx","source":"a1ba02cf", "message":{"qname":"10.10.10.10.in-addr.arpa.","qclass":"IN","qtype":"PTR", "response_ip":"10.10.10.10","soa_rrname":"102.in-addr.arpa.",
"response_time_sec":1578937543,"response_time_nsec":441845000}}If you want to display a pretty-printed output of ND-JSON formatted records, we recommend using jq, a lightweight and flexible command-line JSON processor.
The open source software package is available on Debian and can be installed using $ sudo apt-get install jq. The output from nmsgtool in JSON format [-J -] can be piped to
jq using the following:
$ nmsgtool -r ch221_nxdomain.nmsg -c 1 -J - | jq '.'
{
"time": "2020-01-13 17:46:47.995494902",
"vname": "SIE",
"mname": "dnsnx",
"source": "a1ba02cf",
"message": {
"qname": "10.10.10.10.in-addr.arpa.",
"qclass": "IN",
"qtype": "PTR",
"response_ip": "10.10.10.10",
"soa_rrname": "102.in-addr.arpa.",
"response_time_sec": 1578937543,
"response_time_nsec": 441845000
}
}SIE Access Methods
Data from SIE can be accessed and acquired using the following methods:
- Direct Connect: Connect a system to the SIE network. This 1.) requires a server to be installed in a data center where Farsight has a point of presence, and 2.) then ordering a network cross connect between your server and the SIE network. Customers can optionally, and prefer to, lease a blade server from Farsight
- SIE Remote Access (SRA): Remotely connect to the SIE network using an encrypted tunnel from your workstation or a server in your local data center
- SIE Batch: Provides on-demand access for downloading data from SIE channels using a RESTful API or web-based interface. You select the channel and duration of time you are interested in, and then download the data for analysis The duration of available data is dependent on the channel, but is typically the most recent 12-18 hours
For additional information about SIE access methods, please see the SIE Technical Overview document.
Direct Connect
SIE Direct Connect allows a customer to physically connect a server to the Farsight SIE network for maximum data throughput. This can be done in one of two ways:
- Blade Server: Pre-configured blade servers co-located in one of Farsight’s data centers that can be leased by customers for direct access to SIE channels
- Customer Server: Customer (owned, managed, and operated) servers that can be installed in one of Farsight’s data centers and physically connected to the SIE network with a network cross-connect
If a blade server is leased from Farsight, it will be pre-installed with the essential software components needed to acquire, process, compress, buffer, and transfer data from SIE channels to the customer’s data center for additional analysis, enrichment, and storage.
If a customer uses their own server, an order can be submitted for a cross-connect to the SIE switches hosted at select Equinix data centers (Ashburn DC3 and Palo Alto SV8). An FSI account manager can help guide cross-connect provisioning details, hosting, or colocation options.
For additional information about SIE connection methods, please see the SIE Technical Overview document. A Farsight’s sales representatives is happy to share a copy of this document with you. This will help inform and guide you in understanding which connection method will work best for you.
SIE Remote Access (SRA)
SIE Remote Access (SRA) enables a customer to remotely connect to the Security Information Exchange (SIE) from anywhere on the Internet. SRA provides access to SIE channel data on customer’s local servers, allowing their analysis and processing systems to be located in their own data centers rather than physically co-located at a Farsight’s data center.
Due to the technical limitations of transporting high bitrate SIE channels across the Internet, the SRA access method is not available for all SIE channels. Please reference the [SIE Channel Guide https://docs.farsightsecurity.com/sie/sie-channel-guide/) for channels that can be accessed using SRA:
SRA uses the Advanced Exchange Access (AXA) transport protocol which enables SRA sessions to perform the following:
- Select which SIE channel or channels to monitor and acquire data from
- Define user-specified search or filtering criteria to match IP or DNS traffic
- Control rate-limits and other AXA parameters
The streaming search and filtering capabilities of AXA enables SRA to access and acquire meaningful and relevant data from SIE while avoiding the costs of transporting enormous volumes of data across the Internet.
Note: For high volume channels accessed using SRA, it is expected that customer’s will specify a search or filter for IP addresses and DNS domain names or hostnames of interest. The SRA service will only collect and send data matching the specified criteria across the Internet to the customer.
SIE Batch
SIE Batch provides on-demand access for downloading data from SIE channels using a RESTful API or web-based interface. You select the channel and duration of time you are interested in, and then download the data for analysis. The duration of available data is dependent on the channel, but is typically the most recent 12-18 hours. SIE Batch allows you to acquire data from SIE channel using two (2) methods:
- API: Allows you to write tools to programmatically download data from SIE channels for analysis
- Interactively: Web-based interface to the API that enables you to select and download SIE channel data on-demand
Additional Information
Links to Information in this Document
- Domain Name System (DNS) Parameters
- Domain Name System (DNS) Parameters: DNS Response Codes (RCODEs)
- Farsight’s NMSG GitHub Repository
- Farsight’s SIE-NMSG GitHub Repository
SIE:dnsnxDNS Query and Response NX Domain schemajqa lightweight and flexible command-line JSON processor- Description of JSON Lines text format (aka ND-JSON)
NX Domain Links
- What (Besides NXDOMAINs) Do We See on Farsight Security’s DNS Errors Channel?
- Introducing NXD
- Finding Web Proxy Auto Discovery Protocol (WPAD)-related Security Exposures Using Farsight Security’s NXDOMAINs Channel
- DNS is NOT Boring! Using DNS to Expose and Thwart Attacks* from 29th Annual FIRST Conference, 2017
