Channel 212, the SIE Newly Observed Domains (NOD) channel is a source of DNS intelligence for domains observed in DNSDB for the first time. This enables customers to observe and monitor when new domains become active for the first time.

Newly Observed Domains (NOD) is one of several channels that tracks domain observations, creation, and changes. These channels follow:

1: A “base domain” is one label followed by a suffix. See the Public Suffix List for information on the current list of official suffixes. Suffixes are a superset of the Top Level Domains (TLDs).

These channels use Channel 204 Processed DNS Data, that is used by DNSDB, as their authoritative data source. The DNS data available from channel 204 is after the deduplication and verification phases from the Passive DNS Processing “Waterfall” Model. Domains and hostnames are checked for historic observations in DNSDB back to June 2010.

For more information about Channel 204 Processed DNS Data,please refer to the SIE Technical Overview guide.

Table of Contents

About Security Information Exchange (SIE)

The Security Information Exchange (SIE), from Farsight Security® Inc., is a scalable and adaptable real-time data streaming and information sharing platform. SIE collects and provides access to more than 200,000 observations per-second of raw data from its global sensor network. Farsight also applies unique and proprietary methods for improving usability of the data, directly sharing the refined intelligence with SIE customers and DNSDB®, one of the world’s largest passive DNS (pDNS) databases.

The diverse set of data available from SIE includes the following and is relevant and useful for practitioners in various technology roles:

Each unique set of data in SIE is known as a channel and the data acquired from a specific channel can be customized to meet the needs of each customer, enabling you to subscribe to and access only the channels needed to solve your problem. A channel in SIE may be the result from analyzing the data or a subset of data from other channels.

Why Passive DNS (pDNS)?

DNS is a critical component of Internet communication and almost all Internet transactions begin with a DNS query and response.

DNS serves as early warning and detection solution for phishing, spam, malicious and suspicious behaviors, and other attacks. DNS intelligence is considered the only source of “ground truth” information for the Internet.

Passive DNS (pDNS) begins with raw DNS traffic that is observed and collected by passive DNS sensors and contributed to Farsight’s Security Information Exchange (SIE) by pDNS sensor operators. Once the data is sent to SIE, the data then passes through a series of processing phases:

  1. Deduplication: Channel 207, DNSDB Deduplicated Data
  2. Verification: Channel 208, DNSDB Verified Data
  3. Filtering: Channel 204, Processed DNS Data (which used by DNSDB)

The end result is the highest-quality and most comprehensive passive DNS database, DNSDB, of its kind-with more than 100 billion unique DNS resource records since 2010.

Farsight Security’s mission is to make the Internet a safer place. We provide security solutions that empower customers with meaningful and relevant intelligence. This information provides customers with insights about the network configuration of a threat and the surrounding network on the Internet for improving the value and impact of threat intelligence and research.

The Security Information Exchange (SIE), from Farsight Security Inc., is designed with privacy in mind. The passive DNS (pDNS) sensors do not collect Personally Identifiable Information (PII) from client resolvers (also known as stub) by deliberately collecting between recursive resolvers and authoritative servers.

The data from SIE enables security professionals to accurately identify, map, and protect their networks from cybercriminal activity by providing global visibility. It provides immediate access to a real-time global sensor network without the need to develop or deploy your own data collection infrastructure.

About SIE Newly Observed Domains (NOD)

Channel 212 provides insights about DNS activity for base domains observed in DNSDB for the first time. If DNS activity meets this criteria, the intelligence is sent to channel 212.

When is DNS intelligence for NOD sent to channel 212?

The following narrative will inform and guide you in understanding when DNS resource record data is sent to one of the channels that tracks domain observations, creation, and changes.

For a good introduction into the opportunities and possibilities of these channels, see New (and Newly-Changed) Fully Qualified Domain Names (FQDNs): A View of Worldwide Changes to the Internet’s DNS* from Black Hat Europe, 2015

Use Cases for SIE Newly Observed Domains (NOD)

Newly Observed Domains (NOD) is an important tool in recognizing potential bad actor domains and enables customers to observe and monitor in near real-time when new domains become active for the first time.

Why is this important? In general, legitimate domains are added to DNS before they are actively put into service. In most cases, it takes time from creation before you should observe traffic for the domain. A domain that becomes active shortly after creation is more likely to be dangerous and return or host malicious content. The intent of these bad actors is to create a domain and use it as much as possible as soon as possible after creation. When the domain is then added to various malicious block lists, the bad actor abandons it and moves to a new domain.

There is nothing urgent about accessing a new website when it is first created and becomes active. It is better to refrain or wait a few hours or even days before accessing a new domain and avoid malicious sites as they pop up. Because of this, a policy that temporarily denies access to new domains and hostnames can help protect your environment from malware and attacks until the global blocklists can assess them and decide whether they should be permanently blocked.

Channel 212 Newly Observed Domains (NOD) is an essential tool that enables you to identify new domains that are active shortly after creation. This channel empowers an organization to create policies and decide whether to permit or deny access to the new domains and how long access to the potentially malicious site should be prevented.

Channel Information for SIE Channel 212

Channel Name Newly Observed Domains (NOD)
Channel Number 212
Description Passive DNS observations of base domains not previously seen when compared to the DNSDB historical database.
Schema SIE:newdomain
Bit Rate 3Kb/sec
Bit Rate (Max) 25Kb/sec
Payloads 2/sec
Payloads (Max) 20/sec

Note: Quoted bitrates and payloads are representative of SIE traffic as of June, 2021.

Access Methods for SIE Channel 212

SIE Direct Connect Yes
SIE Direct Connect Data Format NMSG
SIE Remote Access (SRA) Yes
SRA Data Format NMSG
SIE Batch Yes
SIE Batch Data Format Newline-Delimited JSON (ND-JSON)
Advanced Exchange Access Middleware Daemon (AXAMD) Yes
AXAMD Data Format NMSG

Data Format for SIE Channel 212

The Newly Observed Domains (NOD) channel data uses the SIE NMSG newdomain DNS Query and Response resource record schema that observes and collects data returned from a query.

The data available from this channel contains NMSG SIE:newdomain type messages that include the following fields:

The NMSG header includes the following fields:

KEY VALUE
time Time when base domain was first observed in Channel 204.
vname Vendor Name, SIE.
mname Message type, newdomain.
message Embedded JSON record describing the observed DNS Query and Response RR.

The embedded NMSG message payload is JSON formatted and includes the following fields:

KEY VALUE
domain Domain name of the query observed by pDNS.
time_seen Time that pDNS observed the base domain.
bailiwick2 The domain under which the RRset answer was given.
rrname Domain name of the query observed by pDNS.
rrclass RR CLASS is always “Internet (IN)”, which is decimal value “1”.
rrtype RR TYPE describes the type of RR, e.g., A(1), NS(2), CNAME(5).
rdata Data that describes the RR type, returned as an array.
keys Always empty or null.
new_rr Always empty or null.

2: DNS data is considered “in bailiwick” if the resource record being returned is the response from a name server that is known to be responsible for answering with authoritative information about that domain. See What is a Bailiwick? for additional information:

Note: Time-based strings are in the YYYY-MM-DD HH:MM:SS format. The month “MM” starts at 01 for January and ends with 12 for December. The hours “HH” are 00-23, and minutes “MM” and seconds “SS” are 00-59. The times are recorded at UTC (GMT) and daylight savings time (DST) is not applicable.

Note: Most DNS data observed on this channel will be “NSrrtype resource records. The associated rrname field will be the “base domain” and the rdata field will include an array of “authoritative name servers”.

Example Message from SIE Channel 212

Data acquired from Channel 212 Newly Observed Domains (NOD) is returned in NMSG format when using the Direct Connect or SIE Remote Access (SRA) access methods. NMSG is an adaptable container format that allows for consistent or variable message types. If data is downloaded for Channel 212 using SIE Batch, the data is already delivered in ND-JSON format, and the nmsgtool step below can be skipped.

The nmsgtool program is a tool for acquiring a variety of different inputs, like data streams from the network, capturing data from network interfaces, reading data from files, or even standard input and making NMSG payloads available to one or more outputs. The nmsgtool program can acquire data from SIE Channel 212 and convert it to a ND-JSON (newline-delimited JSON) text format for display or additional processing and analysis. nmsgtool is a program written by Farsight and released as open source.

See the following pages for instructions on how to install software packages for a specific distribution.

After data for Channel 212 has been acquired, written, and saved to a file, you need to decode it to ND-JSON using nmsgtool. The [-r ch212_nod.nmsg] option tells nmsgtool to read binary NMSG data from a file, [-c 1] limits the output to single NMSG payload, and [-J -] displays the record in ND-JSON format to stdout, which is typically the screen.

$ nmsgtool -r ch212_nod.nmsg -c 1 -J -
{"time":"2020-03-17 18:49:22.939516067","vname":"SIE","mname":"newdomain",
"source":"a1ba02cf","message":{"domain":"example.com.",
"time_seen":"2020-03-17 18:49:22","bailiwick":"com.","rrname":"example.com.",
"rrclass":"IN","rrtype":"NS","rdata":["ns.example.com.","ns3.example.com.",
"ns6.example.com.","ns14.example.com."],"keys":[],"new_rr":[]}}

Once the data has been formatted to ND-JSON, a record from the Newly Observed Domains (NOD) channel will look similar to the following. The following output can be sent to another tool for additional processing.

{"time":"2020-03-17 18:49:22.939516067","vname":"SIE","mname":"newdomain",
"source":"a1ba02cf","message":{"domain":"example.com.",
"time_seen":"2020-03-17 18:49:22","bailiwick":"com.","rrname":"example.com.",
"rrclass":"IN","rrtype":"NS","rdata":["ns.example.com.","ns3.example.com.",
"ns6.example.com.","ns14.example.com."],"keys":[],"new_rr":[]}}

If you want to display a pretty-printed output of ND-JSON formatted records, we recommend using jq, a lightweight and flexible command-line JSON processor. The open source software package is available on Debian and can be installed using $ sudo apt-get install jq. The output from nmsgtool in JSON format [-J -] can be piped to jq using the following:

$ nmsgtool -r ch212_nod.nmsg -c 1 -J - | jq -r '.'
{
  "time": "2020-03-17 18:49:22.939516067",
  "vname": "SIE",
  "mname": "newdomain",
  "source": "a1ba02cf",
  "message": {
    "domain": "example.com.",
    "time_seen": "2020-03-17 18:49:22",
    "bailiwick": "com.",
    "rrname": "example.com.",
    "rrclass": "IN",
    "rrtype": "NS",
    "rdata": [
      "ns.example.com.",
      "ns3.example.com.",
      "ns6.example.com.",
      "ns14.example.com."
    ],
    "keys": [],
    "new_rr": []
  }
}

SIE Access Methods

Data from SIE can be accessed and acquired using the following methods:

For additional information about SIE access methods, please see the SIE Technical Overview document.

Direct Connect

SIE Direct Connect allows a customer to physically connect a server to the Farsight SIE network for maximum data throughput. This can be done in one of two ways:

If a blade server is leased from Farsight, it will be pre-installed with the essential software components needed to acquire, process, compress, buffer, and transfer data from SIE channels to the customer’s data center for additional analysis, enrichment, and storage.

If a customer uses their own server, an order can be submitted for a cross-connect to the SIE switches hosted at select Equinix data centers (Ashburn DC3 and Palo Alto SV8). An FSI account manager can help guide cross-connect provisioning details, hosting, or colocation options.

For additional information about SIE connection methods, please see the SIE Technical Overview document. A Farsight’s sales representatives is happy to share a copy of this document with you. This will help inform and guide you in understanding which connection method will work best for you.

SIE Remote Access (SRA)

SIE Remote Access (SRA) enables a customer to remotely connect to the Security Information Exchange (SIE) from anywhere on the Internet. SRA provides access to SIE channel data on customer’s local servers, allowing their analysis and processing systems to be located in their own data centers rather than physically co-located at a Farsight’s data center.

Due to the technical limitations of transporting high bitrate SIE channels across the Internet, the SRA access method is not available for all SIE channels. Please reference the [SIE Channel Guide](() for channels that can be accessed using SRA:

SRA uses the Advanced Exchange Access (AXA) transport protocol which enables SRA sessions to perform the following:

The streaming search and filtering capabilities of AXA enables SRA to access and acquire meaningful and relevant data from SIE while avoiding the costs of transporting enormous volumes of data across the Internet.

Note: For high volume channels accessed using SRA, it is expected that customer’s will specify a search or filter for IP addresses and DNS domain names or hostnames of interest. The SRA service will only collect and send data matching the specified criteria across the Internet to the customer.

SIE Batch

SIE Batch provides on-demand access for downloading data from SIE channels using a RESTful API or web-based interface. You select the channel and duration of time you are interested in, and then download the data for analysis. The duration of available data is dependent on the channel, but is typically the most recent 12-18 hours. SIE Batch allows you to acquire data from SIE channel using two (2) methods:

Advanced Exchange Access Middleware Daemon (AXAMD)

Farsight also provides a RESTful middleware layer in front of its AXA service. This service is called the AXA Middleware Daemon (AXAMD) and provides a RESTful capability that adds a streaming HTTP interface on top of the AXA toolkit. This enables web-application developers to interface with SIE using SRA. Farsight also published a command line tool and Python extension library called axamd_client. This toolkit is licensed under the Apache 2.0 license.

The Advanced Exchange Access (AXA) toolkit contains tools and a C library to bring Farsight’s real-time data and services directly from the Farsight Security Information Exchange (SIE) to the customers network.

Advanced Exchange Access Middleware Daemon (AXAMD) is a suite of tools and library code to bring Farsight’s real-time data and services directly from the Farsight Security Information Exchange (SIE) to the customers network.

Due to the technical limitations of transporting high bitrate SIE channels across the Internet, the AXAMD access method is not available for all SIE channels.

DNS Response Policy Zone (RPZ) & Realtime Blackhole List (RBL)

Newly Observed Domains (NOD) intelligence is available as DNS Response Policy Zones (RPZ) or Real-time Blackhole List (RBL). DNS RPZ data can be acquired using DNS zone transfers for implementing “DNS Firewalls” or RBL data can be downloaded for anti-spam filtering.

NOD RPZ can automatically prevent access to newly observed domains for a period of time when a base domain is first observed in DNSDB. Pre-configured durations are five (5) minutes to 24 hours. For more information about NOD RPZ, see:

RPZ enables DNS recursive resolvers to perform the role of a “DNS Firewall”. RPZ is a method for expressing DNS response policy information inside specially constructed DNS zones and for recursive resolvers to use the policy information and return modified results to DNS clients. The modified DNS results can prevent access to selected HTTP servers, redirect users to “walled gardens”, block objectionable email, and otherwise defend against attack. These “DNS Firewalls” are widely used in fighting Internet crime and abuse.

Rules in a RPZ consist of triggers or filters that identify what responses to modify and the policy actions for the responses. Each rule can use one of five policy triggers and specify one of eight policy actions.

Response Policy Triggers

  1. QNAME: Query name.
  2. RPZ-IP: IP address present in a truthful response.
  3. RPZ-NSDNAME: Name of authoritative name server responsible for publishing the original response.
  4. RPZ-NSIP: IP address of authoritative name server responsible for publishing the original response.
  5. RPZ-CLIENT-IP: IP address of the DNS client.

Response Policy Actions

  1. NXDOMAIN: Return a “domain does not exist” response.
  2. NODATA: Return a “name exists but there are no records of the requested type” response.
  3. CNAME example.org: Redirect the user using a CNAME RR to a walled garden.
  4. Local Data: Replace the response with specified data.
  5. CNAME rpz-tcp-only: Require the client to re-query using TCP.
  6. DISABLED: Exempt the response from further policy processing.
  7. CNAME rpz-passthru: Exempt the response from further policy processing.
  8. CNAME rpz-drop Drop the query without any response to the client.

Recursive resolvers with support for RPZ include but are not limited to the following: ISC Bind, BlueCat DNS, InfoBlox DNS Firewall, Unbound, Knot Resolver (partial support), PowerDNS Recursor, Akamai AnswerX, EfficientIP SolidServer.

Additional Information

About Farsight Security

Farsight Security, Inc. is the world’s leading provider of historical and real-time DNS intelligence solutions. We enable security teams to qualify, enrich, and correlate all sources of threat data and ultimately save time when it is most critical - during an attack or investigation. Our solutions provide enterprise, government, and security industry personnel and platforms with unmatched global visibility, context, and response. Farsight Security is headquartered in San Mateo, California, USA. To learn more about how we can empower your security, threat, and intelligence platforms and security organization with Farsight Security passive DNS (pDNS) and threat intelligence solutions, please visit us at www.farsightsecurity.com or follow us on Twitter at @FarsightSecInc.