Introduction

Farsight Security® Inc.’s Newly Observed Hostnames (NOH) leverages Farsight’s Passive DNS database (DNSDB) to determine if a hostname or Fully Qualified Domain Names (FQDNs) is newly seen.

Delivery

NOH is delivered one of two ways:

  1. As a real time stream of intelligence using the Security Information Exchange (SIE)
  2. As a CSV file generated hourly using RSYNC and SSH

SIE

For instructions on how to consume NOH as a real time stream, please see SIE documentation.

RSYNC CSV Files

Farsight provides twenty five CSV files of NOH data available via RSYNC + SSH. Twenty four of the files correspond to hourly roll ups of the previous twenty four hours of NOH data. The twenty fifth file is the current hour updated every minute with newly seen hostnames seen within the last sixty seconds.

The file names are created using the format ‘fqdn-YYYYMMDD-HHMM.csv’.

During the provisioning process you will be asked to submit a public SSH key. Once provisioning has been completed you should receive an email similar to the one below with a username specific to you.

Your NOH access has now been provisioned on the Farsight servers.

To access the data, you will use the username of FSI-XXXX-X. You may
find a configuration like the one below useful on your end, placing it
in either ~/.ssh/config, or /etc/ssh/ssh_config:

Host rsync.dns-nod.net
    User FSI-XXXX-X
    IdentityFile /path/to/sshkey
    Port 49222

NOTE: If you do not edit your ssh configuration you will need to use
extended command-line options for rsync.

You may now synchronize files from the rsync.dns-nod.net rsync server
accessed via ssh on port 49222.

Example:

rsync -az -e 'ssh -p 49222 -i/path/to/sshkey' \
FSI-XXXX-X@rsync.dns-nod.net:nod/ /srv/nod

NOH files will be found in the subdirectory named 'csv'. New NOH files
will be delivered once an hour and the previous twenty four hours of
NOH files will be available. For this reason we ask you to poll the
server at most once an hour.

If you have any questions, please let me know. Please direct future
support requests to nod@farsightsecurity.com.

Data Structures

The data set available for NOH differs depending on the delivery mechanism chosen (RSYNC and SSH or SIE). SIE provides the complete data set whereas the CSV file provides a reduced data set. You can see example data structures below per delivery method.

Security Information Exchange (SIE)

Real time stream of newly observed domains for security intelligence and enrichment

  "message": {
    "domain": "ampproject.net.",
    "time_seen": "2017-02-27 18:23:55",
    "bailiwick": "ampproject.net.",
    "rrname": "d-22069952994228697343.ampproject.net.",
    "rrclass": "IN",
    "rrtype": "A",
    "rdata": [
      "216.58.194.78"
    ],
    "keys": [],
    "new_rr": []
  }

Hourly CSV

Columns:

  1. domain (hostname)
  2. first_seen (Unix time)
domain,first_seen
ztvxm.pgdbz.com.,1488210985
www.ferselogistic.com.,1488211039
dyaxukuqsm.spotilocal.com.,1488211005

Limits

When using RSYNC to synchronize the CSV files, we recommend running RSYCN once an hour; at maximum do not run RSYNC more than once a minute as the current hour file is not updated any faster than once every sixty seconds.

About Farsight Security

Farsight Security, Inc. is the world’s largest provider of historical and real-time DNS intelligence solutions. We enable security teams to qualify, enrich and correlate all sources of threat data and ultimately save time when it is most critical - during an attack or investigation. Our solutions provide enterprise, government and security industry personnel and platforms with unmatched global visibility, context and response. Farsight Security is headquartered in San Mateo, California, USA. Learn more about how we can empower your threat platform and security team with Farsight Security passive DNS solutions at www.farsightsecurity.com or follow us on Twitter: @FarsightSecInc.