Resolvers

ResolverBase

A base class for handling resolution of IP addresses.

This class includes elements common across resolver resolution data sources. This includes common parameters, such as field names, and functions to handle querying the data sources. Additionally, this includes writers for CSV and JSON formats.

Module Documentation

class libchickadee.resolvers.ResolverBase

Generic base class for use by other resolvers.

Parameters

  • fields (list) – Collection of fields to include in query.

  • lang (str) – Language for results.

Returns

(ResolverBase)

batch()

Base method to handle batch queries.

Not implemented in base class.

static defang_ioc(ioc)

Modify the display of IOCs to limit automated hyperlinking or access of unsafe resources

Parameters

ioc (str) – Content to scan for values to defang

Returns

Defanged value.

Return type

(str)

static flatten_objects(data, headers)

Flatten complex fields in to simple columns for CSV usage

Parameters

  • data (list) – List of dictionaries to write to file

  • headers (list) – Header row to use.

Returns

Tuple containing a list of rows to write, followed by a list of headers to use.

Return type

(tuple, list)

static normalize_data_headers(data, headers)

Filter content from response that is not requested in output

Parameters

  • data (list) – Response from API

  • headers (list) – List of user-provided headers to return

Returns

Updated API response with limited headers

Return type

(dict)

query(data)
Generic query handler to decide to use single or batch

method for querying.

Parameters

data (list, tuple, set, str) – One or more IPs to resolve

Returns

(list) List of collected records.

Example

>>> records = ['1.1.1.1', '2.2.2.2']
>>> resolver = ResolverBase()
>>> resolved_data = resolver.query(records)
>>> print(resolved_data)
[
 {"query": "1.1.1.1", "country": "Australia", ...},
 {"query": "2.2.2.2", "country": "France", ...}
]
single()

Base method to handle single queries.

Not implemented in base class.

static write_csv(outfile, data, headers=None)

Writes a list of dictionaries to a CSV file.

Parameters

  • outfile (str or file_obj) – Path to output file

  • data (list) – List of dictionaries to write to file

  • headers (list) – Header row to use. If empty, will use the first dictionary in the data list.

Returns

None

Example

>>> records = [{'query': '1.1.1.1', 'count': 2}]
>>> resolver = ResolverBase()
>>> resolver.write_csv('test.csv', records, ['query', 'count'])
static write_json(outfile, data, headers=None, lines=False)

Writes output in JSON format

Parameters

  • outfile (str or file_obj) – Path to or already open file

  • data (list) – List of dictionaries containing resolved data

  • headers (list) – List of column headers. Will use the first element of data if not present.

  • lines (bool) – Whether to export 1 dictionary object per line or a whole json object.

Returns

None

Examples

Generation of a JSON report

>>> records = [{'query': '1.1.1.1', 'count': 2}]
>>> resolver = ResolverBase()
>>> resolver.write_json('test.json', records, ['query', 'count'])

Generation of a JSON Lines report

>>> recs = [{'query': '1.1.1.1', 'count': 2}]
>>> resolver = ResolverBase()
>>> resolver.write_json('out.json', recs, ['query', 'count'], True)

IP-API Resolver

Resolver leveraging the ip-api.com JSON API.

This is a third-party data source that provides GeoIP, ASN, Organization, ISP, and other resolution information for IPv4 and IPv6 addresses.

Data Source Information

This data source is hosted at ip-api.com and requires an internet connection to use. It offers a free API, with terms described on their site. Please refer to the service’s website for an authoritative source on API specifications. This documentation summarizes a point in time understanding of the data source though since it is a third party service, it may change in a manner that breaks this tool or causes this documentation to become inaccurate. In no way is inclusion of a data source in libchickadee an endorsement of the data source.

Data source documentation: https://ip-api.com/docs

Endpoints

The API supports a number of formats, though this resolver implementation leverages the JSON APIs. This includes both the single and batch APIs depending on the number of IP addresses requested for resolution.

  • http://ip-api.com/json/{query}

  • http://ip-api.com/batch

Fields

These fields are in no particular order.

  • status

  • message

  • continent

  • continentCode

  • country

  • countryCode

  • region

  • regionName

  • city

  • district

  • zip

  • lat

  • lon

  • timezone

  • currency

  • isp

  • org

  • as

  • asname

  • mobile

  • proxy

  • hosting

  • query

Limitations

This service has a free tier for non-commercial use, and is rate limited to 15 requests per minute. The returned HTTP header X-Rl contains the number of requests remaining in the current rate limit window. X-Ttl contains the seconds until the limit is reset.

The professional service is supported by chickadee and allows the execution of unlimited requests, commercial use, and https endpoints. More details are available on the data source website.

Module Documentation

class libchickadee.resolvers.ipapi.ProResolver(api_key, fields=None, lang='en')

GeoIP resolver using the ip-api.com paid subscription.

Sets endpoint to paid API, confirms support for requested language, and disables sleep functionality.

Parameters

  • api_key (str) – IP-API.com paid API key for requests.

  • fields (list) – Collection of fields to request in resolution.

  • lang (str) – Language for returned results.

class libchickadee.resolvers.ipapi.Resolver(fields=None, lang='en')

Class to handle ip-api.com API queries for IP addresses.

Sets endpoint to free API, confirms support for requested language, configures rate limit sleep timers.

Parameters

  • fields (list) – Collection of fields to request in resolution.

  • lang (str) – Language for returned results.

batch()

Handle batch query operations.

Generally not called directly, should be called by self.query() to allow for the logic to handle which endpoint is preferred.

Returns

List of resolved IP address records with specified fields.

Return type

(list)

rate_limit(headers)

Method to check rate limiting. Checks X-Rl and X-Ttl headers.

Will set timer for rate limiting if X-Rl is less than 1 for duration of X-Ttl + 1 second.

Parameters

headers (dict, CaseInsensitiveDict) – Request header information.

Returns

None

single()

Handle single item query operations.

Generally not called directly, should be called by self.query() to allow for the logic to handle which endpoint is preferred.

Returns

List of resolved IP address records with specified fields.

Return type

(list)

sleeper()

Method to sleep operations for rate limiting. Executes sleep.

Returns

None

VirusTotal Resolver

Resolver leveraging the VirusTotal JSON API.

This is a third-party data source that provides GeoIP, ASN, associated detected file samples, detections on web hosted content, known resolutions, and other resolution information for IPv4 and IPv6 addresses.

Data Source Information

This data source is hosted at virustotal.com and requires an internet connection to use. It offers a free API with registration and approval, with terms described on their site. Please refer to the service’s website for an authoritative source on API specifications.

This documentation summarizes a point in time understanding of the data source though since it is a third party service, it may change in a manner that breaks this tool or causes this documentation to become inaccurate. In no way is inclusion of a data source in libchickadee an endorsement of the data source.

Data source documentation: https://developers.virustotal.com/reference

Endpoints

The API supports a number of reports, though for IP addresses only allows the submission of a single IP address per request. This means bulk lookups will take at most 1 minute per 4 IP addresses, due to rate limiting.

  • https://www.virustotal.com/vtapi/v2/ip-address/report?apikey=<apikey>&ip=<ip>

Fields

These fields are in no particular order.

  • query

  • count

  • asn

  • country

  • subnet

  • resolution_count

  • detected_sample_count

  • detected_samples

  • undetected_sample_count

  • undetected_samples

  • detected_url_count

  • detected_urls

  • undetected_url_count

  • undetected_urls

  • status

  • message

Limitations

This service has a free tier for non-commercial use, and is rate limited to:

  • 4 requests per minute

  • 5760 requests per day

  • 172800 requests per month

A 204 response code indicates that you have exceeded your rate limit. The script will sleep for 15 seconds if a 204 is returned.

Module Documentation

class libchickadee.resolvers.virustotal.ProResolver(api_key, fields=None, lang='en')

VirusTotal authenticated API resolver. Requires an API key. Full API documentation here: https://developers.virustotal.com/reference

batch()

Resolve multiple IP addresses.

Due to API limitations, each IP address must be resolved individually.

Returns

List of resolved results

Return type

(list)

parse_vt_resp(query, vt_resp)

Transform the raw response from VirusTotal in to a dictionary easier for analysis

Parameters

  • query (str) – Queried value

  • vt_resp (dict) – Raw API response

Returns

Formatted response for analysis

Return type

(dict)

single()

Gathers VirusTotal report data for a single IP address.

Returns

Report information

Return type

(list)

sleeper()

Method to sleep operations for rate limiting.

Will ensure that 4 requests per minute limit is not exceeded if a 204 encountered.

Returns

None

Indices and tables