Announcing The IP Context V2 Schema

Spur Engineering

November 21, 2021

Context-API

UPDATE 2022-02-09: Checkout our new dev portal and corresponding context api documentation.

Our Next Gen IP Context

Spur’s IP Context API was originally released in early 2020 with 10 proxy and 50 VPN services that we actively tracked and attributed. Today, we track over 40 different proxy providers and 600 VPN services. As we grew our service attribution, we felt a few pain points:

Adding new metadata surrounding VPN services was nearly impossible with our current schema
Customers want to make categorized judgements about connection risks without building complex logic
Customers of the API had a difficult transition to our bulk data due to the schemas being different despite the data itself being the same
Confusion over the difference between Maxmind’s GeoLite2 data and Spur’s GeoPrecision locations
Residential proxies were co-located in the schema next to datacenter proxies which led to conflating the two despite the difference in risks they present

Version 2.0

We are excited to announce our version 2 schema. All customers have access to our new v2 context data. To make the switch, simply change v1 to v2 in your API requests for both the feed service and the Context API.

➜  ~ curl -s -H "token: $API_TOKEN" "https://api.spur.us/v2/context/185.65.135.235" | jq
{
  "as": {
    "number": 39351,
    "organization": "31173 Services AB"
  },
  "client": {
    "behaviors": [
      "TOR_PROXY_USER",
      "FILE_SHARING"
    ],
    "concentration": {
      "city": "Rasht",
      "country": "IR",
      "density": 1,
      "geohash": "tnek0",
      "skew": 3325,
      "state": "Gīlān"
    },
    "count": 1,
    "countries": 1
  },
  "infrastructure": "DATACENTER",
  "ip": "185.65.135.235",
  "location": {
    "city": "Stockholm",
    "country": "SE",
    "state": "Stockholm County"
  },
  "organization": "31173 Services AB infrastructure in Stockholm, Sweden.",
  "risks": [
    "VPN_TUNNEL",
    "GEO_MISMATCH"
  ],
  "tunnels": [
    {
      "anonymous": true,
      "entries": [
        "185.65.135.70"
      ],
      "operator": "MULLVAD_VPN",
      "type": "VPN"
    }
  ]
}

To summarize the major changes:

The bulk data feeds and API have an identical schema
The schema introduces more logical structures which will make feature additions simpler to integrate for customers
We have clustered many of the old device metadata into a client object that encapsulates this information more clearly
Residential/Call-back proxies are now split off from other proxy services and included in the client object
All traditional proxy services are now included with the VPNs in a new tunnels object
We now label risks to be wary of when interacting with these IP addresses at a high-level

Client Metadata

{
  "client": {
    "behaviors": [
      "TOR_PROXY_USER",
      "FILE_SHARING"
    ],
    "concentration": {
      "city": "Rasht",
      "country": "IR",
      "density": 1,
      "geohash": "tnek0",
      "skew": 3325,
      "state": "Gīlān"
    },
    "proxies": [
      "911_PROXY",
      "LUMINATI_PROXY"
    ],
    "count": 1,
    "countries": 1
  },
  ...
}

behaviors: Formerly deviceBehaviors in the v1 schema. The clients utilizing this IP address have been observed performing these behaviors
concentration: Formerly geoPrecision in the v1 schema. New fields include skew and density which represents the distance from the traditional IP geo location and the percentage of users located in this box respectfully
proxies : A list of the residential/malware/call-back proxies that we have observed on clients using this IP address. The existence of these proxies does mean that a customer of the listed service could be using this IP address as their own
count : Formerly devices.estimate in the v1 schema. This has been modified to represent the average number of clients we observe daily on this address
countries : A new field representing the number of countries where we have observed users

Tunnels

One of the biggest improvements is the data surrounding VPN/proxies.

{
  "tunnels": [
    {
      "anonymous": true,
      "entries": [
        "185.65.135.70"
      ],
      "operator": "MULLVAD_VPN",
      "type": "VPN"
    }
  ],
  ...
}

entries : A list of IP addresses that represent the entry into this proxy tunnel. For example, users of this IP address were connecting to 185.65.135.70
operator : The name of the service that runs this tunnel

{
  "services": [
    "WIREGUARD"
  ],
  "tunnels": [
    {
      "anonymous": true,
      "exits": [
        "185.65.135.235",
        "185.65.135.240",
        "185.65.135.220"
      ],
      "operator": "MULLVAD_VPN",
      "type": "VPN"
    }
  ],
  ...
}

services : The specific VPN/proxy technology that is running on this IP address. For example, the above IP address is running a wireguard service
exits : The exit sides of the tunnel that we have labeled. For example, users connecting to the WIREGUARD service on this IP address would exit onto the internet from one of those three IP addresses listed