If you build it, they will come
Putting anything public-facing on the Internet will result in abuse over a long enough time span. As a basic example, fire up a “hello world” web application on a Raspberry Pi, punch a hole in your router’s firewall and port forward to your new web server, and watch the logs roll in. On second thought, don’t do this. But the point remains: nothing is immune. The only qualifiers are the type and scale of abuse your service receives.
Spur, like any other company with a public-facing service, receives abusive traffic nearly every second of every day. One example are the attempts to register for our free community dashboard via the use of throwaway email addresses. Accounts such as these frequently intend to abuse our community dashboard by obfuscating their identity while scraping our data and potentially degrading our service’s availability.
Maintaining a blocklist of known throwaway email domains is effective at combatting this problem but is, predictably, a constant cat-and-mouse game. There will never be an end to new instances of such services. In order to block them at the domain level, we have to see it first, and by that point the damage could already be done.
It’s not lost on us that the same can also be said for anonymizing proxy and VPN services. In addition to the problem of large gateways, there will never be an end of new IPs to block as new proxy and VPN services are stood up and marketed towards users with malice on their mind. Spur’s ever-transient catalog of tens of millions of anonymous infrastructure IPs is well-utilized to analyze fraudulent behavior at scale, or to give better context to abnormal or abusive activity, but as a blocklist, it will never be complete.
Anonymous proxies, especially residential proxies, have been a massive boon to the types of users intent on abusing a service. Now, for the low cost of a couple dollars worth of Bitcoin, your average up-to-no-good user can have access to a vast pool of “clean” IP addresses anywhere in the word, often with the ability to target city or even town-level IP addresses as their needs require. This effectively render’s traditional IP blocking (such as fail2ban) obsolete.
A customer of Monocle once told us, “I don’t have a bot problem. I have a people problem”. Traditional captchas work well to raise the barrier of entry to bot-enabled (automated) abuse, but do nothing to thwart real users behind real keywords with access to an infinite pool of burner IP and email addresses. Still, bots indeed often make use of residential proxy services to elude IP bans or rate-limiting. While active and passive captchas are an excellent starting point for a many use cases, they will always be missing the full picture.
Fortunately for Spur, most of the abuse we see is limited to web scrapers. Nonetheless, there are ample other real-life cases of abuse related to free accounts that raise more pressing concerns:
- Cloud virtualization services suffering from new accounts abusing free credits to mine cryptocurrency
- E-commerce sites (such as sneaker shops or event ticket services) dealing with new stock immediately being sold off to private resellers (I’m still pissed about my T-Swizzle tickets)
- Banking, investment, or gambling platforms who offer a sign-up bonus to new accounts having to pay out to customers whose intent is to collect this bonus en masse
- Social media sites dealing with platform abuse or manipulation from a botnet of accounts
So what’s the answer to the proxy problem when IP-based blocking is no longer reliable or even feasible? Essentially, proxied connections must be detected in real-time at the session level. To this end, Spur uses Monocle on our sign-up form to detect and block account creation attempts from anonymous VPNs and proxies in real-time, without prior intelligence on the IP in question.
This chart represents a 7-day snapshot of Monocle results attached to account registration attempts for our free community dashboard.
Let’s go over these results a bit:
- 12% originated from non-anonymizing (e.g: corporate) VPNs
- 16% from non-anonymous proxies such as ZScaler
- 9.5% from traditional VPNs the likes of which you might find advertised on every YouTube video
- A staggering 11% from anonymous (datacenter and residential) proxies
Just barely half of the new account requests we receive are from IPs that have no indication of being a proxy or VPN. More than 1-in-5 registration attempts are from infrastructure specifically intended to anonymize.
These numbers were a shocking revelation to us, but really not that shocking. If you build it, they will come.
Barking up the wrong tree
We receive alerts on all sorts of suspicious/abusive activity on our platform, and one such alert notifies us when we’ve blocked a quantity of sign-up attempts over a specific timeframe.
Let’s go over a recent example where one user spent over three hours attempting to create an account and being stymied by Monocle at every turn.
To be clear, this was a real user (not a bot), sitting at a keyboard, trying multiple different throwaway email providers and residential proxy services in an effort to get past the bulwarks at our registration form (including Monocle, most importantly). This user went so far as to even register an entirely new domain in an attempt to elude our email blocklist, but with Monocle enabled, it didn’t matter.
As shown in the following logs, it’s important to note that Spur context data already labels many of these IPs as having been an exit node for a residential proxy or VPN. However, Monocle detects proxies at the session level, not the IP level. In other words, the proxy labels from the IP context are coincidental, but definitely lend a large amount of credence to Monocle’s proxy detection.
Spur blocks registrations from anonymous proxy and VPN services, and also in instances where Monocle fails to load (if scripts are disabled in the browser, for example). These instances are indicated in the logs below with empty values for “Prox”, “Anon”, and “VPN”. Ultimately, the implementing user is able to decide how they wish to action (or not) Monocle’s results.
Time Email IP Loc Prox Anon VPN Context 17:36 user1@domain1 x.x.130.236 US yes yes no FACELESS 17:47 user1@domain2 x.x.162.161 CO no yes yes WINDSCRIBE 17:48 user1@domain3 x.x.162.161 CO no yes yes WINDSCRIBE 17:52 user1@domain4 x.x.128.171 US yes yes no YILU 17:52 user1@domain5 x.x.128.171 US yes yes no YILU 17:53 user2@domain3 x.x.128.171 US yes yes no YILU 17:53 user2@domain3 x.x.128.171 US YILU 17:53 user2@domain3 x.x.128.171 US yes yes no YILU 17:54 user3@domain3 x.x.128.171 US yes yes no YILU 17:55 user3@domain3 x.x.128.171 US yes yes no YILU 17:56 user1@domain6 x.x.90.38 US yes yes no TRUESOCKS 17:56 user1@domain6 x.x.90.38 US yes yes no TRUESOCKS 17:59 user1@domain7 x.x.180.216 US yes yes no IPSHARKK 18:00 user2@domain6 x.x.162.161 CO no yes yes WINDSCRIBE 18:00 user3@domain6 x.x.162.161 CO no yes yes WINDSCRIBE 18:01 user2@domain7 x.x.162.161 CO no yes yes WINDSCRIBE 18:02 user1@domain8 x.x.162.161 CO no yes yes WINDSCRIBE 18:02 user1@domain9 x.x.162.161 CO no yes yes WINDSCRIBE 18:20 user1@domain10 x.x.114.94 US no yes yes WINDSCRIBE 18:34 user1@domain11 x.x.220.46 US yes yes no NSOCKS 18:39 user1@domain7 x.x.180.216 US yes yes no IPSHARKK 18:40 user1@domain12 x.x.180.216 US yes yes no IPSHARKK 18:40 user2@domain12 x.x.19.230 US yes yes no TRUESOCKS 18:41 user3@domain12 x.x.221.51 US yes yes no TRUESOCKS 18:42 user4@domain12 x.x.252.143 US yes yes no TRUESOCKS 18:42 user1@domain13 x.x.252.143 US yes yes no TRUESOCKS 18:43 user2@domain13 x.x.192.151 US yes yes no TRUESOCKS 18:44 user1@domain14 x.x.45.165 US yes yes no TRUESOCKS 18:47 user2@domain14 x.x.166.152 US IPSHARKK 18:48 user1@domain15 x.x.185.19 US yes yes no FACELESS 18:53 user3@domain13 x.x.166.152 US IPSHARKK 18:53 user3@domain13 x.x.166.152 US yes yes no IPSHARKK 18:55 user4@domain13 x.x.166.152 US yes yes no IPSHARKK 18:57 user5@domain13 x.x.110.54 US IPSHARKK 18:58 user6@domain13 x.x.38.174 US yes yes no IPSHARKK 18:59 user7@domain13 x.x.38.174 US IPSHARKK 18:59 user7@domain13 x.x.73.214 US IPSHARKK 19:01 user8@domain13 x.x.180.44 US yes yes no UNKNOWN 19:14 user1@domain16 x.x.203.152 US yes yes no FACELESS 19:24 user1@domain17 x.x.74.220 US yes yes no FACELESS 19:42 user1@domain18 x.x.71.42 US yes yes no FACELESS 20:10 user9@domain13 x.x.61.133 US yes yes no UNKNOWN 20:12 user10@domain13 x.x.202.36 US yes yes no IPSHARKK 20:27 user3@domain3 x.x.154.167 US yes yes no GEONODE 20:27 user11@domain13 x.x.154.167 US yes yes no GEONODE 20:29 user12@domain13 x.x.105.101 US yes yes no IPSHARKK 20:30 user13@domain13 x.x.16.28 US ASTROPROXY 20:37 user14@domain13 x.x.16.28 US ASTROPROXY 20:39 user13@domain13 x.x.16.28 US ASTROPROXY 20:52 user15@domain13 x.x.129.2 US IPSHARKK
There are some key takeaways from these ~50 registration attempts:
- The user tried to register with 18 different domains across 27 different distinct IP addresses, most being residential proxies located around the US. They likely switched proxy providers a number of times
- In some cases, they apparently attempted to use the malware proxies Faceless and NSocks in order to get past Monocle, likely assuming their failed attempts were due to the more popular proxies (like Oxylabs) being on a blocklist
- It’s obvious this was a user very comfortable with using residential proxy services for their day-to-day duplicity
- In several instances, Spur data lacked context for an IP that Monocle determined was acting as a proxy
- More than once, the user likely attempted to disable scripts loading in their browser in order to make it through the form submission. As mentioned above, Spur chooses to “fail closed” in these scenarios
- Monocle renders access to an endless amount of anonymous proxies inconsequential
The ability to detect proxied connections at a session level is a game changer in the battle against fraud and abuse. This new level of insight is especially powerful when coupled with Spur’s IP context data (via our API or raw feeds) to form a fuller picture of any dubious traffic or behavior you may be dealing with on your platform.
Old dog, new tricks
As experts in anonymizing infrastructure, Spur already tracks hundreds of proxy services across millions of IP addresses. Still, we deal with the tangible effects of residential proxy usage all the same. We trust Monocle to be the first line of defense against abuse on our platform.
Residential proxies — and the threats they pose — are here to stay. The security industry is waking up to this new frontier, with everyone from Fortune 500 companies, to all manner of e-commerce and banking, to small MSSPs coming to understand the risk posed by a seemingly infinite number of cheap, burnable, and readily available proxies in residential IP space.
Up until now, those suffering the measurable outcomes of this new frontier have had to rely on blocklists and/or IP reputation to give insight to anomalous or outright malicious user activity, a methodology that is growing more obsolete with each passing day.
It’s far past time for a proactive arrow in the anti-fraud team’s quiver. Depending on the use case, Monocle has a place alongside or in lieu of traditional captchas that specialize in bot detection, and is generally recommended to be implemented on high-risk form submissions. For instance:
- account creations
- password resets
- payment submissions