A Botnet Primer for Display Advertisers

This is the unabridged version of a guest post on AdExchanger.

Botnets are the biggest contributor to online display advertising fraud today. Rentable botnets are the most unnerving and the most surprising contributor.

DirectorsLive.com provides an illustrative example of the escalating botnet problem. According to Whois records, DirectorsLive was registered in August 2009; and the Wayback Machine shows its first snapshot for DirectorsLive in September 2009. Since then DirectorsLive has been reporting traffic growth which rivals Pinterest.com, arguably the fastest growing standalone website ever. At the beginning of this year six billion display ad impressions were being served across DirectorsLive each month—six billion display ad impressions being more than most of the largest demand-side platforms (DSPs) and performance advertisers buy each month. The Chameleon botnet was responsible for almost every single one of the six billion display ad impressions being served across DirectorsLive.

In this article we provide some context for online display advertising fraud. We then review the mechanics of botnet-driven fraud across display advertising.

The monetisation engine of the Web is fragile
The monetisation engine of the Web is advertising. More specifically the monetisation engine of the Web is display advertising.

According to recent predictions, display advertising spend will have overtaken search pay-per-click advertising spend by 2016 in the US. To boot, search pay-per-click advertising is largely the monetisation engine of just Google. Over 80% of current search pay-per-click spend in the US is through Google’s pay-per-click ad network and the lion’s share of this spend—at least 70% (and probably much more)—goes toward pay-per-click ads which are shown on websites owned by Google. The rest of the Web predominantly monetises through display advertising—Facebook, Yahoo!, The New York Times and millions of blogs across the Web.

Despite the importance of display advertising to the Web, efforts to combat display advertising fraud are still in their infancy. This is troubling because there is a strong financial motive for publishers and networks to game the system. It is troubling because of the ease with which nefarious and negligent parties can exploit advertisers. But it is perhaps most troubling because of how poorly the industry understands the mechanics of fraud—because without this understanding it is difficult to implement appropriate defences.

Unchecked fraud across display advertising will continue to increase uncertainty for advertisers. The greater this uncertainty, the more advertisers will discount their expected return on investment across display advertising. And the more they discount, the more spend will shift to other advertising channels.

In 2004 Google’s CFO warned: “Something has to be done about [click fraud] really, really quickly, because I think, potentially, it threatens our business model.”

Today the display advertising ecosystem needs to act swiftly to tackle impression fraud.

What is a botnet?
Many in the display advertising industry mistakenly regard botnet traffic as meaning all automated website traffic. There are, in fact, two distinct ways to programmatically surf the Web.

The first way is to deploy your programmatic surfer across computers you own or control legitimately. Googlebot is an example of this type of programmatic surfer. The Alexa crawler is another example. The Alexa crawler surfs the Web from Amazon EC2 IP addresses. Both Googlebot and the Alexa crawler are well-behaved in that they announce themselves as automated agents when they visit websites. They do this by including Googlebot and ia_archiver in their respective user-agent headers. Not all programmatic surfers deployed across legitimately controlled machines are well-behaved. Some have user-agent headers suggesting that they are human-powered browsers. However, even if they are not well-behaved, they are often easy to identify as they typically operate over a finite set of IP addresses and these IP addresses are typically either cloud IP addresses or Tor IP addresses.

The second way to programmatically surf the Web is to deploy your programmatic surfer across an illegal botnet. Botnets are collections of illegitimately hijacked PCs. Cybercriminals use these hijacked PCs to perform various tasks without the owners of the computers being aware.

Historically cybercriminals would have hijacked PCs by tricking users into clicking on some Trojan-horse email attachment. Indications today are that PCs are increasingly being hijacked via pornographic and file-sharing websites. In much the same way that mainstream websites sell ad space to advertisers, some pornographic websites sell space to botnet controllers (herders). This allows the botnet controller to embed exploit kits in the pornographic websites, so that when a user clicks somewhere on a pornographic webpage, the exploit kit is downloaded, the defences of the PC are breached, and control of the PC is ceded to the botner controller.

Programmatic surfers deployed across botnets are markedly more troubling than programmatic surfers deployed across legitimately owned/rented computers. This is because botnet surfers are deployed across the PCs of real people. This means that botnet surfers have residential or corporate IP addresses. They typically have regular browser user-agent headers. They may even surf websites using the cookies of the unknowing owner of the PC. If a botnet controller has taken control of someone’s PC, all manner of disturbing things are possible on that PC.

Enterprise-grade botnets for rent
In a research paper published late last year, Russian Underground 101, some unnerving details were revealed about the state of botnet use today. These details were subsequently explored in an article, A Beginner’s Guide to Building Botnets.

The paper and the article show that it is now possible to rent enterprise-grade botnets in much the same way that one would rent cloud computing resources from, say, Amazon Web Services or Google Compute Engine or Windows Azure. This has become possible because the controllers of the most infamous botnets, like Zeus, Carberp and SpyEye, have moved on from conducting criminal activity themselves to being crimeware vendors. According to the paper, $595 would be a typical setup cost for the first month of botnet rental, and a typical monthly cost thereafter would be $225.

The rentable botnets are disturbingly enterprise-grade to the extent that they come with 24/7 technical support, monitoring services and auto-patching.

We have learnt that some botnets may come with A-B test harnesses and partial roll-out facilities. These allow the renters of botnets to respond quickly to any new defensive efforts taken to combat botnet activity. For example, some social networks have seen their defences being probed in an effort to reverse-engineer the rules the social networks use to identify and block fake profiles. Once the rules have been discovered, and a social network’s defences have been learned, the full force of the botnet is then used to generate fake profiles at scale.

There are indications that some botnets also come with a form of software virtualisation, so that when renters upload code to the botnets, this code is rotated periodically across machines. This would reduce the chance of the careless renter exposing the PC as being hijacked, as no task is run for long on the same PC. Across the Chameleon botnet, for example, we have seen activity move from machine to machine every two or three days.

An app marketplace for cybercriminals
Not only are there enterprise-grade botnets for rent, there is also a disturbingly rich app marketplace for these rentable botnets.

Cybercriminals can buy apps (injector kits) for denial-of-service attacks, apps for spam emails, apps for credit-card theft, apps for banking fraud, apps for fake profile generation across social networks, apps for click fraud and apps for display advertising fraud. These apps typically cost less than $100, and on-going support for an app can also often be bought for less than $10 per month.

These apps mean that very little technical ability is now required to commit botnet-driven cybercrime.

Apps for display advertising fraud
The botnet apps for display advertising fraud are already surprisingly sophisticated, and they will doubtless only become more sophisticated over time. These botnet apps comprise their own web browsers, and they are set up to manipulate the metrics that display advertisers use to optimise their buying.

Some of these apps are exploiting the retargeting strategies of specific advertisers. This involves the programmatic surfer first visiting some specific product webpage, where the retargeting advertiser is running an advertising campaign for this product. The programmatic surfer’s visit to the product webpage is intended to look to the advertiser like an incomplete purchase. The retargeting advertiser will then subsequently look to buy ad space on any website visited by the programmatic surfer with the mistaken aim of getting the programmatic surfer to complete the purchase.

There are strong indications that some botnet apps are already gaming CPA metrics—as many unwanted things are possible when a real person’s PC has been hijacked—and we are investigating this currently.

Do botnets only affect long-tail websites?
Many in the industry have asked whether botnets only impact the websites of nefarious publishers. Indications are that this is not always the case.

Following the disclosure of the Chameleon botnet, someone from an affected publisher group came forward to explain not just how the publisher group had inadvertently bought Chameleon botnet traffic, but also how this publisher group had then subsequently resold Chameleon botnet traffic to two of the Web’s most high-profile websites. This person provided details of a network of traffic laundering. Indications are that cybercriminals are renting botnets and selling fake traffic on to others in the form of cheap pay-per-click traffic, much like the pay-per-click traffic that is sold to text-link advertisers by Google.com. The buyer of the botnet-generated traffic may sell this on to someone else, who in turn may sell it on to someone else. Ultimately a publisher will buy the traffic, and this publisher may or may not know that the traffic is fake.

We are currently investigating the traffic laundering details provided to us.

Summary thoughts
In this article we reviewed the mechanics of botnet fraud across display advertising. We considered how botnets based on the code of some of the most infamous botnets, like Zeus, Carberp and SpyEye, have now become rentable. We considered the rich app marketplace for these rentable botnets. This marketplace means that very little technical ability is now required for cybercriminals to defraud display advertisers. Finally we considered what appears to be a network of traffic laundering, whereby botnet traffic impacts high profile websites as well as long tail websites.

In the early 2000s click farms were regarded as the biggest threat to the integrity of online advertising. Today botnets are the big threat.