2019年网络广告欺诈流量报告.pdf

1 Bot Baseline 2018-2019 May 2019 2018-2019 Bot Baseline Fraud in Digital Advertising2 Bot Baseline 2018-2019 Table of Contents 2 Table of Contents 2 A special thank you to the following ANA Member company participants 4 A Letter from the Authors 5 About the Study 6The Size and Scope of the Study 7 Major Findings 8For the First Time, More Fraud Will Be Stopped This Year than Will Succeed 8Projected Fraud Losses in 2019 by Category* 9How the Industry Has Made These Gains against Fraud 10The Best Buyers are Doing Better than Ever 10Its Still Too Hard to Know What Youre Buying 11 The Good News 12Fraud Volumes Are Growing in Newer Frontiers, Such as Mobile and OTT, 12But Reductions in Desktop Fraud Prove this Fight Is Winnable Structural Improvements to the Fight Against Ad Fraud 13More Money Is Now Being Spent Through Ad Tech Platforms with 15Built-In Fraud Prevention Measures T r affic V endor s G o Under gr ound 16Cyber security Measur es Lik e A ds. txt Ha v e R educ ed Deskt op Spoofing 18The Formation of Industry Coalitions 19 The Bad News 20Less than Half of All Impressions are Fully, Transparently Validatable 21Most Video Ads Still Dont Support VAST4, the Gold Standard in Transparency 22The Quality of Real-Time Independent Monitoring is Still Inconsistent 23Outdated Technology 23Technical Limitations 23Walled Gardens 243 Bot Baseline 2018-2019 3Human Error 25Intentional Evasion 25In Practice, High-Fidelity Validation is Surprisingly Uneven 25 The Frontiers in Fraud 26Mobile 26F r aud on the Mobile W eb: Eas y t o Do , Har d t o Pr ofit 26In-App F r aud: Gr o wing and Inno v a tiv e 26Server-Side Ad Insertion (SSAI) 27Connected TV (CTV) 27 Recommendations 28Steps to Take to Reduce Fraud 28Steps to Take to Increase Measurability 30 Ad Fraud Glossary: An Exhaustive List 31 About the Study Partners 38About the ANA 38About White Ops 384 Bot Baseline 2018-2019 A special thank you to the following ANA Member company participants 45 Bot Baseline 2018-2019 A Letter from the Authors This is our fourth Bot Baseline report, and in some important ways, things are better than they ha v e e v er been . Illegitima t e tr affic sour cing is declining. Les s sophis tica t ed c yber criminals ha v e abandoned their fraud schemes. Industry coalitions have formed to combat more sophisticated threats. Advertisers are spending more on channels with robust fraud protection measures. We are seeing our founding faith turn into validated fact: the battle against fraud is winnable. However, the battle is not won yet. Ad fraud persists in new, creative forms. Every year, cybercrime becomes more advanced, evolving against the selection pressure of fraud detection practices. We sa w this clearly with 3v e , the fr aud oper a tions tha t implement ed sophis tica t ed tag e v asion - wher e v erifica tion tag s w er e b ypas sed - as w ell as IP hijacking - wher e gr oups o f IP addr es ses w er e tak en o v er - in or der t o b ypas s security measur es. Though transparency issues have always plagued digital advertising, they have become more pr es sing than e v er . In this r epor t, w e r e v eal the impor tanc e - and per v asiv enes s - o f the limitations in complete third party auditability of ad impressions. Now is the time for advertisers to push for ability to hold all ad impressions to the same high standard of validatability. Imagine if every CAPTCHA on the internet was the same. It wouldnt work very well. Validation with only a pixel is like serving a CAPTCHA that never changes. Only a dynamic challenge can be used to catch a dynamic adversary. 1Despit e these challenges, w e ar e op timis tic about the futur e . A ds. txt, and the r eduction in spoofing it achieved, has proven the power of industry-wide cooperation. Federal indictments such as the ones handed down for the 3ve and Methbot operations demonstrate that cybercriminals can be held accountable, and face real consequences, for their actions. 1A C APT CH A is a pr ogr am or s y s t em int ended t o dis tinguish entities, specifically those entities who ar e humans fr om those that are automated; typically this is a method to prevent spam and automated extraction of data from websites. As an indus tr y , w e no w ha v e the momentum, if w e per se v er e , t o fix the fr aud pr oblem in a deeply meaningful way. We believe that marketers can help bring about a future where ad fraud is not pr ofitable enough t o be w or th the risk. 5 Michael Tiffany Co-Founder the r emaining 24 par ticipa t ed f or the fir s t time. Our study examines advertising by brand marketers. It does not include search or paid social media campaign data. 7 Domains 606K Participants 50 Campaigns 2,400 Placements 130K Impressions 27B8 Bot Baseline 2018-2019 Major Findings For the First Time, More Fraud Will Be Stopped This Year than Will Succeed Today, fraud attempts amount to 20 to 35 percent of all ad impressions throughout the year, but the fraud that gets through and gets paid for now is now much smaller. We project losses to fraud to reach $5.8 billion globally 3in 2019. In our prior study, we projected losses of $6.5 billion for 2017. That 11 percent decline in two years is particularly impressive considering that digital ad spending increased by 25.4 percent between 2017 and 2019. A detailed breakdown by device and media type is below. F or the fir s t time , the majority o f fr aud a tt emp ts ar e ge tting s tymied be f or e the y ar e paid f or , b y DSP s and SSP s filt ering fr audulent bid r eques ts, b y cla wback s, or b y o ther preventative measures. Absent those measures, losses to fraud would have grown to at least $14 billion annually. Fraud is an evolving threat. Fraud rates have been growing in new formats. Continued vigilance is needed. But there is no denying the structural gains the industry has made in this fight. 8 3Loss estimates are based strictly on digital spending in the categories included in the study: video, display, and other CPM formats for desktop and mobile devices.9 Bot Baseline 2018-2019 Projected Fraud Losses in 2019 by Category* Desktop study sample size: 13.6 Billion Impressions Mobile study sample size: 13.5 Billion Impressions 4As in prior years, video still shows more fraud. 8% Display down from 9% in 2017 14% Video 4 down from 22% in 2017 12% Other Rich Media, Takeovers, etc. 3% Display 8% In-App Video 14% Web Video 7% Other 910 Bot Baseline 2018-2019 How the Industry Has Made Gains against Fraud Anti- fr aud measur es ha v e made tr affic sour cing mor e difficult and e xpensiv e . Than k s t o tr affic sour cing tr anspar enc y e ff or ts led b y the T rus tw or th y A c c ountability Gr oup (T A G), tr affic v endor s ha v e gone fur ther under gr ound , r educing “ r e tail” bo t buying on the open web. A ds. txt has r educ ed domain spoofing. 5Additionally, TAG requires publishers to have c omple t ed ads. txt files if the y w ant t o be C er tified A gains t F r aud . Far more dollars are now being spent through programmatic platforms with built-in fraud prevention measures. Arrests, like those of the alleged masterminds behind the 3ve and Methbot operations, have brought real consequences to botnets operating overseas. The Best Buyers are Doing Better than Ever In our fir s t s tudy in 20 14, fr aud a ff ect ed e v er y one , those with s tr aightf or w ar d media plans and very sophisticated ones alike. This year, every buyer in the study knew about fraud risk; 90 percent had MRC- ac cr edit ed fr aud v erifica tion measur es in plac e t o deal with this risk. Performance was loosely correlated with study participation; long-time participants performed better than they did in previous years. For the top quintile of buyers, fraud was nearly nonexistent. However, discrepancies be tw een ad ser v er impr es sions number s and v erifica tion impr es sions number s w er e present for even the most sophisticated buyers. 5Ads.txt, created by the IAB, stands for Authorized Digital Sellers. The mission of ads.txt is to increase transparency in the programmatic advertising ec os y s t em. It is a simple , fle xible , and secur e me thod tha t publisher s and dis tribut or s can use t o publicly declar e the c ompanies the y authorize t o sell their digital inventory. 1011 Bot Baseline 2018-2019 Its Still Too Hard to Know What Youre Buying The ability to hold all ad spending to the same high level of validatability should be one of marketers top concerns in 2019. The time has come for marketers to stop tolerating and stop paying for outdated media formats like VAST 2 that cannot support the highest levels of third-party validation. By selling media under conditions that do not support high third-party validation, even trustworthy media companies are essentially part of the problem, since the vast sea of good but low-transparency inventory provides cover for the fake inventory sold under the same formats. Fraud detection and prevention is only as good as its implementation. Low Invalid T r affic (IVT) 6measurements do not provide a full picture when half of a media plan is highly v alida table with ac cr edit ed thir d-par ty , dynamic Ja v aScrip t v erifica tion , but the rest is covered by a mix of half-measures like 1x1 pixels, list-based protections, and limited, static integrations. 11 6In v alid T r affic is o ft en r e f err ed t o as Non-Human T r affic (NHT). It is the t o tal o f G ener al In v alid T r affic (GIVT) and Sophis tica t ed In v alid T r affic (SIVT). Case Study One CPG participant purchased 44 million video impressions from an agency trading desk. Of the 44 million impressions, only 48 percent were validatable at the highest standard, with a dynamic fraud test served via JavaScript. Of the 48 percent, over 10 percent was SIVT. 12 Bot Baseline 2018-2019 The Good News Fraud Volumes Are Growing in Newer Frontiers, Such as Mobile and OTT, But Reductions in Desktop Fraud Prove this Fight Is Winnable The rates for desktop ad fraud were the lowest in the history of the Bot Baseline study. Strides ha v e been made in deskt op displa y , wher e buying unde t ectable bo t tr affic is bec oming t oo e xpensiv e t o be pr ofitable . F r aud in deskt op displa y w as 1 1 per c ent in 20 14, nine percent in 201617, and eight percent in 201819. This is how we win: if the industry makes cybercrime unprofitable, cybercriminals will lose the incentive to invest in ad fraud. 12 Case Study 5.8 million display impressions were purchased from a century-old nonprofit organization, with just 22 percent validatable at the highest level. Of those 1.2 million impressions, 15 percent were SIVT. 13 Bot Baseline 2018-2019 Structural Improvements to the Fight Against Ad Fraud In the fiv e y ear s sinc e our fir s t s tudy , ad spending in the ca t egories mos t vulner able t o fraud video and display advertising across desktop and mobile devices has more than doubled . But losses to ad fraud have not. Increased awareness of the problem among marketers raised the priority of dealing with it, and leaders throughout the industry have risen t o the challenge . Fir s t, this slo w ed the gr o wth o f ad fr aud . No w , f or the fir s t time , w e ar e pr ojecting los ses this y ear t o be smaller than in the fir s t y ear o f our s tudy . 7P W C Global Ent er tainment & Media Outlook 20 18-22. 13 Dollars lost to ad fraud year over year Projected fraud losses globally $6.3 2014 $7.1 +13% 2015 $6.5 -8% 2017 $5.8 -11% 2019 +20% +40% +25% Dollar s Los t t o F r aud ($B) Growth in global digital ad spending14 Bot Baseline 2018-2019 1. Advertisers are directing more of their money through buying channels with dedicated, independent fraud prevention measures, an option that wasnt available at the time o f our fir s t s tudy . 2. B uying bo t tr affic has become harder. Many bot tr affic v endor s ha v e been driven out of business or gone underground, reducing the once-prevalent a v ailability o f bo t tr affic sold on the open web to anyone with a credit card. 3. B uying bo t tr affic has become more expensive. The market price for sophis tica t ed bo t tr affic has risen, limiting the arbitrage opportunity for buying visitors and showing them enough ad units to mak e a pr ofit. 4. Initiatives like ads.txt have helped to reduce desktop spoofing, with 78 per c ent o f the top volume domains in the s tudy using ads. txt files to prevent their inventory from being successfully spoofed by the time of our study. 5. Consequences are changing the incentives for perpetrating fraud. The business of committing ad fraud has become significantly riskier , as shown by a number of arrests in the last year something that nobody would have dreamed would happen just a few years ago. Cybersecurity companies, industry experts, and governments have come together to dismantle fraud infrastructure and take down botnets and their operators. Ther e ar e fiv e r easons tha t ad fr aud has dr opped in the las t y ear: 14