SOLVING BRAND SUITABILITYMachine Learning Propelled By Brand Preferences2MACHINE LEARNING IS ONLY AS GOOD AS ITS SIGNALS.ESPECIALLY WHEN IT COMES TO VAST AMOUNTS OF VIDEOS, WHERE EACH AND EVERY VIDEO HAS COUNTLESS NUANCES3WHAT HAPPENS WHEN BRAND SIGNALS ARE USED TO FUEL MACHINE LEARNING?BRAND SUITABILITY Brand Suitability is the alignment of an individual brands advertising with content that makes sense for their image, customer base, and business objectivesBRAND PREFERENCESBrand Preferences are signals brands communicate about what content is best for them. Examples include inclusions lists, exclusion lists, content descriptions, and preferences about individual pieces of content.HUMAN IN THE LOOPHuman in the Loop (HIL) is a process of guiding machine learning with human supervision. People review content with brands' preferences as guides in order to train machine learning algorithms, creating a cycle that consistently improves its models.4GLOSSARYImportant Terms To Know5123RESEARCH QUESTIONS What are consumer attitudes toward video ad and content alignment?How does “human in the loop” machine learning perform compared to traditional targeting methods? Can “human in the loop” machine learning prevent ad/content misalignments? RECRUITRecruited YouTube users for participationn=3,858VIDEO INTERESTSParticipants selected online video topics based on personal interests; those not interested screened out to ensure natural audienceRANDOMIZATIONRandomization into test and control groups Test = Brand Ad (15s) Control = Public Service AnnouncementYOUTUBE EXPERIENCEParticipants visit YouTube testing page, where participants select and play video content based on their interestsBRAND KPISPost-exposure survey to measure traditional branding metrics and perceptions of advertising6METHODOLOGYRigorous Testing Through Experimental Design7WHAT WE MEASUREDIsolating Targeting EffectsDEMO CHANNEL KEYWORD “HUMAN IN THE LOOP”Reflects typical demographic buy on YouTubeWho: Brands demographic targetWhat: Popular content on YouTubeReflects typical channel buy on YouTubeWho: General YouTube audienceWhat: YouTube content based on channels the brand typically targetsReflects typical keyword buy on YouTubeWho: General YouTube audienceWhat: YouTube content based on keywords the brand typically targetsReflects buy on YouTube based on brand-determined suitability signals Who: General YouTube audienceWhat: YouTube content selected via machine learning + human review based on brand-determined signals for suitability8WE ALSO MEASUREDIsolating the Impact of Content QualityLOW QUALITY CONTENTReflects what happens when ads appear next to what are traditionally considered low quality videosWho: General YouTube audienceWhat: YouTube content identified via machine learning + human review based on what is traditionally considered low quality contentReflects what happens when ads appear next to what are traditionally considered high quality videosWho: General YouTube audienceWhat: YouTube content identified via machine learning + human review based on what is traditionally considered high quality contentHIGH QUALITY CONTENTADADVIDEO SELECTION FOR TESTING”Human In The Loop” Curated Videos 93,858 consumers selected content based on their interestsVideos segmented by targeting typeVideos randomly selected for testing Human review guided the machine learning to identify preferencesThe marketer provided signals for the best types of content for the brand to appear next toMachine learning identified brand suitable and/or high quality videosZefr scanned 3.5 billion videos on YouTube10BRANDS WE INCLUDEDThree Industry Verticals