Artificial IntelligenceIndex Report 20222Artificial IntelligenceIndex Report 2022INTRODUCTION TO THE AI INDEX REPORT 2022Welcome to the fifth edition of the AI Index Report! The latest edition includes data from a broad set of academic, private, and nonprofit organizations as well as more self-collected data and original analysis than any previous editions, including an expanded technical performance chapter, a new survey of robotics researchers around the world, data on global AI legislation records in 25 countries, and a new chapter with an in-depth analysis of technical AI ethics metrics. The AI Index Report tracks, collates, distills, and visualizes data related to artificial intelligence. Its mission is to provide unbiased, rigorously vetted, and globally sourced data for policymakers, researchers, executives, journalists, and the general public to develop a more thorough and nuanced understanding of the complex field of AI. The report aims to be the worlds most credible and authoritative source for data and insights about AI.FROM THE CO-DIRECTORSThis years report shows that AI systems are starting to be deployed widely into the economy, but at the same time they are being deployed, the ethical issues associated with AI are becoming magnified. Some of this is naturalafter all, we tend to care more about the ethical aspects of a given technology when it is being rolled out into the world. But some of it is bound up in the peculiar traits of contemporary AIlarger and more complex and capable AI systems can generally do better on a broad range of tasks while also displaying a greater potential for ethical concerns. This is bound up with the broad globalization and industrialization of AIa larger range of countries are developing, deploying, and regulating AI systems than ever before, and the combined outcome of these activities is the creation of a broader set of AI systems available for people to use, and reductions in their prices. Some parts of AI are not very globalized, though, and our ethics analysis reveals that many AI ethics publications tend to concentrate on English-language systems and datasets, despite AI being deployed globally. If anything, we expect the above trends to continue: 103% more money was invested in the private investment of AI and AI-related startups in 2021 than in 2020 ($96.5 billion versus $46 billion). Jack Clark and Ray Perrault3Artificial IntelligenceIndex Report 2022Private investment in AI soared while investment concentration intensified: The private investment in AI in 2021 totaled around $93.5 billionmore than double the total private investment in 2020, while the number of newly funded AI companies continues to drop, from 1051 companies in 2019 and 762 companies in 2020 to 746 companies in 2021. In 2020, there were 4 funding rounds worth $500 million or more; in 2021, there were 15. U.S. and China dominated cross-country collaborations on AI: Despite rising geopolitical tensions, the United States and China had the greatest number of cross-country collaborations in AI publications from 2010 to 2021, increasing five times since 2010. The collaboration between the two countries produced 2.7 times more publications than between the United Kingdom and Chinathe second highest on the list.Language models are more capable than ever, but also more biased: Large language models are setting new records on technical benchmarks, but new data shows that larger models are also more capable of reflecting biases from their training data. A 280 billion parameter model developed in 2021 shows a 29% increase in elicited toxicity over a 117 million parameter model considered the state of the art as of 2018. The systems are growing significantly more capable over time, though as they increase in capabilities, so does the potential severity of their biases.The rise of AI ethics everywhere: Research on fairness and transparency in AI has exploded since 2014, with a fivefold increase in related publications at ethics-related conferences. Algorithmic fairness and bias has shifted from being primarily an academic pursuit to becoming firmly embedded as a mainstream research topic with wide-ranging implications. Researchers with industry affiliations contributed 71% more publications year over year at ethics-focused conferences in recent years.AI becomes more affordable and higher performing: Since 2018, the cost to train an image classification system has decreased by 63.6%, while training times have improved by 94.4%. The trend of lower training cost but faster training time appears across other MLPerf task categories such as recommendation, object detection and language processing, and favors the more widespread commercial adoption of AI technologies. Data, data, data: Top results across technical benchmarks have increasingly relied on the use of extra training data to set new state-of-the-art results. As of 2021, 9 state-of-the-art AI systems out of the 10 benchmarks in this report are trained with extra data. This trend implicitly favors private sector actors with access to vast datasets. More global legislation on AI than ever: An AI Index analysis of legislative records on AI in 25 countries shows that the number of bills containing “artificial intelligence” that were passed into law grew from just 1 in 2016 to 18 in 2021. Spain, the United Kingdom, and the United States passed the highest number of AI-related bills in 2021 with each adopting three. Robotic arms are becoming cheaper: An AI Index survey shows that the median price of robotic arms has decreased 4-fold in the past six yearsfrom $50,000 per arm in 2016 to $12,845 in 2021. Robotics research has become more accessible and affordable. TOP TAKEAWAYS4Artificial IntelligenceIndex Report 2022Steering CommitteeStaff and ResearchersCo-DirectorsMembersResearch Manager and Editor in Chief Research AssociateAffiliated ResearchersGraduate ResearcherJack ClarkAnthropic, OECDDaniel ZhangStanford UniversityErik BrynjolfssonStanford UniversityJohn EtchemendyStanford UniversityTerah LyonsJames ManyikaGoogle, University of OxfordJuan Carlos NieblesStanford University, SalesforceMichael SellittoStanford UniversityYoav Shoham (Founding Director)Stanford University, AI21 LabsRaymond PerraultSRI InternationalNestor MaslejStanford UniversityBenjamin Bronkema-BekkerStanford UniversityAndre BarbeThe World BankLatisha HarryIndependent ConsultantHelen NgoCohereEllie SakhaeeMicrosoft5Artificial IntelligenceIndex Report 2022How to Cite This ReportPublic Data and ToolsAI Index and Stanford HAIDaniel Zhang, Nestor Maslej, Erik Brynjolfsson, John Etchemendy, Terah Lyons, James Manyika, Helen Ngo, Juan Carlos Niebles, Michael Sellitto, Ellie Sakhaee, Yoav Shoham, Jack Clark, and Raymond Perrault, “The AI Index 2022 Annual Report,” AI Index Steering Committee, Stanford Institute for Human-Centered AI, Stanford University, March 2022. The AI Index 2022 Annual Report by Stanford University is licensed under Attribution-NoDerivatives 4.0 International. To view a copy of this license, visit /creativecommons/licenses/by-nd/4.0/.The AI Index 2022 Report is supplemented by raw data and an interactive tool. We invite each reader to use the data and the tool in a way most relevant to their work and interests. Raw data and charts: The public data and high-resolution images of all the charts in the report are available on Google Drive. Global AI Vibrancy Tool: We redesigned the Global AI Vibrancy Tool this year with a better visualization to compare up to 29 countries across 23 indicators.The AI Index is an independent initiative at the Stanford Institute for Human-Centered Artificial Intelligence (HAI). We welcome feedback and new ideas for next year. Contact us at AI-Index-Reportstanford.edu.The AI Index was conceived within the One Hundred Year Study on AI (AI100). 6Artificial IntelligenceIndex Report 2022Supporting PartnersAnalytics and Research Partners7Artificial IntelligenceIndex Report 2022ContributorsWe want to acknowledge the following individuals by chapter and section for their contributions of data, analysis, advice, and expert commentary included in the AI Index 2022 Report:Research and DevelopmentSara Abdulla, Catherine Aiken, Jack Clark, James Dunham, Nezihe Merve Grel, Nestor Maslej, Ray Perrault, Sarah Tan, Daniel ZhangTechnical PerformanceJack Clark, David Kanter, Nestor Maslej, Deepak Narayanan, Juan Carlos Niebles, Konstantin Savenkov, Yoav Shoham, Daniel ZhangTechnical AI EthicsJack Clark, Nestor Maslej, Helen Ngo, Ray Perrault, Ellie Sakhaee, Daniel ZhangThe Economy and EducationBetsy Bizot, Erik Brynjolfsson, Jack Clark, John Etchemendy, Murat Erer, Akash Kaura, Julie Kim, Nestor Maslej, James Manyika, Brenden McKinney, Julia Nitschke, Ray Perrault, Brittany Presten, Tejas Sirohi, Bledi Taska, Rucha Vankudre, Daniel ZhangAI Policy and GovernanceAmanda Allen, Benjamin Bronkema-Bekker, Jack Clark, Latisha Harry, Taehwa Hong, Cameron Leuthy, Terah Lyons, Nestor Maslej, Ray Perrault, Michael Sellitto, Teruki Takiguchi, Daniel ZhangConference AttendanceTerri Auricchio (ICML), Christian Bessiere (IJCAI), Meghyn Bienvenu (KR), Andrea Brown (ICLR), Alexandra Chouldechova (FAccT), Nicole Finn (ICCV, CVPR), Enrico Gerding (AAMAS), Carol Hamilton (AAAI), Seth Lazar (FAccT), Max Qing Hu Meng (ICRA), Jonas Martin Peters (UAI), Libor Preucil (IROS), MarcAurelio Ranzato (NeurIPS), Priscilla Rasmussen (EMNLP, ACL), Hankz Hankui Zhuo (ICAPS)Global AI Vibrancy ToolAndre Barbe, Latisha Harry, Daniel ZhangRobotics SurveyPieter Abbeel, David Abbink, Farshid Alambeigi, Farshad Arvin, Nikolay Atanasov, Ruzena Bajcsy, Philip Beesley, Tapomayukh Bhattacharjee, Jeannette Bohg, David J. Cappelleri, Qifeng Chen, I-Ming Chen, Jack Cheng, Cynthia Chestek, Kyujin Cho, Dimitris Chrysostomou, Steve Collins, David Correa, Brandon DeHart, Katie Driggs-Campbell, Nima Fazeli, Animesh Garg, Maged Ghoneima, Tobias Haschke, Kris Hauser, David Held, Yue Hu, Josie Hughes, Soo Jeon, Dimitrios Kanoulas, Jonathan Kelly, Oliver Kroemer, Changliu Liu, Ole Madsen, Anirudha Majumdar, Genaro J. Martinez, Saburo Matunaga, Satoshi Miura, Norrima Mokhtar, Elena De Momi, Chrystopher Nehaniv, Christopher Nielsen, Ryuma Niiyama, Allison Okamura, Necmiye Ozay, Jamie Paik, Frank Park, Karthik Ramani, Carolyn Ren, Jan Rosell, Jee-Hwan Ryu, Tim Salcudean, Oliver Schneider, Angela Schoellig, Reid Simmons, Alvaro Soto, Peter Stone, Michael Tolley, Tsu-Chin Tsao, Michiel van de Panne, Andy Weightman, Alexander Wong, Helge Wurdemann, Rong Xiong, Chao Xu, Geng Yang, Junzhi Yu, Wenzhen Yuan, Fu Zhang, Yuke Zhu8Artificial IntelligenceIndex Report 2022Bloomberg GovernmentAmanda Allen, Cameron LeuthyCenter for Security and Emerging Technology, Georgetown UniversitySara Abdulla, Catherine Aiken, James DunhamComputing Research AssociationBetsy BizotEmsi Burning GlassJulia Nitschke, Bledi Taska, Rucha VankudreIntentoGrigory Sapunov, Konstantin Savenkov LinkedIn Murat Erer, Akash Kaura McKinsey Global InstituteBrenden McKinney, Brittany PrestenMLCommonsDavid KanterNetBase QuidJulie Kim, Tejas SirohiWomen in Machine LearningNezihe Merve Grel, Sarah TanWe thank the following organizations and individuals who provided data for inclusion in the AI Index 2022 Report:We also would like to thank Jeanina Casusi, Nancy King, Shana Lynch, Jonathan Mindes, Stacy Pea, Michi Turner, and Justin Sherman for their help in preparing this report, and Joe Hinman, Travis Taylor, and the team at Digital Avenues for their efforts in designing and developing the AI Index and HAI websites. Organizations9Artificial IntelligenceIndex Report 2022Table of ContentsREPORT HIGHLIGHTS 10CHAPTER 1 Research and Development 13CHAPTER 2 Technical Performance 47CHAPTER 3 Technical AI Ethics 100CHAPTER 4 The Economy and Education 139CHAPTER 5 AI Policy and Governance 172APPENDIX 196ACCESS THE PUBLIC DATA10Artificial IntelligenceIndex Report 2022CHAPTER 1: RESEARCH AND DEVELOPMENT Despite rising geopolitical tensions, the United States and China had the greatest number of cross-country collaborations in AI publications from 2010 to 2021, increasing five times since 2010. The collaboration between the two countries produced 2.7 times more publications than between the United Kingdom and Chinathe second highest on the list. In 2021, China continued to lead the world in the number of AI journal, conference, and repository publications63.2% higher than the United States with all three publication types combined. In the meantime, the United States held a dominant lead among major AI powers in the number of AI conference and repository citations. From 2010 to 2021, the collaboration between educational and nonprofit organizations produced the highest number of AI publications, followed by the collaboration between private companies and educational institutions and between educational and government institutions. The number of AI patents filed in 2021 is more than 30 times higher than in 2015, showing a compound annual growth rate of 76.9%.CHAPTER 2: TECHNICAL PERFORMANCE Data, data, data: Top results across technical benchmarks have increasingly relied on the use of extra training data to set new state-of-the-art results. As of 2021, 9 state-of-the-art AI systems out of the 10 benchmarks in this report are trained with extra data. This trend implicitly favors private sector actors with access to vast datasets. Rising interest in particular computer vision subtasks: In 2021, the research community saw a greater level of interest in more specific computer vision subtasks, such as medical image segmentation and masked-face identification. For example, only 3 research papers tested systems against the Kvasir-SEG medical imaging benchmark prior to 2020. In 2021, 25 research papers did. Such an increase suggests that AI research is moving toward research that can have more direct, real-world applications. AI has not mastered complex language tasks, yet: AI already exceeds human performance levels on basic reading comprehension benchmarks like SuperGLUE and SQuAD by 1%5%. Although AI systems are still unable to achieve human performance on more complex linguistic tasks such as abductive natural language inference (aNLI), the difference is narrowing. Humans performed 9 percentage points better on aNLI in 2019. As of 2021, that gap has shrunk to 1.REPORT HIGHLIGHTS11Artificial IntelligenceIndex Report 2022 Turn toward more general reinforcement learning: For the last decade, AI systems have been able to master narrow reinforcement learning tasks in which they are asked to maximize performance in a specific skill, such as chess. The top chess software engine now exceeds Magnus Carlsens top ELO score by 24%. However, in the last two years AI systems have also improved by 129% on more general reinforcement learning tasks (Procgen) in which they must operate in novel environments. This trend speaks to the future development of AI systems that can learn to think more