1 HOW TO BUILD A REGULATORY SANDBOX Ivo Jenk and Schan Duff A Practical Guide for Policy Makers September 2020Consultative Group to Assist the Poor 1818 H Street NW, MSN F3K-306 Washington DC 20433 Internet: cgap Email: cgapworldbank Telephone: +1 202 473 9594 Cover photo by Artur Verkhovetskiy from . CGAP/World Bank, 2020 RIGHTS AND PERMISSIONS This work is available under the Creative Commons Attribution 4.0 International Public License (creativecommons/licenses/by/4.0/). Under the Creative Commons Attribution license, you are free to copy, distribute, transmit, and adapt this work, including for commercial purposes. See license for conditions. AttributionCite the work as follows: Jenk, Ivo, and Schan Duff. 2020. “How to Build a Regulatory Sandbox: A Practical Guide for Policy Makers.” Technical Guide. Washington, D.C. : CGA P. All queries on rights and licenses should be addressed to CGAP Publications, 1818 H Street, NW, MSN F3K-306, Washington, DC 20433 USA; e-mail: cgapworldbankCONTENTS Executive Summary 1 Section 1: Introduction 2 Section 2: Objective Setting: Why Set Up a Regulatory Sandbox? 4 Section 3: How to Design a Regulatory Sandbox to Advance the Regulatory Objective 12 Section 4: How to Successfully Run a Regulatory Sandbox 15 Section 5: Alternatives to a Regulatory Sandbox 20 Annex 1: Setting Objectives for a Regulatory Sandbox 22 Annex 2: Templates for Sandbox Teams 23 Annex 3: Sandbox Simulation Guidance 32 Glossary 33 References 34Acknowledgments The authors of this Technical Guide are Ivo Jenk, financial sector specialist, and Schan Duff, senior policy consultant. Stefan Staschen, senior financial sector specialist, and Gregory Chen, lead financial sector specialist, oversaw this effort. Peer reviews were provided by Sharmista Appaya, senior financial sector specialist (World Bank), Denise Dias, senior policy consultant (CGAP), and Eric Duflos, senior financial sector specialist (CGAP). The authors thank many other experts who have contributed to the paper, provided feed- back, shared their experience, or cooperated with CGAP on in-country projects, including Tamara Cook, Gitau Mburu, Rafe Mazer, and Francis Gwer of FSD Kenya; Joseph Lutwama of FSD Uganda; and experts at the World Bank. Other experts provided feedback through a workshop held in April 2020: Christopher Calabia (Bill market engagement; and testing, evalua- tion, and exit. It is followed by a detailed description of complementary options and alternatives to regulatory sandboxes. The Annexes provide tools and sample documentation for each stage of the process. It is important to note that this guide is neither a blueprint for launching a sandbox in any specific jurisdiction nor a recommendation that regulators should adopt sandbox programs in all cases. Rather, it is a summary of common questions and decision points faced by regulators around the world in a wide range of local contexts. To the extent this Guide offers a strong recommendation, it is to encourage regulators to view sandboxes as a specialized toolbest used in conjunction with other innovation facilitatorsto enable evidence-based, outcome-oriented regulation. 3 See Jenk and Lauer (2017) and Appaya and Gradstein (2020). The impact of sandboxes on innovation generally and financial inclusion specifically has not been proven yet. 3 Our hypothesis is that regulatory sandboxes may help advance financial inclusion by enabling evidence-based, outcome-oriented regulation of new, consumer-friendly innovation that helps advance financial inclusion. To achieve this potential, regulators must implement sandboxes appropriately. They should leverage the unique insights from live-testing environments in close coordination with other complementary tools and programs. We created this Guide in service of that objective4 HOW TO BUILD A REGULATORY SANDBOX SECTION 2 OBJECTIVE SETTING: WHY SET UP A REGULATORY SANDBOX? A REGULATORY SANDBOX IS A TOOL for developing evidence about how a new product, technology, or business model (innovation) works and the outcomes it produces. Evidence gathering can help assuage (or confirm) regulatory concerns about the impact of innovations, allowing beneficial innovations to reach the marketplace. Consider the following real-life examples: Alternative data underwriting. A fintech lender proposed using alternative data (education and employ- ment history) and machine learning to underwrite consumer credit (CFPB 2017). While acknowledging the potential for expanding access to credit and lower- ing costs, the regulator worried that the underwriting model would discriminate among borrowers based on race, ethnicity, gender, or age. Using a sandbox test, the regulator determined the technology expanded access to credit and reduced costs relative to a traditional under- writing model, resulting in (i) approval of 27 percent more applicants and 16 percent lower average annual percentage rates overall and (ii) substantially higher approval rates for applicants under age 25 and consumers with incomes under $50,000. Moreover, the test identi- fied no discrimination in approvals. Insurance coverage optimization. Insurtech start- up PolicyPal offered a tool that used natural language processing (NLP) to assess policy coverage and gaps in consumer insurance policies. The regulator recognized the potential benefits of helping consumers understand and navigate complex insurance policies but also the potential harm if the tool made recommendations that exposed individuals to uncovered risks or liabilities. The regulator therefore used a sandbox to compare the results of NLP recommendations with an in-person review of the same policies by an insurance professional. The sandbox results left the regulator satisfied that NLP identified coverage issues and gaps similar to what the live reviewer identified, and the regulator approved the product for launch in the marketplace. In both cases, the new technology presented potential risks that were difficult to assess in the abstractit also identified potential consumer benefits that would be lost if the technology was not allowed to enter the marketplace. Experimenting with live data helped to show that these technologies operated as promised and yielded potential consumer benefits. Most innovations do not require this level of evidence-based regulatory analysis before launch. Regulators should have a clear understanding of the circumstances that warrant a sandbox test and when other tools and frameworks may suffice because sandboxes are time and resource intensive and generally not available to all market participants (Jenk and Appaya 2019). The sandbox decision process illustrated in Figure 1 can help to facilitate this analysis. This decision process is intended to prompt regulators to closely scrutinize the need to conduct a live test, exploring the underlying hypotheses for adopting a sandbox. When the answer is unclear, regulators should gather additional insights from innovation facilitators that do not use sandboxes and care- fully evaluate both complementary and alternative tools. Although the decision process is highly generalized, it envisions a sandbox test as the final stage of an iterative engagement between the regulator and innovator. As a prac- tical matter, this exchange helps to sharpen the questions that must be resolved by evidence developed in the sandbox testing environment. More broadly, the decision process5 d Ev ELOPin G y Our rE m OTE SEn Sin G c APABi Li Ti ES FIGURE 1. Decision process: Overview Does the new tech promise significant benefits? Can be resolved with rule update or clarification? RATIONALE FOR SANDBOX KEY Y Answer to question is “Yes” N Answer to question is “No” Consider Innovation Facilitation STEP 1 STEP 2 STEP 3 Learn about marketplace developments Address regulatory barriers to beneficial innovation Promote competition or innovation Within scope of mandate? Is the strategy to test commericial viability? Is the strategy to reduce regulatory barriers? Compliance is costly Regulation prohibits innovation Can you update the rules? Can be resolved through industry consultation? Regulation creates uncertainty Is live testing necessary? RULE OR POLICY UPDATE NO SANDBOX YES SANDBOX BARRIERS TO INNOVATION N N N Y Y Y Y Y Y Y N N N N N Y OTHER REGULATORY TOOLS START HERE OTHER INNOVATION FACILITATORchannels questions about the regulatory status of an innova- tion (i.e., whether it is permitted or prohibited) to either (i) informal resolution through a low(er) cost, broadly acces- sible innovation facilitator or (ii) a regulatory or licensing process that already is in place. This process is intended to reduce the risk that the sandbox will inadvertently (i) distort normal market mechanisms (unless that is what the regulator is mandated to do), (ii) act as an imperfect substitute for other regulatory enablers and/ or a regulatory change, (iii) create an uneven playing field, or (iv) act as a de facto gatekeeper or substitute for interac- tions with traditional licensing or supervisory processes. 4 Step One: Determine the sandbox objective Regulators frequently cite three overarching objectives for their sandbox programs: 1. Promote innovation and/or competition. 2. Address regulatory barriers to innovation. 3. Learn about developments in the marketplace. 4 For more details on design elements see Jenk and Lauer (2017). 5 This is one of the key metrics tracked in FCA (2017). Considering each objective separately can help regulators decide whether a sandboxor an alternative or complemen- tary approachis the best strategy for advancing the policy objective. See Figure 2. PROMOTE INNOVATION AND COMPETITION Regulators often begin with the assumption that a sandbox should be used to promote innovation and competition by accelerating market entry for new firms or products. To achieve this objective, sandboxes sometimes are positioned as “safe spaces” to defer the up-front time and expense of licensing and registration until after commercial viability of the innovation has been confirmed. While bringing more firms into the market by lowering the ex ante cost of regula- tion may be an important objective, sandboxes are not the best tool to achieve it. Most financial regulators do not have an express statutory mandate to promote competition or innovation. For the few financial regulators with a competition or innovation mandate, regulatory sandboxes have been used to simulta- neously “test” large cohorts of firms with the goal of moving as many as possible into the regulated marketplace. 5 In these cases, the testing activity relates more to assessing HOW TO BUILD A REGULATORY SANDBOX 6 Does the new tech promise significant benefits? Can be resolved with rule update or clarification? RATIONALE FOR SANDBOX KEY Y Answer to question is “Yes” N Answer to question is “No” Consider Innovation Facilitation STEP 1 STEP 2 STEP 3 Learn about marketplace developments Address regulatory barriers to beneficial innovation Promote competition or innovation Within scope of mandate? Is the strategy to test commericial viability? Is the strategy to reduce regulatory barriers? Compliance is costly Regulation prohibits innovation Can you update the rules? Can be resolved through industry consultation? Regulation creates uncertainty BARRIERS TO INNOVATION N N N Y Y Y Y Y N N START HERE STEP 3 Does the new tech promise significant benefits? Can you resolve with new rulemaking? STEP 3 Can you resolve by updating the rules? Can you resolve through industry consultation? FIGURE 2. Decision process: Step one Y Y Y Y N N N N7 OBj Ec Tiv E SETTin G: Why S ET uP A rEG u LATOry S And BOx? commercial viabilityand accelerating time to market than developing outcomes-based evidence to assess risk or make a regulatory determination. However, the regulators role in promoting competition and innovation is far more complex and nuanced than helping a handful of early-stage firms gain market traction. A potentially harmful effect of sandboxes is their potential to distract regulators from providing clear guidance on more foundational enablers of innovation, such as cloud computing, data protection, tiered know-your-customer (KYC) requirements, and open data access, which could benefit more firms overall. 6 ADDRESS REGULATORY BARRIERS TO INNOVATION Alternatively, the regulator may believe that a sandbox will help lower regulatory barriers to beneficial innovation. This may be the most compelling rationale for a regula- tory sandbox and the most critical for understanding the comparative advantage of sandboxes over other regulatory tools. The perceived regulatory barriers to innovation can be varied, but they often include (i) compliance cost, (ii) limited understanding of rules and regulations, (iii) rules or regulations that prohibit an innovation, and (iv) regula- tory uncertainty. At the outset it may be difficult to determine which of these barriers is most pressing. Therefore, the regulator may use an innovation facilitator (other than a sandbox) to gather additional data from the marketplace to determine whether a sandbox is needed. These facilitatorswhether dedicated fintech offices, innovation hubs, hotlines, or similar initia- tivescan be reliable substitutes for sandboxes by providing a venue for both regulators and innovators to understand how a new technology might fit within a regulatory regime. If the regulator ultimately decides to launch a sandbox, the facilitators can provide an important venue for vetting potential sandbox applications. Many jurisdictions with sandboxes have adopted complementary innovation facilita- tors to provide informal regulatory engagement channels for market participants (Jenk and Appaya 2019a). 6 See Jenk and Lauer (2017, 10): “Regulators will remain responsible for supporting the creation of an enabling environment for digital financial ser- vices based on basic regulatory enablers. A regulatory sandbox should not be a distraction for policy makers.” LEARN ABOUT DEVELOPMENTS IN THE MARKETPLACE A CGAP and World Bank survey of financial regulators showed that 81 percent of respondents said the primary reason for launching an innovation facilitator (including a sandbox) was to learn more about emerging innovations in the marketplace (Jenk and Appaya 2019a). Although this may be a valuable policy objective and help inform the regulators viewpoint on the risks and benefits of emerging technologies and business models, this type of general learning is better accomplished through a less formal inno- vation facilitator rather than a sandbox. The regulator can craft a more nimble and broad-reaching learning mecha- nism outside of the hypothesis-driven, resource-intensive confines of a sandbox. If insights from informal interac- tions with market participants ultimately point to using a sandbox, the regulator can proceed with a clear under- standing of market demand and potential use