The role of pilot studies in financial regulation

Financial regulators considering the desirability of a new rule or regulation sometimes use pilot studies as a tool for evidence-based decision-making. Although pilot studies can generate new knowledge, they also can be expensive and potentially subject to serious selection biases, spillover problems, and the infeasibility of setting up a blind design. Alternatively, regulators often can effectively evaluate a proposed regulation’s potential impact by analyzing archival data or applying theory based on well-accepted economic principles. We discuss why pilot studies can be useful, but also why regulators and industry participants sometimes favor pilot studies with little scientific value. We illustrate these issues with a discussion of various SEC pilot studies.