Adaptability and the Pivot Penalty in Science and Technology

Researchers set the direction of their work amidst an evolving landscape of questions, opportunities, and challenges, and the adaptability of researchers helps determine the collective capacity of science and society to confront emergent needs. This paper introduces a measurement framework for research “pivots,” quantifying how far scientists and inventors move from their existing research portfolios when producing new works, and investigates the impact of the research that results. We apply this framework to XXM scientific publications across XX disciplines and XXM patents across XX technology classes, and uncover a pervasive “pivot penalty,” where the impact of the new research steeply declines the further a researcher moves from their prior work. While conceptual frameworks often emphasize novel perspectives and outsider advantages in driving scientific and technological progress, we find that the pivot penalty applies nearly universally across the sciences, and in patenting, and has been growing in magnitude over the past five decades. We then apply this framework to study the COVID-19 pandemic as a high-scale case study, where scientists pivot toward an area with extremely high demand. We find that, despite scientists’ widespread engagement with COVID-19 and a clear impact premium for COVID-related research, the pivot penalty remains severe, characterized by a steep decline in high-impact research the further the scientists moved to engage the pandemic. Additional features condition pivoting, including a scientist’s career stage, prior impact, collaborative scale, the use of new coauthors, and funding, but we find that the pivot penalty persists and remains substantial regardless of these features. Overall, the findings point to large and increasing challenges in adapting given personnel to new opportunities and threats. The results have implications for individual researchers, research organizations, science policy, and the capacity of science and society as a whole to confront emergent demands.