Race to the Bottom: Competition and Quality in Science [Job Market Paper] (joint with Ryan Hill)
Abstract: This paper investigates how competition to publish first and establish priority impacts the quality of scientific research. We begin by developing a model where scientists decide whether and how long to work on a given project. When deciding how long to let their projects mature, scientists trade off the marginal benefit of higher quality research against the marginal risk of being scooped. The most important (highest potential) projects are the most competitive because they induce the most entry. Therefore, the model predicts these projects are also the most rushed and lowest quality. We test the predictions of this model in the field of structural biology using data from the Protein Data Bank (PDB), a repository for structures of large macro-molecules. An important feature of the PDB is that it assigns objective measures of scientific quality to each structure. As suggested by the model, we find that structures with higher ex-ante potential generate more competition, are completed faster, and are lower quality. Consistent with the model, and with a causal interpretation of our empirical results, these relationships are mitigated when we focus on structures deposited by scientists who — by nature of their employment position — are less focused on publication and priority.
Press coverage: MIT News
Scooped! Estimating Rewards for Priority in Science (joint with Ryan Hill)
Abstract: The scientific community assigns credit or “priority” to individuals who publish an important discovery first. We examine the impact of losing a priority race (colloquially known as getting “scooped”) on subsequent publication and career outcomes. To do so, we take advantage of data from structural biology where the nature of the scientific process together with the Protein Data Bank — a repository of standardized research discoveries — enables us to identify priority races and their outcomes. We find that race winners receive more attention than losers, but that these contests are not winner-take-all. Scooped teams are 2.5 percent less likely to publish, are 18 percent less likely to appear in a top-10 journal, and receive 20 percent fewer citations. As a share of total citations, we estimate that scooped papers receive a credit share of 45 percent. This is larger than the theoretical benchmark of zero percent suggested by classic models of innovation races. We conduct a survey of structural biologists which suggests that active scientists are more pessimistic about the cost of getting scooped than can be justified by the data. Much of the citation effect can be explained by journal placement, suggesting editors and reviewers are key arbiters of academic priority. Getting scooped has only modest effects on academic careers. Finally, we present a simple model of statistical discrimination in academic attention to explain how the priority reward system reinforces inequality in science, and document empirical evidence consistent with our model. On the whole, these estimates inform both theoretical models of innovation races and suggest opportunities to re-evaluate the policies and institutions that affect credit allocation in science.
Press coverage: Nature News, MIT News
Research in Progress
Big Money, Small Science? Soft Money and Risk Taking in the Academic Life Sciences (joint with Joey Anderson)
Abstract: Tenure in the academic life sciences is often not accompanied by a guaranteed salary. Rather, salaries are comprised of two components: “hard money,” which is paid by the university, and “soft money” which is funded by extramural grants and is therefore contingent on successful grant applications. Scientists have expressed concern that an increasing reliance on soft money discourages scientists from pursuing risky but potentially transformative work. This project seeks to empirically test these claims. Data on the hard versus soft money composition of salaries has been notoriously difficult to find. We use detailed grant- and transaction-level data from UMETRICS, a consortium of 31 U.S. research universities, in order to reconstruct the hard/soft salary compositions for NIH principle investigators. We find that principle investigators draw about 40 percent of their salary on average from grants, although there is substantial variation across universities and departments. Assistant professors are more dependent on soft money (45 percent) than full professors (39 percent). We plan to test whether scientists who are paid more in soft money pursue less risky projects, using a variety of measures to proxy for project risk.
The Great Pivot: How Science has Shifted in Response to the COVID-19 Pandemic (joint with Ben Jones, Ryan Hill, Dashun Wang, and Yian Yin)
Abstract: How have scientists responded to the COVID-19 pandemic? In this paper we aim to answer three related questions: who pivoted to work on COVID-19 topics, how large were these pivots, and how impactful has this work been? We use a database of scientific publications to identify COVID-related papers and their authors. We compare the reference lists of these papers to references in prior work to assess how “close” or “distant” these COVID papers are in idea space from the authors’ typical work. Early results suggest that senior researchers working in large teams are the most likely to pivot, and that their COVID research represents a significant departure from their prior body of work. Moreover, larger pivots are associated with lower-impact research, highlighting the importance of having researchers already working in adjacent areas of science. This underscores the idea of diverse basic scientific research as a form of insurance against catastrophes.
Heat Check: New Evidence of the Hot Hand in Basketball (joint with Andrew Bocskoscky and John Ezekowitz)
Much of the literature on the hot hand fallacy in basketball rests on the assumption that shot selection is independent of past performance. In this paper, we challenge that assumption using a novel dataset of over 83,000 shots from the 2012-2013 NBA season, combined with detailed optical tracking data from SportVU. We use these data to show that shot selection does depend on past performance: players who have exceeded their expected shooting percentage over recent shots shoot from significantly further away, face tighter defense, are more likely to take their team’s next shot, and attempt overall more difficult shots. Once we attempt to control for shot difficulty, a small but statistically significant hot hand effect emerges.
Press coverage: WSJ, Boston Globe, Grantland, WBUR Only A Game