References

Abadie, Alberto, Susan Athey, Guido W. Imbens, and Jeffrey Wooldridge. 2017. “When Should You Adjust Standard Errors for Clustering?” NBER Working Paper 24003.
Abell, Peter, and Ofer Engel. 2019. “Subjective Causality and Counterfactuals in the Social Sciences: Toward an Ethnographic Causality?” Sociological Methods & Research, 0049124119852373.
Alrababa’h, Ala’, Scott Williamson, Andrea Dillon, Jens Hainmueller, Dominik Hangartner, Michael Hotard, David Laitin, Duncan Lawrence, and Jeremy Weinstein. 2020. “Learning from Null Effects: A Bottom-up Approach.” SocArXiv. https://doi.org/10.31235/osf.io/5ebpy.
Alvarez, R. Michael, Ellen M. Key, and Lucas Núñez. 2018. Research Replication: Practical Considerations.” PS: Political Science & Politics 51 (2): 422–26.
Angrist, Joshua D., and Jörn-Steffen Pischke. 2008. Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton: Princeton University Press.
Aronow, Peter M., Jonathon Baron, and Lauren Pinson. 2019. “A Note on Dropping Experimental Subjects Who Fail a Manipulation Check.” Political Analysis 27 (4): 572–89.
Aronow, Peter M., Donald P. Green, and Donald K. K. Lee. 2014. Sharp Bounds On The Variance In Randomized Experiments.” The Annals of Statistics 42 (3): 850–71.
Aronow, Peter M., and Benjamin T. Miller. 2019. Foundations of Agnostic Statistics. Cambridge University Press.
Aronow, Peter M., and Cyrus Samii. 2017. “Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment.” The Annals of Applied Statistics 11 (4): 1912–47.
Baird, Sarah, J. Aislinn Bohren, Craig McIntosh, and Berk Ozler. 2018. Optimal Design of Experiments in the Presence of Interference.” Review of Economics & Statistics 5 (100): 844–60.
Banerjee, Abhijit, Esther Duflo, Amy Finkelstein, Lawrence F Katz, Benjamin A Olken, and Anja Sautmann. 2020. “In Praise of Moderation: Suggestions for the Scope and Use of Pre-Analysis Plans for RCTs in Economics.” Working Paper 26993. Working Paper Series. National Bureau of Economic Research. https://doi.org/10.3386/w26993.
Bansak, Kirk, Jens Hainmueller, Daniel J. Hopkins, and Teppei Yamamoto. 2019. “Beyond the Breaking Point? Survey Satisficing in Conjoint Experiments.” Political Science Research and Methods, 1–19. https://doi.org/10.1017/psrm.2019.13.
Baron, Hannah, and Lauren Young. 2020. “From Principles to Practice: Methods for Increasing the Transparency of Research Ethics in Violent Contexts.”
Baumgartner, Michael, and Alrik Thiem. 2017. “Often Trusted but Never (Properly) Tested: Evaluating Qualitative Comparative Analysis.” Sociological Methods & Research.
Benjamin, Daniel J., James O. Berger, Magnus Johannesson, Brian A. Nosek, E. -J. Wagenmakers, Richard Berk, Kenneth A. Bollen, et al. 2018. “Redefine Statistical Significance.” Nature Human Behaviour 2 (1): 6–10.
Bennett, Andrew. 2015. “Appendix.” In Process Tracing: From Metaphor to Analytic Tool, edited by Andrew Bennett and Jeffrey Checkel. New York: Cambridge University Press.
Bennett, Andrew, and Jeffrey Checkel. 2015. “Process Tracing: From Philosophical Roots to Best Practices.” In Process Tracing: From Metaphor to Analytic Tool, edited by Andrew Bennett and Jeffrey Checkel, 3–37. New York: Cambridge University Press.
Blair, Graeme, Jasper Cooper, Alexander Coppock, and Macartan Humphreys. 2020. “Declaring and Diagnosing Research Designs.” American Political Science Review.
Blair, Graeme, Alexander Coppock, and Margaret Moor. 2020. “When to Worry about Sensitivity Bias: A Social Reference Theory and Evidence from 30 Years of List Experiments.” American Political Science Review 114 (4): 1297–1315.
Blair, Graeme, Kosuke Imai, and Yang-Yang Zhou. 2015. “Design and Analysis of the Randomized Response Technique.” Journal of the American Statistical Association 110 (511): 1304–19.
Bonilla, Tabitha, and Alvin B. Tillery. 2020. “Which Identity Frames Boost Support for and Mobilization in the #BlackLivesMatter Movement? An Experimental Test.” American Political Science Review 114 (4): 947–62. https://doi.org/10.1017/S0003055420000544.
Bowers, Jake. 2011. Six steps to a better relationship with your future self.” https://cpb-us-e1.wpmucdn.com/blogs.rice.edu/dist/d/2418/files/2013/09/tpm{\_}v18{\_}n2.pdf.
Brady, Henry E. 2004. “Data-Set Observations Versus Causal-Process Observations: The 2000 US Presidential Election.” Rethinking Social Inquiry: Diverse Tools, Shared Standards, 267–72.
Butler, Daniel M., and Charles Crabtree. 2017. “Moving Beyond Measurement: Adapting Audit Studies to Test Bias-Reducing Interventions.” Journal of Experimental Political Science 4 (1): 57–67.
Calonico, Sebastian, Matias D Cattaneo, and Rocio Titiunik. 2014. “Robust Nonparametric Confidence Intervals for Regression-Discontinuity Designs.” Econometrica 82 (6): 2295–2326.
Casey, Katherine, Rachel Glennerster, and Edward Miguel. 2012. “Reshaping Institutions: Evidence on Aid Impacts Using a Pre-Analysis Plan.” The Quarterly Journal of Economics 127 (4): 1755–1812.
Christensen, Garret, and Edward Miguel. 2018. “Transparency, Reproducibility, and the Credibility of Economics Research.” Journal of Economic Literature 56 (3): 920–80. https://doi.org/10.1257/jel.20171350.
Clingingsmith, David, Asim Ijaz Khwaja, and Michael Kremer. 2009. “Estimating the Impact of the Hajj: Religion and Tolerance in Islam’s Global Gathering.” The Quarterly Journal of Economics 124 (3): 1133–70.
Coffman, Lucas C., and Muriel Niederle. 2015. “Pre-Analysis Plans Have Limited Upside, Especially Where Replications Are Feasible.” The Journal of Economic Perspectives 29 (3): 81–97.
Collier, David, David A Freedman, James D Fearon, David D Laitin, John Gerring, and Gary Goertz. 2008. “Symposium: Case Selection, Case Studies, and Causal Inference.” Qualitative & Multimethod Research 6 (2).
Coppock, Alexander. 2019. Avoiding Post-Treatment Bias in Audit Experiments.” Journal of Experimental Political Science 6 (1): 1–4.
Coppock, Alexander, and Dipin Kaur. 2021. Qualitative Imputation of Missing Potential Outcomes.” American Journal of Political Science.
Creighton, Mathew J., and Amaney Jamal. 2015. “Does Islam Play a Role in Anti-Immigrant Sentiment? An Experimental Approach.” Social Science Research 53: 89–103. https://doi.org/https://doi.org/10.1016/j.ssresearch.2015.04.001.
Dawid, A. Philip. 2000. “Causal Inference Without Counterfactuals.” Journal of the American Statistical Association 95 (450): 407–24.
De Chaisemartin, Clément, and Xavier d’Haultfoeuille. 2020. “Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects.” American Economic Review 110 (9): 2964–96.
Deaton, Angus S. 2010. “Instruments, Randomization, and Learning about Development.” Journal of Economic Literature 48 (2): 424–55.
Druckman, James N, and Cindy D Kam. 2011. “Students as Experimental Participants.” Cambridge Handbook of Experimental Political Science 1: 41–57.
Dunning, Thad. 2012. Natural Experiments in the Social Sciences: A Design-Based Approach. Cambridge: Cambridge University Press.
Egami, Naoki, and Erin Hartman. 2021. “Elements of External Validity: Framework, Design, and Analysis.” Unpublished Manuscript.
Elman, Colin, Diana Kapiszewski, and Arthur Lupia. 2018. Transparent Social Inquiry: Implications for Political Science.” Annual Review of Political Science 21 (1): 29–47. https://doi.org/10.1146/annurev-polisci-091515-025429.
Fairfield, Tasha, and Andrew E Charman. 2017. “Explicit Bayesian Analysis for Process Tracing: Guidelines, Opportunities, and Caveats.” Political Analysis 25 (3): 363–80.
Fang, Albert H, Andrew M Guess, and Macartan Humphreys. 2019. “Can the Government Deter Discrimination? Evidence from a Randomized Intervention in New York City.” The Journal of Politics 81 (1): 127–41.
Fearon, James D, and David D Laitin. 2008. “Integrating Qualitative and Quantitative Methods.” In The Oxford Handbook of Political Science.
Fenno, Richard F. 1978. Home Style: House Members in Their Districts. Pearson College Division.
Findley, Michael G., Nathan M. Jensen, Edmund J. Malesky, and Thomas B. Pepinsky. 2016. “Can Results-Free Review Reduce Publication Bias? The Results and Implications of a Pilot Study.” Comparative Political Studies 49 (13): 1667–1703.
Fisher, Ronald A. 1935. “The Logic of Inductive Inference.” Journal of the Royal Statistical Society 98 (1): 39–82.
Fisher, Ronald Aylmer. 1937. The Design of Experiments. Oliver And Boyd; Edinburgh; London.
Frangakis, Constantine E, and Donald B Rubin. 2002. “Principal Stratification in Causal Inference.” Biometrics 58 (1): 21–29.
Freedman, David A. 2008. “On Regression Adjustments to Experimental Data.” Advances in Applied Mathematics 40 (2): 180–93.
Geddes, Barbara. 2003. Paradigms and Sand Castles: Theory Building and Research Design in Comparative Politics. Ann Arbor, Michigan: University of Michigan Press.
Gelman, Andrew, and John Carlin. 2014. “Beyond Power Calculations Assessing Type s (Sign) and Type m (Magnitude) Errors.” Perspectives on Psychological Science 9 (6): 641–51.
Gelman, Andrew, Jennifer Hill, and Aki Vehtari. 2020. Regression and Other Stories. Cambridge University Press.
Gelman, Andrew, and Guido Imbens. 2017. “Why High-Order Polynomials Should Not Be Used in Regression Discontinuity Designs.” Journal of Business & Economic Statistics, no. Forthcoming.
Gerber, Alan S., and Donald P. Green. 2012. Field Experiments: Design, Analysis, and Interpretation. New York: W.W. Norton.
Gerring, John, and Lee Cojocaru. 2016. “Selecting Cases for Intensive Analysis: A Diversity of Goals and Methods.” Sociological Methods & Research 45 (3): 392–423.
Goertz, Gary. 2008. “Choosing Cases for Case Studies: A Qualitative Logic.” Newsletter of the APSA Section on Qualitative & Multi-Method Research 6 (2): 11–14.
Goertz, Gary, and James Mahoney. 2012. A Tale of Two Cultures: Qualitative and Quantitative Research in the Social Sciences. Princeton: Princeton University Press.
Green, Donald P., and Winston Lin. 2016. “Standard Operating Procedures: A Safety Net for Pre-Analysis Plans.” PS: Political Science & Politics 49 (3): 495–99.
Green, Donald P., and Andrej Tusicisny. 2012. “Statistical Analysis of Results from Laboratory Studies in Experimental Economics: A Critique of Current Practice.” Available at SSRN 2181654.
Gulzar, Saad, and Muhammad Yasir Khan. 2021. ‘Good Politicians:’ Experimental Evidence on Motivations for Political Candidacy and Government Performance.”
Hainmueller, Jens, Daniel J. Hopkins, and Teppei Yamamoto. 2014. Causal Inference in Conjoint Analysis: Understanding Multidimensional Choices via Stated Preference Experiments.” Political Analysis 22 (1): 1–30.
Halpern, Joseph Y. 2000. “Axiomatizing Causal Reasoning.” Journal of Artificial Intelligence Research 12: 317–37.
Hartman, Erin, and F. Daniel Hidalgo. 2018. “An Equivalence Approach to Balance and Placebo Tests.” American Journal of Political Science 62 (4): 1000–1013.
Hedayat, A. S., Hansheng Cheng, and Jennifer Pajda-De La O. 2019. “Existence of Unbiased Estimation for the Minimum, Maximum, and Median in Finite Population Sampling.” Statistics & Probability Letters 153: 192–95.
Herron, Michael C., and Kevin M. Quinn. 2016. “A Careful Look at Modern Case Selection Methods.” Sociological Methods & Research 45 (3): 458–92.
Humphreys, Macartan. 2015. “Reflections on the Ethics of Social Experimentation.” Journal of Globalization and Development 6 (1): 87–112.
Humphreys, Macartan, and Alan Jacobs. 2017. “Qualitative Inference from Causal Models.” Draft Manuscript (Version 0.2). Retrieved November 27: 2017.
Humphreys, Macartan, and Alan M. Jacobs. 2015. “Mixing Methods: A Bayesian Approach.” American Political Science Review 109 (4): 653–73.
Humphreys, Macartan, Raul de la Sierra, and Peter van der Windt. 2013. Fishing, Commitment, and Communication: A Proposal for Comprehensive Nonbinding Research Registration.” Political Analysis 21 (1): 1–20.
Imai, Kosuke. 2011. Multivariate Regression Analysis for the Item Count Technique.” Journal of the American Statistical Association 106 (494): 407–16.
Imai, Kosuke, Gary King, and Clayton Nall. 2009. “The Essential Role of Pair Matching in Cluster-Randomized Experiments, with Application to the Mexican Universal Health Insurance Evaluation.” Statistical Science 24 (1): 29–53.
Imai, Kosuke, Gary King, and Elizabeth A. Stuart. 2008. “Misunderstandings Between Experimentalists and Observationalists about Causal Inference.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 171 (2): 481–502.
Imbens, Guido W. 2010. “Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009).” Journal of Economic Literature 48 (2): 399–423.
Imbens, Guido W., and Donald B. Rubin. 2015. Causal Inference in Statistics, Social, and Biomedical Sciences. Cambridge: Cambridge University Press.
Jamison, Julian C. 2019. “The Entry of Randomized Assignment into the Social Sciences.” Journal of Causal Inference 7 (1): 1–16.
Johnson, Noel D, and Alexandra A Mislin. 2011. “Trust Games: A Meta-Analysis.” Journal of Economic Psychology 32 (5): 865–89.
King, Gary, Emmanuela Gakidou, Nirmala Ravishankar, Ryan T Moore, Jason Lakin, Manett Vargas, Martha Marı́a Téllez-Rojo, Juan Eugenio Hernández Ávila, Mauricio Hernández Ávila, and Héctor Hernández Llamas. 2007. “A ‘Politically Robust’ Experimental Design for Public Policy Evaluation, with Application to the Mexican Universal Health Insurance Program.” Journal of Policy Analysis and Management 26 (3): 479–506.
King, Gary, and Melissa Sands. 2015. “How Human Subjects Research Rules Mislead You and Your University, and What to Do about It.” Unpublished Manuscript.
Kirkland, Patricia A., and Alexander Coppock. 2018. Candidate Choice Without Party Labels: New Insights from Conjoint Survey Experiments.” Political Behavior 40 (3): 571–91.
Kling, Jeffrey R, Jeffrey B Liebman, and Lawrence F Katz. 2007. “Experimental Analysis of Neighborhood Effects.” Econometrica 75 (1): 83–119.
Levy, Jack S. 2008. “Case Studies: Types, Designs, and Logics of Inference.” Conflict Management and Peace Science 25 (1): 1–18.
Lieberman, Evan S. 2005. “Nested Analysis as a Mixed-Method Strategy for Comparative Research.” American Political Science Review 99 (3): 435–52.
Lin, Winston. 2013. “Agnostic Notes on Regression Adjustments to Experimental Data: Reexamining Freedman’s Critique.” Annals of Applied Statistics 7 (1): 295–318.
Lyall, Jason. 2020. “Preregister Your Ethical Redlines.”
Martin, Lisa. 1992. Coercive Cooperation: Explaining Multilateral Economic Sanctions. Princeton: Princeton University Press.
McPhetres, Jonathon, Nihan Albayrak-Aydemir, Ana Barbosa Mendes, Elvina C. Chow, Patricio Gonzalez-Marquez, Erin Loukras, Annika Maus, et al. 2020. “A Decade of Theory as Reflected in Psychological Science (2009-2019).”
Mellon, Jonathan. 2021. “Rain, Rain, Go Away: 176 Potential Exclusion-Restriction Violations for Studies Using Weather as an Instrumental Variable.” SocArXiv. https://doi.org/10.31235/osf.io/9qj4f.
Middleton, Joel A. 2008. “Bias of the Regression Estimator for Experiments Using Clustered Random Assignment.” Statistics & Probability Letters 78 (16): 2654–59.
Miguel, Edward, Colin Camerer, Katherine Casey, Joshua Cohen, Kevin M Esterling, Alan S. Gerber, Rachel Glennerster, et al. 2014. “Promoting Transparency in Social Science Research.” Science 343 (6166): 30.
Mill, John Stuart. 1884. A System of Logic, Ratiocinative and Inductive: Being a Connected View of the Principles of Evidence and the Methods of Scientific Investigation. Harper.
Miller, Judith Droitcour. 1984. “A New Survey Technique for Studying Deviant Behavior.” PhD thesis, George Washington University.
Montgomery, Jacob M., and Erin L. Rossiter. 2020. “So Many Questions, so Little Time: Integrating Adaptive Inventories into Public Opinion Research.” Journal of Survey Statistics and Methodology 8 (4): 667–90.
Morris, Tim P., Ian R. White, and Michael J. Crowther. 2019. “Using Simulation Studies to Evaluate Statistical Methods.” Statistics in Medicine.
Mousa, Salma. 2020. “Building Social Cohesion Between Christians and Muslims Through Soccer in Post-ISIS Iraq.” Science 369 (6505): 866–70.
Nosek, Brian A., George Alter, George C. Banks, Denny Borsboom, Sara D. Bowman, Steven J. Breckler, Stuart Buck, et al. 2015. “Promoting an Open Research Culture: Author Guidelines for Journals Could Help to Promote Transparency, Openness, and Reproducibility.” Science 348 (6242): 1422.
Offer-Westort, Molly, Alexander Coppock, and Donald P. Green. 2021. Adaptive Experimental Design: Prospects and Applications in Political Science.” American Journal of Political Science.
Ofosu, George K., and Daniel N. Posner. 2020. “Do Pre-Analysis Plans Hamper Publication?” AEA Papers and Proceedings 110 (May): 70–74.
Ofosu, George, and Daniel Posner. 2021. Pre-analysis Plans: A Stocktaking.” Perspectives on Politics.
Olken, Benjamin A. 2015. Promises and Perils of Pre-Analysis Plans.” Journal of Economic Perspectives 29 (3): 61–80.
Pearl, Judea. 1999. “Probabilities of Causation: Three Counterfactual Interpretations and Their Identification.” Synthese 121 (1-2): 93–149.
———. 2009. Causality: Models, Reasoning and Inference. Second Edition. Cambridge: Cambridge University Press.
Pearl, Judea, and Dana Mackenzie. 2018. The Book of Why: The New Science of Cause and Effect. Basic Books.
Peer, Limor, Lilla Orr, and Alexander Coppock. 2021. Active Maintenance: A Proposal for the Long-term Computational Reproducibility of Scientific Results.” PS: Political Science & Politics.
Plümper, Thomas, Vera E. Troeger, and Eric Neumayer. 2010. “Case Selection and Causal Inference in Qualitative Research.” British Journal of Political Science.
Porter, Ethan, and Yamil Velez. 2021. “Placebo Selection in Survey Experiments: An Agnostic Approach.” Policy and Society.
Reiss, Peter C, and Frank A Wolak. 2007. “Structural Econometric Modeling: Rationales and Examples from Industrial Organization.” Handbook of Econometrics 6: 4277–4415.
Rennie, Drummond. 2004. Trial registration.” JAMA: The Journal of the American Medical Association 292 (11): 1359–62.
Rohlfing, Ingo. 2018. “Power and False Negatives in Qualitative Comparative Analysis: Foundations, Simulation and Estimation for Empirical Studies.” Political Analysis 26 (1): 72–89.
Rubin, Donald B. 1980. “Randomization Analysis of Experimental Data: The Fisher Randomization Test Comment.” Journal of the American Statistical Association 75 (371): 591–93.
———. 1984. Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician.” The Annals of Statistics 12 (4): 1151–72.
Rubinstein, Ariel. 1982. “Perfect Equilibrium in a Bargaining Model.” Econometrica: Journal of the Econometric Society, 97–109.
Samii, Cyrus. 2016. “Causal Empiricism in Quantitative Research.” The Journal of Politics 78 (3): 941–55.
Samii, Cyrus, and Peter M. Aronow. 2012. “On Equivalencies Between Design-Based and Regression-Based Variance Estimators for Randomized Experiments.” Statistics & Probability Letters 82 (2): 365–70.
Schrag, Zachary M. 2010. Ethical Imperialism: Institutional Review Boards and the Social Sciences, 1965–2009. Johns Hopkins University Press.
Schwarz, Susanne, and Alexander Coppock. 2020. What Have We Learned About Gender From Candidate Choice Experiments? A Meta-analysis of 67 Factorial Survey Experiments.” Journal of Politics.
Seawright, Jason, and John Gerring. 2008. “Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options.” Political Research Quarterly 61 (2): 294–308.
Shadish, William, Thomas D. Cook, and Donald Thomas Campbell. 2002. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin.
Sinclair, Betsy, Margaret McConnell, and Donald P. Green. 2012. “Detecting Spillover Effects: Design and Analysis of Multilevel Experiments.” American Journal of Political Science 56 (4): 1055–69.
Skocpol, Theda. 1979. States and Social Revolutions: A Comparative Analysis of France, Russia and China. Cambridge University Press.
Slough, Tara. 2020. “The Ethics of Electoral Experimentation: Design-Based Recommendations.” Unpublished Manuscript.
Swank, Duane. 2002. Global Capital, Political Institutions, and Policy Change in Developed Welfare States. New York: Cambridge University Press.
Teele, Dawn. 2021. “Virtual Consent: A Bronze Standard for Experimental Ethics.” In Cambridge Handbook of Experimental Political Science, edited by Donald P. Green and James N. Druckman. Cambridge University Press.
Van Evera, Stephen. 1997. Guide to Methods for Students of Political Science. Ithaca: Cornell University Press.
Wager, Stefan, and Susan Athey. 2018. “Estimation and Inference of Heterogeneous Treatment Effects Using Random Forests.” Journal of the American Statistical Association 113 (523): 1228–42.
White, Ariel R., Noah L. Nathan, and Julie K. Faller. 2015. What Do I Need to Vote? Bureaucratic Discretion and Discrimination by Local Election Officials.” American Political Science Review 109 (1): 129–42.
Wilke, Anna M., Donald P. Green, and Jasper Cooper. 2020. “A Placebo Design to Detect Spillovers from an Education–Entertainment Experiment in Uganda.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 183 (3): 1075–96.
Wilke, Anna M., and Macartan Humphreys. 2020. “Field Experiments, Theory, and External Validity.” SAGE Handbook of Research Methods in Political Science; International Relations.
Wood, Elisabeth Jean. 2006. The Ethical Challenges of Field Research in Conflict Zones.” Qualitative Sociology 29 (3): 373–86.
Yamamoto, Teppei. 2012. “Understanding the Past: Statistical Analysis of Causal Attribution.” American Journal of Political Science 56 (1): 237–56.
Zarin, Deborah A., and Tony Tse. 2008. “Moving Towards Transparency of Clinical Trials.” Science 319 (5868): 1340–42.
Zhang, Junni L., and Donald B. Rubin. 2003. “Estimation of Causal Effects via Principal Stratification When Some Outcomes Are Truncated by ‘Death’.” Journal of Educational and Behavioral Statistics 28 (4): 353–68.