Structured Ethics Appendix

Social science researchers engaged in primary data collection often consider a range of ethical issues during the planning of their research, but such considerations are rarely articulated upfront or in subsequent articles generated from the research. We believe that building explicit steps for this can lead to better research, better communications about research, and thus better impact of research as well. To facilitate this, we propose authors include a structured ethics appendix in working papers and published online appendices, and we provide a framework below for guidance.

We consider this a living document, and plan to update it periodically upon receiving feedback and observing usage. Please email us at dean.karlan@gmail.com and christopher.udry@northwestern.edu if you have any suggestions. If you have included such an appendix in a working or published paper, please let us know so that we can observe usage as part of our effort to maintain, update and improve these guidelines.

Structured Ethics Appendix Questions: Brief Explanations

For a more complete discussion of each item, see Asiedu et al. (2021).

1. Policy Equipoise

Is there policy equipoise? That is, is there uncertainty regarding participants’ net benefits from each arm of the study relative to the other arms and to the best possible policy to which participants could have access? If not, ethical randomization requires two conditions related to scarcity: (1) Was there scarcity, i.e., did the inclusion of multiple arms change the expected aggregate value of the programs delivered? (2) Do all ex-ante identifiable participants have equal moral or legal claims to the scarce programs?

If there is no reasonable expectation that one arm of the study produces more benefits to participants than any other arm or than the best possible alternative policy, then randomization is ethically unproblematic. If not, then excluding some participants from the superior treatment arm can only be justified by scarcity. Scarcity conditions are two-fold: (1) resources are not sufficient, given constraints, to include all participants in the superior treatment arm; (2) no ex-ante identifiable participants are excluded from the superior arm and have a greater claim to those resources than any participant assigned to the superior arm. See MacKay 2018, 2020 for more complete discussions of policy equipoise.

2. Role of researchers with respect to implementation

Are researchers “active” researchers, i.e. did the researchers have direct decision making power over whether and how to implement the program? If YES, what was the disclosure to participants and informed consent process for participation in the program? Providing IRB approval details may be sufficient but further clarification of any important issues should be discussed here. If NO, i.e., implementation was separate, explain the separation.

A researcher should be considered “active” if, for example, the implementing staff are employed by an institution at which the PI is employed, and the staff report either directly or indirectly to the PI at this institution with regard to this project. Or if researchers control funding for implementation, or have direct decision-making power over key implementation decisions.

Some key factors that help illuminate whether the researchers are “active” or not (here “researchers” are defined as the PIs and the staff that report directly or indirectly to the PIs): Did researchers directly provide any of the interventions, or parts thereof, to participants? Did researchers interact directly with participants and implicitly endorse one or more of the interventions?

3. Potential harms to participants or nonparticipants from the interventions or policies

Does the intervention, policy or product being studied pose potential harm to participants or non-participants? Related, are participants or likely affected non-participants particularly vulnerable? Also related, are participants’ access to future services or policies changed because of participation in the study? If yes to any of the above, what is being done to mitigate such risks?

It may be important to consider whether the researchers are “active” (see above) or not for this discussion. If the researchers are “active”, then they are responsible for the potential harms, and thus a robust discussion is appropriate. If the researchers are not “active”, then while they may not be responsible for potential harms, a discussion of this would be appropriate here.

There will almost always be some potential harms, if nothing else because of complementary investments such as time that participants in an intervention necessarily redirect from one activity to another. Quantifying these risks and complementary investments may be difficult ex-ante, but a discussion of what they are here would help the reader assess their likely importance relative to the potential benefits of the tested intervention. Also note that measuring any harms ex-post may be the exact reason for the study, particularly when the intervention is common.

If risks to nonparticipants exist, discuss the mechanisms through which the risk arises from the study and provide an estimate of the magnitude of the risk and the probability of harm.

4. Potential harms to research participants or research staff from data collection (e.g., surveying, privacy, data management) or research protocols (e.g., random assignment)

Are data collection and/or research procedures adherent to privacy, confidentiality, risk-management, and informed consent protocols with regard to human subjects? Are they respectful of community norms, e.g., community consent not merely individual consent, when appropriate? Are there potential harms to research staff from conducting the data collection that are beyond “normal” risks?

Example of sub-questions to consider as part of the broad question: Are there any risks that could ensue because of the data collection process or storage, e.g. discomfort to being asked certain questions or breach of confidentiality? If so, what are the mitigation strategies? Are there costs to the participant for the data collection process, such as their time, and if so, what is the strategy or rationale for offsetting this cost?

Because these are all issues covered by most IRB processes, a sufficient explanation for a “yes” response may be to provide the IRB approval numbers for all IRBs that have approved the project. However, if there are particular issues that are important to discuss, please do so here.

Harms to research staff could include, e.g., exposure to political violence, exposure to unusual levels of a communicable disease, mistrust due to lack of perceived lack of community consent, or emotional wellbeing from surveying about difficult subject matters. This would not include, e.g., traffic accidents.

5. Financial and reputational conflicts of interest

Do any of the researchers have financial conflicts of interest with regard to the results of the research? Do any of the researchers have potential reputational conflicts of interest?

We define financial conflicts of interest as that used by the researcher’s institutional (e.g., their university) guidelines. We define a reputational conflict of interest as one in which prior writing or advocacy could be contradicted by specific results pursued in this study, and such contradiction would pose reputational risks to the author.

6. Intellectual freedom

Were there any contractual limitations on the ability of the researchers to report the results of the study? If so, what were those restrictions, and who were they from?

This could include, for example, approval of release of the paper and restrictions on data release, but does not include things such as a “comment period” during which interested parties have a right to review and provide comments prior to release but not to control the outputs of the study.

7. Feedback to participants or communities

Is there a plan for providing feedback on research results to participants or communities? If yes, what is the plan? If not, why not?

Engaging in post-study feedback is a way of acknowledging the agency of participants and communities, and is thus a desired practice. However, it may be impractical due to costs, timing, challenges communicating the results, or potential harms if such communication may itself change behavior in undesirable ways.

8. Foreseeable misuse of research results

Is there a foreseeable and plausible risk that the results of the research will be misused and/or deliberately misinterpreted by interested parties to the detriment of other interested parties? If yes, please explain any efforts to mitigate such risk.

In settings with strong imbalances of power between interested parties, there may be foreseeable risks that a powerful party could use deliberately selected research findings to their advantage and to the harm of participants or non-participants, including for general public policy. For example, if the research might reveal the vulnerability of some that can be exploited for the gain of the more powerful party, what steps does the researcher plan to mitigate this risk?

9. Other Ethics Issues to Discuss

Are there any other issues to discuss?

Samples

Sample 1: Tying Odysseus to the Mast

Sample 2: A Better Vision for Development

Sample 3: Randomizing Religion

Sample 4: Agricultural Decisions after Relaxing Credit and Risk Constraints

Archive of Prior Versions

Version 1.0, November 2020:

Link to doc
Link to blog post

Version 2.0, January 2021:

Link to doc
Link to paper posted contemporaneously with doc

Feedback

Please email us if you have any suggestions. If you have included such an appendix in a working or published paper, please let us know so that we can observe usage as part of our effort to maintain, update and improve these guidelines.

Email Dean Karlan and Chris Udry