ESA POSITION STATEMENT: Position Statement on Scientific Peer Review

Peer review is an integral component of scientific research and publishing. It allows the scientific community to maintain quality control of research through the review of research proposals, journal manuscripts and other reports. Academic peer review, although far from perfect, is the best tool scientists have to ensure high standards for their professional work.

This idea has been translated into the policy arena through ‘scientific peer review’ – the review, by scientific experts, of in-house agency science or the body of science underlying management decisions. These types of reviews are critically important tools for policy makers. They allow experts from both inside and outside the federal government to provide technical advice and analysis, increasing public confidence in federal science, and ensuring that the best quality information is used in decision making.

However, it is critical that scientific peer review programs be carefully designed to maintain objectivity, quality and thoroughness. While scientific peer review is an important tool for decision makers, a poorly designed process can do more harm than good. It is for this reason that we endorse the following list of important considerations for government scientific peer review of agency-produced science and the body of science underlying management decisions.

• The first priority in choosing reviewers should be to engage the most competent scientists.Therefore, conflict of interest exclusions must be carefully designed to balance barring those with a direct conflict of interest and the reality of a finite pool of suitable reviewers. The key issue in selecting reviewers is whether they bring the necessary scientific knowledge and objectivity to reviewing the matter at hand.

  Scientific peer review should be insulated from politics as much as possible. Oversight of scientific peer review should be vested in scientists and science managers within the agencies. This adds assurance that the composition of panels is not being unduly influenced by politics and constitutes a representative subset of the scientists most competent to review and assess the topic. The agencies must be trusted to perform the task of constituting and overseeing fair and independent scientific peer review efforts, without interference from political entities.

• Even the best scientific peer review cannot give policy makers the ‘right’ answer. Scientific peer review can provide assurances that rigorous, transparent and respected methods were followed, that the data were reasonably interpreted, and that the stated conclusions logically follow from the results. However, often more than one interpretation of the data set can be made, and there may be no way to determine which interpretation is ‘best’. Where data are limited or other uncertainties abound, scientific peer review can point these problems out, but it cannot overcome them.
• Scientific peer review must maintain programmatic flexibility. While guidelines can help to ensure that certain standards are met and maintained, an overly rigid process, particularly for scientific peer review of the body of science underlying policy decisions, will result in inefficient use of time and resources. It may be overly prescriptive to stipulate the number of reviewers, the questions they must answer, or the type of report they must produce for the broad range of agency scientific work.
• All scientific peer review must be based upon an assumption of integrity. While commonsense measures can be taken to weed out direct conflicts of interest, an implementable system can never be fully cleared of all potential conflicts of interest. Instead, fair reviews are the product of professional standards of conduct that are a fundamental component of training in scientific research. Scientific peer review must ultimately rest on the presumed integrity of the reviewers.
• Efforts to revise the process of peer review should acknowledge the differences in professional culture that often divide scientists, policy makers, and the public. The academic model of peer review calls on reviewers to be as critical as possible. This is done so that authors are able to make improvements where they can and so that the weaknesses of the work are understood and acknowledged. Thus, results from scientific peer review that highlight uncertainties, questions and alternative explanations do not mean that the science was not well done or that its findings are invalid. Science is inherently uncertain and there will always be unanswered questions and areas where more research is needed. However, acknowledging uncertainty should not be equated with an inability to draw conclusions; managers often must act without complete certainty. Scientific peer review, properly carried out by competent peer scientists, can reassure managers, decision makers, and the public that such difficult decisions are based on research that represents the current state of our scientific understanding.