Research Program Peer Review: Purposes, Principles, Practices, Protocols

Abstract

The purposes, principles, practices, and protocols of research program peer review are described. While the principles are fundamentally generic, and apply to peer review across the full spectrum of performing institutions as well as manuscript, proposal, and program peer review, the focus of this report is peer review of proposed and ongoing research programs in federal agencies. Following the self-contained Executive Summary of factors for high-quality peer reviews, the report addresses potential implications of the implementation of the Government Performance and Results Act of 1993 on federal agency research program peer review practices. Then, the report describes strengths and weaknesses of major peer review components and issues, including the following: Objectives and Purposes of Peer Review; Quality of Peer Review; Impact of Peer Review Manager on Quality; Selection of Peer Reviewers; Selection of Evaluation Criteria; Secrecy (Reviewer and Performer Anonymity); Objectivity, Bias, and Fairness of the Peer Review; Normalization of Peer Review Panels; Repeatability and Reliability of the Peer Review; Effectiveness and Predictability of Peer Review; Global Data Awareness; Costs of Performing a Peer Review; Ethical Issues in Peer Review; and Alternatives to Peer Review. The report then presents different federal agency peer review practices as well as sample protocols and processes for conducting a successful research program peer review. Some peer review variants, such as the Science Court and Network-Centric Peer Review, are described, and research requirements to improve peer review are discussed. The final section is an extensive bibliography of over 3,000 references that includes text references and related references for further reading. (49 refs.)

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jun 01, 2004
Accession Number
ADA424141

Entities

People

  • Ronald Neil Kostoff

Organizations

  • Office of Naval Research

Tags

Communities of Interest

  • Biomedical

DTIC Thesaurus Topics

  • Business Administration
  • Geography
  • Health Services
  • Information Science
  • Medical Personnel
  • Organizational Structure
  • Patient Care
  • Psychiatry

Readers

  • Library and Information Science
  • Organizational Process Management (OPM).
  • Psychometric Testing or Psychological Assessment.