Optimal Allocation of Proposals to Reviewers to Facilitate Effective Ranking

Abstract

Peer review of research proposals and articles is an essential element in research and development processes worldwide. Here we consider a problem that, to the best of our knowledge, has not been addressed until now: how to assign subsets of proposals to reviewers in scenarios where the reviewers supply their evaluations through ordinal ranking. The solution approach we propose for this assignment problem maximizes the number of proposal pairs that will be evaluated by one or more reviewers. This new approach should facilitate meaningful aggregation of partial rankings of subsets of proposals by multiple reviewers into a consensus ranking. We offer two ways to implement the approach: an integer-programming set-covering model and a heuristic procedure. The effectiveness and efficiency of the two models are tested through an extensive simulation experiment.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Apr 01, 2005
Accession Number
ADA495145

Entities

People

  • Boaz Golany
  • Michal Penn
  • Moshe Kress
  • Tal Raviv
  • Wade D. Cook

Organizations

  • Naval Postgraduate School

Tags

Communities of Interest

  • Materials and Manufacturing Processes

DTIC Thesaurus Topics

  • Acquisition
  • Algorithms
  • Computer Programming
  • Coverings
  • Data Acquisition
  • Engineering
  • Heuristic Methods
  • Industrial Engineering
  • Integer Programming
  • Mathematical Models
  • Models
  • Numbers
  • Operations Research
  • Simulations
  • Social Sciences
  • Square Roots
  • Test And Evaluation

Fields of Study

  • Computer science

Readers

  • Life Cycle Cost Analysis
  • Operations Research
  • Organizational Process Management (OPM).