Provably Impartial Peer Assessment for Expert Hiring
Abstract
The goal of the proposed research is to develop a theoretically-grounded, impartial method for using peer assessment to hire experts (workers) in online outsourcing markets. We further aim to develop interfaces such that workers contribute truthful assessments. Online outsourcing enables many individuals and businesses to hire experts forparticular tasks, and for short periods of time. For employers, expert online outsourcing provides broader access to specialized skills; for workers, this has created new opportunities to access and compete in global job markets, from anywhere at any time, as long as they have computer and Internet access. Yet, employers who wish to hire online experts, and researchers who create innovative systems around expert crowdwork, are limited because they are unable to discern qualified workers effectively. A lack of expertise in areas for hiring prevents employers from discerning high-quality workers (Consider a hairdresser hiring a web designer to make the business website.) Over time, employers offer lower wages to offset their risk of lowquality results, and workers respond to lower payment with lower quality work. Lower wages also discourage qualified workers from participating, in turn driving away potential employers. Even employers who know enough to assess workers well still face significant friction. On online expert outsourcing platforms like Upwork, it takes employers three days to screen, interview, and hire candidates. The effort of hiring processes can be understood as a search friction, encouraging employers to satisfice (hiring workers who are ~good enough~), and discouraging hiring workers entirely.This proposal will introduce novel methods for hiring workers by leveraging peer assessment. In this scheme, workers applying to a position assess each other using our novel mechanism. This can make hiring much faster and reduce the need for employers to understand their hiring domain closely. The thrust of the proposed research is threefold:1. Designing algorithms for approximate impartial peer assessment of workers, that relies on pairwise comparisons that workers make about each others~ capabilities and expertise.2. Designing user interfaces that encourage workers to assess each other accurately and truthfully3. Using real-world evaluation, test the performance of our algorithms in practice, and to develop meaningful practical extensions such as adapting our work to dynamic settings.Impact on Naval Capabilities. Effectively finding expertise is a cornerstone to the success of any organization, including the Navy. In our setting, we consider how a non-expert could find expertise from a pool of experts quickly and with theoretical guarantees about effectiveness. To give a few examples where Navy officers act as employers are hiring experts in an area as nonexperts, consider the task of finding translators in languages that are unfamiliar to Americans, local scouts and similar personnel. Similarly, hiring could also be internal to organizations (including the navy) and may seek personnel who have relevant experience and skills for a particular project. This is exactly the what we propose to address, creating methods for non-experts to hire experts quickly, and with theoretical guarantees on effectiveness.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- May 05, 2017
- Source ID
- N000141712428
Entities
People
- Chinmay Kulkarni
Organizations
- Massachusetts Institute of Technology
- Office of Naval Research
- United States Navy