Tools for the Precision Medicine Era: How to Develop Highly Personalized Treatment Recommendations From Cohort and Registry Data Using Q-Learning
Abstract
Q-learning is a method of reinforcement learning that employs backwards stagewise estimation to identify sequences of actions that maximize some long-term reward. The method can be applied to sequential multiple-assignment randomized trials to develop personalized adaptive treatment strategies (ATSs)—longitudinal practice guidelines highly tailored to time-varying attributes of individual patients. Sometimes, the basis for choosing which ATSs to include in a sequential multiple-assignment randomized trial (or randomized controlled trial) may be inadequate. Nonrandomized data sources may inform the initial design of ATSs, which could later be prospectively validated. In this paper, we illustrate challenges involved in using nonrandomized data for this purpose with a case study from the Center for International Blood and Marrow Transplant Research registry (1995–2007) aimed at 1) determining whether the sequence of therapeutic classes used in graft-versus-host disease prophylaxis and in refractory graft-versus-host disease is associated with improved survival and 2) identifying donor and patient factors with which to guide individualized immunosuppressant selections over time. We discuss how to communicate the potential benefit derived from following an ATS at the population and subgroup levels and how to evaluate its robustness to modeling assumptions. This worked example may serve as a model for developing ATSs from registries and cohorts in oncology and other fields requiring sequential treatment decisions.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Jun 22, 2017
- Source ID
- 10.1093/aje/kwx027
Entities
People
- Amin Alousi
- Brent Logan
- Daniel R. Couriel
- Elizabeth F Krakow
- Erica E M Moodie
- Joseph Pidala
- Michael Hemmer
- Michael Last
- Mukta Arora
- Silvy Lachance
- Stephen R. Spellman
- Tao Wang
Organizations
- Actinium Pharmaceuticals (United States)
- Amgen
- Angiocrine Bioscience (United States)
- Astellas Pharma (United States)
- AstraZeneca (United States)
- Atara Biotherapeutics, Inc.
- Celgene
- Center for International Blood and Marrow Transplant Research
- Fred Hutchinson Cancer Center
- Genzyme
- Gilead Sciences
- Health Resources and Services Administration
- HistoGenetics (United States)
- Incyte
- Janssen Scientific Affairs (United States)
- Jazz Pharmaceuticals
- Karyopharm Therapeutics (United States)
- Kite Pharma
- Medical College of Wisconsin
- Merck & Co.
- Meso Scale Diagnostics (United States)
- Miltenyi Biotec
- National Cancer Institute
- National Heart, Lung, and Blood Institute
- National Institute of Allergy and Infectious Diseases
- National Marrow Donor Program
- Novartis (Canada)
- Office of Naval Research
- Otsuka Pharmaceutical
- Pfizer
- Seagen
- Shire (Ireland)
- Spectrum Pharmaceuticals
- Swedish Orphan Biovitrum
- Takeda Oncology
- University of Minnesota