Reinforcement Learning for Generative Models
Abstract
This proposal aims to develop new Reinforcement Learning (RL) algorithms for training modern large-scale generative models. The proposal contains two main research thrusts. In the first thrust, we aim to leverage standard supervised learning techniques, such as regression, to design efficient and scalable RL algorithms for optimizing Large language models with billions of parameters. We willalso extend our algorithms to allow them to learn directly from general ranking-based feedback. In the second thrust, we propose tobuild a strong connection between the field of diffusion model training and Imitation Learning (IL). We propose to understand the fundamental limits of the current approaches of training diffusion models via this connection. We also aim to develop new algorithms for training diffusion models via leveraging the rich set of tools developed in the imitation learning, RL, and control communities.Finally, we propose to extend the algorithms proposed in the first thrust to diffusion model training and thus enable RL from humanfeedback (e.g., ranking feedback) for training image generative models.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Apr 10, 2025
- Source ID
- N000142512267
Entities
People
- Wen Sun
Organizations
- Cornell University
- Office of Naval Research
- United States Navy