Frontstage-Backstage Learning Framework to Address the Multi-objective Personalized Generation Problem in Task Support Generative AI Services
Abstract
Personalized generation in task support AI services can be defined as a multi-objective optimization problem. This is because it demands results that align not only with the general task objective but also with user-specific goals. From this perspective, the AI model must identify a feasible region where multiple objectives are satisfied, and generate optimal solutions. However, due to objective mismatches when conflicting objectives arise, the model often misidentifies this region, leading to sub-optimal results. While existing studies try to address this by updating model parameters using user feedback data, this strategy often fails to resolve the objective mismatch problem and does not adequately adapt to users’ specific needs and preferences. In this research project, we seek to overcome these limitations. We aim to discover theoretical insights for identifying feasible regions in multi-objective optimization problems of generation and design an efficient algorithm enabling task-support generative AI models to fulfill objectives optimally. Specifically, we will develop a frontstage-backstage learning framework. In this framework, the edit model scrutinizes and refines the base model’s outputs to align with user preferences, similar to real-world services where frontstage employees adapt the work of backstage employees for end-users. Central to our approach is the adoption of multi-objective reinforcement learning to produce solutions that satisfy multiple objectives simultaneously, including likelihood and reward maximization. Furthermore, our research will delve into the identification of users’ latent objective dimensions and the estimation of their preference vectors from feedback data. We will assess the robustness and efficacy of our framework in real-world contexts, like Korean-English translation and diet planning. By corroborating our theoretically grounded, high-performance models with practical case studies, we believe our contributions will elevate the capabilities of task support generative AI services across diverse sectors, from healthcare and education to defense.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Feb 22, 2024
- Source ID
- FA23862314121
Entities
People
- Chiehyeon Lim
Organizations
- Air Force Office of Scientific Research
- Ulsan National Institute of Science and Technology
- United States Air Force