Specification technologies to enable complex, data-intensive collaborative systems for assessment and evaluation
Abstract
This project is about a new approach for defining programs in a domain which enjoys broad demand yet today suffers high costs in imp lementation: collaborative systems, by which we refer to distributed applications which implement peer review processes. Industry of fers such products, of course, but they have little flexibility, and often leave data of value isolated in different silos where the y cannot be leveraged with one another. We thus miss opportunities to wring value from our collaborations. We attribute this state o f the practice to a lack of powerful ways to specialize tools which might improve outcomes in nuanced settings. Designing collaborat ive systems demands that we tame complexity in the coordination of many interoperating threads of control while preserving integrity of the data they exchange. In the absence of expressive tools, this complexity is what drives cost: our field has great tools for d efining interaction between small assortments of programs but designing collaborative systems at scale demands experts who must reas on about models using essentially manual techniques. We propose a novel specification-driven technology to raise the level of discou rse about collaborative systems and enable automatic generation of configurations to directly execute those specifications. In this way we will bring modern quality improvement practices which require us to anticipate effects of a policy decision before paying t o build it to an important domain. Direct execution offers such insight. It will solve many collaboration problems directly, and f or the rest it will result in high quality blueprints for build out at scale. This is as much a data problem as control problem, sin ce blending value from tools which cross administrative boundaries (as is essential in realistic scenarios) requires a reliable way to relate data to one another, not just move bits around. We cant just build bigger closed systems.We characterize this technology as enabling becauseof the climate of innovation it should create. The state of the practice is that data scientists, managers and bu siness leaders have content they cant share and work flows that cant meld, and because the complexity of modelling improved collab orations is high, the economic winds blow against early exploration of better requirements which is when decisions for a product wou ld be inexpensive. To contain cost, developers thus make best guesses during product definition (hoping not to have to walk them bac k after paying large build costs) or just design for a low common denominator. It is this way with peer review systems which serve a s our driving example: there is strong motivation for far more expressive systems yet essential properties of novel work flows cant be vetted without costly implementation. What we propose is a tems inexpensively, allowing technical consequences of early decisions to become known quickly. By raising the level of abstraction and lowering cost of experimentation, our approach will promote agile quality improvement processes in this important domain.We are well-positioned to study this based on results of a prior project. We manually crafted pilot peer review systems and learned quite a lot about what engines are necessary to drive such tools. Motivation for a spec-based approach came from recognizing that much prog ram organization might be generated automatically. We would like to capitalize on this work and continue the momentum. Peer review w ill be our driving example in on-going research, and it is our intention to deploy serious pilots to test and illustrate the depth o f our work.Approved for Public Release
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Aug 20, 2021
- Source ID
- N000142112821
Entities
People
- James Purtilo
Organizations
- Office of Naval Research
- United States Navy
- University of Maryland