Tsamados et al. 2024 and Calboreanu 2026 mark a shift in the unit of analysis for human control of AI: away from a human supervising a single AI system and toward evaluating and managing humans teamed with, or individually orchestrating, portfolios of AI tools.
Tsamados et al. explicitly urge moving from benchmarking AI systems against humans to assessing human–AI systems as wholes, and Calboreanu explicitly shifts the conceptual unit from teams of humans working with a single AI system to individual practitioners composing portfolios of distinct tools.
Written by Kimi K3 via Ollama Cloud · checked by GLM-5.3 via Ollama Cloud · 4 Oct, 04:46
Source chain
AEvery quote below was checked, without a model, to appear verbatim in its source.
- 01
“moves from benchmarking progress on whether AI systems can outperform humans in given tasks to assessing the performance of human–AI systems as a whole”
Full-text passage · no page number · evidence E32
This requires that the AI research community moves from benchmarking progress on whether AI systems can outperform humans in given tasks to assessing the performance of human–AI systems as a whole, including their interactive, organisational, and collaborative capabilities as agents of a more extensive system [46, 47]. For example, training programs that focus on testing human–machine interactions in realistic settings––either through high-fidelity simulations or deployment in contained environments––can reveal insights about agents’ capabilities and the corresponding levels of authority they should be given. Indeed, this is a growing trend among organisations spearheading the development of AI systems for complex socio-technical environments, such as the Defense Advanced Research Projects Agency (DARPA). The agency has launched several programs, including ASIST [48], SAIL-ON [49], and CAML [50] and associated software for assessing and improving human–AI systems’ performance during deployment or in computer-simulated environments, as well as to test AI systems in complex socio-technical environments (more on this in the next section).
Passage read from www.ebi.ac.uk, which may be a preprint rather than the published version.
- 02
“Augment engineering shifts both dimensions: the organizational unit is an individual practitioner, and the AI component is a portfolio of distinct tools”
Full-text passage · p. 6 · evidence E5
Human-AI Teaming Human-AI collaboration research has established that complementary human-AI teams can exceed human-only performance [14, 15, 16], with design guidelines for interaction, explanation, and graceful degradation. The conceptual unit in this literature, however, is a team of humans working with a single AI system. Augment engineering shifts both dimensions: the organizational unit is an individual practitioner, and the AI component is a portfolio of distinct tools. The concern is not decision accuracy within one system but the practitioner’s ability to select, configure, and compose tools across domain boundaries. 2.4. Prompt Engineering Prompt engineering has matured into a learnable discipline with replicable techniques: pattern catalogs [2], prompt programming [3], and chain-ofthought reasoning [17] demonstrate that structured interaction design measurably improves model output. The underlying assumption, however, is that prompt optimization is tool-specific.
Passage read from arxiv.org, which may be a preprint rather than the published version.