Tsamados et al. 2024 argue that as AI systems become less predictable, evaluation must shift "from benchmarking progress on whether AI systems can outperform humans in given tasks to assessing the performance of human–AI systems as a whole," moving the control paradigm from supervision toward teaming.
The evaluation-shift quote appears verbatim in E36, but the cited chunk neither attributes the shift to AI systems becoming less predictable nor describes moving a control paradigm 'from supervision toward teaming.'
Written by Kimi K3 via Ollama Cloud · checked by GLM-5.3 via Ollama Cloud · 3 Oct, 04:46
Source chain
AEvery quote below was checked, without a model, to appear verbatim in its source.
- 01
“moves from benchmarking progress on whether AI systems can outperform humans in given tasks to assessing the performance of human–AI systems as a whole”
Full-text passage · no page number · evidence E36
This requires that the AI research community moves from benchmarking progress on whether AI systems can outperform humans in given tasks to assessing the performance of human–AI systems as a whole, including their interactive, organisational, and collaborative capabilities as agents of a more extensive system [46, 47]. For example, training programs that focus on testing human–machine interactions in realistic settings––either through high-fidelity simulations or deployment in contained environments––can reveal insights about agents’ capabilities and the corresponding levels of authority they should be given. Indeed, this is a growing trend among organisations spearheading the development of AI systems for complex socio-technical environments, such as the Defense Advanced Research Projects Agency (DARPA). The agency has launched several programs, including ASIST [48], SAIL-ON [49], and CAML [50] and associated software for assessing and improving human–AI systems’ performance during deployment or in computer-simulated environments, as well as to test AI systems in complex socio-technical environments (more on this in the next section).
Passage read from www.ebi.ac.uk, which may be a preprint rather than the published version.