Mapping physical AI safety evaluations

1. Introduction

Robot safety has traditionally focused primarily on collision avoidance, force limits and avoiding direct physical harm to people. Generalist robots controlled by foundation models extend the set of relevant hazards, as "the safety of bits cannot be separated from the safety of atoms" anymore [1]. Physical AI safety must account for context, user intent and hard-to-model physical consequences across the robot's lifecycle. Empirical results support this concern: Language-model-controlled robots can be jailbroken into harmful physical actions [2], and frontier models appear to refuse harmful instructions far less often when in control of a robot arm than when the same instructions are given in text [3].

A redefinition of safety requires corresponding instruments of measurement. Safety evaluations for embodied systems are published across robotics, computer vision, natural-language processing and AI safety venues, with heterogeneous task designs and lacking a shared vocabulary. Three recent surveys organise the attack and defence literature for embodied AI [4], vision-language-action (VLA) models [5] and foundation-model-powered embodied agents [6]. They do not assess how well the existing evaluations cover the risks that motivate the field, in particular those with potential for catastrophic harm.

Our interactive Eval Atlas catalogues published physical AI evaluations. Based on this map, we identify the current epistemic gaps in the evals landscape, and motivate directions for future benchmarks and evaluation methodologies. For an overview of the results, skip ahead to Sections 4 and 5.

2. The Eval Atlas

The Eval Atlas is a public, interactive catalogue of physical AI safety evaluations. Each entry records an evaluation's kind, venue, a short summary, headline results, links to the paper and code, where available. Each entry is scored on task complexity, dexterity demand, cognitive demand, and relevance to catastrophic risk scenarios. The atlas is intended as a reference for researchers designing new evaluations, for funders and developers assessing what current evaluations do and do not cover, and for standardisation work that requires an overview of existing test methods proposed by the academic community.

The atlas provides five views:

• Coverage map: A heatmap that cross-tabulates any two dimensions, for example harm class against degree of embodiment or architecture against failure mode.
• Dexterity vs cognition: A map of the evaluation landscape organised by the degree to which models are evaluated on dexterity- vs. cognition-bounded tasks.
• Catalogue: The full catalogue, searchable by name, summary and results, filterable by domain, architecture, harm class and kind, and sortable by relevance to catastrophic risk, date or score.
• Gaps: The predominant gaps in the current literature.
• Method: Inclusion criteria, scoring rubrics and sources (see Section 3).

The atlas is periodically updated. Please contact us for corrections and suggested additions!

4. The evaluation landscape

Table 1 summarises the distribution of the 94 entries over the tagged dimensions. Since an evaluation can carry several tags per dimension, counts within a dimension can sum to more than 94.

Table 1. Distribution of evaluations, n = 94

The literature concentrates on accidents and misuse that cause local physical harm, predominantly evaluated in simulation. Ten evaluations score 2 or more for relevance to catastrophic risk; only ABC-Bench [7] reaches 3. For 62 of 94 evaluations, reported failures are predominantly attributable to cognition rather than lack of dexterity.

Evaluations fall predominantly into two groups. Evaluations of VLM and LLM agents typically abstract motion into primitives and test planning, hazard recognition and safety judgement. VLA benchmarks typically include short tabletop tasks that primarily test for dexterity and related capabilities. Of the 15 evaluations that are demanding on both axes, 6 are capability suites, and the 9 dedicated safety evaluations all sit at cognition level 2. No current safety evaluation combines dexterous manipulation with causal or strategic reasoning.

Recent work is beginning to fill this gap. Of the 9 safety evaluations demanding on both axes, 7 were first posted between September and November 2026, among them NegaBench [9] and RoboHarm [3]. Because recent months were searched more thoroughly than earlier ones (see Section 6), this observation may be due to our literature sampling methodology.

5. Gaps in current coverage

We identify nine gaps in the current evaluation landscape (see Gaps in the atlas), ordered here by their potential impact on the safety of deployed systems. Most of them concern aspects of deploying general-purpose foundation models on real-world, general-purpose hardware on long-horizon, open-ended tasks, or deploying frontier models on narrow-scoped but safety-critical tasks with the potential of large-scale downstream harm.

5.1 Limited overlap between embodied and catastrophic-risk evaluations

ABC-Bench [7] is the only surveyed evaluation that physically evaluates a robot capability relevant to a catastrophic-risk scenario: Agents wrote scripts that a liquid-handling robot executed to assemble a benign DNA construct, while the dual-use steps were evaluated digitally. Nine further evaluations address precursors of catastrophic pathways, comprising text and code evaluations, cyber attacks on robot software, jailbreak demonstrations, a runtime governance check and an insider attack on a team of language-model-controlled robots. Closed-loop manipulation benchmarks, including recent work on real hardware, address everyday accidents or deliberate, localised harm through a single robot.

The gap reflects a corresponding epistemic gap in threat models, where it is often unclear how exactly advanced AI systems may cross the cyberphysical barrier, as well as the cost and dual-use concerns of building such evaluations. Whether physical execution is a bottleneck on a catastrophic pathway can only be established by evaluations that carry the pathway through to execution. Biological laboratories are the most tractable starting point, since laboratory manipulation suites and a dual-use capability evaluation already exist (Section 5.6).

5.2 Absence of embodied misalignment and loss-of-control evaluations

Most surveyed work addresses accidents, misuse or attacks. Few studies address cases in which a model exhibits dangerous behaviour on its own, e.g. due to competing goals. EAPrivacy [24] reports that agents prioritised task completion over a stated privacy constraint in up to 86% of cases. What Happens to the Goal? [25] traces goal misgeneralisation inside trained policies on navigation and manipulation tasks, but does not study frontier robot models. Loss-of-control scenarios, in which humans can no longer meaningfully oversee, correct or stop AI systems [10], are reflected in only two of the surveyed works. EmbodiedGovBench [11] tests whether a robot runtime honours human overrides, but not whether a model is able or willing to evade them; ForesightSafety Bench [12] assesses misalignment and loss of control through text questions only. For non-embodied agents, related behaviour has been evaluated directly, including autonomous replication [13], self-replication risk in realistic scenarios [14] and the propensity for misaligned behaviour [15]. We did not find an embodied counterpart that evaluates resistance to being stopped, unprompted self-modification, concealment of actions, or evaluation awareness that arises without instruction.

Unlike misuse, which deployers can partly control through access restrictions, these behaviours originate in the model itself. Without such evaluations, developers have no evidence that a robot policy will accept correction or shutdown.

5.3 Limited evidence that safety judgements carry through to physical action

A model's safety judgement when asked about a task can differ from its behaviour when executing the same task. Whether refusal at the language level prevents the corresponding physical action has been measured directly only recently, and predominantly for language-model agents. RoboHarm [3] varies the interface for a fixed model and reports refusal of 73% in text against 18–25% under robot control, with comparable refusal on a real arm and in simulation. OSMA-Bench [17] and AgentSafe [18] report a corresponding gap between recognition and action in household settings. Whether these findings extend to end-to-end VLAs, which have no language-level refusal mechanism, remains open.

If safety training applied at the language level does not carry over to action, refusal rates measured in text overstate the safety of robots built on the same models, and safety cases that rely on them may not hold.

5.4 Absence of safety evaluations that combine dexterity and cognition

As Figure 1 shows, no current safety benchmark combines demanding manipulation with causal or strategic reasoning. The only evaluation demanding in both respects at the highest level, EmbodiedLab [16], is a capability suite that does not include a safety measure. HazardArena [8], which matches motor demand across safe and unsafe scenarios, is the clearest controlled design we identified.

More evaluations in the high-cognition, high-dexterity domain would allow safety-critical failures to be attributed either to a lack of physical capability or to cognitive phenomena such as misalignment. Notably, the tasks that are most relevant to catastrophic risk scenarios, such as wet-laboratory protocols, often require both advanced cognitive and dexterous capabilities (see Section 5.1).

5.5 Real-hardware evaluation and certification

Of the 23 evaluations that involve real hardware, most report a single validation run on one arm alongside a larger simulated benchmark, without repeated trials across initial conditions or robot units. RoboHarm, built on the Inspect Robots harness [19], combines public code, frontier models and a real arm, and is the nearest approximation to a standing real-robot evaluation suite. We did not find benchmarks designed to certify or stress-test an individual embodied system, and most benchmarks are designed to compare models. Singh [20] shows that finite testing cannot exclude adversarial behaviour that begins after certification, and evaluates monitoring with revocation as an alternative.

Standards and certification bodies require test methods that are repeatable on a specific system. Current benchmarks do not yet provide such methods.

5.6 Absence of a safety layer for laboratory robots

Six evaluations measure laboratory manipulation capability in simulation and, in some cases, on real arms. Neither the laboratory safety benchmark LABSHIELD [21], which is text-based, nor ABC-Bench is executed on these platforms, so no laboratory safety evaluation executes a multi-step workflow on a robot. Adding hazards and dual-use requests to an existing laboratory simulator would connect these lines of work, and would allow dual-use risk in automated laboratories to be measured end to end.

5.7 Limited coverage of psychosocial harm

Social robots are increasingly deployed with children, older adults and patients, although not yet widely [26]. The human–robot interaction literature identifies these groups as particularly susceptible to social influence from robots. Only two of the surveyed works address harm to people beyond physical injury, and both concern discrimination [27, 28]. Established paradigms measure children's conformity to a group of robots [29], overtrust in a robot during an emergency [30] and obedience to a robot experimenter [31]. These studies use scripted or teleoperated robots, so they measure human susceptibility rather than the behaviour of an AI system. Child-safety benchmarks for language models [32, 33] are text-only. The forthcoming standard IEEE P7027 [34] (Institute for Humane Robotics) addresses the identification and assessment of relational risks such as over-attachment and behavioural influence in domestic, educational and child-facing settings. We did not find evaluations that measure manipulation, deception or emotional dependence in interactions with robots controlled by frontier models.

5.8 Further gaps

• 5.8.1

  Humanoids and multi-robot systems. We did not find safety benchmarks that evaluate learned policies on real humanoids. Multi-robot work largely consists of attack studies.

• 5.8.2

  Human oversight at scale. As decisions and physical work are delegated to large numbers of robots, the ability of people to direct and correct them at the level of fleets or deployments becomes a safety property in its own right [22, 23]. We found no instrument that measures it.

6. Limitations

All scores are ordinal judgements made by our team from published material. The rubrics compress heterogeneous task designs and evaluation methodologies into four scales, and are not a replacement for nuanced judgement from reading the original works. Displaying evaluations as datapoints on a map obscures the fact that results within individual evaluations often vary widely across tasks and models. Coverage is limited to published work with a public description; evaluations conducted internally by model developers or robot manufacturers are not represented. The search combined surveys, targeted queries and regular arXiv searches. Recent months were searched more thoroughly than earlier periods, so the atlas does not allow confident extrapolation of temporal trends. Counts by tag should be read as a description of the surveyed set rather than of the field as a whole.

7. Conclusion

The current generation of physical AI safety evaluations measures everyday physical harm in simulation well. As model and hardware capabilities improve, the evaluations landscape must evolve to subject physical AI systems to new kinds of testing that will increasingly resemble frontier LLM evaluations in that it must involve open-ended, long-horizon tasks in adversarial and non-adversarial contexts. Physical AI evaluations will need to deviate from LLM evaluations in important ways, however. One fundamental issue that remains unaddressed is that building real-world evaluation environments is expensive, resetting real-world environments is cumbersome, and real-world harm is often irreversible. Exploring the viability of simulated environments resistant to eval awareness (if possible), as well as establishing paradigms for real-world evaluation that reflect real-world deployments while affording the scale and replicability needed for rigorous evaluation, are open questions that remain to be addressed.

The Eval Atlas will be updated periodically as new work appears. If we have missed or mis-scored an evaluation, please let us know.