The White House AI accord calls for audits without defining the tests

Suppose an AI assistant finds the perfect gap in your calendar—and books it when you only asked for a suggestion. Its answer was accurate; its action exceeded your permission. For anyone considering handing tasks to software, that distinction matters: useful performance and reliable safeguards need separate checks.

The AI agreement released by President Donald Trump on September 29 puts those safeguards at the center of its proposed audits. Reuters reports that Trump and leaders of Google, Anthropic, Meta, OpenAI, X and Nvidia signed the document, which he described as “morally binding.” It is a voluntary commitment that contemplates possible future laws or regulations. The announcement does not establish that the promised audits have been completed.

The published accord, reproduced by Forbes Australia, sets out four layers:

1. Operational safeguards: Monitor models during training and deployment for cybersecurity, biological and chemical risks, aiming to prevent unintended access to technical systems.
2. Internal review: Empower a company team to check that safeguards, monitoring and detection work, and ensure problems are corrected.
3. External evaluation: Bring in an independent auditor or evaluator to assess whether those protections work as intended.
4. Board oversight: Designate an independent board committee to receive reports from the operating teams and reviewers, and ensure identified problems are addressed.

The text does not specify conflict-of-interest safeguards, evaluator access to systems and records, testing methods, passing thresholds, implementation deadlines, audit frequency or publication of findings. Companies commit to meeting regularly to develop standards and best practices. These are gaps in the agreement’s detail; they do not establish which safeguards companies already have. Published accord.

There is a useful comparison available in guidance from the National Institute of Standards and Technology, or NIST. Its voluntary AI Risk Management Framework 1.0 calls for documented test cases, measurements and tools; evaluation under conditions resembling actual use; and continued monitoring after deployment. It also calls for documenting limits on how far results can be generalized, and risks that cannot be measured. The accord does not say it incorporates that framework. But the comparison explains what would turn an oversight structure into evidence about a particular protection.

Return to the hypothetical calendar assistant. Imagine a calendar with a meeting from noon to 1 p.m., followed by an empty half-hour. Ask the assistant to suggest a 30-minute appointment between noon and 1:30 p.m. Identifying 1 p.m. tests whether it understands the schedule. Keeping the calendar unchanged tests whether it respects the request’s boundary.

Now imagine the assistant tries to create the appointment anyway. A separate permission control could block that attempt. An evaluator could inspect both the attempted action and the unchanged calendar to determine whether that protection worked in this case. If the assistant merely answers that it did nothing, the evaluator would still need records showing what actually happened.

This example reflects a concern identified in NIST’s February 2026 draft concept paper on AI agent identity and authorization: giving software agents access to data, tools and applications requires appropriate controls over their identity and permitted actions. The paper outlines a potential project, rather than a completed standard or a finding about any product.

An audit of the calendar safeguard would therefore need a defined scope. Which assistant version and calendar connection were examined? What permissions were enabled? Did the evaluator see approval records and action logs? A successful check with one setup would support a conclusion about that setup. It would leave questions about shared calendars, different permissions or later software changes—the kind of limits NIST’s framework asks organizations to document.

NIST’s evaluation playbook adds two practical details: define acceptable performance limits, and assess whether evaluators have sufficient independence and resources. Independence needs examination alongside the work itself. A reviewer’s title alone cannot tell readers how thoroughly a safeguard was assessed.

For future claims that an AI system has been audited, three pieces of information would make the claim interpretable:

• What was examined: The named system, its version, the particular safeguards and the conditions tested.
• How it was examined: The evaluator’s independence, access to evidence, methods and criteria for acceptable performance.
• What was found: Results, failures, corrective action and disclosed limits on the conclusions.

For the calendar assistant, those details could establish that a permission barrier blocked an unapproved change under specified conditions. They would give the user a concrete reason to trust that barrier, while leaving the assistant’s other abilities and risks to their own evidence.