Pre-release model review is procurement leverage, not a safety standard
The new executive order buys the government up to 30 days of confidential pre-release access to frontier models. Without published pass criteria, that is leverage dressed as assurance — buyers should read it that way.
This week's executive order on advanced AI directs agencies to stand up a voluntary framework under which frontier-model developers can grant the federal government up to 30 days of confidential pre-release access, with the framework itself due by August 1. The order is explicit about what it is not: no mandatory licensing, no preclearance, no permitting for new models.
The newsletter framing doing the rounds — "the NSA now grades your AI model" — is ahead of the text. I could not verify any provision for an actual grade, and the order publishes no pass criteria, no evaluation suite, and no remediation triggers. What exists is an access mechanism and a promise of collaboration. Everything else is inference.
So what is 30 days of quiet access actually worth? Read it as a procurement instrument and it makes complete sense.
Access without criteria is leverage
A review with published pass criteria and consequences is a standard. A review with neither is a relationship — and in government, structured relationships are how leverage works. The state gets threat intelligence on frontier capabilities before the public does, which is genuinely valuable for defence planning. The lab gets something too: the ability to say, truthfully, that the government saw the model before release and didn't object.
That sentence is where the danger lives for the rest of us. "Reviewed under the federal framework" will start appearing in sales decks within a quarter of the framework going live — my bet, and I will happily be wrong. It will sound like certification. It will be an attendance record. Voluntary access with confidential findings and no published bar is an informal market gate that looks much stronger than it is, and the vendors who lean on it hardest will be the ones who benefit most from the ambiguity.
There is a real tension here, and it is worth being fair about it: early access probably does improve national threat intelligence, and the no-licensing language keeps the door open for small labs. The order is not a bad instrument. It is just not the instrument your risk register thinks it is.
What this changes for a regulated buyer: nothing
At work, the question landed within days of the order: a client in a regulated industry asked whether the federal review would "count" toward their own model-risk obligations. The answer I gave them is the whole post: a review whose methods, findings, and thresholds you cannot see transfers zero obligation away from you. Your regulator will not accept "the government looked at it" any more than it accepts "the vendor tested it". Context is the whole game — the government did not test the model on your data, your workflows, or your failure costs.
My rule for any external review badge, government or otherwise: it reduces your testing burden only to the extent that its methods and pass criteria are public and its scope covers your use. Score it zero on both today. That may change — if the August framework ships with published evaluation suites and consequence triggers, I will upgrade it happily. The order as signed commits to neither.
Steal this for your vendor file: add a line item called "external reviews claimed", and next to each one record two fields — methods public? consequences defined? Anything with two nos is marketing, and gets weighted accordingly in the risk assessment.
Thirty days of confidential access buys the government intelligence and the lab a talking point — your own eval on your own risks is still the only review that transfers.
