When the Fire Is Already Out, You’re Already Behind

By Don Norbeck | Dark AI Defense LLC | July 10, 2026

On August 17, 1975, a fire broke out at Gulf Oil’s Girard Point refinery complex on the Schuylkill River near its confluence with the Delaware, at the southern tip of the South Philadelphia peninsula. The original tank fire was actually declared under control by mid-morning. What killed eight Philadelphia firefighters came hours later, when the refinery’s sewage system failed to drain the foam and water mixture accumulating in the avenues between the oil tanks, naphtha hidden beneath the foam ignited, and a second explosion trapped firefighters in the flooded refinery.¹ The fire burned for nearly a week. My father was on the department. He had traded a shift to be down the shore with us that weekend. I was three years old. I remember the radio.

About fifteen miles down the Delaware, the data systems division of Sun Company was quietly running the calculations that would eventually produce a different kind of response to catastrophic failure. John Ryan, president of Sun Information Services, had looked at his company’s growing dependence on computers and worked out that a mainframe failure would cost Sun roughly $3 million by the third day of downtime. He built a contingency plan. Daily transaction tapes, offsite storage, an alternate mainframe on standby. In 1978, he turned that internal program into the world’s first commercial hot site, and SunGard Data Systems was eventually born from it.² The disaster recovery industry didn’t emerge from an abstract risk model. It emerged from people who had watched, from close enough to smell the smoke, what systemic failure looked like when the preparedness infrastructure didn’t exist.

This week, Illinois Governor JB Pritzker signed SB 315, the Artificial Intelligence Safety Measures Act, what advocates are calling the strongest AI safety and accountability legislation in the country. The law requires the largest AI developers to publicly disclose their safety practices, submit to independent third-party audits, and extends whistleblower protections to employees who raise safety concerns.³ It is a serious piece of legislation, and it landed in the same week that Axios reported on the chaotic behind-the-scenes scramble that preceded both OpenAI’s GPT-5.6 release and the export controls placed on Anthropic following concerns about Fable 5’s jailbreak vulnerabilities.⁴ The news cycle made the argument the legislation was trying to make: when there is no framework in place before a crisis, what you get is improvisation under pressure, and improvisation under pressure produces results that satisfy nobody and protect nobody systematically.

The Accountability Gap

The Axios account of what happened with Anthropic is instructive not as a story about one company, but as a preview of what governance failure at scale looks like in practice. The vulnerability that triggered export controls was a jailbreaking concern, which is the kind of safety bypass that should, in a properly functioning governance environment, have had pre-negotiated severity thresholds, defined reporting protocols, and an established process for determining what response it warranted. None of those things existed. There was no shared vocabulary between industry and government for how serious a jailbreak needed to be before it crossed into regulatory territory. So when Amazon Web Services identified something concerning and flagged it, the government reached for the tools it already had: export controls. The instrument was designed for a different kind of risk entirely, and it produced a result nobody had planned for, least of all the companies now navigating what it means for a frontier model to be treated as a potential export-controlled munition.

The Cybersecurity and Infrastructure Security Agency and the Center for AI Standards and Innovation were supposed to be building the standards and technical capacity that would have made this less improvised. CAISI is operating on a $15 million annual budget while needing $84 million to execute the administration’s own AI action plan, according to the Institute for Progress.⁵ That gap is not a funding line item. It is a structural statement about whether the preparedness infrastructure is real. Less than one percent of AI PhD graduates go into government,⁴ which means the agencies nominally responsible for evaluating frontier model safety are doing so without the technical bench to evaluate what they’re looking at.

States Cannot Do This Alone

The Illinois law is doing something meaningful. Mandatory safety practice disclosure and independent third-party audits create an external accountability surface that currently doesn’t exist. The whistleblower protections are worth particular attention because they acknowledge something that disclosure requirements alone cannot: safety failures are often known inside organizations before they are visible outside them. Human observers of model behavior, employees who notice something wrong before a deployment decision goes final, represent an accountability mechanism that doesn’t depend on the model reporting on itself. That matters more than it might appear, for reasons the second piece in this series will develop in detail.

What Illinois cannot fix is what no single state can fix alone. The law operates within Illinois. A company building to Illinois’s requirements while operating nationally is still navigating California’s disclosure standards, New York’s algorithmic accountability provisions, Colorado’s high-risk AI governance framework, and more than thirty other active state bills with varying scope, definitions, and enforcement mechanisms.⁶ The compliance cost of that patchwork falls on businesses, and businesses pass costs through. The institutions least equipped to absorb those pass-through costs are not large technology companies with legal departments that specialize in multi-jurisdiction compliance. They are the smaller organizations, the healthcare providers, the hiring platforms, the financial services firms operating at the margins of AI adoption, deploying systems that make consequential decisions about people who have no visibility into how those decisions are made.

Preparedness Is an Engineering Problem

The SunGard insight, built in the shadow of catastrophic witnessed failure, was that disaster recovery is not a response strategy. It is an infrastructure problem that has to be solved before the disaster, because after the disaster you are already behind. The DR industry that grew from that insight eventually required regulatory backing: in 1983, the Comptroller of the Currency began requiring national banks to have testable backup plans, and SunGard’s customer base more than doubled within a year.² The regulatory mandate didn’t create the industry, it scaled it. The technical infrastructure existed first, the standard came after, and together they produced something that worked.

The technical frameworks for what meaningful safety audits should look like, what severity thresholds for jailbreak vulnerabilities should trigger what responses, what independent audit standards should require, those things are being worked out in real time, in state legislatures and export control proceedings and industry working groups, without a federal platform underneath them to make the pieces composable. Illinois is building load-bearing infrastructure on ground that has no foundation. That produces structures, but not a system.

The people who will pay for the gap between structures and a system are not the companies managing compliance costs. They are the individuals in the consequential decision pipelines those companies are running, people whose housing applications, job screenings, insurance determinations, and healthcare triage decisions are being shaped by systems whose safety properties cannot be independently verified by anyone outside the organization that built them. Governance gaps, when they produce casualties, have never concentrated those casualties among the institutions that created the exposure. The Gulf fire killed eight firefighters, not Gulf Oil executives. The refinery had been cited for fires repeatedly since 1960.¹

We are now two frontier model pulls in eighteen months. The preparedness infrastructure has not been built. The next event will not wait for it.

Sources

1. Wikipedia, 1975 Philadelphia Gulf Refinery Fire; Firehouse Magazine, History As a Teacher: How a Refinery Blaze Killed 8 Firefighters; Philadelphia Inquirer, Philly firefighters honor victims of Gulf Oil refinery fire 50 years ago, August 2025.
2. International Directory of Company Histories, Vol. 11 (1995), SunGard Data Systems Inc. entry. SunGard origin story, John Ryan, Sun Information Services commercial hot site, and 1983 Comptroller of the Currency mandate.
3. Office of Governor JB Pritzker, Gov. Pritzker Signs Nation-Leading Artificial Intelligence Safety Law, July 6, 2026; Capitol News Illinois, Pritzker signs landmark AI regulation bill, July 6, 2026.
4. Axios, Ashley Gold and Maria Curi, Inside the Alternative Playbook to AI Regulation, July 10, 2026. Source for both the Anthropic export control account and the less-than-one-percent AI PhD statistic, attributed to Cato Institute’s Kevin Frazier.
5. Institute for Progress, What Will It Cost for the US to Be Ready for the Next Big AI Breakthrough?, May 2026. CAISI $15 million operating budget and $84 million annual requirement.
6. Capitol News Illinois, Pritzker signs landmark AI regulation bill, July 6, 2026. CA/NY/IL tier-one cluster and 40% US AI market estimate; more than 35 states with active AI legislation as of mid-2026.