AI Governance for Critical Infrastructure

Why critical infrastructure AI governance is different

Critical infrastructure organisations operate services where failure can affect safety, public confidence, economic activity, national security, continuity of government, and the wellbeing of communities. AI governance in this environment is therefore not only a technology-management exercise. It is part of operational resilience. A weak AI-assisted workflow can create more than an incorrect output; it can contribute to delayed restoration, poor incident response, unsafe operational recommendations, supply-chain disruption, privacy exposure, or loss of confidence in essential services.

The Australian Security of Critical Infrastructure Act context is a useful reminder of the breadth of this operating environment. The SOCI page identifies sectors such as communications, financial services and markets, data storage or processing, defence industry, higher education and research, energy, food and grocery, healthcare and medical, space technology, transport, and water and sewerage. It also describes critical infrastructure as interconnected: disruption in one area can flow into others. AI governance must therefore account for dependencies, not only the immediate AI system.

NIST’s concept note for an AI RMF profile on trustworthy AI in critical infrastructure points in the same direction. It recognises that critical infrastructure will increasingly rely on AI across IT, operational technology, and industrial control systems, and that AI in those high-stakes environments must be worthy of trust. For operators, the practical implication is clear: AI must be governed through lifecycle risk management, operational boundaries, trustworthiness requirements, evidence, and supply-chain communication.

The asset and service are the unit of risk

Critical infrastructure AI risk should be assessed at the level of the asset, service, and workflow, not only the model or vendor product. The same AI system can be low risk when used to summarise public maintenance reports and high risk when used to prioritise outage restoration, recommend control-room actions, or guide incident response. The governance record should identify which critical service is affected, which asset or system context is involved, which human role is supported, and what consequence could follow from error, delay, unavailability, or misuse.

The use-case record should connect AI to asset criticality. Does the AI system affect a registered critical infrastructure asset, a system of national significance, a regulated service, an essential operational process, or a dependency that supports service delivery? Does it process protected, sensitive, security-relevant, or operational data? Does it recommend actions near operational technology? Does it influence a cyber incident response? Does it depend on a vendor that could become a concentration risk? These questions determine the required governance tier.

This asset-context approach also prevents underclassification. A back-office AI tool may look harmless until it becomes part of outage communications, workforce dispatch, procurement decisions, or incident reporting. Critical infrastructure operators should treat AI-enabled workflow expansion as a reassessment trigger. A system can move from low impact to material impact when it becomes connected to service continuity.

Operational resilience before optimisation

AI is often introduced to optimise: better forecasts, faster triage, more efficient maintenance, improved customer communication, sharper cyber alerts, or faster document processing. In critical infrastructure, optimisation must sit behind resilience. A system that improves average performance but creates opaque dependency, weak fallback, or unsafe automation may reduce resilience even while improving productivity metrics. Governance should therefore ask what happens when the AI is wrong, unavailable, degraded, compromised, or outside its intended operating conditions.

Resilience governance should define fallback paths. Can operators continue the process manually? Is there a tested procedure for disabling AI assistance? Are staff trained to recognise when AI output is unreliable? Are monitoring thresholds tied to escalation? Can the organisation reconstruct what AI recommended during an incident? Is there a plan for vendor outage or model regression? These questions should be answered before AI becomes embedded in critical workflows.

The most important resilience test is not whether AI works on a normal day. It is whether the organisation can maintain authority and recover when AI does not work. Continuity exercises should include AI failure modes: hallucinated recommendations, stale retrieval data, model drift, vendor service interruption, prompt manipulation, unauthorised tool use, missing logs, and staff overreliance. The exercise output should feed back into risk treatment and control design.

AI in OT and ICS-adjacent environments

AI near operational technology and industrial control systems requires conservative governance. Many critical infrastructure operators are exploring AI for asset monitoring, predictive maintenance, anomaly detection, operator assistance, incident triage, and field-service support. These uses may be valuable, but the governance model must preserve safety, reliability, segmentation, and human authority. AI should not silently expand from advice to action in operationally sensitive environments.

The boundary between IT and OT matters. An AI assistant that reads OT telemetry, maintenance history, or incident logs may create data and security risk even if it cannot directly control equipment. A system that recommends actions to operators may influence decisions even without direct write access. An agent that can call tools, open tickets, send commands, or update operational records creates a different risk profile again. Governance must document the boundary and control movement across it.

For OT-adjacent AI, operators should define approved data sources, network zones, access paths, tool permissions, testing requirements, human approval points, prohibited actions, and emergency disablement procedures. They should also record which outputs are advisory, which are decision support, and which can trigger workflow actions. In critical infrastructure, the difference between recommendation and execution is a governance boundary.

Autonomy and agentic risk

Agentic AI changes critical infrastructure risk because the system may plan, call tools, search systems, create tickets, draft communications, trigger playbooks, or propose operational actions. The risk is no longer only a bad answer. It is a chain of actions. An agent may retrieve stale information, misclassify an incident, recommend the wrong escalation, draft an inaccurate update, or call a tool outside the operator’s intent. Each step may look plausible until the sequence creates harm.

Governance should classify autonomy separately from model capability. A small model with powerful tool access can create more operational risk than a larger model used only for drafting. The assessment should ask what the agent can read, what it can write, which tools it can call, which systems it can affect, which actions require approval, which actions are prohibited, and how the organisation detects boundary violations. Least privilege should be the default, the same principle the OWASP Top 10 for LLM Applications codifies as LLM06:2025 Excessive Agency, which identifies excessive functionality, excessive permissions, and excessive autonomy as the drivers of harm in tool-enabled AI systems.

Human approval should be designed into the workflow, not added as a vague policy statement. Approval should be required for high-impact actions, irreversible changes, external communications during incidents, containment actions, operational changes, and actions affecting essential-service delivery. Logs should show the agent’s recommendation, the human decision, the tool call, the result, and any exception. Without that record, the operator cannot show how human authority was preserved.

Cybersecurity and AI governance overlap

Cybersecurity and AI governance overlap heavily in critical infrastructure. AI systems can be used by defenders, embedded in managed security tools, connected to incident-response workflows, and targeted by attackers. Prompt injection, data exfiltration, unsafe tool use, weak identity, unapproved integrations, and vendor compromise can all become operational risks when AI is connected to critical systems or security processes.

AI security controls should include identity, least privilege, network boundaries, approved providers, data classification, prompt and retrieval controls, logging, monitoring, secure change management, incident response, and vendor assurance. These controls should be mapped to the AI use case. A generic security review is not enough if it does not examine model access, prompts, tools, retrieval sources, output handling, and autonomous action.

Operators should also govern defensive AI carefully. A security copilot that summarises alerts is different from an agent that can isolate hosts, revoke credentials, alter firewall rules, or trigger containment. Speed is useful in incident response, but speed without bounded authority can create cascading harm. The governance model should define when AI can recommend, when it can prepare an action for approval, and when automation is allowed.

Third-party and supply-chain exposure

Critical infrastructure operators often depend on vendors for asset management, managed security, cloud services, telemetry, maintenance platforms, customer systems, data processing, field-service tools, and operational analytics. AI features can appear inside these platforms before the operator has a complete inventory. A vendor may add summarisation, anomaly detection, recommendation, optimisation, or agentic functions through a product update. That creates governance risk because the operator may not know where AI is influencing the service.

Vendor governance should require disclosure of AI functionality, data handling, model providers, subcontractors, training or retention practices, change notices, incident notification, audit support, evidence export, and exit options. Operators should ask whether the vendor can support resilience obligations: what happens if the AI feature is disabled, degraded, or changed? Can the operator opt out? Can it test changes before deployment? Can it obtain logs needed for post-incident review?

Concentration risk should be visible. A single AI provider or vendor platform may support multiple essential workflows. The failure or change of that provider could affect operations, customer service, cyber response, and reporting simultaneously. Critical infrastructure operators should track AI vendor dependencies at the enterprise level, not only within individual projects. This supports resilience planning and executive risk reporting.

Protected information and operational data

Critical infrastructure AI governance must treat operational data as a control concern, not merely an input. Asset information, system architecture, outage details, telemetry, security alerts, maintenance records, incident notes, and dependency maps can reveal sensitive information about the operation and resilience of essential services. The SOCI page also highlights that protected information about critical infrastructure assets has specific disclosure constraints. AI workflows that summarise, retrieve, transform, or transmit this information need controls that match its sensitivity.

The data question is broader than whether personal information is present. Operational data can be sensitive because it reveals how a service works, where it is vulnerable, how incidents unfold, which assets are connected, or what recovery options exist. A generative AI assistant used for incident summaries may expose protected operational details through prompts, logs, vendor systems, or generated outputs. A retrieval system may surface information to users who should not see it. An agent may combine low-sensitivity records into a high-sensitivity operational picture. Governance should classify these data flows explicitly.

That classification should be visible to operators, developers, security teams, and vendors so the same data is not treated differently across connected workflows.

Operators should define approved data sources, approved AI environments, retention periods, logging rules, export controls, access groups, and redaction requirements for critical infrastructure data. They should also decide which information must never be sent to external AI services and which uses require a controlled environment. Technical controls matter because training alone may not reliably stop staff or agents from moving sensitive operational information into the wrong place during a fast-moving incident.

Risk management and SOCI alignment

For Australian critical infrastructure operators, AI governance should align with the broader SOCI risk-management context rather than sitting beside it. The SOCI page identifies positive security obligations that can include operational and ownership information, cyber incident reporting, and maintaining a written risk management program depending on asset class. Under the Act’s mandatory reporting scheme, a responsible entity must notify the Australian Cyber Security Centre within 12 hours of a critical cyber security incident that has a significant impact, and within 72 hours of a cyber security incident that has a relevant impact. It also identifies enhanced cyber security obligations for systems of national significance, including incident response plans, cyber exercises, vulnerability assessments, and system information.

AI governance can support these obligations by making AI-enabled dependencies visible, documenting risk treatment, retaining evidence of control operation, and feeding AI failure modes into incident response and exercises. If an AI system affects essential service delivery, cyber response, or operational continuity, it should be considered in the operator’s risk management program. The governance question is whether AI changes likelihood, impact, detection, recovery, or dependency for the critical asset.

Operators outside Australia can still use the same principle. AI governance should connect to existing critical infrastructure risk programs, cyber resilience programs, safety management, business continuity, vendor risk, and incident response. Creating a separate AI governance silo will fragment accountability. The better pattern is to add AI-specific risk dimensions and evidence requirements into the governance structures that already manage critical services.

Assurance and internal audit

Assurance should test whether AI governance is designed appropriately and operating effectively. Design testing asks whether the controls are strong enough for the use case: asset-context assessment, authority boundaries, fallback procedures, vendor obligations, monitoring, incident response, and evidence retention. Operating testing asks whether those controls actually worked during the review period. Were AI use cases inventoried? Were high-impact actions approved? Were vendor changes reviewed? Were exceptions time bound? Were incidents recorded? Were fallback procedures tested?

Internal audit should be able to test AI governance without reconstructing the control environment manually. Evidence should be retained in a form that shows who approved the use case, which controls were required, which policy decisions occurred, which exceptions were granted, which incidents or near misses were recorded, and which reassessments happened after change. For systems connected to critical services, auditability is not a compliance nicety. It is part of resilience because post-incident learning depends on a reliable record.

Assurance should also include exercises. A tabletop can test whether executives, operators, security teams, vendors, and control owners understand the AI dependency and their responsibilities. A technical exercise can test whether AI can be disabled, whether humans can take over, whether logs are complete, and whether blocked actions are visible. These exercises help reveal weak controls before an incident forces the same discovery under pressure.

Monitoring and key risk indicators

Monitoring should combine AI signals, operational signals, and resilience signals. AI signals include policy violations, blocked prompts, tool-call failures, model changes, retrieval-source errors, evaluation drift, and exception volumes. Operational signals include outage events, restoration delays, operator overrides, maintenance backlog, service complaints, incident-response timing, and near misses. Resilience signals include fallback use, vendor outage, overdue reassessments, control-test failures, and missing evidence.

Key risk indicators should drive action. If AI recommendations are frequently overridden, the model, workflow, or operator guidance may need review. If exceptions are increasing, teams may be bypassing controls or the policy may not match operational reality. If evidence completeness is low, audit and post-incident review will be weak. If a vendor changes an AI feature without reassessment, third-party risk controls need strengthening.

Critical infrastructure AI monitoring should also watch for silent expansion. A tool initially used for drafting may become used in operational decision support. An assistant may receive new data access. A vendor feature may become enabled by default. An agent may move from read-only to write access. Monitoring should detect these shifts because they can change risk tier faster than a scheduled annual review.

Incidents, exercises, and recovery

AI incidents in critical infrastructure should include more than outages. They can include harmful recommendations, unsafe autonomous actions, unauthorised tool use, prompt injection, data leakage, vendor failure, inaccurate incident summaries, missed alerts, degraded performance, missing records, or failure of human oversight. If teams do not know what counts as an AI incident, they will under-report the very events governance needs to learn from.

Exercises should test the AI governance system. Can the operator disable the AI function? Can humans take over? Can the team identify which use cases depended on the vendor? Can incident responders reconstruct AI recommendations and tool calls? Can control owners show which policies fired? Can executives understand exposure quickly? These questions matter because crisis conditions are when opaque AI dependencies become most dangerous.

Recovery should include reassessment. After an AI-related incident or near miss, the operator should update the risk assessment, control design, monitoring thresholds, user guidance, vendor requirements, and maturity target where needed. The lesson should not remain trapped in a post-incident report. It should change the governance system.

Conclusion: Helixar perspective

Helixar’s view is that critical infrastructure operators should enforce AI governance at the point where AI activity can affect resilience, safety, and continuity. Operators often need controls that are difficult to operate manually across providers, copilots, SaaS platforms, security tools, internal workflows, and agents. The governance layer should evaluate AI activity against policy, support proportionate governance responses, and retain evidence of what happened.

In critical infrastructure contexts, this governance pattern can include sensitive-data restrictions, approved-provider policies, human approval for high-impact actions, agent tool-permission boundaries, blocked actions, exception handling, and reassessment triggers. A low-risk internal prompt may be observed and logged. An OT-adjacent recommendation may require approval. An agent attempting an unapproved action may be blocked. A vendor or model change may be routed for review. These controls help preserve human authority and operational visibility.

This research also emphasises assurance and recovery. Evidence of approvals, policy decisions, blocked actions, exceptions, tool calls, and incident signals can be retained for audit, resilience exercises, post-incident review, and executive reporting. That matters because critical infrastructure operators need to show not only that AI was useful, but that it remained inside defined operating boundaries when service continuity was on the line.

For operators with mixed IT, OT, cloud, and vendor environments, the value is the shared record. Security teams, control-room leaders, resilience teams, risk owners, and audit can review the same AI activity trail and decide whether additional containment, reassessment, or recovery action is needed.

This research framing is also useful during exercises. Operators can test whether policy decisions, blocked actions, review gates, and fallback procedures are visible enough for incident commanders and executives to act quickly. That evidence can then feed resilience planning without turning every exercise into a manual reconstruction project. The result is a practical feedback loop between AI governance, incident response, operational continuity, and board-level resilience oversight for the services communities rely on. It keeps lessons visible after the exercise ends for teams, owners, auditors, resilience leaders, executives, and operators.

In critical infrastructure the highest-consequence moment is rarely the prompt; it is the action an OT-adjacent assistant, a defensive security copilot, or an agent takes near service restoration, incident response, or a control-room decision. Helixar sits in front of or in place of the AI gateway and enforces policy at that exact moment, across every model provider, so the same boundary governs internal copilots, embedded vendor AI, and agentic tool calls alike. At each action it verifies the identity and context of both the user and the agent, evaluates the requested action against policy, and applies a graduated response: it observes and logs a low-risk internal prompt, requires operator approval for an OT-adjacent recommendation, blocks an agent reaching for an unapproved tool or protected operational data, and contains when a boundary is crossed during a fast-moving incident. Because it is fail-closed by default and enforces organisation-wide cost caps, a degraded provider or a silent vendor model change cannot quietly expand autonomy, and every decision is recorded in a tamper-evident, independently verifiable evidence trail that feeds resilience exercises and post-incident review. That trail produces framework-aligned evidence packs, with SOC 2 and ISO 27001 available today and ISO 42001, EU DORA, PCI DSS v4, APRA CPS 234, RBNZ BS-11, and the NZ Privacy Act 2020 mapped and delivered at implementation.

Implementation roadmap

The first phase is inventory and classification. Identify AI use across planning, operations, cyber, maintenance, field service, customer operations, procurement, and vendor platforms. Map each use case to critical assets, essential services, data classes, vendors, human roles, autonomy level, and operational impact. Identify which use cases are read-only, advisory, decision-support, or action-capable.

The second phase is risk treatment. Define risk tiers, prohibited uses, authority boundaries, human approval requirements, fallback procedures, vendor obligations, evidence requirements, and incident criteria. Integrate AI risk into resilience, cyber, safety, vendor-risk, and SOCI-aligned risk management processes. High-impact or OT-adjacent uses should receive stronger review, testing, monitoring, and executive visibility.

The third phase is operationalisation and assurance. Deploy operational controls where needed, monitor KRIs, test incident and fallback procedures, review vendor changes, and retain evidence. Internal audit, risk, security, and operational leaders should test whether controls are designed and operating effectively. The roadmap should prioritise workflows where AI can affect service continuity, safety, or incident response.

Leadership questions

Boards and executives should ask which critical services depend on AI today. Which AI use cases affect service continuity, operational technology, incident response, customer communication, or safety-relevant workflows? Which vendors provide AI-enabled capabilities? Which systems have agentic tool access? Which use cases are outside appetite? Which controls are incomplete? Which AI dependencies are included in continuity exercises?

Operational leaders should ask whether humans remain in authority. Can operators override AI? Are approval points clear? Are fallback procedures tested? Are AI recommendations recorded? Are staff trained to detect unreliable output? Are model, vendor, and data changes reviewed before operational deployment? Can the organisation disable AI assistance without losing service continuity?

Risk, security, and audit teams should ask whether evidence is complete. Can they inspect the inventory, risk assessments, approvals, policy decisions, vendor reviews, incident records, exercises, exceptions, and reassessments? Can they link AI controls to critical asset risk? If the answer is no, the organisation is relying on assumptions where it needs evidence.