NVIDIA OpenShell: 6 Critical SRE Checks After Agent Containment

NVIDIA launched its Open Agent Safety Platform on September 28, 2026. The design combines OpenShell, an open-source runtime that constrains what an autonomous agent can access and do, with NVIDIA Sentry, an out-of-band monitoring and enforcement design for BlueField-4 DPUs. NVIDIA launch announcement

Most of the immediate discussion is about whether those controls can stop an agent from escaping its intended permissions.

That is the right security question.

SRE has another one to answer immediately afterward:

What changed before the control stopped the agent?

A containment event can prevent the next action. It does not tell you whether the previous write succeeded, whether production state changed after approval, whether work remains queued, or whether the service actually recovered.

My production-agent rule is to authorize a bounded action against a defined state, then verify the outcome independently.

NVIDIA OpenShell makes that approach more enforceable. It also makes the post-containment path more important.

Security containment stops future execution. It does not reconcile past effects.

That is the operational problem worth solving.

What NVIDIA OpenShell Actually Controls

OpenShell runs an agent inside an isolated sandbox and applies policy outside the agent's own reasoning loop. NVIDIA's documentation describes kernel-level controls over filesystem access, system calls, processes, and outbound network connections. Each agent receives an isolated sandbox, while provider controls can keep real credentials outside the agent workload and attach them only to requests that pass the applicable authorization checks. OpenShell runtime and credential controls

OpenShell also includes a policy prover. Its guarantees apply to the features represented in its model, not to the safety of every possible agent behavior. Proposed policy changes can be formally checked for access they would introduce before they are applied. Risky expansion can therefore be surfaced for review instead of relying on the agent to determine whether its own new permissions are appropriate. OpenShell policy prover documentation

There is an important operational distinction inside the current policy model. Filesystem and process controls are startup-time controls. Network policy can be changed dynamically after successful validation, while current OpenShell documentation also describes provider attachments as runtime-updatable. OpenShell policy architecture

That detail matters during recovery because not every blocked capability can or should be added to an existing execution environment.

Sentry adds another enforcement layer. NVIDIA describes it as an out-of-band watchdog on BlueField-4 DPUs that continuously monitors agent behavior and can quarantine an agent that attempts to move outside its permitted boundary. NVIDIA says that containment can occur in milliseconds. That timing is NVIDIA's claim, not an independent AIOpsSRE benchmark. NVIDIA Sentry announcement

Sentry is also an optional layer in NVIDIA's reference architecture. OpenShell itself is open source and can be used outside the full Vera and BlueField design. NVIDIA layered reference architecture

That gives teams something valuable: enforcement that does not depend entirely on the agent deciding to behave.

It does not remove the need to operate what happens after enforcement.

A Blocked Agent Can Leave a Valid Change Behind

Consider a hypothetical Kubernetes operations agent.

The agent is authorized to scale checkout-api from 12 replicas to no more than 20 when an approved saturation condition is present. Its decision was evaluated while the Deployment had resource version 8841.

It sends:

PATCH Deployment checkout-api

current replicas: 12
requested replicas: 16
state precondition: resourceVersion 8841

The Kubernetes API accepts the write.

Before the agent verifies the rollout, it makes another outbound request that violates OpenShell policy. The request is denied, or an independent control quarantines the agent.

The security control worked.

The operational task is still unfinished.

At that point, the responder needs to determine whether the scale operation committed, whether 16 replicas are actually available, whether another controller or human changed the Deployment afterward, whether another action remains queued, whether the original approval still describes current state, and whether the customer-facing symptom improved.

Restarting the agent answers none of those questions.

Broadening the policy answers none of them either.

The next useful move is evidence collection.

1. Decide Whether the Control Event Is Operational

Not every OpenShell denial should wake an SRE.

If a coding agent attempts an unapproved write to an external API, the policy blocks it before any consequential action occurs, and the workflow can continue safely, the deny may simply prove that the control worked.

The response should change when containment intersects with production consequence.

Condition	Operational consequence
A consequential write may have happened first	Destination state may be unknown
The agent is quarantined	The workflow cannot safely continue without reconciliation
Required production work is interrupted	Reliability is now affected even if the security control succeeded
The agent requests unexpected additional authority	The task or policy may no longer describe actual behavior
The same unexpected deny repeats after correction	The workflow may be drifting or the policy remains wrong
Destination state cannot be inspected	Safe recovery cannot yet be established

This distinction prevents a safety control from becoming another alert-noise generator.

A deny count is useful telemetry.

It is not automatically a page.

2. Freeze New Writes Without Destroying Evidence

When containment follows a possible production side effect, stop new consequential actions from that workflow.

Preserve enough identity to reconstruct what happened:

containment_event:
  agent_id:
  sandbox_id:
  task_id:
  policy_version:
  event_time:
  blocked_operation:
  last_confirmed_operation:
  possible_inflight_operation:
  target:
  target_state_version:
  execution_id:
  queued_actions:

This is an SRE operating template, not an OpenShell-native schema.

The important property is continuity. The team needs to connect the security event to the exact agent execution and to any destination state that execution may have modified.

If safe read-only inspection can remain available while writes stay frozen, preserve it.

Losing observation at the same moment you stop execution makes the recovery problem harder than necessary.

This is where Evidence Before Motion matters. The first response should improve what you know before another action changes the scene.

3. Reconcile Destination State Before Retrying

The agent's local history is not authoritative for a remote side effect.

The destination is.

For the Kubernetes example, inspect the Deployment's current resource version, desired replicas, available replicas, rollout conditions, and relevant audit or controller evidence.

Different evidence requires different decisions:

Evidence	Next decision
Approved write is confirmed and remains current	Verify the service postcondition before another change
Write occurred, but state changed afterward	Reassess from current state
Destination proves the write did not occur	Decide whether a fresh attempt is still justified and authorized
Evidence is incomplete or conflicting	Preserve an explicit unknown state
Destination cannot be inspected safely	Hold further writes and assign the unresolved action to an owner

An empty lookup is not universally proof that nothing happened. Its strength depends on the destination's consistency and operation model.

That is why unknown must remain a real execution state.

A production agent becomes dangerous when it translates:

I did not receive proof that the write succeeded

into:

The write failed, so send it again

The same failure mode exists with ordinary automation timeouts. The implementation pattern is covered in unknown write reconciliation.

4. Reject Authority That Belongs to Old State

Containment can last seconds or minutes.

Production state does not wait.

A person may intervene. An autoscaler may act. Another controller may reconcile the object. A different agent may receive related work.

Suppose the original scale action was approved against:

replicas = 12
resourceVersion = 8841
requested replicas = 16

After containment, the responder finds:

replicas = 18
resourceVersion = 8897

The original request may have been appropriate when the Deployment had 12 replicas.

It is no longer the same decision.

Do not let the old approval silently become permission to operate against the new state.

Read current state, determine whether the original objective still exists, and obtain whatever fresh decision the workflow requires.

This principle applies regardless of where the model runs. A local model, a cloud model, and a highly capable model do not inherit authority merely from being available. Model execution and action permission remain separate decisions. See local models and agent permission boundaries for the deployment implications.

5. Resume With the Smallest Capability That Solves the Block

A blocked task creates immediate pressure to broaden the policy.

That is exactly when the change should become more specific.

If the agent legitimately needs read access to one additional endpoint, the response should not be unrestricted outbound access. Add the narrow destination, operation, and credential scope the task requires, then let the policy-validation path evaluate that change.

Current OpenShell behavior makes this technically relevant.

If the missing capability is network access, the network policy can be changed dynamically after validation.

If the agent now needs a filesystem or process permission that was not available when its sandbox started, the current OpenShell model treats those as startup-time controls. A new sandbox with a newly reviewed policy is therefore a materially different and often cleaner recovery path than trying to mutate the existing environment. OpenShell policy lifecycle

The smallest useful recovery can mean one endpoint instead of a network range, one allowed method instead of read and write access, one namespace instead of a cluster, one repository instead of an organization, or one reconciled operation instead of replaying the complete plan.

The objective is not to make the agent finish at any cost.

It is to restore legitimate work without turning the containment event into uncontrolled permission expansion.

6. Verify the Service Outcome Independently

A successfully resumed agent is not proof that the service recovered.

Return to the Kubernetes example. If checkout-api reaches 16 available replicas, that establishes something useful about the Deployment.

It does not prove why the action was taken or whether the original customer impact disappeared.

An independent verification might look like this:

verification:
  deployment_available_replicas: 16
  request_error_rate: recovered
  p95_latency: recovered
  synthetic_checkout: pass
  unexpected_policy_denials_after_resume: 0

Those are illustrative checks, not universal thresholds.

The important property is independence.

Verification must be capable of disagreeing with the agent.

If the Deployment is healthy but the customer path still fails, the operational task remains unresolved.

If the service recovers but the agent immediately reaches the same unexpected policy boundary, the execution path remains unresolved.

This is part of the larger production-agent problem described in authorizing actions and independently verifying outcomes.

Measure the State Between Containment and Recovery

OpenShell and Sentry make another operational interval visible: the time between intervention and verified recovery.

These are proposed SRE measures, not native OpenShell metric names:

Measure	Question it answers
Expected policy denials	Is policy enforcing known restrictions?
Unexpected policy denials	Are normal workflows hitting unplanned limits?
Quarantined agents	How often does execution reach stronger containment?
Unknown in-flight actions	How much consequential state still lacks a known outcome?
Reconciliation latency	How long does it take to establish what actually changed?
Stale-authority rejections	How often does production state move before execution or resume?
Repeat denial after resume	Did the recovery actually correct the control path?
Containment to verified recovery	How long until both service state and execution state are understood?

Operation IDs and other high-cardinality identifiers belong in controlled logs or traces rather than metric labels.

Do not make "zero denials" the reliability objective.

If a policy never rejects anything, the policy may simply be too broad.

A healthier objective is that expected denials remain cheap, unexpected denials are explainable, and unresolved consequential state never gets replayed blindly.

Put Post-Containment Recovery in the Runbook

A production OpenShell deployment should document what happens after enforcement, not only how policy is configured.

A minimal recovery record can look like this:

containment_runbook:
  classify:
    expected_or_unexpected:
    possible_prior_side_effect:
    production_work_interrupted:

  preserve:
    agent_id:
    sandbox_id:
    task_id:
    policy_version:
    last_confirmed_operation:
    possible_inflight_operation:

  reconcile:
    destination_source_of_truth:
    current_state:
    operation_evidence:
    result: confirmed | rejected | unknown

  authority:
    original_precondition:
    current_precondition:
    fresh_approval_required:

  resume:
    minimum_policy_change:
    new_sandbox_required:
    queued_work_disposition:

  verify:
    service_postcondition:
    control_postcondition:
    accountable_owner:

Rehearse the process in a synthetic or nonproduction workflow before the agent gains broader production authority.

A useful rehearsal should include containment before a write, containment after a confirmed write, an unknown write result, state movement while quarantined, a runtime network-policy change, a static policy change requiring a fresh sandbox, failed service verification, and a repeated deny after resume.

The important test is not whether the agent eventually completes its task.

It is whether the team can explain the state throughout interruption and recovery.

Avoid the Five Easy Recovery Mistakes

Immediate restart is dangerous when the first run may have produced a side effect. Policy expansion is dangerous when the reason for the requested access is still unclear. Quarantine does not prove that earlier actions did nothing. Agent-reported success does not replace destination and service verification. Finally, paging every expected denial converts a useful safety control into operational noise.

Those mistakes share one cause: treating containment as a terminal state.

It is not.

Containment stops an execution path. Recovery still has to establish what happened on that path.

When a Lighter Response Is Enough

The complete recovery procedure is unnecessary when evidence is strong that no consequential state could have changed.

For example, imagine an agent in a disposable test sandbox attempts a forbidden outbound connection before any write-capable tool is available. OpenShell denies the connection, no production work is affected, and the event matches the intended policy.

Record it. Fix the task or policy if appropriate. Do not manufacture an incident.

The heavier SRE response belongs where containment intersects with production authority, incomplete evidence, stale state, or customer impact.

That boundary keeps runtime security useful without turning every blocked action into ceremony.

Post-Containment Checklist

1. Classify the denial or quarantine as expected, unexpected, or unresolved.
2. Stop new consequential writes while preserving execution evidence.
3. Identify every operation that may have been in flight.
4. Read destination state instead of trusting the agent's local history.
5. Keep unresolved effects explicitly unknown.
6. Reject approvals tied to stale state.
7. Add only the minimum capability needed for legitimate work.
8. Create a fresh sandbox when a startup-time policy must change.
9. Dispose of queued work that no longer matches current state.
10. Verify the production outcome independently.
11. Confirm that the same unexpected control event does not recur after resume.
12. Close the event only when both production state and execution authority are understood.

Containment Is the Start of Recovery

NVIDIA OpenShell is a meaningful change in agent infrastructure because it moves important controls outside the model's own judgment. Sentry extends that architecture with an independent hardware enforcement domain for environments that deploy it. NVIDIA reference architecture

For SRE teams, the operational value depends on what happens next.

A security control can stop an agent in exactly the right place while leaving an earlier operation unresolved. A fast quarantine can still be followed by a duplicate write. A valid policy expansion can still be applied to stale state. A successfully resumed agent can still fail the customer outcome.

The recovery rule is therefore straightforward:

Preserve the evidence, reconcile what already changed, refresh authority against current state, then resume with the smallest safe capability.

Containment limits the next move.

Reliable recovery explains the last one.

Related AIOpsSRE reading

• Unknown write reconciliation
• Production agents: authorize the action, then verify the outcome
• Evaluate incident-response agents against service outcomes

Primary sources

• NVIDIA Sentry announcement
• OpenShell runtime and credential controls
• OpenShell policy prover documentation
• OpenShell policy architecture
• NVIDIA reference architecture
• OpenShell policy lifecycle
• OpenShell provider and credential documentation

Evidence and scope: Primary sources reviewed September 29, 2026. NVIDIA performance and containment claims are vendor statements, not independent benchmarks. The Kubernetes scenario, recovery templates and proposed metrics are illustrative SRE guidance, not native OpenShell schemas or reports of a production incident.