GitHub Copilot gains computer use for desktop workflows

GitHub Copilot can now operate desktop applications when the work has no suitable command-line, API or Model Context Protocol interface. The October 1 release puts computer use in public preview in the existing Copilot app and Copilot CLI on macOS and Windows. The new capability is desktop interaction, rather than a launch of the app itself.

For a platform engineer, that opens a possible route into a legacy administration console whose useful controls still live behind a graphical interface. Copilot can inspect application context and interact with its controls. The engineering question becomes which application and operation it is allowed to use, and how someone can establish what happened afterward.

How Copilot reaches the interface

GitHub's concept reference describes a combination of accessible application content, visual context and UI interaction. It also records limitations: an agent can select the wrong control, lose its place when a window changes or repeat an action. A structured API or command-line tool remains preferable where one is available, because it gives the task a more explicit input and response than locating controls on a changing screen.

Computer use is disabled by default. In the app, it runs in local sessions and is enabled under Settings, Computer Use; on macOS, the system also requires Accessibility and Screen Recording permissions. The feature does not turn a cloud coding session into a desktop operator, and the announced platform scope does not include Linux.

The CLI instructions use /computer on to enable the bundled integration and /computer show to inspect it. Managed subscribers also depend on their organization's CLI policy. A local preference cannot override that policy, so a missing capability needs an eligibility and configuration check before it is treated as an agent failure.

A test console with a known record

In a hypothetical exercise, a platform engineer asks Copilot to inspect an incident record in a GUI-only test console. The account uses synthetic data, and the requested record has a known identifier. The engineer can compare the application Copilot opens, the record it selects and the status it reports against that known target. This is an inspection exercise, not evidence of a completed production deployment.

A later test could allow a bounded update to that record. Our agent-action approach treats approval as permission for a particular action against a defined state. Here, that means specifying the test record and intended change, then checking the destination's actual state. Permission to operate the console alone cannot establish that the agent selected the right record or completed the update.

If an operation is interrupted after a click, the record may already have changed. Inspect the destination before asking for the same update again. If the console offers no reliable way to establish the result, keep the outcome unresolved and return to a human-controlled operation rather than treating another click as recovery.

Stopping work and ending access

The app instructions distinguish stopping an operation from ending its session. Stop or Esc interrupts current work; ending the session revokes application access granted to that session. Removing a saved application approval affects future sessions, including the app and CLI on that machine, but does not revoke access already granted to a running session.

Approval prompts depend on the active tool-permission settings or CLI permission mode. Saved approvals and managed deny rules also affect what can run, so the preview should not be described as requiring a fresh confirmation for every individual UI action. Review the mode and approved applications before a test, then use the control that actually matches the goal: interrupt the operation, end the session's access, or remove future approval.

Computer use can make a previously unreachable console accessible to an agent. The useful first result for an SRE team is narrower and observable: the intended test record was found, any permitted change can be read back, and the session's application access can be ended. That gives the team something concrete to evaluate before extending the workflow to consequential administration.