surface — targeting the desktop itself

Applications are not the whole screen. The menu bar, the taskbar/panel/dock, tray status items, the desktop underneath, and transient flyouts (notification centers, shell context menus) all live OUTSIDE any application’s tree. surface makes them a first-class targeting root: pass surface instead of name/pid to the addressing tools and their selectors resolve from the shell’s element tree.

surface::resolve(kind: SurfaceKind, timeout: Duration) -> Result<ShellSurface, String>
surface::resolve_with(provider, kind, timeout) -> Result<ShellSurface, String>   // injected
surface::surface_label(kind) -> &'static str                                     // human label

Consumers rarely call these directly — snapshot, act and screenshot accept surface as a scope field and route through here.

The model: the third root

The vocabulary — SurfaceKind, snake_case on the wire, mirroring xa11y’s ShellSurfaceKind exactly:

Flyouts exist only while open

The one timing rule worth memorizing: a flyout (Notification Center, Quick Settings, a shell context menu) has elements only while it is open. The caller performs the press that opens it on a real element first, THEN re-enumerates — which is also why a flyout snapshot can legitimately be empty.

Edges: platform honesty

Surface enumeration is only as rich as the backend. On macOS and GNOME-family desktops the full vocabulary is reachable. On X11 window managers like i3 the backend classifies only AT-SPI dock frames — a legitimately empty result, not a bug. That honesty is built into the errors: when enumeration finds nothing, the error carries a live census of what the platform DID report, so “0 surfaces” is diagnosable from the message alone (see errors).

// The taskbar's tree, same outline as an app snapshot:
computer.snapshot(&ComputerSnapshot {
    surface: Some(SurfaceKind::Taskbar),
    ..Default::default()
}).await?;

// Press a taskbar icon by selector:
computer.act(&ComputerAct {
    surface: Some(SurfaceKind::Taskbar),
    selector: Some("button[name='Files']".into()),
    ..Default::default()
}).await?;

See also

snapshot / act for the tools that consume it · cargo run --example computer-surface