[lahfir]

Summary

An agent can now observe and identify real Windows applications: snapshot returns reffed trees with honest completeness, list-windows/list-apps/list-displays/focused_window are live with recycling-safe identity, and the dense-app cases actually work — Chromium/Electron trees are detected and settled, transparent web wrappers stop eating depth budget, drill-down (snapshot --root @ref) and skeleton (--skeleton) resolve fail-closed, and a Chromium modal is reachable as a sheet surface.

This sub-phase (2.4 of docs/phases.md) lands the full read path on Windows — the first time snapshot returns refs on this platform. It also ships the four P2-O8 evidence fields (subrole, role_description, placeholder, dom_classes) as optional core schema with Windows producers from named, measured sources — they were previously absent on every platform — and the --timeout-ms knob for cold Chromium settles, per probe A16-11.

Related issues

Related: none. Sub-phase branch per docs/phases.md Platform Delivery Model — this PR targets feat/windows-adapter, not main.

Type of change

• feat: — new feature

How tested

Read-path correctness was established by running it, not by the test suite alone (the sub-phase's carried principle): the release binary was pointed at Notepad, Explorer, the WinForms and WPF fixtures, and Obsidian (Chromium/Electron), and the JSON read and judged. The durable record is docs/dogfood-reports/2026-08-01-feat-windows-2-4-observation-dogfood.md. Verified live:

• snapshot returns reffed trees on all four UI stacks and Chromium; --skeleton clamps to depth 3 with reffed boundary counts; --root @ref re-resolves and drills down; a missing ref returns STALE_REF.
• list-windows / list-apps / list-displays return real inventories with corroborated process-generation tokens; the single 96-DPI display reads scale 1.0.
• Obsidian's settled tree reads 57 refs at full depth via the wrapper skip and no activation loop; the pre-activation shell shape was caught and fixed during the run (tree-end signal, not raw-depth ceiling).
• The two ref-able arms from 2.3: switch now resolves (fixture ToggleButton); cell still does not on WPF DataGrid cells (they present ControlType.Custom), with the measured reason recorded for the reader.

Gates run on the dev box (Windows Server 2019):

• cargo fmt --all -- --check
• cargo clippy --locked -p agent-desktop-core -p agent-desktop-windows -p agent-desktop -p agent-desktop-ffi --all-targets -- -D warnings
• cargo test --locked -p agent-desktop-core -p agent-desktop-windows --lib — 969 core, 299 windows
• cargo test --locked -p agent-desktop
• cargo test --locked -p agent-desktop-ffi --tests
• cargo check --locked -p agent-desktop-windows --all-targets --target x86_64-unknown-linux-gnu (cross-compile lane)
• probes/windows/13-ledger-check.ps1 (hunk index bijective)
• Release binary size 2.03 MB (12.9% of the 15 MiB cap)
• Note: bash tests/e2e/run.sh is macOS-only and not runnable off-macOS; the Windows read path was instead exercised by driving the release binary directly (the dogfood report).

The macOS lane runs on this PR's CI and is the detector for byte-identical goldens after the core schema change — the U2 unit asserts tests/fixtures/ is untouched.

Checklist

• PR title uses a conventional-commit prefix (feat: — lowercase, imperative, no capital after colon)
• cargo fmt and cargo clippy -D warnings are clean
• All tests pass
• No unwrap() in non-test code (a deny lint on the crate enforces it)
• No inline comments added — only /// doc-comments on public items
• All modified files are within the 400 LOC limit
• Skill docs updated for the new flags and output fields (skills/agent-desktop/references/commands-observation.md)
• README / other docs updated when the public interface changed (docs/phases.md corrected in place by U10; CONCEPTS.md gained Window Identity and Web Wrapper)

Session-settled decisions carried from planning: 2.4 planned as docs/phases.md defines it with contradictions corrected rather than planned around (user-directed); correctness established by running it rather than unit tests alone (user-directed); no test asserts a machine-specific or application-specific fact (user-directed). The plan's carries_target_text classification of LocalizedControlType/AriaRole as provider vocabulary (not withheld on secure fields) is the settled decision the reviewer flag on that point was reconciled against; the AriaRole author-text verbatim sub-question is recorded as a WebView2-capable-environment follow-up in probe A16-6.

Known Residuals

Recorded from code review, accepted as low-risk with rationale:

1. cell refinement on WPF grid cells is unsettled (P2, design). WPF DataGrid cells carry GridItem/TableItem but present ControlType.Custom (resolves unknown). Whether Custom + those patterns should refine to cell is a decision recorded in the dogfood report for the reader; not changed here because it involves the role-map contract, not the read path.
2. AriaRole/LocalizedControlType are not withheld on secure fields (P3, design, reconciled). The plan classifies them as provider vocabulary (carries_target_text=false, backed by A16-6's measurement that author tokens did not pass through), so they are not in VALUE_BEARING. A hypothetical hostile provider placing text in AriaRole would bypass the secure gate; the plan's settled classification stands and this is documented.
3. Resolver sibling-cap silent truncation (P3). The resolver stops searching a sibling list at the 10k cap with no incompleteness marker, yielding STALE_REF for a bounded search. Fails closed (never a wrong target); low probability (10k siblings on one parent). Left as-is for the 2.4/2.5 boundary.
4. list_apps/list_windows identity corroboration tightened but differs from macOS's full two-source merge (P3). Windows now fails the inventory when the window token disagrees with a fresh process read; macOS additionally merges a full app inventory. Windows 2.4's shape is documented as the shipped contract.

Post-Deploy Monitoring & Validation

No production/runtime impact — agent-desktop is a stateless CLI/adapter library; there is no server, daemon, or always-on service. The observable runtime surface (CLI JSON output) is covered by the unit/live tests and the committed dogfood report. No additional operational monitoring required; the macOS CI lane on this PR is the only always-running gate and it verifies the golden fixtures remain byte-identical.

[greptile-apps]

Greptile Summary

The PR implements the Windows observation read path, including live inventories, strict ref resolution, Chromium settling, surface discovery, and optional descriptor evidence.

• Adds Windows UI Automation tree observation and drill-down support.
• Adds live application, window, display, focus, and process-generation discovery.
• Threads snapshot timeout and renderer-accessibility options through the CLI and core.
• Extends node output with optional subrole, role description, placeholder, and DOM class fields.

Confidence Score: 4/5

The PR is not yet safe to merge because stored references can resolve unintended elements or recycled windows, and Chromium activation state can leak across FFI snapshots.

Evidence-free refs still accept a sole unrelated candidate, stored HWND resolution does not verify the handle's current owner, and the adapter-wide activation flag causes a reused FFI adapter to treat a new Chromium target as already settled.

Files Needing Attention: crates/windows/src/tree/resolve.rs, crates/windows/src/tree/resolve_match.rs, crates/windows/src/system/window_identity.rs, crates/windows/src/adapter.rs, crates/windows/src/tree/observe.rs

Important Files Changed

Filename	Overview
crates/windows/src/tree/resolve.rs	Implements stored-ref root verification, bounded traversal, candidate selection, and strict-resolution error handling.
crates/windows/src/tree/resolve_match.rs	Defines candidate identity matching and stale or ambiguous resolution errors.
crates/windows/src/adapter.rs	Connects Windows observation and inventory operations while retaining renderer-activation state on the adapter instance.
crates/windows/src/tree/observe.rs	Builds Windows observed trees and coordinates Chromium shell detection, settle behavior, and root re-verification.
crates/windows/src/system/window_identity.rs	Introduces strict and stored window identity verification using HWND, PID, process-generation, and title evidence.
crates/core/src/commands/snapshot.rs	Threads timeout and renderer-accessibility options into fresh snapshots and ref-based drill-down.
crates/core/src/node_descriptor.rs	Adds optional serializable descriptor fields while preserving the legacy output shape when absent.

Sequence Diagram

sequenceDiagram
    participant CLI
    participant Core
    participant Win as WindowsAdapter
    participant UIA
    CLI->>Core: snapshot options
    Core->>Win: observe_tree(root, request)
    Win->>UIA: enumerate and read properties
    alt Chromium shell requires activation
        Win-->>Core: activation required
        Core->>Win: activate renderer accessibility
        Core->>Win: retry observation
        Win->>UIA: settle and walk tree
    end
    Win-->>Core: ObservedTree
    Core-->>CLI: reffed snapshot JSON

Loading

_{Reviews (3): Last reviewed commit: "fix: strip remaining plan ids and versio..." | Re-trigger Greptile}

[greptile-apps]

Recycled HWND crosses window identity

When a source window closes and Windows recycles its HWND to another window in the same still-running process, this check accepts the unchanged process-generation token and resolves the current UIA root for the recycled handle, causing drill-down or a later action to target a matching element in the wrong window instead of returning STALE_REF.

Context Used: CLAUDE.md (source)

[greptile-apps]

Too many files changed for review. (101 files found, 100 file limit)

Bypass the limit by tagging @greptile-apps to review.