Skip to content

CHANGELOG.md

Latest commit

 

History

History
540 lines (306 loc) · 38.1 KB

CHANGELOG.md

File metadata and controls

540 lines (306 loc) · 38.1 KB
Raw

Changelog

1.1.0 (2026-11-02)

⚠ BREAKING CHANGES

  • The C ABI major version is now 7. ad_resolve_element_exact and ad_execute_ref_action_exact_with_policy always refuse with ACTION_NOT_SUPPORTED (details.kind: exact_entry_unsupported): a C caller cannot supply the capture proof a ref now resolves by, so act through a snapshot ref instead. (#442) by @lahfir
  • The response envelope moves to 2.6. On Windows and Linux, a ref to one of several identical controls, a ref from a window the snapshot could not read whole, and a ref from an earlier snapshot now return AMBIGUOUS_TARGET with details.kind: identity_not_unique where some used to act. A snapshot that could not read its window whole to prove its refs now says so with data.identity_unproven: true. See the version 2.6 migration in docs/json-output.md. (#442) by @lahfir
  • A surface kind the OS does not have now returns INVALID_ARGS instead of PLATFORM_NOT_SUPPORTED, and names the kinds to use. FFI callers see the matching AdResult code. An adapter that has no snapshot surfaces at all still returns PLATFORM_NOT_SUPPORTED. (#480) by @lahfir

Features

  • Reach the macOS Dock and menu bar status items with snapshot --surface dock and --surface menu-bar-extras. With --app, menu-bar-extras reads that app's own status item. open-system-surface returns either surface at once, headless, with no window in its data. (#480) by @lahfir
  • Observe a macOS background app that has no window, such as a menu bar app, the Dock, or the Control Center and Notification Center panels, through its process. Its snapshot leaves out data.window. --app also matches an app's executable name or bundle ID, so --app ControlCenter works. (#480) by @lahfir
  • Add type --interval-ms to type one keystroke every N ms (1 to 2000), verified against the field after the last keystroke. (#348) by @ctate
  • Show the insertion caret in the cursor overlay after type and set-value, on macOS and Windows. (#349, #350) by @ctate
  • Draw the cursor label as a black pill sized to its text, on macOS and Windows. (#357) by @ctate
  • Colour the cursor body with the accent by default. --fill sets a different body colour. (#361) by @ctate
  • Draw --image and --pointer-image cursors on Windows. (#345) by @ctate
  • Start a headless action's cursor travel before it takes the interaction lease, so the cursors of parallel agents move at the same time. (#347) by @ctate

Bug Fixes

  • Remove the macOS command host. Every CLI call runs in its own process, as on Windows and Linux. The npm install and every CLI call stop any command host an earlier version left running. After an install that skips npm scripts, or a manual install, run agent-desktop version once. (#414) by @lahfir
  • Refs from snapshots taken before this release carry no capture proof, so take a new snapshot. On Windows and Linux they return AMBIGUOUS_TARGET with details.kind: identity_not_unique; on macOS a ref still resolves while the element its token names is alive. (#442) by @lahfir
  • A ref does not act on a different element after the UI changes. On macOS a ref follows its own element: a ref to a removed row returns STALE_REF, and identical controls can each be used by ref. On Windows and Linux a ref resolves only if it was unique when captured and exactly one element matches it now; a ref to one of several identical controls returns AMBIGUOUS_TARGET. Identity is the element's role and stable fields, not its instance: an identical element that replaced the original is acted on, and a toolkit that keeps a removed widget listed (GTK4, Qt6 on Linux) can still receive the action. (#441, #442) by @lahfir
  • On macOS, a --headed click on a status item or on a background app's control sends a real click, without first focusing a window the app does not have. (#480) by @lahfir
  • On macOS, a press whose return code proves nothing counts as delivered when focus moves, or the focused element changes, inside the pressed element's window. Notes' New Note no longer reports ACTION_FAILED after creating the note. (#480) by @lahfir
  • On macOS, set-value on a stepped control re-reads the value after each step before giving up, and says when the steps cannot reach the target. (#480) by @lahfir
  • Keep the overlay caret on the insertion point after key presses. On macOS, place it on an indented new line. (#353, #354) by @ctate
  • On macOS, draw overlay controls as they arrive, hide a cleared label, and stop clipping the click highlight's glow. (#346, #352) by @ctate
  • On Windows, stop every command ending in a crash report and about 3 s of error reporting at exit. (#342) by @ctate
  • On Windows, type CR LF as one line break. (#368) by @Railly
  • On Windows, stop scaling the overlay's coordinates twice, which sent the cursor off screen at 200% scaling, and stop the overlay flashing a black square when it starts. (#343, #344) by @ctate

1.0.2 (2026-10-30)

Features

  • Tell npm users when the Node launcher slows each call. When npm blocks install scripts, the launcher prints a one-time notice to stderr with the command that links the native binary.

Bug Fixes

  • Make permissions --guide on macOS offer the app that launched agent-desktop, such as a terminal that LaunchServices does not know, instead of the agent-desktop binary.
  • Restart the command host after it reports a missing permission, so commands succeed after the user grants access.
  • Point the permission hints and skills at permissions --guide instead of --request.
  • Point the npm package and repository links at vercel-labs/agent-desktop.

1.0.1 (2026-10-27)

Bug Fixes

  • Remove the version line from the skills. In 1.0.0 it still said 0.11.1; agent-desktop version reports the version.
  • Publish npm and GitHub releases automatically when the Cargo workspace version changes on main.
  • Remove the custom Homebrew tap, the Scoop bucket, and their package-update pull requests.

1.0.0 (2026-10-27)

Release

  • Add Linux support: the CLI, the C library, and the npm package now run on x86_64 and aarch64 Linux (glibc 2.35 or later) through AT-SPI.
  • Release the integrated macOS, Windows, and Linux CLI and C library as 1.0.0.
  • Bundle native CLI executables and platform helpers in npm installations, including installations with lifecycle scripts disabled.
  • Verify release archives, checksums, and build provenance through the manual release workflow.

0.10.1 (2026-10-18)

Bug Fixes

  • windows: let launch take a bare name without .exe (#276) (483427e)

0.10.0 (2026-10-18)

⚠ BREAKING CHANGES

  • the release carries these contract changes.
    • The C ABI major version is 5. The legacy entrypoints that each have an *_exact replacement are removed: ad_list_surfaces, ad_surface_list_count, ad_surface_list_get, ad_surface_list_free, ad_list_windows, ad_window_list_count, ad_window_list_get, ad_window_list_free, ad_launch_app, ad_release_window_fields, ad_get_tree, ad_find, ad_is, ad_focus_window, ad_window_op, ad_resolve_element and ad_execute_ref_action_with_policy, with the AdSurfaceList and AdWindowList handles. AdScreenshotKind loses WINDOW (FULL_SCREEN keeps 2 and 1 is rejected); AdScreenshotTarget loses pid; AdExactSurfaceInfo is version 2, 48 bytes, with a trailing unclassified field; the snapshot_id argument of ad_execute_by_ref and ad_execute_by_ref_timeout and the AdWaitPredicate.snapshot_id field are removed, so AdWaitPredicate is 40 bytes and AdWaitArgs 104. Consumers must rebuild against the regenerated header.
    • Bare @eN ref input is removed from every ref-taking CLI command, wait --element, snapshot --root, find --root, drag --from/--to, batch entries and the FFI. Only the snapshot-qualified @<snapshot_id>:eN is accepted, and a bare ref fails with INVALID_ARGS. The --snapshot flag that only qualified a bare ref is removed from every command, and batch entries carrying a snapshot or snapshot_id field are rejected as unknown fields.
    • get --property text no longer returns the same value as --property value. It prefers the value for textfield, combobox, listbox, datefield and timefield and falls back to the accessible name; every other role prefers the accessible name and falls back to the value. Blank strings count as absent. Ask for --property value explicitly to keep the previous answer.
    • An io error whose kind is PermissionDenied answers PERM_DENIED instead of INTERNAL, in the CLI envelope and in the FFI envelope, return code and last error alike.
    • A ref action whose element is gone no longer ends in TIMEOUT. A non-retryable resolution failure answers at once; a retryable one keeps polling, and if no later resolution succeeds the action answers the recorded STALE_REF or AMBIGUOUS_TARGET at the deadline.

Features

  • support Windows 10 1809+ and Windows Server 2019+ on x64 and ARM64 (483a31d)

0.9.4 (2026-09-23)

Bug Fixes

  • sync cargo lock with the 0.9.3 release (#217) (8c1502e)

0.9.3 (2026-09-22)

Bug Fixes

  • harden native accessibility targeting and delivery (#210) (882c4c4)

0.9.2 (2026-09-17)

Bug Fixes

  • make headless desktop actions report what actually happened, and add jev-desktop (#206) (0691eba)

0.9.1 (2026-09-13)

Bug Fixes

  • stop the inspector shutdown test failing on an expected reset (22cad9e)
  • drag: scope --wait-for to the drop target by default, with --wait-for-scope from|to (cefc80a)
  • preserve wait_timeout envelope when an inherited deadline expires (07913a7)
  • prevent wait --count false positives on incomplete snapshots (319e9c6)
  • split acronym boundaries in trace key redaction (3042e0c)
  • cursor: gate agent_id stamping on multi-agent mode (58612ea)
  • skip ended-session batch entries before wait baseline pre-capture (899ce27)
  • restore cursor overlay lifecycle for keyboard and clipboard actions (c827cf9)
  • clear remembered cursor landing when the overlay is hidden (6adff7a)
  • reject percent-encoded NUL in file_url_to_path (de5df38)

0.9.0 (2026-09-12)

⚠ BREAKING CHANGES

  • the response envelope moves to 2.4. Stateful ref actions can now return ACTION_FAILED for an observed postcondition contradiction, and ActionResult::from_execution takes only action and steps.

Features

  • verify stateful ref actions against fresh platform state (9bc91bc)

Bug Fixes

  • guard the debug viewer image sink and unpin a flaky test deadline (fca24f2)
  • report unread expand state as unverified rather than failed (a7131c2)

0.8.5 (2026-09-06)

Features

  • combine multi-agent cursors with core and macOS optimizations (#171) (922bb7d)

0.8.4 (2026-08-28)

Features

  • rebuild the agent cursor overlay and fix seen-but-unactionable elements (#145) (5ec2504)

0.8.3 (2026-08-22)

Features

  • harden native automation and add cursor overlay (#137) (25a0087)

0.8.2 (2026-08-20)

Features

  • relocate the state root with AGENT_DESKTOP_HOME (#135) (a336b01)

0.8.1 (2026-08-14)

Features

  • drive Chromium apps through a verified DevTools endpoint from launch --cdp (#129) (366d348)

0.8.0 (2026-08-13)

⚠ BREAKING CHANGES

  • the response envelope is version 2.3. launch returns { app, pid, process_instance, window? } instead of a bare window object, and an application that presents no window is ok:true with window omitted rather than WINDOW_NOT_FOUND. The C ABI is unchanged: it still writes one window and reports WINDOW_NOT_FOUND when there is none.

Features

  • decide macOS delivery by observation and stop launch waiting on an uncaused event (#125) (298f1ff)

0.7.0 (2026-08-02)

⚠ BREAKING CHANGES

  • ENVELOPE_VERSION is now 2.2. data.complete is present on every successful snapshot, and a snapshot that exhausts its budget returns ok: true with complete: false where it previously returned a TIMEOUT error. Callers that branched on TIMEOUT to detect an oversized tree must read complete instead.

Bug Fixes

  • return observed trees and stop demanding renderer activation from shallow walks (#117) (32175e4)

0.6.0 (2026-07-26)

⚠ BREAKING CHANGES

  • private artifacts on Windows are no longer written through ACL-hardened handles. That hardening guarded refmap, trace, and session files on a platform where no command can produce them, and it had never executed. It returns in Phase 2.1, built on Windows against a CI lane that runs it, under the constraints recorded in docs/solutions/best-practices/never-ship-platform-code-that-ci-cannot-execute.md.

Refactoring

  • remove speculative Win32 private-file layer from core, add real Windows/Linux test lanes (#106) (8ad66b8)
    • deleted 1,062 LOC of Windows-only unsafe Win32 file I/O from agent-desktop-core and dropped the windows-sys dependency; Windows now uses the same portable std::fs path as every other non-unix target
    • this is a structural fix, not a downgrade: std::fs::OpenOptions defaults share_mode to FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, precisely the flag the deleted code omitted and the cause of its sharing-violation failures
    • cargo test now runs on windows-latest and ubuntu-latest, so every platform-conditional branch in core is executed on each PR instead of only compiled

Bug Fixes

  • the new platform lanes exposed four pre-existing defects that were unreachable while only macOS ran tests:
    • trace files were opened append-only while trace.rs locks that handle; Windows LockFileEx rejects append-only handles, so all trace writing failed with Access is denied
    • refs_lock tests placed lock files directly in /tmp, which is root-owned and world-writable, so the private-parent check correctly refused them
    • a trace test interpolated a raw path into a JSON string literal, so Windows paths produced invalid escapes and the event was silently dropped instead of skipped
    • the default SystemOps::permission_report returned Denied, so CLI preflight reported PERM_DENIED on Windows and Linux when the adapter is simply unimplemented; it now reports Unknown and surfaces the honest PLATFORM_NOT_SUPPORTED, including over the FFI
  • the pinned toolchain gained clippy and rustfmt, which profile = "minimal" had omitted

0.5.0 (2026-07-20)

⚠ BREAKING CHANGES

  • default-on auto-wait changes the timing of every previously-untouched ref-action call (bounded 5000 ms default; --timeout-ms 0 restores single-shot). ENVELOPE_VERSION is now 2.1 (adds the APP_UNRESPONSIVE code and process state in error details). FFI ABI major is 3 (append-only struct evolution; wait --event is intentionally not exposed over FFI). The legacy string clipboard API is removed in favor of typed content. key-down/key-up fail closed until daemon-owned held input exists. close-app verifies termination and the osascript fallback path is removed. --text matching is subtree containment: find --text X --first returns the outermost matching container.

Features

  • implement Playwright-grade foundation contract (3f32272)
    • default-on auto-wait: every ref action waits for actionability (visible, enabled, stable, unoccluded) before dispatching, under a bounded budget
    • live find locator plus a serializable LocatorQuery, and honest is --property visible
    • three-way hit_test occlusion gate and core scroll_into_view before element actions
    • list-displays and honest --screen with per-display scale factor; native_id identity spine; window-id-first resolution
    • ProcessState classification with the APP_UNRESPONSIVE code (envelope 2.1); typed ActionStep delivery tier
    • LaunchOptions (--arg/--env/--cwd/--no-attach); baseline-diff desktop signals via wait --event
    • typed clipboard (Text/Image/FileUrls); mouse modifier chords and a mouse-wheel primitive
    • capability-supertrait PlatformAdapter split with not_supported() defaults, so Windows and Linux inherit the contract

0.4.7 (2026-07-02)

Features

  • add trace viewer and replay artifacts (e3e1872)

0.4.6 (2026-07-02)

Features

  • make sessions the first-class trace container (35fa914)

0.4.5 (2026-06-30)

Features

  • add --wait-for selector polling flags (#86) (ce23278)

0.4.4 (2026-06-29)

Features

  • macos,core: harden adapter and core foundation with caller-controllable guardrails (#82) (94ce6c5)

0.4.3 (2026-06-28)

Bug Fixes

  • macos: harden retained_handle null guard against release-only CFRetain(null) (#80) (a708fa0)

0.4.2 (2026-06-27)

Features

  • ffi: Phase B and C — Python smoke harness, parity gates, build.rs codegen (#77) (9023f33)

0.4.1 (2026-06-26)

Features

  • complete FFI C-ABI surface (Phase A): load-time ABI handshake, session-scoped adapter, JSON-envelope command entrypoints (version, status, snapshot, wait, execute-by-ref), an optional tracing log callback, and a unified error-envelope contract (#67)

0.4.0 (2026-06-24)

⚠ BREAKING CHANGES

  • the version command no longer accepts --json; it always emits the standard JSON envelope.

Refactoring

0.3.1 (2026-06-21)

Bug Fixes

  • harden macos stale ref resolution (#62) (9f144c2)

0.3.0 (2026-06-20)

⚠ BREAKING CHANGES

  • ffi: AD_POLICY_KIND_PHYSICAL is now AD_POLICY_KIND_HEADED (discriminant 2 unchanged).
  • ffi: AdRefEntry and AdDragParams include additional reliability metadata; C ABI consumers must zero-initialize and validate with AD_REF_ENTRY_SIZE, AD_DRAG_PARAMS_SIZE, and the ad_*_size() accessors.
  • close-app graceful responses now return { "method": "graceful", "requested": true } instead of claiming closed: true before the app has exited.

Features

  • add Playwright-grade ref reliability with strict late resolution, session-scoped snapshots, deterministic stale/ambiguous target handling, and skeleton drill-down preservation (#54).
  • add Playwright-style headed/headless interaction policy: accessibility-first dispatch remains default, while --headed enables focused physical fallbacks only when needed (#54).
  • add retrying wait predicates, JSONL traces with secret redaction, richer actionability reports, and shared CLI/FFI execution through the same resolver/actionability/dispatch ladder (#54).

Bug Fixes

  • harden macOS ref actions across focus/window raising, disclosure expansion, scroll, drag, numeric set-value, menu detection, stale-ref recovery, protected-process errors, and close-app confirmation (#54).
  • fail closed on unsafe fallback ambiguity and stale refs, including symlink-safe refstore/latest-snapshot reads and deadline-bound resolver work (#54).

Performance

  • reduce repeated live actionability reads and add deadline-bound resolver behavior, with large-app snapshot and per-command CLI wall-clock gates in the E2E suite (#54).

Tests

  • add real-app E2E coverage for headless/headed ref actions, sessions, snapshots, traces, waits, skeleton drill-down, menus, surfaces, drag, expand/collapse, and performance checks (#54).

0.2.3 (2026-06-06)

Bug Fixes

  • harden macos ax window fallback (3b266fd)
  • resolve fullscreen AX tree retrieval returning ref_count: 0 (a52b7c7)

0.2.2 (2026-06-02)

Bug Fixes

  • macos: guard CFArray casts with type-ID check (fixes Mail.app crash) (#50) (c02cb5e)

0.2.1 (2026-05-23)

Bug Fixes

  • stabilize empty accessibility identity refs (1fb5a7d)

0.2.0 (2026-05-20)

⚠ BREAKING CHANGES

  • chain execution deadlines now return TIMEOUT instead of ACTION_FAILED when the target app does not respond before the chain deadline.

Refactoring

  • unify command execution contracts (1291a9c)

0.1.14 (2026-05-04)

Features

  • bundle skill docs and refactor --help for AI agents (#36) (b04d6f9)

0.1.13 (2026-04-17)

Features

  • ffi: ship C-ABI cdylib with review fixes and release pipeline (#26) (3cffbd6)

0.1.12 (2026-04-16)

Features

  • progressive skeleton traversal with ref-rooted drill-down (#20) (c17f2fa)

0.1.11 (2026-03-03)

Bug Fixes

  • show skill install prompt on all success paths (39b2bc6)

0.1.10 (2026-03-03)

Bug Fixes

  • fix skill publication authentication in CI (208af12)

0.1.9 (2026-03-03)

Features

  • scalable skill architecture with publication automation (#14) (9766520)

0.1.8 (2026-03-01)

Features

  • add electron/web app compatibility for accessibility tree traversal (a19c1b5)
  • implement --compact flag to collapse single-child unnamed nodes (4a300c8)

0.1.7 (2026-02-28)

Features

  • add notification command types, adapter trait, and CLI wiring (c5b05ba)
  • add notification management commands (macOS) (b1fd368)
  • macos: add NC session RAII guard and notification adapter wiring (0d55c21)
  • macos: implement dismiss and notification action commands (53d697d)
  • macos: implement notification list via AX tree traversal (53549a3)

Bug Fixes

  • macos: remove AXPress from dismiss action list (27ef4f3)
  • macos: restore frontmost app after notification center interaction (3881bc8)
  • macos: use pgrep and async osascript for NC lifecycle (9797585)

0.1.6 (2026-02-24)

Bug Fixes

  • handle null bounds in refmap and improve sidebar click resolution (d4197e8)

0.1.5 (2026-02-23)

Features

  • add fallback chains for set-value, clear, focus, scroll-to, type and post-action state hints (11f8da0)
  • add structured verbose logging across all layers (c7316e8)

Bug Fixes

  • add dwell time before drag release for drop target recognition (2a52d62)

0.1.4 (2026-02-23)

Features

  • add agent-desktop skill for universal AI agent support (ef45135)
  • add Claude Code skills for agent-desktop automation (ad91cd3)

0.1.3 (2026-02-23)

Bug Fixes

  • correct GitHub Release download URL and simplify tag format (8f66a93)
  • include README and CHANGELOG in npm package (084fc8c)

Performance

  • use curl for binary download in postinstall (ebafb71)

0.1.2 (2026-02-23)

Bug Fixes

  • use macos-latest for both build targets (91c7677)

0.1.1 (2026-02-23)

Features

  • 10-step scroll chain, focus guards, enhanced click chain, bounds fix (595ccb6)
  • add 19 new commands, AX-first rewrites, LOC compliance (d3f7e03)
  • add 19 new commands, AX-first rewrites, LOC compliance (eca04e8)
  • add release automation with GitHub Releases and npm distribution (18fc50c)
  • add structural hints to splitter columns in snapshots (48f8470)
  • AX-first right-click chain with inline context menu capture (cddc5d3)
  • Phase 1 foundation — workspace scaffold, core engine, macOS adapter, 31 commands (a346f24)
  • smart AX-first click chain + macOS crate restructure (4616c8f)
  • surface-targeted snapshot, menu wait, list-surfaces command (39178b2)

Bug Fixes

  • add menubar surface, fix press --app crash and modifier mapping (a231962)
  • address code review findings (double-free, CF leaks, injection) (2f495ff)
  • align error codes with spec (APP_NOT_FOUND, PERM_DENIED) and add -i shorthand (6dc567a)
  • ancestor-path cycle detection + CGEvent click fallback (198d7d7)
  • detect open menus via AXMenuBarItem.AXSelected, not AXMenus attribute (7f0d610)
  • make all 30 commands work end-to-end on macOS (1d98ab8)
  • remove AXShowDefaultUI from activation chain, fix child walk (74242f5)
  • resolve all 47 code review findings from Phase 1 audit (218503a)
  • right-click uses AXShowMenu; context menus detected via focused element (2c9aee3)
  • suppress dead_code lint on BatchCommand deserializer struct (608d4aa)
  • use simple release strategy for workspace version bumps (0ab78dd)