Skip to content

fix: resolve refs only through a unique recorded identity - #429

Closed
lahfir wants to merge 51 commits into
laf/remove-macos-hostfrom
laf/identity-review-fixes
Closed

lahfir wants to merge 51 commits into
laf/remove-macos-hostfrom
laf/identity-review-fixes

Conversation

@lahfir

@lahfir lahfir commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator
  1. On macOS, a ref to an element whose identifier it shares with others, such as a row in a list, could act on a different element after the list changed.
  2. Each ref now records which identifier, name or description proved it unique when captured, and resolves only when a complete live count finds exactly one match; otherwise it returns STALE_REF or AMBIGUOUS_TARGET.
  3. Builds on fix: remove the macOS command host #414. Replaces fix: resolve refs only through a unique recorded identity #427 and test: prove identity resolution on every platform #428.

Before

After the Alpha row is filtered out, set-value on its ref writes "poisoned" into the row that took its place.

before.mp4

After

The same command is refused with AMBIGUOUS_TARGET, and the remaining row keeps "Beta".

after.mp4

lahfir added 30 commits October 31, 2026 20:41
The churn scenario now relaunches the fixture per case and checks the
fixture readouts for removal, replacement, swap and duplicate-identifier
cases. Host-kill assertions are gone, and the removal case accepts
STALE_REF or AMBIGUOUS_TARGET.
Ref entries gain an optional basis that records which snapshot evidence
singles the entry out from look-alikes: a unique native id, a unique name or
description, ambiguous, or positional. Every constructor sets it to None for
now; the allocator and matcher start reading it in the following commits.
basis_gate refuses a ref before any live search when its basis cannot
single out one element: an entry recorded before bases existed returns
STALE_REF and an ambiguous one returns AMBIGUOUS_TARGET with kind
identity_not_unique. requires_live_uniqueness and bounds_may_decide tell
every platform resolver when a complete live count is required and when
geometry may decide, so the three resolvers apply one rule.
Without the command host, a ref to a removed row could resolve to a
surviving look-alike that shared its identifier, path and bounds.

Snapshots now record each ref's identity basis: a native identifier unique
in the snapshot, or a name or description unique among the elements that
share its identifier (or, for id-less elements, its role). Ambiguous refs
and refs from incomplete snapshots record Ambiguous; refs with no evidence
record Positional. Look-alikes are counted across the whole tree, so
cousins under separate rows are compared.

At action time macOS, Windows and Linux call the shared basis gate, then
count live candidates in the source scope with the full predicate. Exactly
one resolves, none returns STALE_REF, several return AMBIGUOUS_TARGET, and
a count cut short by a budget or an unreadable subtree never resolves.
Bounds never pick or confirm a candidate for a ref that carries identity;
only positional refs keep the path-and-bounds rules. Values never identify
an element. Refmaps written before this change return STALE_REF.

The live count walks the source window instead of following the stored
path, so identity-based actions read more of the tree; latency is measured
separately.
The first native E2E run showed find refusing its own matches. The macOS
locator-anchor path applied the stored-ref basis gate, hydration rebuilt
entries with an unproven basis, and the locator survey treated any element
with an unread field, including a value, as making every proof incomplete.

A locator anchor is now found again by the fields and path it was observed
with in the same call, on every platform, and an unread live field still
never resolves. A hydrated match keeps the basis its survey proved, and the
survey withdraws only the proofs that rest on a field an element left
unread. Refs that find returns resolve later through the normal identity
rules.
CONCEPTS, AGENTS, the JSON contract, the cross-platform skill and the
identity solution note now say how refs re-identify after the command host
removal: a ref resolves only to the one live element matching its recorded
identity, AMBIGUOUS_TARGET with kind identity_not_unique covers look-alikes
that were never unique, pre-upgrade refmaps return STALE_REF, and find
--value is the way to act on current content. The pinned-namespace guard
messages and the permission guide doc no longer name a command host.
A labeled pair that shares an identifier resolves through its unique
name, accepts repeated writes, and returns STALE_REF once its own field
is removed even though the look-alike now sits in its place. An unlabeled
look-alike is refused before any change.
The second native E2E run showed every ref that find returned recorded as
ambiguous, so the next action through it was refused. Selected-match
evaluation never counted look-alikes, and any unread field anywhere in the
survey withdrew every proof.

Selected matches now take their basis from the same survey count the full
refmap uses. An unread field withdraws only proofs for elements of the same
role, and with identifiers unread a name or description unique across the
whole role still proves itself, since the live predicate is never broader.
A name query also reads values so a text field's stable name can be
decided; role-only queries keep their narrow reads.
Snapshot allocation reads each entry's basis from the completed counts
instead of a second traversal and map. The locator survey counts identity
fields without building whole ref entries. Same-call anchor confirmation no
longer clones the anchor, since it never reads the basis. The Linux
resolver reuses core's typed-identifier matcher and the FFI ref decoder
reuses the existing string decoders, with the same limits and errors.
On macOS 26 a set-value changes a field's own accessibility value but not
the SwiftUI binding behind its readout, so readouts alone cannot show a
wrong-target write. Every refusal now compares the live values of all text
fields before and after, and positive writes check the written field's own
value.
A ref must never resolve to an element other than the one it was captured
from, so every proof the capture cannot make now records Ambiguous or is
refused at action time.

- Matching compares each basis only with its recorded field and requires
  every recorded stable field and identifier, in resolution and in the
  actionability check.
- Snapshot projection keeps which fields were unreadable, skeleton and
  shallow observations no longer count as a full census, identifier-free
  text identities are counted across the whole role, and every typed
  identifier a live match would accept is counted.
- Locator proofs from an element-rooted survey are withheld, unreadable
  roles withdraw proofs before addressability filtering, and hydration
  rechecks every observed field and keeps an unproven basis when it adds
  identity the survey never proved.
- macOS counts identity matches only in the captured window. Linux counts
  every identity after the menu-context check and runs the basis gate
  before geometry. Windows matches the snapshot depth boundary and stops at
  a proven duplicate.
- A unit test proves the duplicate verdict never consults bounds.
A second field with the same identifier and label at a different position
must make the original ref ambiguous rather than letting bounds choose it,
and a ref to an id-less field must not land in a same-named field that has
an identifier once its own field is removed. Effects are proven from live
field values.
- A snapshot records the AX window it actually observed and refuses a
  pinned window whose focused surface belongs to another window.
- A same-call locator anchor is confirmed by its identifier and every
  observed stable field, compared field by field.
- Linux counts every unique basis with the shared full predicate, keeps the
  menu-context check separate, and propagates unread evidence.
- Windows no longer lets a recorded value narrow the live count, stops
  traversal at a proven duplicate, and uses one depth rule for traversal.
- Capture counts every node a stored ref could match, not only nodes that
  receive refs.
- Doc comments added on this branch are cut to one or two sentences.
The focused-surface capture now goes through one owner decision that is
tested for a matching, foreign and unreadable owner. The Windows depth
boundary fixture seeds every ancestor with a known non-matching type, so it
fails only if the depth rule regresses.
A field whose name mirrors its value was left out of the census, so a
look-alike could win a name basis and match after a value change. Every
observed name now counts; only the target still needs a stable name.
Hydration kept the counted basis but could swap in a newly preferred
identifier that no whole-scope count had checked. It now keeps the
proven identifier whenever it keeps the proven basis.
A focused capture counted only the sheet, while resolution searched the
whole owner window, so a match outside a closed sheet could take its
ref. Capture now traverses the owner window, the scope resolution uses.
The uniqueness search mapped E_POINTER, E_INVALIDARG and
UIA_E_NOTSUPPORTED to settled absence, so it could resolve a ref
without reading the subtree holding a look-alike. Only the end of the
list now completes a count; other failures leave it unread.
A skeleton ref recorded no unique identity, so --root refused it, and
every drilled ref was marked ambiguous. A drill root now resolves when a
complete live count finds exactly one match. Drilled refs count only
their observed subtree, and acting on one resolves its root first, then
counts the target inside it: a twin outside the root is ignored, a twin
inside returns AMBIGUOUS_TARGET.
Focused capture verified the owner window, then looked it up again by
id with a title fallback, which could return another window with the
same title. It now walks the verified handle itself.
When hydration could not read a text field's value, it still saved the
name, which may mirror that value. A later read of the unchanged field
then contradicted the saved name and returned STALE_REF. The name is
left out unless the survey proved it.
The FFI exact find did not request value evidence, so a unique text
field came back ambiguous, and FFI basis derivation treated a name that
mirrors the value as stable. Find now requests the same evidence as the
CLI and the basis uses core's stable_name.
Scoped resolution compared only role, bounds and text, so a replaced
element with another retained object at the same place resolved. It
now compares the retained object at selection and after hydration.
find --root counted every element-rooted census as incomplete and
dropped the root chain from its refmap; a complete subtree now counts
and the root chain is copied in.
Drilling promoted an ambiguous skeleton ref to a unique basis from a
census of one entry, so a surviving twin could be selected. A skeleton
anchor now records the depth its census covered and resolves only by a
live count over that same depth: a deeper twin is never counted on
either side, so a removed anchor is STALE_REF. Screenshot preparation
and legacy tray refs now go through the shared resolver and its gate.
lahfir added 11 commits October 31, 2026 21:03
Following a moved root through the window added four ways for drilled
and rooted refs to go stale or misread paths. A root that left its
captured position now returns STALE_REF with a suggestion to drill it
again, and the count over omitted children stays incomplete. Selected
locator entries also withhold an unverified mutable name, like every
other path that stores a basis.
Boundary probes treated E_ACCESSDENIED as an unread region, so it ended
as TIMEOUT after retries instead of PERM_DENIED; they now share the
enumeration's verdict and propagate terminal errors. Windows test
fixtures without an identity basis hit STALE_REF before the adapter.
An explicit pick among identical elements is a fresh same-call
observation, but ad_find_exact sent it through strict resolution, which
refused it as ambiguous. It now resolves through the same positional
anchor that selected-match hydration uses; saved ambiguous refs are
still refused.
… replay

The census path matched the listed window by id and pid only, so after
an app restart that reused both, an old skeleton ref could resolve in
the new process. The window's process generation must now match. A
focused capture replays its census over the recorded window, so a
focus change no longer turns an unchanged ref into WINDOW_NOT_FOUND.
find --root and snapshot --root resolved a moved root by identity but
read children under its captured path, so they could return another
root's child. Both now pin the root at its captured position before
observing and before accepting results, and fail closed with STALE_REF
and a recovery suggestion when it moved.
Scoped and census anchors cleared the value but kept a name that
mirrored it, so the platform treated that name as stable and refused an
unchanged field. The shared stable_name rule now runs on the original
pair first.
First, last and nth picks published whatever handle the anchor resolver
returned, and Linux checks only the role there, so a swapped element
could be handed back. They now use the verified anchor that scoped
resolution uses.
Tray resolution checked only that the owner and menu entry existed, so
a retitled item still resolved. It now requires the recorded role,
identity and retained object to match.
A failed ExpandCollapse, GetState or similar read made the role fall
back to a plain role, so the count dropped the original element without
marking uncertainty and could resolve a look-alike. The role is now
unknown when a read it depends on failed, and the count stays
incomplete.
Many identity tests repeated a kept test with different data or tested
a helper a behavioral test already covers. They are deleted or merged
into the test that exercises the same rule through real code paths;
each merged test was checked to fail when its rule is broken. Every
production rule keeps at least one behavioral guard per platform.
An explicit pick among look-alikes records no unique identity, so the
landed element was checked only by role and bounds, and a replacement
with another name at the same place was accepted. Any stable field the
entry recorded must now match the landed element too.
@vercel

vercel Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
agent-desktop Ignored Ignored Preview, v0 Nov 1, 2026 6:36am UTC

@lahfir
lahfir added this pull request to stack #430 November 1, 2026 05:16
@lahfir lahfir changed the title laf/identity review fixes fix: close the remaining wrong-target paths in ref resolution Nov 1, 2026
Census and drill replay looked the recorded window up in the normal
window list, which leaves out shell surfaces such as the taskbar, tray
and Start menu, so those refs failed with WINDOW_NOT_FOUND. They now
find the surface the way capture does, with the same process checks.
@lahfir
lahfir removed this pull request from stack #430 November 1, 2026 05:37
@lahfir
lahfir changed the base branch from laf/identity-platform-proofs to laf/remove-macos-host November 1, 2026 05:37
@lahfir lahfir changed the title fix: close the remaining wrong-target paths in ref resolution fix: resolve refs only through a unique recorded identity Nov 1, 2026
@lahfir
lahfir added this pull request to stack #431 November 1, 2026 05:38
Strict resolution reads nodes that cannot be the target without their
states, and treating those unread fields as an unknown role left every
such node unread, so a unique button click timed out. A node whose role
number can never reach the recorded role is refuted as before; a
candidate whose refinement read fails still reads as unknown.
The log callback is global, and other FFI tests running in parallel log
through it, so the delivery and scope tests saw extra messages and
failed at random. They now look only at their own event; delivery,
redaction and scoping are still checked.
Collapse duplicated code and fixtures across core, macOS, Windows, Linux
and FFI: shared RefEntry test builders, one fields helper, one verdict
combinator, the macOS identity-roots module folded into root dispatch,
and a single direct-and-batch refusal helper in ref_churn.

Fixes from review:
- depth-cut default snapshots record their census, so unique elements
  in deep trees resolve instead of coming back ambiguous
- twin skeleton anchors are no longer offered as drill targets
- transient read failures in scoped resolution stay retryable
- FFI exact entries trust only a native id; strict finds land through
  the verified anchor
- Linux skips the path tier for a clean unique match
- the identity_not_unique recovery now says what works, and that a
  headed coordinate click is the only way to pick one of two twins
@lahfir

lahfir commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator Author

Replaced by #441 (macOS element addresses) and #442 (Linux and Windows capture rule, stacked on #441).

@lahfir lahfir closed this Nov 1, 2026
@lahfir
lahfir removed this pull request from stack #431 November 1, 2026 11:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant