Skip to content

Keep the best ASTRA iterate and log the scale-free objective - #619

Open
luciaquirke wants to merge 1 commit into
mainfrom
feat/astra-best-iterate
Open

luciaquirke wants to merge 1 commit into
mainfrom
feat/astra-best-iterate

Conversation

@luciaquirke

@luciaquirke luciaquirke commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

What: the ASTRA SGD solver returns the iterate at the step whose mean objective over the best_window steps ending there is lowest (default 5, 0 returns the last iterate). The kept iterate is saved with the solver state, so a resumed solve keeps it. Each step logs, next to the raw objective, the objective at the best rescaling of x, -(q.x)^2 / (2 x.(H+D)x), computed from the products already in hand. It is independent of the scale of x, as scoring is, so it compares the directions of iterates and of different solvers.

Why: a step size that is a little too large reaches a good solution and then climbs. On a 9B whole-rollout fit at absolute damping 5,000 the objective fell from 1.97 to 0.13 by step 13 and rose to 16 by step 38, and the returned vector was the step-38 one.

Checks: tests/test_astra_best.py runs the solver on a 2-D quadratic with a step size that converges along the soft direction and diverges along the stiff one; the kept iterate is the visited one with the lowest objective.

@luciaquirke
luciaquirke force-pushed the feat/astra-best-iterate branch from 891b37c to 1726b4e Compare November 19, 2026 07:15
The SGD solver returns the iterate whose mean objective over the last
best_window steps is lowest, carried through checkpoints. Each step also
logs the objective at the best rescaling of x, -(q.x)^2 / (2 x.(H+D)x),
which is independent of the scale of x.
@luciaquirke
luciaquirke force-pushed the feat/astra-best-iterate branch from 1726b4e to cad6821 Compare November 19, 2026 07:23

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant