Repository navigation
Keep the best ASTRA iterate and log the scale-free objective - #619
Open
luciaquirke wants to merge 1 commit into
Open
luciaquirke wants to merge 1 commit into
luciaquirke wants to merge 1 commit into
Conversation
luciaquirke
force-pushed
the
feat/astra-best-iterate
branch
from
November 19, 2026 07:15
891b37c to
1726b4e
Compare
The SGD solver returns the iterate whose mean objective over the last best_window steps is lowest, carried through checkpoints. Each step also logs the objective at the best rescaling of x, -(q.x)^2 / (2 x.(H+D)x), which is independent of the scale of x.
luciaquirke
force-pushed
the
feat/astra-best-iterate
branch
from
November 19, 2026 07:23
1726b4e to
cad6821
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What: the ASTRA SGD solver returns the iterate at the step whose mean objective over the
best_windowsteps ending there is lowest (default 5, 0 returns the last iterate). The kept iterate is saved with the solver state, so a resumed solve keeps it. Each step logs, next to the raw objective, the objective at the best rescaling ofx, -(q.x)^2 / (2 x.(H+D)x), computed from the products already in hand. It is independent of the scale ofx, as scoring is, so it compares the directions of iterates and of different solvers.Why: a step size that is a little too large reaches a good solution and then climbs. On a 9B whole-rollout fit at absolute damping 5,000 the objective fell from 1.97 to 0.13 by step 13 and rose to 16 by step 38, and the returned vector was the step-38 one.
Checks:
tests/test_astra_best.pyruns the solver on a 2-D quadratic with a step size that converges along the soft direction and diverges along the stiff one; the kept iterate is the visited one with the lowest objective.