Repository navigation
Evaluate the KL divergence of the spliced model on an eval split - #147
Open
flyingdoubleG wants to merge 1 commit into
Open
flyingdoubleG wants to merge 1 commit into
flyingdoubleG wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #100.
Adds
--eval_split. Everysave_everysteps, the trainer splices all the SAEs into the model at once (excluded tokens keep their original activations) and logs the mean KL divergence between the next-token distributions of the clean and the spliced model aseval/kl. This is the actual KL, unlike thekltraining loss, which drops the entropy term of the clean distribution.Notes:
Trainertakes an optionaleval_dataset;__main__loads it from the same dataset as the training split. The eval set is sharded across ranks under DDP and the sums are all-reduced.distribute_modules, multipleinit_seeds, and theceloss with transcoders. It also raises for memmap datasets, which have no splits.I know #117 was closed because you planned to write this yourself. If you'd rather keep doing that, feel free to close this.
Tests (
tests/test_kl_eval.py, CPU): the KL matches a reference that splices GPT-2 block by block, it is zero when every token is excluded or the SAEs reconstruct perfectly,fitevaluates at every save, and the unsupported configurations raise.