Skip to content

fix: let int() decide what an int is in handle_arg_string - #146

Open
abhishekcpp007 wants to merge 1 commit into
EleutherAI:mainfrom
abhishekcpp007:abhishekcpp007/fix-arg-string-int-parsing
Open

abhishekcpp007 wants to merge 1 commit into
EleutherAI:mainfrom
abhishekcpp007:abhishekcpp007/fix-arg-string-int-parsing

Conversation

@abhishekcpp007

Copy link
Copy Markdown

What this fixes

handle_arg_string in sparsify/utils.py uses str.isnumeric() to decide whether a value is an
integer. That predicate disagrees with int() in both directions, so the function gets two
different things wrong.

A negative integer silently becomes a float. "-5".isnumeric() is False, so -5 skips the
int branch and comes back out of float() as -5.0. simple_parse_args_string feeds directly
into load_dataset(**kwargs) in __main__.py, so --data_args num_proc=-1 hands a float to a
parameter that wants an int. No error, just the wrong type.

A Unicode numeral raises. "½".isnumeric() is True but int("½") raises ValueError, and
nothing catches it on that branch. Same for "²" and for Roman numerals such as "Ⅷ". At the
call site that ValueError is swallowed by the load_from_disk handler and resurfaces as a
dataset error, which points at the wrong thing.

Current behaviour:

input    isnumeric  int() works  handle_arg_string returns
5        True       yes          5 (int)
-5       False      yes          -5.0 (float)      <- wrong type
+5       False      yes          5.0  (float)      <- wrong type
½        True       NO           ValueError        <- crash
²        True       NO           ValueError        <- crash
Ⅷ        True       NO           ValueError        <- crash

Fix

Try int() and fall through on ValueError, instead of asking isnumeric() first. The predicate
and the parser are then the same thing, so they cannot disagree.

Values that already parsed keep parsing the same way, fullwidth digits ("123" to 123)
included. "3.5" still reaches float, and anything else still comes back as the original string.

Tests

tests/test_utils.py covers signed integers, floats including 1e3, the three Unicode numerals
above, booleans, plain strings, and the end-to-end simple_parse_args_string path with a negative
num_proc. The signed-integer and Unicode cases fail on main.

The module it tests is pure stdlib, so the file needs no GPU and no model.

Checks I ran

  • ruff check sparsify/utils.py tests/test_utils.py — clean
  • black --check on both — no changes
  • I could not install the project in my environment, so I did not run pytest against the repo.
    Saying that rather than reporting a pass I did not see. The two functions under test are pure
    stdlib, so I re-ran every assertion in the new test file against the patched implementations in
    isolation and all 20 pass. CI is the real run.

`handle_arg_string` used `str.isnumeric()` to decide whether a `--data_args`
value was an integer. That predicate disagrees with `int()` in both directions:

- `"-5".isnumeric()` is False, so a negative integer skipped the int branch and
  came back from `float()` as `-5.0`. `simple_parse_args_string` feeds straight
  into `load_dataset(**kwargs)`, so `--data_args num_proc=-1` silently handed a
  float to a parameter that wants an int. No error, just the wrong type.
- `"½".isnumeric()` is True but `int("½")` raises, so a Unicode numeral raised an
  unhandled ValueError. The same applies to `"²"` and to Roman numerals such as
  `"Ⅷ"`. At the call site in `__main__.py` that ValueError is caught by the
  `load_from_disk` handler and resurfaces as a dataset error, which points at
  the wrong thing.

Trying `int()` and falling through on ValueError removes both. Values that
already parsed keep parsing the same way, including fullwidth digits.

Adds tests/test_utils.py covering signed integers, floats, Unicode numerals,
booleans, plain strings, and the end-to-end `simple_parse_args_string` path.
@CLAassistant

CLAassistant commented Nov 1, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants