Repository navigation
Resolve macOS proxy settings before forking - #619
Open
roberstreams wants to merge 2 commits into
Open
roberstreams wants to merge 2 commits into
roberstreams wants to merge 2 commits into
Conversation
On macOS, urllib.request.getproxies() falls back to the SystemConfiguration framework when no *_proxy variable is set. requests, httpx and aiohttp all call it, and the framework is not fork-safe, so any test that builds an HTTP client crashes in the forked child and every mutant reports as a segfault. Resolve the system settings once in the main process and export them as *_proxy / no_proxy variables, which urllib reads first. Explicit proxy variables and other platforms are left untouched.
Copying macOS bypass exceptions into no_proxy changed their meaning: "*.local" and "169.254/16" use macOS's own syntax, which urllib's environment matcher does not understand, so hosts the system would bypass went through the proxy. Query SystemConfiguration once in the parent and replace urllib.request._get_proxies / _get_proxy_settings with the cached answers instead; forked children inherit them, and urllib's own macOS matcher gives the same results. The environment is no longer modified.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On macOS,
urllib.request.getproxies()andproxy_bypass()query the SystemConfiguration framework through_scproxywhen no*_proxyvariable is set.requests,httpxandaiohttpall call them. That framework isn't fork-safe, so any test that builds an HTTP client crashes in the forked child, and every mutant reports assegfault.Change:
_pin_macos_proxy_settings()runs at the start of_run, before any fork. On macOS it callsurllib.request._get_proxies()and_get_proxy_settings()once, in the unforked parent. It then replaces both with functions that return those cached answers. Forked children inherit the patched module, so they never reach SystemConfiguration. Proxy and bypass results stay identical, because urllib's own macOS matcher still evaluates the system's exception list. The environment is not modified. Other platforms are untouched.A first version exported the settings as
*_proxy/no_proxyvariables instead. That changed bypass behavior: macOS exceptions such as*.localand169.254/16use syntax that urllib's environment matcher doesn't understand. The second commit replaces that approach.Verification:
mainboth mutants reportsegfault. With this patch both reportkilled.getproxies()andproxy_bypass()forbuild.local,169.254.1.2,localhost,example.com,10.0.0.5andprintergive identical results before and after_pin_macos_proxy_settings().tests/test_macos_proxy.py. The lookups are answered from one pre-fork query, and bypass results come from urllib's real_proxy_bypass_macosx_sysconf(*.localand169.254/16match,example.comdoesn't). Other platforms are left alone. Both tests run on any OS.ruff checkandruff formatare clean.mypyis clean for__main__.pywith both--platform darwinand--platform linux, andmainhas 1 error there already.pytest tests --ignore=tests/e2egives 466 passed and 1 failed. The failure,test_excluded_lines_reports_only_the_first_line_of_each_statement, fails the same way on unpatchedmain.Fixes #618
🤖 Generated with Claude Code