Skip to content

gh-92041: Avoid a second sys.modules scan in inspect.getmodule - #156685

Open
eendebakpt wants to merge 2 commits into
python:mainfrom
eendebakpt:inspect_getmodule_norecurse
Open

gh-92041: Avoid a second sys.modules scan in inspect.getmodule#156685
eendebakpt wants to merge 2 commits into
python:mainfrom
eendebakpt:inspect_getmodule_norecurse

Conversation

@eendebakpt

@eendebakpt eendebakpt commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

When inspect.getmodule() computes its filename cache key, getabsfile() consults getsourcefile(), which recurses into getmodule() — scanning sys.modules a second time on every cache miss. Cache hits also pay getsourcefile()'s linecache probe and os.path.exists() call. This PR passes the filename explicitly, so the key is computed without the recursion.

Benchmark

10,000 synthetic modules in sys.modules

Workload main PR Speedup
Cached filename hit 4.13 µs 0.85 µs 4.9x
Unregistered frame (repeated miss) 3.19 ms 1.55 ms 2.1x
Bare code object (repeated miss) 3.06 ms 1.53 ms 2.0x
Miss after a registry mutation 3.14 ms 1.67 ms 1.9x
Function fast path 0.168 µs 0.168 µs no change
Benchmark script
import inspect
import sys
import timeit
import types

for i in range(10_000):
    m = types.ModuleType(f"fake_{i}")
    m.__file__ = f"/tmp/fake/mod_{i}.py"
    sys.modules[m.__name__] = m

real_file = sys.modules["fake_1"].__file__
code_hit = compile("pass", real_file, "exec")
code_miss = compile("pass", "<generated>", "exec")
ns = {"sys": sys, "out": {}}
exec(compile("out['frame'] = sys._getframe()", "<generated>", "exec"), ns)
frame = ns["out"]["frame"]

def a_function():
    pass

inspect.getmodule(code_hit)  # warm the cache

for label, fn in [
    ("cached filename hit", lambda: inspect.getmodule(code_hit)),
    ("unregistered frame (miss)", lambda: inspect.getmodule(frame)),
    ("bare code object (miss)", lambda: inspect.getmodule(code_miss)),
    ("function fast path", lambda: inspect.getmodule(a_function)),
]:
    number, elapsed = timeit.Timer(fn).autorange()
    print(f"{label:28s} {elapsed / number * 1e6:10.3f} us")

Pass the filename to getabsfile() explicitly, so that computing the
cache key does not recurse into getmodule() through the loader check
in getsourcefile().

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A frame whose filename belongs to a zipimported module must resolve
through the filename fallback so getsource() can use the module's
loader.  Test adapted from PR python#92042.

Co-authored-by: Mike Decker <mrd999@gmail.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant