← 返回首页
GH-108716: Turn off deep-freezing of modules. by markshannon · Pull Request #108722 · python/cpython · GitHub
Skip to content

Navigation Menu

Toggle navigation
Sign in
Appearance settings
Search or jump to...

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted
Cancel Submit feedback

Saved searches

Use saved searches to filter your results more quickly

Cancel Create saved search
Appearance settings
Resetting focus

GH-108716: Turn off deep-freezing of modules.#108722

Merged
markshannon merged 5 commits into
python:mainfrom
faster-cpython:no-deep-freeze-3
Sep 8, 2023
Merged

GH-108716: Turn off deep-freezing of modules.#108722
markshannon merged 5 commits into
python:mainfrom
faster-cpython:no-deep-freeze-3

Conversation

markshannon commented Aug 31, 2023 •
edited by bedevere-bot
Loading

Copy link
Copy Markdown
Member

Performance impact is neutral, maybe a little positive, but noisy

Startup might be a bit slower: no-site is 2% faster, but normal startup 5% slower.

However, without deepfreeze it is much easier to implement things like: faster-cpython/ideas#566 or #99555 which should more than compensate.

bedevere-bot mentioned this pull request Aug 31, 2023
Closed
brandtbucher self-requested a review August 31, 2023 16:19
brettcannon removed their request for review August 31, 2023 17:11

brandtbucher commented Aug 31, 2023

Copy link
Copy Markdown
Member

We may want to benchmark this on the other machines too, but in general I'm excited by this!

markshannon commented Sep 1, 2023

Copy link
Copy Markdown
Member Author

Windows 0% slower, Mac 1% faster

markshannon mentioned this pull request Sep 1, 2023
Merged

vstinner commented Sep 6, 2023

Copy link
Copy Markdown
Member

FYI I merged my PR #108741 to fix a race condition in make regen-all in the main branch. Backports to 3.11 and 3.12 are on-going.

vstinner commented Sep 6, 2023

Copy link
Copy Markdown
Member

Windows 0% slower, Mac 1% faster

Oh right, for the comparison to the main branch, I see:

Windows:

  • python_startup_no_site: 16.4 ms => 16.2 ms: 1.01x faster
  • python_startup: 19.3 ms => 19.8 ms: 1.02x slower

macOS:

  • python_startup_no_site: 9.40 ms => 9.15 ms: 1.03x faster
  • python_startup: 11.5 ms => 11.8 ms: 1.02x slower

To be honest, it's surprising that disabling an optimization supposed to make Python startup actually... has no effect (or might be faster, but that can be noise of the benchmark?).

vstinner commented Sep 6, 2023

Copy link
Copy Markdown
Member

What I can say is that it has a nice impact on build performance! :-)

Comparison main => this PR:

  • gcc -O0 (pydebug): 0m20,152s => 0m13,200s
  • gcc -O3 (release): 0m39,349s => 0m35,819s

Python configured with ./configure --with-pydebug or ./configure (without LTO, without PGO). I ran time make -j14 to measure build time on my laptop with 12 threads (6 CPU cores). Quick-and-dirty non scientific benchmark, I only ran make once to measure :-)

brandtbucher approved these changes Sep 7, 2023
View reviewed changes

brandtbucher left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

Looks good to me!

markshannon merged commit 15d4c9f into python:main Sep 8, 2023

vstinner commented Sep 8, 2023

Copy link
Copy Markdown
Member

It's unclear to me if Python/deepfreeze/deepfreeze.c is still used or not. When I build Python with make, it's not built. But make-regen creates the file. Is it intended?

# Dependencies which can add and/or remove _Py_ID() identifiers: # - deepfreeze.c # - "make clinic" .PHONY: regen-global-objects regen-global-objects: $(srcdir)/Tools/build/generate_global_objects.py $(DEEPFREEZE_C) clinic $(PYTHON_FOR_REGEN) $(srcdir)/Tools/build/generate_global_objects.py

markshannon commented Sep 8, 2023

Copy link
Copy Markdown
Member Author

Python/deepfreeze/deepfreeze.c shouldn't be used any more. I don't want to remove the tools yet, but we probably should remove the build target.

vstinner commented Sep 11, 2023

Copy link
Copy Markdown
Member

It seems like the Windows build system was not updated. deepfreeze.c is still generated on Windows, no?

This rule in PCbuild/_freeze_module.vcxproj:

<Target Name="_RebuildDeepFrozen" AfterTargets="_RebuildFrozen" DependsOnTargets="FindPythonForBuild" Condition="$(Configuration) != 'PGUpdate'"> <!-- BEGIN deepfreeze rule --> <Exec Command='$(PythonForBuild) "$(PySourcePath)Tools\build\deepfreeze.py" ^ "$(PySourcePath)Python\frozen_modules\importlib._bootstrap.h:importlib._bootstrap" ^ "$(PySourcePath)Python\frozen_modules\importlib._bootstrap_external.h:importlib._bootstrap_external" ^ "$(PySourcePath)Python\frozen_modules\zipimport.h:zipimport" ^ "$(PySourcePath)Python\frozen_modules\abc.h:abc" ^ "$(PySourcePath)Python\frozen_modules\codecs.h:codecs" ^ "$(PySourcePath)Python\frozen_modules\io.h:io" ^ "$(PySourcePath)Python\frozen_modules\_collections_abc.h:_collections_abc" ^ "$(PySourcePath)Python\frozen_modules\_sitebuiltins.h:_sitebuiltins" ^ "$(PySourcePath)Python\frozen_modules\genericpath.h:genericpath" ^ "$(PySourcePath)Python\frozen_modules\ntpath.h:ntpath" ^ "$(PySourcePath)Python\frozen_modules\posixpath.h:posixpath" ^ "$(PySourcePath)Python\frozen_modules\os.h:os" ^ "$(PySourcePath)Python\frozen_modules\site.h:site" ^ "$(PySourcePath)Python\frozen_modules\stat.h:stat" ^ "$(PySourcePath)Python\frozen_modules\importlib.util.h:importlib.util" ^ "$(PySourcePath)Python\frozen_modules\importlib.machinery.h:importlib.machinery" ^ "$(PySourcePath)Python\frozen_modules\runpy.h:runpy" ^ "$(PySourcePath)Python\frozen_modules\__hello__.h:__hello__" ^ "$(PySourcePath)Python\frozen_modules\__phello__.h:__phello__" ^ "$(PySourcePath)Python\frozen_modules\__phello__.ham.h:__phello__.ham" ^ "$(PySourcePath)Python\frozen_modules\__phello__.ham.eggs.h:__phello__.ham.eggs" ^ "$(PySourcePath)Python\frozen_modules\__phello__.spam.h:__phello__.spam" ^ "$(PySourcePath)Python\frozen_modules\frozen_only.h:frozen_only" ^ "-o" "$(PySourcePath)Python\deepfreeze\deepfreeze.c"'/> <!-- END deepfreeze rule --> </Target>

If you want to keep the infra for now, maybe this rule should just be commented with a reference to gh-108716?

markshannon deleted the no-deep-freeze-3 branch September 26, 2023 12:56

gvanrossum commented Mar 16, 2024 •
edited
Loading

Copy link
Copy Markdown
Member

This PR probably should also have updated Python/deepfreeze/README.txt to indicate that it is (currently) no longer used. It should also have updated various sections in Makefile.pre.in (including some comments) that still reference deepfreeze -- those are just distractions at this point (until we decide to actually deep-freeze some other objects, which doesn't look like it's going to happen for 3.13 at least). EDIT: See gh-116919

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

brandtbucher brandtbucher approved these changes
kumaraditya303 Awaiting requested review from kumaraditya303 kumaraditya303 is a code owner
ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner
ncoghlan Awaiting requested review from ncoghlan ncoghlan is a code owner
warsaw Awaiting requested review from warsaw warsaw is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

Footer

© 2026 GitHub, Inc.