← 返回首页
gh-103295: expose API for writing perf map files by gsallam · Pull Request #103546 · python/cpython · GitHub
Skip to content

Navigation Menu

Toggle navigation
Sign in
Appearance settings
Search or jump to...

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted
Cancel Submit feedback

Saved searches

Use saved searches to filter your results more quickly

Cancel Create saved search
Appearance settings
Resetting focus

gh-103295: expose API for writing perf map files#103546

Merged
pablogsal merged 60 commits into
python:mainfrom
gsallam:main
May 21, 2023
Merged

gh-103295: expose API for writing perf map files#103546
pablogsal merged 60 commits into
python:mainfrom
gsallam:main

Conversation

gsallam commented Apr 14, 2023 •
edited by bedevere-bot
Loading

Copy link
Copy Markdown
Contributor

#96123 added support for CPython to write /tmp/perf-.map files, associating instruction address ranges with a human-readable frame name for the Linux perf profiler.

Two external Python JIT compilers, Cinder and Pyston, both also independently write to perf map files.

Since perf map files are one-per-process, multiple separate libraries trying to write perf map entries independently can lead to file corruption from simultaneous writes.

It's unlikely for both Cinder and Pyston JITs to be used in the same process, but it's quite reasonable to use one of these JITs along with CPython's native perf trampoline support.

This PR add a C-API to write to the perf map file. It also update the perf trampoline to use the new API to write to the perf map file.

ghost commented Apr 14, 2023 •
edited by ghost
Loading

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

bedevere-bot commented Apr 14, 2023

Copy link
Copy Markdown

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

bedevere-bot mentioned this pull request Apr 14, 2023
Closed

bedevere-bot commented Apr 14, 2023

Copy link
Copy Markdown

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

4 similar comments

bedevere-bot commented Apr 14, 2023

Copy link
Copy Markdown

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

bedevere-bot commented Apr 14, 2023

Copy link
Copy Markdown

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

bedevere-bot commented Apr 14, 2023

Copy link
Copy Markdown

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

bedevere-bot commented Apr 14, 2023

Copy link
Copy Markdown

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

gsallam marked this pull request as ready for review April 16, 2023 20:50
arhadthedev added the interpreter-core (Objects, Python, Grammar, and Parser dirs) label Apr 19, 2023

czardoz commented Apr 19, 2023

Copy link
Copy Markdown
Contributor

Looks like the CI is having trouble installing dependencies, I will update this branch with rebase and try again in a bit

92 hidden items Load more…
sunmy2019 requested changes May 19, 2023
View reviewed changes
Comment thread Python/perf_trampoline.c Show resolved Hide resolved
facebook-github-bot pushed a commit to facebookincubator/cinder that referenced this pull request May 19, 2023
Backport perf trampoline …
a88f497
Summary: Backport the perf-trampoline introduced in python/cpython#96123. The perf trampoline doesn't work properly with the JIT, so we have submitted a PR to have a C-API to unify writing to the perf-map files python/cpython#103546. Reviewed By: czardoz Differential Revision: D45419843 fbshipit-source-id: 16bd13d7981e48c9eb7bc0e5eef1c1f4748965f6
facebook-github-bot pushed a commit to facebookincubator/cinder that referenced this pull request May 19, 2023
Perfmap C-API and JIT integration …
c475629
Summary: With the perf trampoline writing to the perf-map files, we want to have a C-API to unify writing to the perf-map files to avoid file corruption from simultaneous writes. We are trying to upstream the API here python/cpython#103546. More details about the motivation is in the PR. In addition to introducing the C-API, we also change JIT to utilize the new C-API. Reviewed By: czardoz Differential Revision: D45421966 fbshipit-source-id: d270cc753a245f93cbfe3d723d0880595fef45f2
carljm requested a review from sunmy2019 May 20, 2023 03:41
carljm added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 20, 2023

bedevere-bot commented May 20, 2023

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @carljm for commit 10630e8 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 20, 2023
sunmy2019 approved these changes May 20, 2023
View reviewed changes
Comment thread Python/perf_trampoline.c Outdated Show resolved Hide resolved
Comment thread Python/perf_trampoline.c Outdated Show resolved Hide resolved
carljm added 4 commits May 19, 2023 21:41
apparently :c:data:errno doesn't exist
386dd42
use PyMem_Raw* instead of malloc/free
6585f32
Merge branch 'main' into gsallam/main …
ea5965a
* main: (30 commits) pythongh-103987: fix several crashes in mmap module (python#103990) docs: fix wrong indentation causing rendering error in dis page (python#104661) pythongh-94906: Support multiple steps in math.nextafter (python#103881) pythongh-104472: Skip `test_subprocess.ProcessTestCase.test_empty_env` if ASAN is enabled (python#104667) pythongh-103839: Allow building Tkinter against Tcl 8.7 without external libtommath (pythonGH-103842) pythongh-85984: New additions and improvements to the tty library. (python#101832) pythongh-104659: Consolidate python examples in enum documentation (python#104665) pythongh-92248: Deprecate `type`, `choices`, `metavar` parameters of `argparse.BooleanOptionalAction` (python#103678) pythongh-104645: fix error handling in marshal tests (python#104646) pythongh-104600: Make type.__type_params__ writable (python#104634) pythongh-104602: Add additional test for listcomp with lambda (python#104639) pythongh-104640: Disallow walrus in comprehension within type scopes (python#104641) pythongh-103921: Rename "type" header in argparse docs (python#104654) Improve readability of `typing._ProtocolMeta.__instancecheck__` (python#104649) pythongh-96522: Fix deadlock in pty.spawn (python#96639) pythonGH-102818: Do not call `PyTraceBack_Here` in sys.settrace trampoline. (pythonGH-104579) pythonGH-103545: Add macOS specific constants for ``os.setpriority`` to ``os`` (python#104606) pythongh-104623: Update macOS installer to SQLite 3.42.0 (pythonGH-104624) pythongh-104619: never leak comprehension locals to outer locals() (python#104637) pythongh-104602: ensure all cellvars are known up front (python#104603) ...
fix PyMem_Free to PyMem_RawFree
e6be2a7
carljm added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 20, 2023

bedevere-bot commented May 20, 2023

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @carljm for commit e6be2a7 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 20, 2023

carljm commented May 20, 2023 •
edited
Loading

Copy link
Copy Markdown
Member

Refleak buildbot failures are unrelated, they are all failing in test_mmap, which is a known issue on main: #104698. Otherwise buildbots look happy.

I think this is ready for merge; @pablogsal I will plan to merge it tomorrow evening unless you have objections!

pablogsal merged commit be0c106 into python:main May 21, 2023

pablogsal commented May 21, 2023

Copy link
Copy Markdown
Member

I think this is ready for merge; @pablogsal I will plan to merge it tomorrow evening unless you have objections!

I went ahead and land it myself after another review pass 😉

Great job everyone!

facebook-github-bot pushed a commit to facebookincubator/cinder that referenced this pull request May 24, 2023
User error codes and fix 32-bit system issues …
0a7cc0b
Summary: Port the latest changes from the [perf-map C-API PR](python/cpython#103546). This includes using error codes instead of exceptions to avoid the need to hold the GIL. Additionally, there was an issue with Fedora on a 32-bit system that was fixed in this [PR](python/cpython#104811), so I also ported the fix. Reviewed By: carljm Differential Revision: D46041553 fbshipit-source-id: 6502189f489ca6cf11949ecd93c044e5ebe4fd2c
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

carljm carljm left review comments
pablogsal pablogsal left review comments
gpshead gpshead approved these changes
sunmy2019 sunmy2019 approved these changes
+1 more reviewer
czardoz czardoz left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

interpreter-core (Objects, Python, Grammar, and Parser dirs)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

Footer

© 2026 GitHub, Inc.