← 返回首页
coerce dtype=np.int64 for np.prod calls in subsample by FlynnOConnell · Pull Request #806 · fastplotlib/fastplotlib · GitHub
Skip to content

Navigation Menu

Toggle navigation
Sign in
Appearance settings
Search or jump to...

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
Resetting focus

coerce dtype=np.int64 for np.prod calls in subsample#806

Merged
kushalkolar merged 4 commits into
mainfrom
fix-int-overflow
May 20, 2025
Merged

coerce dtype=np.int64 for np.prod calls in subsample#806
kushalkolar merged 4 commits into
mainfrom
fix-int-overflow

Conversation

FlynnOConnell commented Apr 25, 2025
edited
Loading

Copy link
Copy Markdown
Collaborator

Some of my datasets are overflowing during histogram calculation.

raw_scan.shape Out[1]: (5632, 14, 448, 448) np.prod(raw_scan.shape) Out[2]: -1354760192 np.prod(raw_scan.shape, dtype=np.int64) Out[3]: 15825108992

Also, ignore_dims won't work due to tuple item assignment. Changed that to a list

github-actions Bot commented Apr 25, 2025
edited
Loading

Copy link
Copy Markdown

📚 Docs preview built and uploaded! https://www.fastplotlib.org/ver/fix-int-overflow

Comment thread fastplotlib/utils/functions.py Outdated Show resolved Hide resolved
kushalkolar merged commit 0e1a308 into main May 20, 2025
kushalkolar deleted the fix-int-overflow branch May 21, 2025 08:39
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Footer

© 2026 GitHub, Inc.