add ImageVolumeGraphic#791

kushalkolar

actually going to merge it this time

TextureArrays supports 3D textures
implement simple ImageVolumeGraphic
implement graphic features for all the different volume materials
check and document sharing the data buffer between an object that uses mip/ray material and another object that uses slice material.
examples
- basic for all material types
- 3D video, maybe generated calcium toy data?
- multi channel zarr
make hlut tool work with this, account for the dims so that the cmap bar shows up
make this work with ImageWidget, image widget is a mess, need to refactor iw before adding volume support
tests: screenshots, API, new graphic, TextureArray for 2d and 3d, and any new graphic features.
test multi-channel
fix some typos in graphics.image
scripts/generate_add_graphic_methods.py is better, removed Graphic.type attribute which was only used in the generate mixin script and use regex instead to create the add_<graphic> method names.
improve docstring for quick_min_max()

@almarklein are RGB volumes supported? Is it even a thing?

selector tools for slicing, cube slicing, etc., non-orthogonal slicing to come later

note: implementing ImageVolumeGraphic as a subclass of ImageGraphic wasn't a good idea because the __init__ gets messy. Might make a BaseImageGraphic class which just has properties which are almost identical between ImageGraphic and ImageVolumeGraphic, but not a big deal right now.

kushalkolar

took some 3d movie gen code from caiman to generate a 3d movie:

@FlynnOConnell

3d-2025-04-12_03.09.08.mp4

FlynnOConnell

File [~\repos\fastplotlib\fastplotlib\graphics\features\_image.py:147](http://localhost:8888/lab/tree/demos/notebooks/~/repos/fastplotlib/fastplotlib/graphics/features/_image.py#line=146), in TextureArray._fix_data(self, data)
def _fix_data(self, data):
   if data.ndim not in (2, 3):
--> 147         raise ValueError(
           "image data must be 2D with or without an RGB(A) dimension, i.e. "
           "it must be of shape [rows, cols], [rows, cols, 3] or [rows, cols, 4]"
       )
   # let's just cast to float32 always
   return data.astype(np.float32)

ValueError: image data must be 2D with or without an RGB(A) dimension, i.e. it must be of shape [rows, cols], [rows, cols, 3] or [rows, cols, 4]

data = np.random.randn(50, 2, 448, 448)
data.shape
>> (50, 2, 448, 448)

The ability to view a time-series with a 4th dimension would be nice here

FlynnOConnell

Also how difficult is something like scale=[1, 4, 1] to account for anisotropy in XY and Z? Here I have a volume of [14, 448, 448] ZXY, but from the side its a flat plane. This dataset is [2um x 2um x 17um]

FlynnOConnell

Also how difficult is something like scale=[1, 4, 1] to account for anisotropy in XY and Z? Here I have a volume of [14, 448, 448] ZXY, but from the side its a flat plane. This dataset is [2um x 2um x 17um]

nevermind :)

volume.world_object.world.scale_z = 15.0

Some other helpful tidbits:

hlut = fpl.HistogramLUTTool(volume.data.value, volume)
volume.interpolation = "linear"
volume.cmap = "gnuplot2"

fig[0, 0].docks["right"].add_graphic(hlut)
fig[0, 0].docks["right"].size = 80

fig[0, 0].docks["right"].controller.enabled = False
fig[0, 0].docks["right"].camera.maintain_aspect = False
fig[0, 0].docks["right"].auto_scale(maintain_aspect=False)

almarklein

@almarklein are RGB volumes supported? Is it even a thing?

Yes, looks like it is; it uses the same logic (in the shader) as the image. I have never encountered it in the wild though.

kushalkolar

ok all the basics are down, switchable materials, can change material props:

vol-2025-06-04_03.34.28.mp4

Need to figure out why regex is turning VolumeSliceMaterial into slic instead of slice

kushalkolar

In other news, the latest pycharm is full of bugs, wtf is this. Also it wanted to auto insert the old rst structured docstring template even though I configured it to do numpydoc format 🤮

kushalkolar

All that's left is tests, 2D over time example, and RGB(A) examples. I also think I want to merge #849 and then rebase this so the examples will be much simpler.

Also pygfx has non-orthogonal slicing:

vol_slicing-2025-06-05_00.53.40.mp4

kushalkolar

ok there's a bug in the TextureArray or tiling somewhere, because when a dim is greater than the limit it is tiling along the wrong dim 😆

kushalkolar

The rendering at the edges is weird if one of the dims is larger than the 3d texture size limit so let's just not support this, I think the uses are very much rare edge cases. Also if the 3d texture size limit is 16,383, then 16384x16384x16384 with float32 is 17 TB which is more than any GPU that I know of can handle 😂 .

github-actions

📚 Docs preview built and uploaded! https://www.fastplotlib.org/ver/volume-image

kushalkolar

More tests for the TextureArray for volumes would be nice, but can do that later. I think tiling of volume is a rare edge case so if there is a bug it will affect very rare usecases not already covered by the tests.

clewis7

Just some small comments, otherwise looks good to me

Very cool stuff!

FlynnOConnell

Amazing! @kushalkolar I'll test this out!

kushalkolar

@clewis7 gtg! 🥳 and then we can merge #904 once I make new ground truth screenshots for that and then #883

add volume graphic, basic stuff works

2eeea6e

kushalkolar requested a review from clewis7 as a code owner April 12, 2025 06:20

kushalkolar mentioned this pull request Apr 12, 2025

basic image volume #401

Closed

kushalkolar added 2 commits April 12, 2025 02:21

remove a useless file I accidentally added

874cd3c

black

b2b9b10

kushalkolar force-pushed the volume-image branch from ce9c197 to b2b9b10 Compare April 12, 2025 06:30

add volume movie

6feec31

linear interpolation is better

ac76f6a

kushalkolar mentioned this pull request Apr 17, 2025

Roadmap 2026 #55

Open

78 tasks

merge conflict

1e61b64

kushalkolar marked this pull request as draft June 4, 2025 04:58

kushalkolar added 3 commits June 4, 2025 01:10

cmap shown for volume image in hlut tooL

4e616bb

volume render modes work

1ed2794

example works

b0f26ff

kushalkolar added 5 commits June 5, 2025 00:51

some updates to ImageGraphic for sharing buffesr

9d059c7

update volume and add examples

d27cd31

update example

98a09f7

update add_graphics mixin

61be715

bah

b6e29cd

separate TextureArrayVolume for volumes, fix indexing for tiling volumes

36d5abb

kushalkolar added 7 commits September 28, 2025 05:15

fix

25562b6

upate example

7bac111

better way to detect images

26dfd05

texture array for volume image is 3D

599c6dd

fix test

e9b7db9

rename to figure so pygfx gallery scraper picks it up

8698c3e

allow longer time for actions, I guess volumes take a while

e23bf44

kushalkolar added 6 commits September 28, 2025 07:05

rename

5095612

update screenshots

c2f2923

for some reason using black via python gives different results than b… …

0566f1b

…lack on commandline

add ImageVolume api docs

5de4a23

replace image_widget_grid yet again...

9b57677

forgot to commit event tables diff

78b4e96

clewis7 requested changes Sep 29, 2025

View reviewed changes

kushalkolar added 7 commits September 30, 2025 03:16

hlut tool can manage multiple ImageGraphics or ImageVolumeGraphics

616a223

remove property that's no longer part of the API

c8e44f6

docs

e9c9a7a

docstring fix

39344c3

better text on hlut

dbe5056

black

8ff0f7d

update api docs

d92b24a

kushalkolar added 2 commits September 30, 2025 23:57

update CI timeouts because of new downloaded volume image

201532a

update screenshots because hlut line is more gray

ce4570b

clewis7 approved these changes Oct 1, 2025

View reviewed changes

clewis7 deleted the volume-image branch October 1, 2025 15:38

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

add ImageVolumeGraphic#791