← 返回首页
Fix selector garbage collection by kushalkolar · Pull Request #319 · fastplotlib/fastplotlib · GitHub
Skip to content

Navigation Menu

Toggle navigation
Sign in
Appearance settings
Search or jump to...

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
Resetting focus

Fix selector garbage collection#319

Merged
kushalkolar merged 3 commits into
mainfrom
fix-selector-gc
Oct 26, 2023
Merged

Fix selector garbage collection#319
kushalkolar merged 3 commits into
mainfrom
fix-selector-gc

Conversation

Copy link
Copy Markdown
Member

WIP

Still have some issues in the context of mesviz when switching CNMF items:

Exception ignored in: <function Synchronizer.__del__ at 0x7f515504dd00> Traceback (most recent call last): File "/home/kushal/repos/fastplotlib/fastplotlib/graphics/selectors/_sync.py", line 85, in __del__ self.remove(s) File "/home/kushal/repos/fastplotlib/fastplotlib/graphics/selectors/_sync.py", line 43, in remove selector.selection.remove_event_handler(self._handle_event) ^^^^^^^^^^^^^^^^^^ ReferenceError: weakly-referenced object no longer exists /home/kushal/repos/mesmerize-viz/mesmerize_viz/_cnmf/_viz_container.py:275: FutureWarning: You are trying to use the following experimental feature, this may change in the future without warning: CaimanDataFrameExtensions.get_params_diffs This feature is new and the might improve in the future param_diffs = self._dataframe.caiman.get_params_diffs( 10:16:55 PM --------------------------------------------------------------------------- RecursionError Traceback (most recent call last) File ~/venvs/mescore/lib/python3.11/site-packages/ipywidgets/widgets/widget.py:773, in Widget._handle_msg(self, msg) 771 if 'buffer_paths' in data: 772 _put_buffers(state, data['buffer_paths'], msg['buffers']) --> 773 self.set_state(state) 775 # Handle a state request. 776 elif method == 'request_state': File ~/venvs/mescore/lib/python3.11/site-packages/ipywidgets/widgets/widget.py:650, in Widget.set_state(self, sync_data) 645 self._send(msg, buffers=echo_buffers) 647 # The order of these context managers is important. Properties must 648 # be locked when the hold_trait_notification context manager is 649 # released and notifications are fired. --> 650 with self._lock_property(**sync_data), self.hold_trait_notifications(): 651 for name in sync_data: 652 if name in self.keys: File /usr/lib/python3.11/contextlib.py:144, in _GeneratorContextManager.__exit__(self, typ, value, traceback) 142 if typ is None: 143 try: --> 144 next(self.gen) 145 except StopIteration: 146 return False File ~/venvs/mescore/lib/python3.11/site-packages/traitlets/traitlets.py:1524, in HasTraits.hold_trait_notifications(self) 1522 for changes in cache.values(): 1523 for change in changes: -> 1524 self.notify_change(change) File ~/venvs/mescore/lib/python3.11/site-packages/ipywidgets/widgets/widget.py:701, in Widget.notify_change(self, change) 698 if name in self.keys and self._should_send_property(name, getattr(self, name)): 699 # Send new state to front-end 700 self.send_state(key=name) --> 701 super().notify_change(change) File ~/venvs/mescore/lib/python3.11/site-packages/traitlets/traitlets.py:1539, in HasTraits.notify_change(self, change) 1537 def notify_change(self, change): 1538 """Notify observers of a change event""" -> 1539 return self._notify_observers(change) File ~/venvs/mescore/lib/python3.11/site-packages/traitlets/traitlets.py:1586, in HasTraits._notify_observers(self, event) 1583 elif isinstance(c, EventHandler) and c.name is not None: 1584 c = getattr(self, c.name) -> 1586 c(event) File ~/repos/mesmerize-viz/mesmerize_viz/_cnmf/_wrapper.py:231, in GridPlotWrapper.__init__.<locals>.<lambda>(change) 227 for trait in ["value", "max"]: 228 jslink((self.component_slider, trait), (self.component_int_box, trait)) 230 self.component_int_box.observe( --> 231 lambda change: self.set_component_index(change["new"]), "value" 232 ) 234 # gridplot for each sublist 235 for sub_data in self._data: File ~/repos/mesmerize-viz/mesmerize_viz/_cnmf/_wrapper.py:277, in GridPlotWrapper.set_component_index(self, index) 274 for s in self.heatmap_selectors: 275 # TODO: Very hacky for now, ignores if the slider is currently being moved, prevents weird slider movement 276 if s._move_info is None: --> 277 s.selection = index 279 self.component_int_box.value = index File ~/repos/fastplotlib/fastplotlib/graphics/_base.py:139, in Graphic.__setattr__(self, key, value) 137 attr = getattr(self, key) 138 if isinstance(attr, GraphicFeature): --> 139 attr._set(value) 140 return 142 super().__setattr__(key, value) File ~/repos/fastplotlib/fastplotlib/graphics/_features/_selection_features.py:166, in LinearSelectionFeature._set(self, value) 163 self._parent.position_y = value 165 self._data = value --> 166 self._feature_changed(key=None, new_data=value) File ~/repos/fastplotlib/fastplotlib/graphics/_features/_selection_features.py:192, in LinearSelectionFeature._feature_changed(self, key, new_data) 181 pick_info = { 182 "world_object": self._parent.world_object, 183 "new_data": new_data, (...) 187 "delta": self._parent.delta, 188 } 190 event_data = FeatureEvent(type="selection", pick_info=pick_info) --> 192 self._call_event_handlers(event_data) File ~/repos/fastplotlib/fastplotlib/graphics/_features/_base.py:185, in GraphicFeature._call_event_handlers(self, event_data) 183 func() 184 else: --> 185 func(event_data) 186 else: 187 func() File ~/repos/mesmerize-viz/mesmerize_viz/_cnmf/_wrapper.py:284, in GridPlotWrapper._heatmap_set_component_index(self, ev) 281 def _heatmap_set_component_index(self, ev): 282 index = ev.pick_info["selected_index"] --> 284 self.set_component_index(index) File ~/repos/mesmerize-viz/mesmerize_viz/_cnmf/_wrapper.py:277, in GridPlotWrapper.set_component_index(self, index) 274 for s in self.heatmap_selectors: 275 # TODO: Very hacky for now, ignores if the slider is currently being moved, prevents weird slider movement 276 if s._move_info is None: --> 277 s.selection = index 279 self.component_int_box.value = index File ~/repos/fastplotlib/fastplotlib/graphics/_base.py:139, in Graphic.__setattr__(self, key, value) 137 attr = getattr(self, key) 138 if isinstance(attr, GraphicFeature): --> 139 attr._set(value) 140 return 142 super().__setattr__(key, value) File ~/repos/fastplotlib/fastplotlib/graphics/_features/_selection_features.py:166, in LinearSelectionFeature._set(self, value) 163 self._parent.position_y = value 165 self._data = value --> 166 self._feature_changed(key=None, new_data=value) File ~/repos/fastplotlib/fastplotlib/graphics/_features/_selection_features.py:192, in LinearSelectionFeature._feature_changed(self, key, new_data) 181 pick_info = { 182 "world_object": self._parent.world_object, 183 "new_data": new_data, (...) 187 "delta": self._parent.delta, 188 } 190 event_data = FeatureEvent(type="selection", pick_info=pick_info) --> 192 self._call_event_handlers(event_data) File ~/repos/fastplotlib/fastplotlib/graphics/_features/_base.py:185, in GraphicFeature._call_event_handlers(self, event_data) 183 func() 184 else: --> 185 func(event_data) 186 else: 187 func() File ~/repos/mesmerize-viz/mesmerize_viz/_cnmf/_wrapper.py:284, in GridPlotWrapper._heatmap_set_component_index(self, ev) 281 def _heatmap_set_component_index(self, ev): 282 index = ev.pick_info["selected_index"] --> 284 self.set_component_index(index) [... skipping similar frames: Graphic.__setattr__ at line 139 (489 times), GridPlotWrapper.set_component_index at line 277 (489 times), GraphicFeature._call_event_handlers at line 185 (488 times), LinearSelectionFeature._feature_changed at line 192 (488 times), GridPlotWrapper._heatmap_set_component_index at line 284 (488 times), LinearSelectionFeature._set at line 166 (488 times)] File ~/repos/fastplotlib/fastplotlib/graphics/_features/_selection_features.py:166, in LinearSelectionFeature._set(self, value) 163 self._parent.position_y = value 165 self._data = value --> 166 self._feature_changed(key=None, new_data=value) File ~/repos/fastplotlib/fastplotlib/graphics/_features/_selection_features.py:192, in LinearSelectionFeature._feature_changed(self, key, new_data) 181 pick_info = { 182 "world_object": self._parent.world_object, 183 "new_data": new_data, (...) 187 "delta": self._parent.delta, 188 } 190 event_data = FeatureEvent(type="selection", pick_info=pick_info) --> 192 self._call_event_handlers(event_data) File ~/repos/fastplotlib/fastplotlib/graphics/_features/_base.py:185, in GraphicFeature._call_event_handlers(self, event_data) 183 func() 184 else: --> 185 func(event_data) 186 else: 187 func() File ~/repos/mesmerize-viz/mesmerize_viz/_cnmf/_wrapper.py:284, in GridPlotWrapper._heatmap_set_component_index(self, ev) 281 def _heatmap_set_component_index(self, ev): 282 index = ev.pick_info["selected_index"] --> 284 self.set_component_index(index) File ~/repos/mesmerize-viz/mesmerize_viz/_cnmf/_wrapper.py:277, in GridPlotWrapper.set_component_index(self, index) 274 for s in self.heatmap_selectors: 275 # TODO: Very hacky for now, ignores if the slider is currently being moved, prevents weird slider movement 276 if s._move_info is None: --> 277 s.selection = index 279 self.component_int_box.value = index File ~/repos/fastplotlib/fastplotlib/graphics/_base.py:139, in Graphic.__setattr__(self, key, value) 137 attr = getattr(self, key) 138 if isinstance(attr, GraphicFeature): --> 139 attr._set(value) 140 return 142 super().__setattr__(key, value) File ~/repos/fastplotlib/fastplotlib/graphics/_features/_selection_features.py:163, in LinearSelectionFeature._set(self, value) 161 self._parent.position_x = value 162 else: --> 163 self._parent.position_y = value 165 self._data = value 166 self._feature_changed(key=None, new_data=value) File ~/repos/fastplotlib/fastplotlib/graphics/_base.py:142, in Graphic.__setattr__(self, key, value) 139 attr._set(value) 140 return --> 142 super().__setattr__(key, value) File ~/repos/fastplotlib/fastplotlib/graphics/_base.py:114, in Graphic.position_y(self, val) 112 @position_y.setter 113 def position_y(self, val): --> 114 self.world_object.world.y = val File ~/repos/pygfx/pygfx/utils/transform.py:381, in AffineBase.y(self, value) 378 @y.setter 379 def y(self, value): 380 x, _, z = self.position --> 381 self.position = (x, value, z) File ~/repos/pygfx/pygfx/utils/transform.py:336, in AffineBase.position(self, value) 334 @position.setter 335 def position(self, value): --> 336 self.matrix = la.mat_compose(value, self.rotation, self.scale) File ~/repos/pygfx/pygfx/utils/transform.py:720, in RecursiveTransform.matrix(self, value) 718 @matrix.setter 719 def matrix(self, value): --> 720 self.own.matrix = self._parent.inverse_matrix @ value File ~/repos/pygfx/pygfx/utils/transform.py:553, in AffineTransform.matrix(self, value) 550 @matrix.setter 551 def matrix(self, value): 552 self.untracked_matrix[:] = value --> 553 self.flag_update() File ~/repos/pygfx/pygfx/utils/transform.py:542, in AffineTransform.flag_update(self) 540 def flag_update(self): 541 self.last_modified = perf_counter_ns() --> 542 super().flag_update() File ~/repos/pygfx/pygfx/utils/transform.py:193, in AffineBase.flag_update(self) 191 """Signal that this transform has updated.""" 192 for callback in list(self.update_callbacks.values()): --> 193 callback(self) File ~/repos/pygfx/pygfx/utils/transform.py:73, in callback.__get__.<locals>.inner(*args, **kwargs) 70 if weak_instance() is None: 71 return ---> 73 return self.callback_fn(weak_instance(), *args, **kwargs) File ~/repos/pygfx/pygfx/utils/transform.py:684, in RecursiveTransform._own_updated(self, own) 682 origin = self.parent.position 683 self._reference_up = la.vec_normalize(new_ref - origin) --> 684 self.flag_update() File ~/repos/pygfx/pygfx/utils/transform.py:667, in RecursiveTransform.flag_update(self) 665 def flag_update(self): 666 self.last_modified = perf_counter_ns() --> 667 super().flag_update() File ~/repos/pygfx/pygfx/utils/transform.py:193, in AffineBase.flag_update(self) 191 """Signal that this transform has updated.""" 192 for callback in list(self.update_callbacks.values()): --> 193 callback(self) File ~/repos/pygfx/pygfx/utils/transform.py:73, in callback.__get__.<locals>.inner(*args, **kwargs) 70 if weak_instance() is None: 71 return ---> 73 return self.callback_fn(weak_instance(), *args, **kwargs) File ~/repos/pygfx/pygfx/utils/transform.py:672, in RecursiveTransform._parent_updated(self, parent) 669 @callback 670 def _parent_updated(self, parent: AffineBase): 671 self._propagate_reference_up() --> 672 self.flag_update() File ~/repos/pygfx/pygfx/utils/transform.py:667, in RecursiveTransform.flag_update(self) 665 def flag_update(self): 666 self.last_modified = perf_counter_ns() --> 667 super().flag_update() File ~/repos/pygfx/pygfx/utils/transform.py:193, in AffineBase.flag_update(self) 191 """Signal that this transform has updated.""" 192 for callback in list(self.update_callbacks.values()): --> 193 callback(self) File ~/repos/pygfx/pygfx/utils/transform.py:73, in callback.__get__.<locals>.inner(*args, **kwargs) 70 if weak_instance() is None: 71 return ---> 73 return self.callback_fn(weak_instance(), *args, **kwargs) File ~/repos/pygfx/pygfx/objects/_base.py:186, in WorldObject._update_uniform_buffers(self, transform) 184 self.uniform_buffer.data["world_transform"] = transform.matrix.T 185 self.uniform_buffer.data["world_transform_inv"] = transform.inverse_matrix.T --> 186 self.uniform_buffer.update_range() File ~/repos/pygfx/pygfx/resources/_buffer.py:199, in Buffer.update_range(self, offset, size) 197 Resource._rev += 1 198 self._rev = Resource._rev --> 199 self._gfx_mark_for_sync() File ~/repos/pygfx/pygfx/resources/_base.py:74, in Resource._gfx_mark_for_sync(self) 73 def _gfx_mark_for_sync(self): ---> 74 resource_update_registry._gfx_mark_for_sync(self) File ~/repos/pygfx/pygfx/resources/_base.py:96, in ResourceUpdateRegistry._gfx_mark_for_sync(self, resource) 94 raise TypeError("Given object is not a Resource") 95 if resource._wgpu_object is not None and resource._gfx_pending_uploads: ---> 96 self._syncable.add(resource) File /usr/lib/python3.11/_weakrefset.py:88, in WeakSet.add(self, item) 86 if self._pending_removals: 87 self._commit_removals() ---> 88 self.data.add(ref(item, self._remove)) RecursionError: maximum recursion depth exceeded in comparison

kushalkolar requested a review from clewis7 October 13, 2023 02:21

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

This is a diff fix, not related to gc

kushalkolar marked this pull request as ready for review October 26, 2023 02:26

Copy link
Copy Markdown
Member Author

@clewis7 this is ready to go! 😄 gc works in mesmerize-viz for CNMF now, I tested by checking out a bunch of different rows, selectors get gc'd, ram usage stays constant, now we can put mesviz in beta :D

clewis7 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

just some clarifying questions but otherwise LGTM

I think it was a good choice to make all the selectors inherit from base

Comment thread fastplotlib/graphics/image.py Show resolved Hide resolved
Comment thread fastplotlib/layouts/_base.py Show resolved Hide resolved
kushalkolar merged commit f4bab4a into main Oct 26, 2023
kushalkolar deleted the fix-selector-gc branch October 29, 2023 19:57
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Footer

© 2026 GitHub, Inc.