One possible way to do automatic rectangle packing: https://github.com/laserson/squarify

@almarklein have you done something like this before? Allow bboxes for subplot viewport rects to be defined, and then when a viewport rect is resized all the other rects should adjust accordinlgy.

One observation is that sometimes you want to position something in absolute pixels, and sometimes as a fraction of the parent's space. In visvis I therefore positioned widget-objects like this (taken from the visvis docs):

Each element (x,y,w,h) can be either: 1) The integer amount of pixels relative to the parent's position; 2) The fractional amount (float value between 0.0 and 1.0) of the parent's width or height. Each value can be negative. For x and y this simply means a negative offset from the parent's left and top. For the width and height the difference from the parent's full width/height is taken.

But this is a bit weird/confusing. Plus very often you want a relative position with some offset.

So in visvis2 (wip) I borrowed an approach from CSS, each item in the rect is a string e.g. "120px", "50%" or "50% - 20px". See https://github.com/pygfx/visvis2/blob/3e3e9d785c34d3170433c78595390f2304185df3/visvis2/_view.py#L33-L39 I've implemented something similar in pygfx/pygfx#954 (not sure if that will stay).

If you want more elaborate layout you should probably implement container objects that position their children in a specific way, similar to CSS flexbox and the like. But that'd be a lot more work (it's harder than it looks).

Each element (x,y,w,h) can be either: 1) The integer amount of pixels relative to the parent's position; 2) The fractional amount (float value between 0.0 and 1.0) of the parent's width or height.

Ah I actually just did this! https://github.com/fastplotlib/fastplotlib/pull/740/files#diff-44de8531560bd88b7488ae543fbd3ff464555037e40be4d4d26b5fec8a17ace5R91-R96

I was wondering if you knew anything about auto-packing/docking algorithms. It's easy to find GUI toolkits that have it but I can't find any explanation of the backend algorithms anywhere.

I was wondering if you knew anything about auto-packing/docking algorithms.

Ah. I did stuff like that in Flexx, but there I could leverage HTML/CSS. I remember that in the early days Jupyterlab used some custom layout engine, but I cannot find it anymore. I suspect they moved to using CSS Flexbox.

I would recommend looking at what Matplotlib does, e.g. here: https://github.com/matplotlib/matplotlib/blob/6fc8169112a42e3f6f55793a6b56159cd8ff69d7/lib/matplotlib/figure.py#L2677

A note of warning: widget layout can be a pretty deep rabbit-hole 😉

Thanks! I think we'll just have a free-form drag & resize subplots on the canvas and block overlaps, and have another mode that uses a simple tiling algorithm maybe from squarify.

This answer on stackoverflow is very good: https://stackoverflow.com/a/306332 and this demo 😄 https://silentmatt.com/rectangle-intersection/

Ok I've got the basics down! 🥳

Next step, integrate this mess into Figure 😂

@almarklein Additional pointer control might be useful to implement in rendercanvas. For example here, the ability to lock the system cursor when the user can done an invalid movement, such as trying to overlap two rects. Instead of the rect stopping and the cursor continuing to move much further away, I think it would be nice to also have the cursor stop moving at the same time. I think cursor-locking behavior is what most people are used to for these types of things 🤔 . There's something about this which just feel slightly off and I think that it's this.

I found how it can be done in the browser: https://developer.mozilla.org/en-US/docs/Web/API/Pointer_Lock_API#locked_state

Idea:

Instead of putting all the layout management logic inside the Figure class we keep it as separate classes. Figure.layout_manager is an instance of a layout manager class. Right now we can have 3 different layout managers, Grid, freeform Flex (the implementation in the video above), and Squarify for the treemap once I figure out how to implement that.

Merge the implementation of the subplot Frame stuff into the Subplot class, I think this makes sense since it will then give all subplots the underlay plane, title, and resize handler. The rect and extent properties can be made read-only since we don't want users to manually change a subplot rect by directly addresses the subplot. Users can "request" changing a subplot rect via Figure.layout_manager if the type of LayoutManager supports it. For example, do not allow users to change the rect if the layout manager is a Grid since the rects are done automatically, but if it is a Flex layout we allow the user to request changing the rect. The request to change the rect is "denied" if the user passes in an invalid rect or a rect that would overlap with another subplot or cause other issues.

Basically to summarize:

We make:

Figure.layout_manager which is an instance of GridLayout, FlexLayout, or SquarifyLayout.

The LayoutManager manages pointer events which allow subplots to move or resize (if a flex layout). In a GridLayout it just auto-sets the subplot rects.

For FlexLayout we have FlexLayout.set_rect(subplot, rect: <x, y, w, h>) which is a user-facing function that allows them to manually define a rect. This function will raise if the user passes an invalid rect.
Similarly we have functions FlexLayout.set_extent(subplot, extent: <x0, x1, y0, y1>) for extent.

To the current Subplot class we add the stuff from Frame in the PR as it looks right now, to give it the plane Mesh, resize handler, subplot title text. Properties like rect, extent are read-only for the user. The LayoutManager however can set the subplot rect.

I think layout manager sounds like a nice way to abstract, and then users can specify the layout they want via string when they instantiate a plot

I think by default, we would want to maintain the grid layout we have now.

Also, would squarify be for making rectangular subplots of different sizes and tiling them all together?

If that's the case, could we not also use squarify for making our grids too? Fixed rectangles that get tiles together in a particular shape. Or maybe I am not understanding what the the squarify layout would be for

Yes squarify is for tiling rects of diff sizes. I think it's overkill for a simple grid plot layout though so we should keep them separate.

In the flex and squarify layouts we can also have a swap method to swap subplot rects

Instead of the rect stopping and the cursor continuing to move much further away, I think it would be nice to also have the cursor stop moving at the same time.

I think we have an issue to be able to hide the cursor, which is also convenient for games. But that means you have to draw some sort of cursor yourself. Is that what you mean? AFAIK You cannot keep the OS's cursor and stop it from moving.

keep the OS's cursor and stop it from moving.

This is what I meant, I'll do another search and see if there is such a thing. Otherwise once we have our own cursors.

I just realized a grid layout can just be managed by a flex layout with fractional extents, this will make the implementations easier.

so what's left is:

1. layout.set_rect() or settable Subplot.rect that respects the overlap allow/not allowed mode
2. mode to allow/not allow overlaps
   • if there are overlaps need to determine which subplot goes on top...
3. cleanup implemention

@almarklein related to pygfx/pygfx#492, would it be possible to have an underlay render pass which also doesn't have a depth butter? For rendering the box frame of the subplot, subplot title and resize handles?

We can allow one inset plot area within a subplot, arbitrary overlaps would be a mess with the screenspace camera and if a user wants many overlapping plots they should probably make them separate anyways

Let's do insets and add/remove subplots and row/columms in another PR.

@almarklein would be great to get your thoughts on this PR! 😄

📚 Docs preview built and uploaded! https://www.fastplotlib.org/ver/figure-rect-mode

@clewis7 should be gtg! 🥳

All tests using pygfx@main passing, docs built!