Is your feature request related to a problem? Please describe.
Bytewax Materialization job tries to write all rows at a time from a single pyarrow parquet dataset
Describe the solution you'd like
Should break into mini-batches before writing to online store, and this number should be configurable
Describe alternatives you've considered
No
Additional context
Reactions are currently unavailable
Is your feature request related to a problem? Please describe.
Bytewax Materialization job tries to write all rows at a time from a single pyarrow parquet dataset
Describe the solution you'd like
Should break into mini-batches before writing to online store, and this number should be configurable
Describe alternatives you've considered
No
Additional context