Is your feature request related to a problem? Please describe.
When working with array-based features that represent sequences of past events per entity (e.g., last N transactions, recent interactions, rolling window of measurements), the current write_to_online_store implementation requires completely replacing the entire feature set for each entity. This is inefficient for use cases where you want to append new events to existing arrays, as it requires:
- Reading the current array via get_online_features
- Modifying it in application code (append, prepend, or maintain sliding window)
- Writing the entire updated array back via write_to_online_store
For DynamoDB specifically, this doesn't leverage the native UpdateItem operation with list_append expressions, which would be more efficient and atomic.
Describe the solution you'd like
Add optional partial update support to the DynamoDB online store that allows in-place list operations. This could be implemented as:
- New method in DynamoDB online store: update_online_store or extend write_to_online_store with an update_mode parameter
- Support for update expressions: Allow specifying DynamoDB update expressions like:
- list_append(feature_array, :new_values) - append to end
- list_append(:new_values, feature_array) - prepend to beginning
- Custom expressions for more complex operations
Example API:
# Option 1: New method
store.update_online_store(
feature_view_name="user_events",
entity_keys=[{"user_id": "123"}],
updates={
"recent_transactions": {
"operation": "list_append",
"values": [new_transaction_dict]
}
}
)
# Option 2: Extended write_to_online_store
store.write_to_online_store(
feature_view_name="user_events",
df=new_events_df,
write_mode="append" # or "replace" (default), "prepend"
)
Implementation Approach:
Since this is DynamoDB-specific functionality, it could be:
- Added as an optional method in the DynamoDBOnlineStore class
- Kept separate from the core online store interface to avoid breaking changes
- Potentially exposed through a DynamoDB-specific utility or extension
Key changes needed:
- Modify sdk/python/feast/infra/online_stores/dynamodb.py to add update operation support
- Use boto3's update_item with UpdateExpression instead of put_item
- Handle the serialization/deserialization of list values appropriately
- Maintain backward compatibility with existing write_to_online_store behavior
Describe alternatives you've considered
- Read-Modify-Write pattern (current workaround): Inefficient and not atomic
- Maintain arrays upstream: Requires duplicating windowing logic in batch/stream processing
- Use OnDemandFeatureView: Doesn't solve the storage efficiency problem
- Custom online store implementation: Requires maintaining a fork
Additional context
This feature would be particularly valuable for:
- Time-series features with sliding windows
- User interaction histories
- Event sequences for sequential models
- Any use case requiring efficient array/list manipulation at the storage layer
Related Slack discussion: https://feastopensource.slack.com/archives/C01M2GYP0UC/p1761052622312959
Technical References:
Is your feature request related to a problem? Please describe.
When working with array-based features that represent sequences of past events per entity (e.g., last N transactions, recent interactions, rolling window of measurements), the current write_to_online_store implementation requires completely replacing the entire feature set for each entity. This is inefficient for use cases where you want to append new events to existing arrays, as it requires:
For DynamoDB specifically, this doesn't leverage the native UpdateItem operation with list_append expressions, which would be more efficient and atomic.
Describe the solution you'd like
Add optional partial update support to the DynamoDB online store that allows in-place list operations. This could be implemented as:
Example API:
Implementation Approach:
Since this is DynamoDB-specific functionality, it could be:
Key changes needed:
Describe alternatives you've considered
Additional context
This feature would be particularly valuable for:
Related Slack discussion: https://feastopensource.slack.com/archives/C01M2GYP0UC/p1761052622312959
Technical References: