← 返回首页
feat: Added support for image search by ntkathole · Pull Request #5577 · feast-dev/feast · GitHub
Skip to content

Navigation Menu

Toggle navigation
Sign in
Appearance settings
Search or jump to...

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted
Cancel Submit feedback

Saved searches

Use saved searches to filter your results more quickly

Cancel Create saved search
Appearance settings
Resetting focus

feat: Added support for image search#5577

Merged
franciscojavierarceo merged 5 commits into
feast-dev:masterfrom
ntkathole:image_support
Sep 7, 2025
Merged

feat: Added support for image search#5577
franciscojavierarceo merged 5 commits into
feast-dev:masterfrom
ntkathole:image_support

Conversation

Copy link
Copy Markdown
Member

ntkathole commented Aug 22, 2025 •
edited
Loading

What this PR does / why we need it:

Added support for image search combining text and image queries and fixed Milvus binary data handling with base64 encoding.

Users can now define Field(name="image_bytes", dtype=ImageBytes)

This PR also fixes the vector field inconsistency issue on online_write_batch with milvus.

  • IMAGE_BYTES Type Support
sdk/python/feast/value_type.py: Added IMAGE_BYTES = 21 sdk/python/feast/types.py: Added ImageBytes type and mappings sdk/python/feast/on_demand_feature_view.py: Added sample JPEG data for testing
  • Image Search Functionality
sdk/python/feast/feature_store.py: Enhanced retrieve_online_documents_v2 with image search parameters sdk/python/feast/vector_store.py: Added query_image_bytes support sdk/python/feast/image_utils.py: New file with ImageFeatureExtractor and combine_embeddings

Which issue(s) this PR fixes:

Fixes #5372 and Fixes #5551

ntkathole self-assigned this Aug 22, 2025
ntkathole requested a review from a team as a code owner August 22, 2025 17:47
ntkathole force-pushed the image_support branch 2 times, most recently from 5ca64fc to e1f19f2 Compare August 22, 2025 18:14
franciscojavierarceo reviewed Aug 22, 2025
View reviewed changes
Comment thread sdk/python/feast/feature_store.py
query: Optional[List[float]] = None,
query_string: Optional[str] = None,
distance_metric: Optional[str] = "L2",
query_image_bytes: Optional[bytes] = None,
Copy link
Copy Markdown
Member

franciscojavierarceo Aug 22, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

is this the right way to do this? See I would have updated the proto values to support ImageBytes like we support PDFBytes and then just queried embedding the standard way.

You can then enrich the image vector embeddings with text and semantic embeddings and then allow a composite search of both. In that sense, you still would use 1 query but searching across multiple vectors.

Copy link
Copy Markdown
Member

franciscojavierarceo Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

Actually I'm realizing my mistake here. ImageBytes is required for retrieval, what you here is appropriate for image search when the image is passed in for the query.

Copy link
Copy Markdown
Member Author

ntkathole Aug 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

Added IMAGE_BYTES as well 👍

ntkathole force-pushed the image_support branch 2 times, most recently from b88d3a6 to e252cdf Compare August 27, 2025 05:13
franciscojavierarceo reviewed Aug 28, 2025
View reviewed changes
Comment thread infra/templates/README.md.jinja2 Show resolved Hide resolved
franciscojavierarceo reviewed Aug 28, 2025
View reviewed changes
Comment thread sdk/python/feast/infra/online_stores/milvus_online_store/milvus.py Outdated Show resolved Hide resolved
franciscojavierarceo reviewed Aug 28, 2025
View reviewed changes
Comment thread sdk/python/feast/infra/online_stores/milvus_online_store/milvus.py Outdated Show resolved Hide resolved
franciscojavierarceo reviewed Sep 6, 2025
View reviewed changes
Comment thread sdk/python/feast/image_utils.py Outdated Show resolved Hide resolved
franciscojavierarceo reviewed Sep 6, 2025
View reviewed changes
Comment thread sdk/python/feast/image_utils.py

for output in outputs:
feature_vector = output.numpy()
normalized = normalize(feature_vector.reshape(1, -1), norm="l2")
Copy link
Copy Markdown
Member

franciscojavierarceo Sep 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

feel like l2 should be configurable but probably not needed yet

Copy link
Copy Markdown
Member Author

ntkathole Sep 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

right, most of the usecase get covered via l2, we can expand as needed

franciscojavierarceo reviewed Sep 6, 2025
View reviewed changes
Comment thread sdk/python/feast/image_utils.py Outdated
else:
raise ValueError(
f"Unknown combination strategy: {strategy}. "
f"Supported strategies: weighted_sum, concatenate, average"
Copy link
Copy Markdown
Member

franciscojavierarceo Sep 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

small nit, technically you should just make the strategy an enum so the valueerror can be maintained easier

Copy link
Copy Markdown
Member Author

ntkathole Sep 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

Made the changes

franciscojavierarceo approved these changes Sep 6, 2025
View reviewed changes
Copy link
Copy Markdown
Member

franciscojavierarceo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

some small nits but this lgtm and i'd like to include it in the next release as this is a killer new feature. great work as always @ntkathole!

feat: Added requirements for image search support …
1746e76
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
feat: Added support for image search …
c4f542e
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
test: Added test for image search …
8ba87f4
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
fix: Resolved comments …
59ece96
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
fix: Fixed comments …
d02913b
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Hide details View details franciscojavierarceo merged commit 56c5910 into feast-dev:master Sep 7, 2025
16 checks passed
franciscojavierarceo pushed a commit that referenced this pull request Sep 30, 2025
chore(release): release 0.54.0 …
99a321a
# [0.54.0](v0.53.0...v0.54.0) (2025-09-30) ### Bug Fixes * Column quoting in query of `PostgreSQLOfflineStore.pull_all_from_table_or_query` ([#5621](#5621)) ([e8eae71](e8eae71)) * Correct column list polars materialization engine ([#5595](#5595)) ([39aeb0c](39aeb0c)) * Fix Go feature server entitykey serialization for version 3 ([#5622](#5622)) ([5ab18a6](5ab18a6)) * Fix hostname resolution for spark tests ([#5610](#5610)) ([8f0e22d](8f0e22d)) * Fixed filtering based on data_source for ODFVs ([#5593](#5593)) ([c3e6c56](c3e6c56)) * Fixed project_description to set in registry and UI ([#5602](#5602)) ([02c3006](02c3006)) * Fixed Registry Cache Refresh Issues ([#5604](#5604)) ([3c7a022](3c7a022)) * Fixed tls issue when running both grpc and rest servers ([#5617](#5617)) ([51c16b1](51c16b1)) * Fixed transaction handling with SQLite registry ([#5588](#5588)) ([0052754](0052754)) * Update the deprecated functions in Go feature server. ([#5632](#5632)) ([a24e06e](a24e06e)) * Updated python packages conflicting with kserve dependencies ([#5580](#5580)) ([d56baf4](d56baf4)) ### Features * Add 'featureView' in global search api result for features. ([#5626](#5626)) ([76590bf](76590bf)) * Add aggregation in OnDemandFeatureView ([#5629](#5629)) ([8715ae8](8715ae8)) * Added codeflare-sdk to requirements ([#5640](#5640)) ([51a0ee6](51a0ee6)) * Added RemoteDatasetProxy that executes Ray Data operations remotely ([7128024](7128024)) * Added support for image search ([#5577](#5577)) ([56c5910](56c5910)) * Enable ingestion without event timestamp ([#5625](#5625)) ([eb51f00](eb51f00)) * Feast dataframe phase1 ([#5611](#5611)) ([2ce4198](2ce4198)) * Feast dataframe phase2 ([#5612](#5612)) ([1d08786](1d08786)) * Feast Namespaces registry for client ConfigMaps availability ([#5599](#5599)) ([728589a](728589a)) * Support hdfs:// uris in to_remote_storage for Spark offline store ([#5635](#5635)) ([5e4b9fd](5e4b9fd))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

franciscojavierarceo franciscojavierarceo approved these changes

Assignees

ntkathole

Labels

ok-to-test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Milvus Online Store Dimension Mismatch Error in Push API and Materialization Support Image Search

2 participants

Footer

© 2026 GitHub, Inc.