← 返回首页
fix: Thread safe Clickhouse offline store by astronautas · Pull Request #5710 · feast-dev/feast · GitHub
Skip to content

Navigation Menu

Toggle navigation
Sign in
Appearance settings
Search or jump to...

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
Resetting focus

fix: Thread safe Clickhouse offline store#5710

Merged
franciscojavierarceo merged 5 commits into
feast-dev:masterfrom
astronautas:fix/thread-safe-clickhouse-offline-store-2
Nov 4, 2025
Merged

fix: Thread safe Clickhouse offline store#5710
franciscojavierarceo merged 5 commits into
feast-dev:masterfrom
astronautas:fix/thread-safe-clickhouse-offline-store-2

Conversation

Copy link
Copy Markdown
Contributor

astronautas commented Nov 4, 2025
edited
Loading

What this PR does / why we need it:

PR adjusts the Clickhouse offline store to be thread safe. Issue with the current implementation results in the issue as follows:

raise ProgrammingError('Attempt to execute concurrent queries within the same session.'

if you bombard it - feast serve - with concurrent requests, the under the hood threadpool is more than 1 thread by default (arbitrary number decided by uvicorn).

as per Clickhouse docs here:

Because each query or insert executed maintains state in its own QueryContext or InsertContext object, respectively, these helper objects are not thread-safe, and they should not be shared between multiple processing streams.

Solution - using threading.local() we can initialize a per-thread container for variables, to make sure we re-use the client on subsequent calls. Can we just drop the caching? Not really, unless the store is heavily refactored.

Which issue(s) this PR fixes:

No issue raised yet.

Misc

astronautas requested a review from a team as a code owner November 4, 2025 15:25
Signed-off-by: lukas.valatka <lukas.valatka@cast.ai>
Copy link
Copy Markdown
Member

franciscojavierarceo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

lgtm

Hide details View details franciscojavierarceo merged commit 5f446ed into feast-dev:master Nov 4, 2025
22 checks passed
HaoXuAI pushed a commit that referenced this pull request Nov 12, 2025
* add pull_all_from_table_or_query for clickhouse, to align with new materialization logic (calling it) Signed-off-by: lukas.valatka <lukas.valatka@cast.ai> * make sure get client is thread-local, since client is thread-unsafe Signed-off-by: lukas.valatka <lukas.valatka@cast.ai> * cleanup Signed-off-by: lukas.valatka <lukas.valatka@cast.ai> --------- Signed-off-by: lukas.valatka <lukas.valatka@cast.ai> Co-authored-by: lukas.valatka <lukas.valatka@cast.ai>
franciscojavierarceo pushed a commit that referenced this pull request Nov 13, 2025
# [0.57.0](v0.56.0...v0.57.0) (2025-11-13) ### Bug Fixes * Improve trino to feast type mapping with (real,varchar,timestamp,decimal) ([#5691](#5691)) ([f855ad2](f855ad2)) * Materialize API - ODFV views not looked-up (thinks views non existant) - crashes materialize ([#5716](#5716)) ([1b050b3](1b050b3)) * Support historical feature retrieval with start_date/end_date in RemoteOfflineStore ([#5703](#5703)) ([ad32756](ad32756)) * Thread safe Clickhouse offline store ([#5710](#5710)) ([5f446ed](5f446ed)) ### Features * Add annotations to cronjob CRDs ([#5701](#5701)) ([be6e6c2](be6e6c2)) * Add batch commit mode for MySQL OnlineStore ([#5699](#5699)) ([3cfe4eb](3cfe4eb)) * Add possibility to materialize only latest values, to increase performance ([#5713](#5713)) ([8d77b72](8d77b72)) * Support table format: Iceberg, Delta, and Hudi ([#5650](#5650)) ([2915ad1](2915ad1))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Footer

© 2026 GitHub, Inc.