← 返回首页
Ray (contrib) | Feast: the Open Source Feature Store
Ctrlk
GitBook Assistant
GitBook Assistant
GitBook Assistant
arrow-up-right-and-arrow-down-left-from-center
GitBook Assistant
Good night

I'm here to help you with the docs.

What is this page about?What should I read next?Can you give an example?
Ctrli
AI Based on your context
Send
v0.64-branch
Powered by GitBook
On this page
For the complete documentation index, see llms.txt. This page is also available as Markdown.
GitBook AssistantAsk
On this page
  1. Reference
  2. Offline stores

Ray (contrib)

⚠️ Contrib Plugin: The Ray offline store is a contributed plugin. It may not be as stable or fully supported as core offline stores. Use with caution in production and report issues to the Feast community.

The Ray offline store is a data I/O implementation that leverages Ray for reading and writing data from various sources. It focuses on efficient data access operations, while complex feature computation is handled by the Ray Compute Engine.

Quick Start with Ray Template

The easiest way to get started with Ray offline store is to use the built-in Ray template:

GitBook AssistantAskCopy
feast init -t ray my_ray_project cd my_ray_project/feature_repo

This template includes:

  • Pre-configured Ray offline store and compute engine setup

  • Sample feature definitions optimized for Ray processing

  • Demo workflow showcasing Ray capabilities

  • Resource settings for local development

The template provides a complete working example with sample datasets and demonstrates both Ray offline store data I/O operations and Ray compute engine distributed processing.

Overview

The Ray offline store provides:

  • Ray-based data reading from file sources (Parquet, CSV, etc.)

  • Support for local, remote, and KubeRay (Kubernetes-managed) clusters

  • Integration with various storage backends (local files, S3, GCS, HDFS, Azure Blob)

  • Efficient data filtering and column selection

  • Timestamp-based data processing with timezone awareness

  • Enterprise-ready KubeRay cluster support via CodeFlare SDK

  • GPU support: schedule worker tasks on GPU nodes via num_gpus config (all modes including KubeRay)

Functionality Matrix

Method
Supported

get_historical_features

Yes

pull_latest_from_table_or_query

Yes

pull_all_from_table_or_query

Yes

offline_write_batch

Yes

write_logged_features

Yes

RetrievalJob Feature
Supported

export to dataframe

Yes

export to arrow table

Yes

persist results in offline store

Yes

local execution of ODFVs

Yes

preview query plan

Yes

read partitioned data

Yes

⚠️ Important: Resource Management

By default, Ray will use all available system resources (CPU and memory). This can cause issues in test environments or when experimenting locally, potentially leading to system crashes or unresponsiveness.

For testing and local experimentation, we strongly recommend:

  1. Configure resource limits in your feature_store.yaml (see Resource Management and Testing section below)

This will limit Ray to safe resource levels for testing and development.

Architecture

The Ray offline store follows Feast's architectural separation:

  • Ray Offline Store: Handles data I/O operations (reading/writing data)

  • Ray Compute Engine: Handles complex feature computation and joins

  • Clear Separation: Each component has a single responsibility

For complex feature processing, historical feature retrieval, and distributed joins, use the Ray Compute Engine.

Configuration

The Ray offline store can be configured in your feature_store.yaml file. It supports three execution modes:

  1. LOCAL: Ray runs locally on the same machine (default)

  2. REMOTE: Connects to a remote Ray cluster via ray_address

  3. KUBERAY: Connects to Ray clusters on Kubernetes via CodeFlare SDK

Execution Modes

Local Mode (Default)

For simple data I/O operations without distributed processing:

Remote Ray Cluster

Connect to an existing Ray cluster:

KubeRay Cluster (Kubernetes)

Connect to Ray clusters on Kubernetes using CodeFlare SDK:

Environment Variables (alternative to config file):

Ray Offline Store + Compute Engine

For distributed feature processing with advanced capabilities:

Local Development Configuration

For local development and testing:

Production Configuration

For production deployments with distributed Ray cluster:

Configuration Options

Ray Offline Store Options

Option
Type
Default
Description

type

string

Required

Must be feast.offline_stores.contrib.ray_offline_store.ray.RayOfflineStore or ray

storage_path

string

None

Path for storing temporary files and datasets

ray_address

string

None

Ray cluster address (triggers REMOTE mode, e.g., "ray://host:10001")

use_kuberay

boolean

None

Enable KubeRay mode (overrides ray_address)

kuberay_conf

dict

None

KubeRay configuration dict with keys: cluster_name (required), namespace (default: "default"), auth_token, auth_server, skip_tls (default: false)

enable_ray_logging

boolean

false

Enable Ray progress bars and verbose logging

ray_conf

dict

None

Ray initialization parameters for resource management (e.g., memory, CPU limits)

broadcast_join_threshold_mb

int

100

Size threshold for broadcast joins (MB)

enable_distributed_joins

boolean

true

Enable distributed joins for large datasets

max_parallelism_multiplier

int

2

Parallelism as multiple of CPU cores

target_partition_size_mb

int

64

Target partition size (MB)

window_size_for_joins

string

"1H"

Time window for distributed joins

num_gpus

float

None

GPUs per worker task. Supported in all modes. See Worker Resource Scheduling.

gpu_batch_format

string

"pandas"

Batch format for map_batches when num_gpus is set ("numpy" or "pyarrow" for GPU-native libs).

worker_task_options

dict

None

Arbitrary Ray .options() kwargs (num_cpus, memory, accelerator_type, resources, runtime_env, …). See Worker Resource Scheduling for the full reference.

Mode Detection Precedence

The Ray offline store automatically detects the execution mode using the following precedence:

  1. Environment Variables (highest priority)

    • FEAST_RAY_USE_KUBERAY, FEAST_RAY_CLUSTER_NAME, etc.

  2. Config kuberay_conf

    • If present → KubeRay mode

  3. Config ray_address

    • If present → Remote mode

  4. Default

    • Local mode (lowest priority)

Ray Compute Engine Options

For Ray compute engine configuration options, see the Ray Compute Engine documentation.

Resource Management and Testing

Overview

By default, Ray will use all available system resources (CPU and memory). This can cause issues in test environments or when experimenting locally, potentially leading to system crashes or unresponsiveness.

Resource Configuration

For custom resource control, configure limits in your feature_store.yaml:

Conservative Settings (Local Development/Testing)

Production Settings

Resource Configuration Options

Setting
Default
Description
Testing Recommendation

broadcast_join_threshold_mb

100

Size threshold for broadcast joins (MB)

25

max_parallelism_multiplier

2

Parallelism as multiple of CPU cores

1

target_partition_size_mb

64

Target partition size (MB)

16

enable_ray_logging

false

Enable Ray progress bars and logging

false

Environment-Specific Recommendations

Local Development

Production Clusters

Usage Examples

Basic Data Source Reading

Direct Data Access

The Ray offline store provides direct access to underlying data:

Batch Writing

The Ray offline store supports batch writing for materialization:

Saved Dataset Persistence

The Ray offline store supports persisting datasets for later analysis:

Remote Storage Support

The Ray offline store supports various remote storage backends:

Using Ray Cluster

Standard Ray Cluster

To use Ray in cluster mode for distributed data access:

  1. Start a Ray cluster:

  1. Configure your feature_store.yaml:

  1. For multiple worker nodes:

KubeRay Cluster (Kubernetes)

To use Feast with Ray clusters on Kubernetes via CodeFlare SDK:

Prerequisites:

  • KubeRay cluster deployed on Kubernetes

  • CodeFlare SDK installed: pip install codeflare-sdk

  • Access credentials for the Kubernetes cluster

Configuration:

  1. Using configuration file:

  1. Using environment variables:

Features:

  • The CodeFlare SDK handles cluster connection and authentication

  • Automatic TLS certificate management

  • Authentication with Kubernetes clusters

  • Namespace isolation

  • Secure communication between client and Ray cluster

  • Automatic cluster discovery

GPU Support

The Ray offline store supports GPU scheduling via the num_gpus and gpu_batch_format config options. This works across all execution modes (local, remote, and KubeRay).

For full configuration details, examples, and KubeRay GPU setup, see the Ray Compute Engine GPU Support section.

Data Source Validation

The Ray offline store validates data sources to ensure compatibility:

Data Sources

RaySource is the recommended data source for the Ray offline store. It is a pure-metadata descriptor that tells Feast how to load a Ray Dataset from any source Ray Data supports — Parquet, CSV, JSON, HuggingFace datasets, MongoDB, binary files, images, TFRecords, WebDataset, SQL, and more.

See the RaySource reference for a full list of reader_type values and configuration options.

Note: FileSource (Parquet) remains supported for backward compatibility but RaySource(reader_type="parquet") is preferred for new projects.

Limitations

The Ray offline store has one known limitation:

  • online_write_batch not implemented: The OfflineStore.online_write_batch() interface is not supported by the Ray offline store. This does not affect materialization — feast materialize writes to the online store correctly via the Ray Compute Engine. The restriction only applies to callers that invoke online_write_batch on the offline store object directly, which is an uncommon pattern outside of custom tooling.

Integration with Ray Compute Engine

For complex feature processing operations, use the Ray offline store in combination with the Ray Compute Engine. See the Ray Offline Store + Compute Engine configuration example in the Configuration section above for a complete setup.

For more advanced troubleshooting, refer to the Ray documentation.

Quick Reference

Configuration Templates

Basic Ray Offline Store (local development):

Ray Offline Store + Compute Engine (distributed processing):

Key Commands

For complete examples, see the Configuration section above.

PreviousClickhouse (contrib)NextOracle (contrib)

Last updated 22 days ago

Was this helpful?