Running Feast Java Server with Redis & calling with python (with registry in GCP)
For this tutorial, we setup Feast with Redis, using the Feast CLI to register and materialize features, and then retrieving via a Feast Java server deployed in Kubernetes via a gRPC call.
👉 for tips on how to run and debug this locally without using Kubernetes, see java/serving/README.md
First, let's setup a Redis cluster
-
Start minikube (minikube start)
-
Use helm to install a default Redis cluster
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
helm install my-redis bitnami/redis
-
Port forward Redis so we can materialize features to it
kubectl port-forward --namespace default svc/my-redis-master 6379:6379
-
Get your Redis password using the command (pasted below for convenience). We'll need this to tell Feast how to communicate with the cluster.
export REDIS_PASSWORD=$(kubectl get secret --namespace default my-redis -o jsonpath="{.data.redis-password}" | base64 --decode)
echo $REDIS_PASSWORD
Next, we setup a local Feast repo
- Install Feast with Redis dependencies pip install "feast[redis]"
- Make a bucket in GCS (or S3)
- The feature repo is already setup here, so you just need to swap in your GCS bucket and Redis credentials.
We need to modify the feature_store.yaml, which has two fields for you to replace:
registry: gs://[YOUR GCS BUCKET]/demo-repo/registry.db
project: feast_java_demo
provider: gcp
online_store:
type: redis
# Note: this would normally be using instance URL's to access Redis
connection_string: localhost:6379,password=[YOUR PASSWORD]
offline_store:
type: file
entity_key_serialization_version: 2
- Run feast apply to apply your local features to the remote registry
- Note: you may need to authenticate to gcloud first with gcloud auth login
- Materialize features to the online store:
CURRENT_TIME=$(date -u +"%Y-%m-%dT%H:%M:%S")
feast materialize-incremental $CURRENT_TIME
Now let's setup the Feast Server
- Add the gcp-auth addon to mount GCP credentials:
minikube addons enable gcp-auth
- Add Feast's Java feature server chart repo
helm repo add feast-charts https://feast-helm-charts.storage.googleapis.com
helm repo update
- Modify the application-override.yaml file to have your credentials + bucket location:
feature-server:
application-override.yaml:
enabled: true
feast:
activeStore: online
stores:
- name: online
type: REDIS
config:
host: my-redis-master
port: 6379
password: [YOUR PASSWORD]
global:
registry:
path: gs://[YOUR BUCKET]/demo-repo/registry.db
cache_ttl_seconds: 60
project: feast_java_demo
- Install the Feast helm chart: helm install feast-release feast-charts/feast --values application-override.yaml
Dev instructions: if you're changing the java logic or chart, you can do
- eval $(minikube docker-env)
- make build-java-docker-dev
- In the application-override.yaml, uncomment the two image: tag: dev blocks
- helm install feast-release ../../../infra/charts/feast --values application-override.yaml
- (Optional): check logs of the server to make sure it’s working
kubectl logs svc/feast-release-feature-server
- Port forward to expose the grpc endpoint:
kubectl port-forward svc/feast-release-feature-server 6566:6566
- Make a gRPC call:
-
Python example
python test.py
-
gRPC cli:
grpc_cli call localhost:6566 GetOnlineFeatures '
features {
val: "driver_hourly_stats:conv_rate"
val: "driver_hourly_stats:acc_rate"
}
entities {
key: "driver_id"
value {
val {
int64_val: 1001
}
val {
int64_val: 1002
}
}
}'
connecting to localhost:6566
metadata {
feature_names {
val: "driver_hourly_stats:conv_rate"
val: "driver_hourly_stats:acc_rate"
}
}
results {
values {
float_val: 0.812357187
}
values {
float_val: 0.379484832
}
statuses: PRESENT
statuses: PRESENT
event_timestamps {
seconds: 1631725200
}
event_timestamps {
seconds: 1631725200
}
}
results {
values {
float_val: 0.840873241
}
values {
float_val: 0.151376978
}
statuses: PRESENT
statuses: PRESENT
event_timestamps {
seconds: 1631725200
}
event_timestamps {
seconds: 1631725200
}
}
Rpc succeeded with OK status
