Vertex AI Feature Store Complete Guide: New BigQuery-based Version & MLOps

Vertex AI Feature Store is a centralized service for managing ML features. The 2024 release evolves the architecture to be BigQuery-based. It lets you reuse the same features for training and production inference, eliminating feature skew (the gap between training and serving).

What a Feature Store Does

• Centralized feature management: reuse features across teams
• Online Store: low-latency reads (millisecond-class) at inference time
• Offline Store: bulk reads for training
• Point-in-time Lookup: prevents data leakage
• Feature skew detection: detects drift between training and production distributions

New Version vs Legacy

Key Concepts (New Version)

• Feature Group: a feature group backed by a BigQuery table or view
• Feature: an individual feature (column)
• Feature View: a feature set for online serving (with an entity specified)
• Online Store: cluster for low-latency serving
• Sync: BigQuery → online store synchronization (cron or manual)

Setup Example

from google.cloud import aiplatform as ai

ai.init(project="my-project", location="asia-northeast1")

# Create the Online Store
online_store = ai.FeatureOnlineStore.create_bigtable_store(
    name="my-online-store",
    min_node_count=1, max_node_count=3,
)

# Create a Feature View (BigQuery as the source)
feature_view = online_store.create_feature_view(
    name="user-features",
    source=ai.FeatureViewBigQuerySource(
        uri="bq://my-project.ml_features.user_features",
        entity_id_columns=["user_id"],
    ),
    sync_config="0 * * * *",  # Hourly sync
)

# Run the sync
feature_view.sync()

# Online read (at inference time, milliseconds)
result = feature_view.read(key="user-123")

Point-in-time Lookup (for Training)

# Batch read for training: features at a specific point in time
df = feature_group.fetch_historical_features(
    entity_df=labels_df,  # user_id, timestamp, label
    feature_groups=[user_features, transaction_features],
)
# Prevents data leakage: only the latest value at or before each timestamp is fetched

Typical Architecture

1. Compute features in BigQuery (Dataform / Dataflow)
2. Register them as Feature Groups / Feature Views
3. Replicate to the Online Store via sync
4. Vertex AI Endpoints fetch features online
5. For training, perform point-in-time reads from the offline store (BigQuery)
6. Model Monitoring detects feature skew

Comparison with Other Feature Stores

Pricing Examples

Best Practices

• Keep all feature computation in BigQuery (no data movement)
• Align the sync schedule with business hours
• Use consistent entity IDs (user_id, device_id, etc.)
• Always enable feature skew monitoring
• Use BigQuery materialized views to cut recomputation cost

Google Cloud is a trademark of Google LLC. For the latest details, see the official Vertex AI Feature Store documentation.