Feast Integration

Feast is an operational data system for managing and serving machine learning features to models in production.

Flyte provides a way to train models and perform feature engineering as a single pipeline. But it provides no way to serve these features to production when the model matures and is ready to be served in production.

This is where Feast comes into the picture.

By leveraging the collective capabilities, Flyte adds the ability to engineer the features, and Feast provides the feature registry and online feature serving system. Moreover, Flyte can help us ensure incremental development of features and enables us to turn on the sync to online stores only when one is confident about the features.

In this tutorial, we’ll walk through how Feast can be used to store and retrieve features to train and test the model through a pipeline curated using Flyte.

Dataset

We will use the horse colic dataset wherein we will determine if the lesion of the horse is surgical or not. This is a modified version of the original dataset.

The dataset will have the following columns:

Horse Colic Features

surgery	Age	Hospital Number	rectal temperature	pulse
respiratory rate	temperature of extremities	peripheral pulse	mucous membranes	capillary refill time
pain	peristalsis	abdominal distension	nasogastric tube	nasogastric reflux
nasogastric reflux PH	rectal examination	abdomen	packed cell volume	total protein
abdominocentesis appearance	abdomcentesis total protein	outcome	surgical lesion	timestamp

The horse colic dataset will be a compressed zip file consisting of the SQLite DB. For this example, we just wanted a dataset available online, but this could be easily plugged into another dataset/data management system like Snowflake, Athena, Hive, BigQuery, or Spark, all of which are supported by Flyte.

Takeaways

The example we are trying to demonstrate is a simple feature engineering job. Here are the nitty-gritties:

1. Source data is from SQL-like data sources

2. Procreated feature transforms

3. Ability to create a low-code platform

4. Feast integration

5. Serve features to production using Feast

Feature Engineering Tasks

Feature Store Dataclass

Flyte Pipeline With Feast