MLFlow

Tags: Integration, Data, Metrics, Intermediate

The MLflow Tracking component is an API and UI for logging parameters, code versions, metrics, and output files when running your machine learning code and for later visualizing the results

First, install the Flyte MLflow plugin:

pip install flytekitplugins-mlflow

To log the metrics and parameters to Flyte deck, add @mlflow_autolog to the task. For example

@task(disable_deck=False)
@mlflow_autolog(framework=mlflow.keras)
def train_model(epochs: int):
...

To log the metric and parameters to a remote mlflow server, add default environment variable MLFLOW_TRACKING_URI to the flytepropeller config map.

kubectl edit cm flyte-propeller-config

plugins:
  k8s:
    default-cpus: 100m
    default-env-vars:
    - MLFLOW_TRACKING_URI: postgresql+psycopg2://postgres:@postgres.flyte.svc.cluster.local:5432/flyteadmin

MLflow Example