Hide navigation sidebar

Hide table of contents sidebar

Toggle site navigation sidebar

Toggle table of contents sidebar

Introduction
Quickstart guide
Getting started with workflow development
Toggle navigation of Getting started with workflow development
Flyte fundamentals
Toggle navigation of Flyte fundamentals
Flyte agents
Toggle navigation of Flyte agents
Core use cases
Toggle navigation of Core use cases

Examples

User Guide
Toggle navigation of User Guide
- Environment setup
- Basics
  Toggle navigation of Basics
- Data Types and IO
  Toggle navigation of Data Types and IO
- Advanced composition
  Toggle navigation of Advanced composition
- Customizing dependencies
  Toggle navigation of Customizing dependencies
- Development lifecycle
  Toggle navigation of Development lifecycle
- Testing
  Toggle navigation of Testing
  - Mocking tasks
- Productionize
  Toggle navigation of Productionize
- Extending Flyte
  Toggle navigation of Extending Flyte
Tutorials
Toggle navigation of Tutorials
- Diabetes Classification
  Toggle navigation of Diabetes Classification
  - Train and Validate a Diabetes Classification XGBoost Model
- House Price Regression
  Toggle navigation of House Price Regression
  - Predicting House Price in a Region Using XGBoost
  - Predicting House Price in Multiple Regions Using XGBoost and Dynamic Workflows
- MNIST Classification With PyTorch and W&B
  Toggle navigation of MNIST Classification With PyTorch and W&B
  - Single Node, Single GPU Training
  - Single Node, Multi GPU Training
- NLP Processing
  Toggle navigation of NLP Processing
  - Word Embeddings and Topic Modelling with Gensim
- Forecasting Rossman Store Sales with Horovod and Spark
  Toggle navigation of Forecasting Rossman Store Sales with Horovod and Spark
  - Data-Parallel Distributed Training Using Horovod on Spark
- EDA, Feature Engineering, and Modeling With Papermill
  Toggle navigation of EDA, Feature Engineering, and Modeling With Papermill
- Feast Integration
  Toggle navigation of Feast Integration
- Nucleotide Sequence Querying with BLASTX
  Toggle navigation of Nucleotide Sequence Querying with BLASTX
  - BLASTX Example
- Weather Forecasting
Integrations
Toggle navigation of Integrations
- SQL
  Toggle navigation of SQL
  - Sqlite3
  - SQLAlchemy
- Great Expectations
  Toggle navigation of Great Expectations
  - Task Example
  - Type Example
- Papermill
  Toggle navigation of Papermill
  - Jupyter Notebook Tasks
  - Simple Papermill Notebook
- Pandera
  Toggle navigation of Pandera
  - Basic Schema Example
  - Validating and Testing Machine Learning Pipelines
- Modin
  Toggle navigation of Modin
  - KNN Classifier
- Dolt
  Toggle navigation of Dolt
  - Quickstart
  - Dolt Branches
- DBT
  Toggle navigation of DBT
  - dbt example
- WhyLogs
  Toggle navigation of WhyLogs
  - whylogs Example
- MLFlow
  Toggle navigation of MLFlow
  - MLflow Example
- ONNX
  Toggle navigation of ONNX
- DuckDB
  Toggle navigation of DuckDB
  - DuckDB Example
- Kubernetes Pods
  Toggle navigation of Kubernetes Pods
  - Pod Example
- Dask
  Toggle navigation of Dask
  - Running a Dask Task
- Spark
  Toggle navigation of Spark
  - Running a Spark Task
  - Converting a Spark DataFrame to a Pandas DataFrame
- PyTorch Distributed
  Toggle navigation of PyTorch Distributed
  - Run PyTorch Distributed
- TensorFlow Distributed
  Toggle navigation of TensorFlow Distributed
  - Run Distributed TensorFlow Training
- MPI
  Toggle navigation of MPI
  - Running Distributed Training Using Horovod and MPI
- Ray
  Toggle navigation of Ray
  - Running Ray Tasks
- Airflow agent
  Toggle navigation of Airflow agent
  - Airflow agent example usage
- BigQuery agent
  Toggle navigation of BigQuery agent
  - BigQuery agent example usage
- ChatGPT agent
  Toggle navigation of ChatGPT agent
  - ChatGPT agent example usage
- Databricks agent example
  Toggle navigation of Databricks agent example
  - Running Spark on Databricks
- Memory Machine Cloud agent
  Toggle navigation of Memory Machine Cloud agent
  - Memory Machine Cloud agent example usage
- AWS SageMaker Inference Agent
  Toggle navigation of AWS SageMaker Inference Agent
  - Deploy and serve an XGBoost model on AWS SageMaker using FastAPI
- Sensor
  Toggle navigation of Sensor
  - File Sensor
- Snowflake agent
  Toggle navigation of Snowflake agent
  - Querying data in Snowflake
- AWS Athena
  Toggle navigation of AWS Athena
  - Athena Query
- AWS Batch
  Toggle navigation of AWS Batch
  - AWS Batch
- FlyteInteractive
  Toggle navigation of FlyteInteractive
  - FlyteInteractive Vscode Decorator
  - FlyteInteractive Jupyter Decorator
- Hive
  Toggle navigation of Hive
  - Hive Tasks
- Airflow Provider
  Toggle navigation of Airflow Provider
  - FlyteOperator Example
Deprecated integrations
Toggle navigation of Deprecated integrations
- BigQuery plugin
  Toggle navigation of BigQuery plugin
  - BigQuery example query
- Databricks plugin
  Toggle navigation of Databricks plugin
  - Running Spark on Databricks
- Memory Machine Cloud plugin
  Toggle navigation of Memory Machine Cloud plugin
  - Memory Machine Cloud
- Snowflake plugin
  Toggle navigation of Snowflake plugin
  - Querying data in Snowflake

Cluster Deployment

Getting Started
Deployment Paths
Toggle navigation of Deployment Paths
Plugin Setup
Toggle navigation of Plugin Setup
- Configure Kubernetes Plugins
- Configure AWS Plugins
  Toggle navigation of Configure AWS Plugins
- Configure GCP Plugins
  Toggle navigation of Configure GCP Plugins
  - Google BigQuery Plugin
- Configure Web APIs
  Toggle navigation of Configure Web APIs
  - Snowflake Plugin
  - Databricks Plugin
Agent Setup
Toggle navigation of Agent Setup
Configuration
Toggle navigation of Configuration
Configuration Reference
Toggle navigation of Configuration Reference
Security Overview
Flyte API Playground: Swagger

API Reference

flytekit
Toggle navigation of flytekit
- Overview
  Toggle navigation of Overview
- Core Flytekit
  Toggle navigation of Core Flytekit
- Configuration
  Toggle navigation of Configuration
- ImageSpec
  Toggle navigation of ImageSpec
  - flytekit.image_spec.ImageSpec
- Remote Access
  Toggle navigation of Remote Access
- Clients
  Toggle navigation of Clients
  - flytekit.clients.friendly.SynchronousFlyteClient
  - flytekit.clients.raw.RawSynchronousFlyteClient
- Unit Testing
  Toggle navigation of Unit Testing
  - flytekit.testing.patch
  - flytekit.testing.task_mock
- Specifying Accelerators
  Toggle navigation of Specifying Accelerators
- Extending Flytekit
  Toggle navigation of Extending Flytekit
- Flyte Deck
  Toggle navigation of Flyte Deck
- Plugin API reference
  Toggle navigation of Plugin API reference
  - AWS Athena
    Toggle navigation of AWS Athena
    - flytekitplugins.athena.AthenaConfig
    - flytekitplugins.athena.AthenaTask
  - AWS Batch
    Toggle navigation of AWS Batch
    - flytekitplugins.awsbatch.AWSBatchConfig
  - Google Bigquery
    Toggle navigation of Google Bigquery
  - Dask
    Toggle navigation of Dask
  - Deck standard
    Toggle navigation of Deck standard
  - Dolt standard
    Toggle navigation of Dolt standard
  - Great expectations
    Toggle navigation of Great expectations
  - Hive
    Toggle navigation of Hive
  - Pod
    Toggle navigation of Pod
    - flytekitplugins.pod.Pod
  - KF MPI
    Toggle navigation of KF MPI
    - flytekitplugins.kfmpi.MPIJob
  - KF Pytorch
    Toggle navigation of KF Pytorch
    - flytekitplugins.kfpytorch.PyTorch
    - flytekitplugins.kfpytorch.Elastic
  - KF Tensorflow
    Toggle navigation of KF Tensorflow
    - flytekitplugins.kftensorflow.TfJob
  - Modin
    Toggle navigation of Modin
    - flytekitplugins.modin.ModinPandasDataFrameTransformer
  - Pandera
    Toggle navigation of Pandera
    - flytekitplugins.pandera.PanderaTransformer
  - Papermill
    Toggle navigation of Papermill
    - flytekitplugins.papermill.NotebookTask
    - flytekitplugins.papermill.record_outputs
  - Snowflake
    Toggle navigation of Snowflake
  - Spark
    Toggle navigation of Spark
  - SQLAlchemy
    Toggle navigation of SQLAlchemy
  - ONNX PyTorch
    Toggle navigation of ONNX PyTorch
    - flytekitplugins.onnxpytorch.PyTorch2ONNXConfig
    - flytekitplugins.onnxpytorch.PyTorch2ONNX
  - ONNX TensorFlow
    Toggle navigation of ONNX TensorFlow
    - flytekitplugins.onnxtensorflow.TensorFlow2ONNXConfig
    - flytekitplugins.onnxtensorflow.TensorFlow2ONNX
  - ONNX ScikitLearn
    Toggle navigation of ONNX ScikitLearn
    - flytekitplugins.onnxscikitlearn.ScikitLearn2ONNXConfig
    - flytekitplugins.onnxscikitlearn.ScikitLearn2ONNX
  - Ray
    Toggle navigation of Ray
  - DBT
    Toggle navigation of DBT
  - Vaex
    Toggle navigation of Vaex
    - flytekitplugins.vaex.VaexDataFrameToParquetEncodingHandler
    - flytekitplugins.vaex.ParquetToVaexDataFrameDecodingHandler
  - MLflow
    Toggle navigation of MLflow
    - flytekitplugins.mlflow.mlflow_autolog
  - DuckDB
    Toggle navigation of DuckDB
    - flytekitplugins.duckdb.DuckDBQuery
  - SageMaker Inference
    Toggle navigation of SageMaker Inference
- Custom Tasks
  Toggle navigation of Custom Tasks
  - flytekit.core.base_task.kwtypes
  - flytekit.core.base_task.PythonTask
  - flytekit.core.base_task.Task
  - flytekit.core.base_task.TaskResolverMixin
  - flytekit.core.base_task.IgnoreOutputs
  - flytekit.core.python_function_task.PythonFunctionTask
  - flytekit.core.python_function_task.PythonInstanceTask
  - Shell Task
    Toggle navigation of Shell Task
    - flytekit.extras.tasks.shell.ShellTask
    - flytekit.extras.tasks.shell.OutputLocation
  - SQLite3 Task
    Toggle navigation of SQLite3 Task
    - flytekit.extras.sqlite3.task.SQLite3Task
    - flytekit.extras.sqlite3.task.SQLite3Config
- Custom Types
  Toggle navigation of Custom Types
  - Flytekit StructuredDataset
    Toggle navigation of Flytekit StructuredDataset
  - Flytekit File Type
    Toggle navigation of Flytekit File Type
  - Flytekit Directory Type
    Toggle navigation of Flytekit Directory Type
  - PyTorch Type
    Toggle navigation of PyTorch Type
  - TensorFlow Type
    Toggle navigation of TensorFlow Type
    - flytekit.extras.tensorflow.TensorFlowRecordFileTransformer
    - flytekit.extras.tensorflow.TensorFlowRecordsDirTransformer
  - Sklearn Type
    Toggle navigation of Sklearn Type
    - flytekit.extras.sklearn.SklearnEstimatorTransformer
- Experimental Features
  Toggle navigation of Experimental Features
- Pyflyte CLI
- Flytekit Contribution Guide
flytectl
Toggle navigation of flytectl
- Overview
- CLI Entrypoint
- Verbs
  Toggle navigation of Verbs
- Nouns
  Toggle navigation of Nouns
  - Project
    Toggle navigation of Project
  - Execution
    Toggle navigation of Execution
  - Workflow
    Toggle navigation of Workflow
    - flytectl get workflow
    - flytectl update workflow-meta
  - Task
    Toggle navigation of Task
    - flytectl get task
    - flytectl update task-meta
  - Task resource attribute
    Toggle navigation of Task resource attribute
  - Cluster resource attribute
    Toggle navigation of Cluster resource attribute
  - Execution cluster label
    Toggle navigation of Execution cluster label
  - Execution queue attribute
    Toggle navigation of Execution queue attribute
  - Plugin override
    Toggle navigation of Plugin override
  - Launchplan
    Toggle navigation of Launchplan
  - Workflow execution config
    Toggle navigation of Workflow execution config
  - Examples
    Toggle navigation of Examples
    - flytectl register examples
  - Files
    Toggle navigation of Files
    - flytectl register files
  - Config
    Toggle navigation of Config
  - Sandbox
    Toggle navigation of Sandbox
  - Demo
    Toggle navigation of Demo
- Contributing Guide
flyteidl
Toggle navigation of flyteidl

Ecosystem

flytekit-java
unionml
pterodactyl
latch sdk

Community

Community Resources
Contributing to Flyte
Contributing Examples
Roadmap
Frequently Asked Questions
Troubleshooting Guide

Glossary

Main Concepts
Toggle navigation of Main Concepts
Control Plane
Toggle navigation of Control Plane
Component Architecture
Toggle navigation of Component Architecture
- FlytePropeller Architecture
- Flyte Native Scheduler Architecture

v: latest

Versions: latest; stable; v1.11.0; v1.10.7; v1.10.6

Downloads

On Read the Docs: Project Home; Builds

Flyte

Toggle table of contents sidebar

flytekit.extend.T¶

flytekit.extend.TypeEngine

flytekit.extend.DictTransformer

Copyright © 2022, Flyte Authors

flytekit: dev, afa1fc7
flytesnacks: dev, 7a11248
flytectl: dev, 46c6751