Contents Menu Expand
Logo
Flyte
Getting Started
User Guide
Tutorials
Concepts
Integrations
Deployment
API Reference
Community
flyte.org
Logo
Flyte
  • Getting Started
  • User Guide
  • Tutorials
  • Concepts
  • Deployment
  • API Reference
  • Community

Flytekit SDK

  • Flytekit Python
  • Overview
    • Model Files
    • Authoring Structure
    • Control Plane
    • Execution Time Support
    • Command Line Interfaces and Clients
  • Core Flytekit
    • flytekit.task
    • flytekit.workflow
    • flytekit.kwtypes
    • flytekit.current_context
    • flytekit.ExecutionParameters
    • flytekit.FlyteContext
    • flytekit.map_task
    • flytekit.core.workflow.ImperativeWorkflow
    • flytekit.conditional
    • flytekit.TaskMetadata
    • flytekit.Resources
    • flytekit.WorkflowFailurePolicy
    • flytekit.dynamic
    • flytekit.CronSchedule
    • flytekit.FixedRate
    • flytekit.Email
    • flytekit.PagerDuty
    • flytekit.Slack
    • flytekit.get_reference_entity
    • flytekit.LaunchPlanReference
    • flytekit.TaskReference
    • flytekit.WorkflowReference
    • flytekit.reference_task
    • flytekit.reference_workflow
    • flytekit.SQLTask
    • flytekit.ContainerTask
    • flytekit.PythonFunctionTask
    • flytekit.PythonInstanceTask
    • flytekit.LaunchPlan
    • flytekit.Secret
    • flytekit.SecurityContext
    • flytekit.AuthRole
    • flytekit.Labels
    • flytekit.Annotations
    • flytekit.WorkflowExecutionPhase
    • flytekit.Blob
    • flytekit.BlobMetadata
    • flytekit.Literal
    • flytekit.Scalar
    • flytekit.LiteralType
    • flytekit.BlobType
  • Configuration
    • flytekit.configuration.Config
    • flytekit.configuration.Image
    • flytekit.configuration.ImageConfig
    • flytekit.configuration.SerializationSettings
    • flytekit.configuration.PlatformConfig
    • flytekit.configuration.StatsConfig
    • flytekit.configuration.SecretsConfig
    • flytekit.configuration.S3Config
    • flytekit.configuration.GCSConfig
    • flytekit.configuration.DataConfig
    • flytekit.configuration.Config
  • Remote Access
    • flytekit.remote.remote.FlyteRemote
    • flytekit.remote.remote.Options
    • flytekit.remote.task.FlyteTask
    • flytekit.remote.workflow.FlyteWorkflow
    • flytekit.remote.launch_plan.FlyteLaunchPlan
    • flytekit.remote.nodes.FlyteNode
    • flytekit.remote.component_nodes.FlyteTaskNode
    • flytekit.remote.component_nodes.FlyteWorkflowNode
    • flytekit.remote.executions.FlyteWorkflowExecution
    • flytekit.remote.executions.FlyteTaskExecution
    • flytekit.remote.executions.FlyteNodeExecution
  • Unit Testing
    • flytekit.testing.patch
    • flytekit.testing.task_mock
  • Extending Flytekit
    • flytekit.extend.get_serializable
    • flytekit.extend.context_manager
    • flytekit.extend.SQLTask
    • flytekit.extend.IgnoreOutputs
    • flytekit.extend.PythonTask
    • flytekit.extend.ExecutionState
    • flytekit.extend.Image
    • flytekit.extend.ImageConfig
    • flytekit.extend.SerializationSettings
    • flytekit.extend.Interface
    • flytekit.extend.Promise
    • flytekit.extend.TaskPlugins
    • flytekit.extend.DictTransformer
    • flytekit.extend.T
    • flytekit.extend.TypeEngine
    • flytekit.extend.TypeTransformer
    • flytekit.extend.PythonCustomizedContainerTask
    • flytekit.extend.ExecutableTemplateShimTask
    • flytekit.extend.ShimTaskExecutor
    • flytekit.extend.DataPersistence
    • flytekit.extend.DataPersistencePlugins
  • Plugin API reference
    • AWS Athena
      • flytekitplugins.athena.AthenaConfig
      • flytekitplugins.athena.AthenaTask
    • AWS Batch
      • flytekitplugins.awsbatch.AWSBatchConfig
    • AWS Sagemaker
      • flytekitplugins.awssagemaker.AlgorithmName
      • flytekitplugins.awssagemaker.AlgorithmSpecification
      • flytekitplugins.awssagemaker.CategoricalParameterRange
      • flytekitplugins.awssagemaker.ContinuousParameterRange
      • flytekitplugins.awssagemaker.DISTRIBUTED_TRAINING_CONTEXT_KEY
      • flytekitplugins.awssagemaker.DistributedProtocol
      • flytekitplugins.awssagemaker.DistributedTrainingContext
      • flytekitplugins.awssagemaker.HPOJob
      • flytekitplugins.awssagemaker.HyperparameterScalingType
      • flytekitplugins.awssagemaker.HyperparameterTuningJobConfig
      • flytekitplugins.awssagemaker.HyperparameterTuningObjective
      • flytekitplugins.awssagemaker.HyperparameterTuningObjectiveType
      • flytekitplugins.awssagemaker.HyperparameterTuningStrategy
      • flytekitplugins.awssagemaker.InputContentType
      • flytekitplugins.awssagemaker.InputMode
      • flytekitplugins.awssagemaker.IntegerParameterRange
      • flytekitplugins.awssagemaker.ParameterRangeOneOf
      • flytekitplugins.awssagemaker.SagemakerBuiltinAlgorithmsTask
      • flytekitplugins.awssagemaker.SagemakerCustomTrainingTask
      • flytekitplugins.awssagemaker.SagemakerHPOTask
      • flytekitplugins.awssagemaker.SagemakerTrainingJobConfig
      • flytekitplugins.awssagemaker.TrainingJobEarlyStoppingType
      • flytekitplugins.awssagemaker.TrainingJobResourceConfig
    • Google Bigquery
      • flytekitplugins.bigquery.BigQueryConfig
      • flytekitplugins.bigquery.BigQueryTask
    • FS Spec
      • flytekitplugins.fsspec.ArrowToParquetEncodingHandler
      • flytekitplugins.fsspec.FSSpecPersistence
      • flytekitplugins.fsspec.PandasToParquetEncodingHandler
      • flytekitplugins.fsspec.ParquetToArrowDecodingHandler
      • flytekitplugins.fsspec.ParquetToPandasDecodingHandler
    • Deck standard
      • flytekitplugins.deck.BoxRenderer
      • flytekitplugins.deck.FrameProfilingRenderer
      • flytekitplugins.deck.MarkdownRenderer
    • Dolt standard
      • flytekitplugins.dolt.DoltConfig
      • flytekitplugins.dolt.DoltTable
      • flytekitplugins.dolt.DoltTableNameTransformer
    • Great expectations
      • flytekitplugins.great_expectations.BatchRequestConfig
      • flytekitplugins.great_expectations.GreatExpectationsFlyteConfig
      • flytekitplugins.great_expectations.GreatExpectationsTask
      • flytekitplugins.great_expectations.GreatExpectationsType
    • Hive
      • flytekitplugins.hive.HiveConfig
      • flytekitplugins.hive.HiveSelectTask
      • flytekitplugins.hive.HiveTask
    • Pod
      • flytekitplugins.pod.Pod
    • KF MPI
      • flytekitplugins.kfmpi.MPIJob
    • KF Pytorch
      • flytekitplugins.kfpytorch.PyTorch
    • KF Tensorflow
      • flytekitplugins.kftensorflow.TfJob
    • Modin
      • flytekitplugins.modin.ModinPandasDataFrameTransformer
    • Pandera
      • flytekitplugins.pandera.PanderaTransformer
    • Papermill
      • flytekitplugins.papermill.NotebookTask
      • flytekitplugins.papermill.record_outputs
    • Snowflake
      • flytekitplugins.snowflake.SnowflakeConfig
      • flytekitplugins.snowflake.SnowflakeTask
    • Spark
      • flytekitplugins.spark.new_spark_session
      • flytekitplugins.spark.ParquetToSparkDecodingHandler
      • flytekitplugins.spark.Spark
      • flytekitplugins.spark.SparkDataFrameSchemaReader
      • flytekitplugins.spark.SparkDataFrameSchemaWriter
      • flytekitplugins.spark.SparkDataFrameTransformer
    • SQLAlchemy
      • flytekitplugins.sqlalchemy.SQLAlchemyConfig
      • flytekitplugins.sqlalchemy.SQLAlchemyDefaultImages
      • flytekitplugins.sqlalchemy.SQLAlchemyTask
  • Build Custom Task Types
    • flytekit.core.base_task.kwtypes
    • flytekit.core.base_task.PythonTask
    • flytekit.core.base_task.Task
    • flytekit.core.base_task.TaskMetadata
    • flytekit.core.base_task.TaskResolverMixin
    • flytekit.core.base_task.IgnoreOutputs
    • flytekit.core.python_function_task.PythonFunctionTask
    • flytekit.core.python_function_task.PythonInstanceTask
  • Extend Type System
    • Flytekit StructuredDataset
      • flytekit.types.structured.StructuredDataset
      • flytekit.types.structured.StructuredDatasetEncoder
      • flytekit.types.structured.StructuredDatasetDecoder
      • flytekit.types.structured.StructuredDatasetTransformerEngine
    • Flytekit File Type
      • flytekit.types.file.FlyteFile
      • flytekit.types.file.HDF5EncodedFile
      • flytekit.types.file.HTMLPage
      • flytekit.types.file.JoblibSerializedFile
      • flytekit.types.file.JPEGImageFile
      • flytekit.types.file.PDFFile
      • flytekit.types.file.PNGImageFile
      • flytekit.types.file.PythonPickledFile
      • flytekit.types.file.PythonNotebook
      • flytekit.types.file.SVGImageFile
    • Flytekit Directory Type
      • flytekit.types.directory.FlyteDirectory
      • flytekit.types.directory.TensorboardLogs
  • Extend Data Persistence layer
    • flytekit.core.data_persistence.DataPersistence
    • flytekit.core.data_persistence.DataPersistencePlugins
    • flytekit.core.data_persistence.DiskPersistence
    • flytekit.core.data_persistence.FileAccessProvider
    • flytekit.core.data_persistence.UnsupportedPersistenceOp
    • flytekit.extras.persistence.GCSPersistence
    • flytekit.extras.persistence.HttpPersistence
    • flytekit.extras.persistence.S3Persistence
  • Flytekit Contribution Guide

Extend Type SystemΒΆ

Flytekit ships with an extensible type system, which makes it easy for anyone to extend and add new types. Feel free to follow the pattern of the built-in types.

  • Flytekit StructuredDataset
  • Flytekit File Type
  • Flytekit Directory Type
Next
Flytekit StructuredDataset
Previous
flytekit.core.python_function_task.PythonInstanceTask
Copyright © 2021, Flyte | Built with Sphinx and the Furo theme. | Show Source | Edit on Github