Contributing to Flyte#

Tags: Contribute, Basic

Thank you for taking the time to contribute to Flyte! Please read our Code of Conduct before contributing to Flyte.

Here are some guidelines for you to follow, which will make your first and follow-up contributions easier.

TL;DR: Find the repo-specific contribution guidelines in the Component Reference section.

💻 Becoming a Contributor#

An issue tagged with good first issue is the best place to start for first-time contributors. You can find them here.

Appetizer for every repo: Fork and clone the concerned repository. Create a new branch on your fork and make the required changes. Create a pull request once your work is ready for review.

Note

To open a pull request, follow this guide.

Example PR for your reference: GitHub PR. A couple of checks are introduced to help maintain the robustness of the project.

  1. To get through DCO, sign off on every commit (Reference)

  2. To improve code coverage, write unit tests to test your code

  3. Make sure all the tests pass. If you face any issues, please let us know

On a side note, format your Go code with golangci-lint followed by goimports (use make lint and make goimports), and Python code with black and isort (use make fmt). If make targets are not available, you can manually format the code. Refer to Effective Go, Black, and Isort for full coding standards.

As you become more involved with the project you may be able to be added as a contributor to the repos you’re working on, but there is a medium term effort to move all development to forks ✨.

📃 Documentation#

Flyte uses Sphinx for documentation. protoc-gen-doc is used to generate the documentation from .proto files.

Sphinx spans multiple repositories under flyteorg. It uses reStructured Text (rst) files to store the documentation content. For API- and code-related content, it extracts docstrings from the code files.

To get started, refer to the reStructuredText reference.

For minor edits that don’t require a local setup, you can edit the GitHub page in the documentation to propose improvements.

The edit option can be found at the bottom of a page, as shown below.

GitHub edit option for Documentation

Intersphinx#

Intersphinx can generate automatic links to the documentation of objects in other projects.

To establish a reference to any other documentation from Flyte or within it, use Intersphinx.

To do so, create an intersphinx_mapping in the conf.py file which should be present in the respective docs repository. For example, rsts is the docs repository for the flyte repo.

For example:

intersphinx_mapping = {
    "python": ("https://docs.python.org/3", None),
    "flytekit": ("https://flyte.readthedocs.io/projects/flytekit/en/master/", None),
}

The key refers to the name used to refer to the file (while referencing the documentation), and the URL denotes the precise location.

Here is an example using :std:doc:

  • Direct reference

    Task: :std:doc:`generated/flytekit.task`

    Output:

    Task: flytekit.task

  • Custom name

    :std:doc:`Using custom words <generated/flytekit.task>`

    Output:

    Using custom words


You can cross-reference multiple Python objects. Check out this section to learn more.


For instance, task decorator in flytekit uses the func role.

Link to flytekit code :py:func:`flytekit:flytekit.task`

Output:

Link to flytekit code flytekit.task()


Here are a couple more examples.

:py:mod:`Module <python:typing>`
:py:class:`Class <python:typing.Type>`
:py:data:`Data <python:typing.Callable>`
:py:func:`Function <python:typing.cast>`
:py:meth:`Method <python:pprint.PrettyPrinter.format>`

Output:

Module

Class

Data

Function

Method

🧱 Component Reference#

To understand how the below components interact with each other, refer to Understand the lifecycle of a workflow

Dependency graph between various flyteorg repos

The dependency graph between various flyte.org repos#

flyte#

Repo

Purpose: Deployment, Documentation, and Issues

Languages: Kustomize & RST

Note

For the flyte repo, run the following command in the repo’s root to generate documentation locally.

make -C rsts html

flyteidl#

Repo

Purpose: Flyte workflow specification is in protocol buffers which forms the core of Flyte

Language: Protobuf

Guidelines: Refer to the README

flytepropeller#

Repo | Code Reference

Purpose: Kubernetes-native operator

Language: Go

Guidelines:

  • Check for Makefile in the root repo

  • Run the following commands:

    • make generate

    • make test_unit

    • make link

  • To compile, run make compile

flyteadmin#

Repo | Code Reference

Purpose: Control Plane

Language: Go

Guidelines:

  • Check for Makefile in the root repo

  • If the service code has to be tested, run it locally:

    • make compile

    • make server

  • To seed data locally:

    • make compile

    • make seed_projects

    • make migrate

  • To run integration tests locally:

    • make integration

    • (or to run in containerized dockernetes): make k8s_integration

flytekit#

Repo

Purpose: Python SDK & Tools

Language: Python

Guidelines: Refer to the Flytekit Contribution Guide

flyteconsole#

Repo

Purpose: Admin Console

Language: Typescript

Guidelines: Refer to the README

datacatalog#

Repo | Code Reference

Purpose: Manage Input & Output Artifacts

Language: Go

flyteplugins#

Repo | Code Reference

Purpose: Flyte Plugins

Language: Go

Guidelines:

  • Check for Makefile in the root repo

  • Run the following commands:

    • make generate

    • make test_unit

    • make link

flytestdlib#

Repo

Purpose: Standard Library for Shared Components

Language: Go

flytesnacks#

Repo

Purpose: Examples, Tips, and Tricks to use Flytekit SDKs

Language: Python (In the future, Java examples will be added)

Guidelines: Refer to the Flytesnacks Contribution Guide

flytectl#

Repo

Purpose: A standalone Flyte CLI

Language: Go

Guidelines: Refer to the FlyteCTL Contribution Guide

🐞 File an Issue#

GitHub Issues is used for issue tracking. The following are available issue types that you could use for filing an issue.

If none of the above fit your requirements, file a blank issue. Also, add relevant labels to your issue. For example, if you are filing a Flytekit plugin request, add the flytekit label.


For feedback at any point in the contribution process, feel free to reach out to us on Slack.