Contributing to Flyte

Thank you for taking the time to contribute to Flyte! Here are some guidelines for you to follow, which will make your first and follow-up contributions easier.

Note

Please read our Code of Conduct before contributing to Flyte.

Code

An issue tagged with good first issue is the best place to start for first-time contributors. Look into them here.

To take a step ahead, check out the repositories available under flyteorg.

Appetizer (for every repo): Fork and clone the concerned repository. Create a new branch on your fork and make the required changes. Create a pull request once your work is ready for review.

Note

Note: To open a pull request, follow this guide.

A piece of good news – You can be added as a committer to any ``flyteorg`` repo as you become more involved with the project.

Example PR for your reference: GitHub PR. A couple of checks are introduced to help in maintaining the robustness of the project.

  1. To get through DCO, sign off on every commit. (Reference)

  2. To improve code coverage, write unit tests to test your code.

Note

Format your Go code with golangci-lint followed by goimports (we used the same in the Makefile), and Python code with black (use make fmt command which contains both black and isort). Refer to Effective Go and Black for full coding standards.

Component Reference

Dependency Graph between various flyteorg repos

The dependency graph between various flyteorg repos

flyte

Repo

Purpose: Deployment, Documentation, and Issues

Languages: Kustomize & RST

flyteidl

Repo

Purpose: The Flyte Workflow specification in protocol buffers which forms the core of Flyte

Language: Protobuf

Setup: Refer to the README

flytepropeller

Repo | Code Reference

Purpose: Deployment, Documentation, and Issues

Languages: Kustomize & RST

Setup

  • Check for the Makefile in the root repo

  • Run the following commands:
    • make generate

    • make test_unit

    • make link

  • To compile, run make compile

flyteadmin

Repo | Code Reference

Purpose: Control Plane

Language: Go

Setup:

  • Check for the Makefile in the root repo

  • If the service code has to be tested, run it locally:
    • make compile

    • make server

  • To seed data locally:
    • make compile

    • make seed_projects

    • make migrate

  • To run integration tests locally:
    • make integration

    • (or, to run in containerized dockernetes): make k8s_integration

flytekit

Repo

Purpose: Python SDK & Tools

Language: Python

Setup: Refer to the Flytekit Contribution Guide

flyteconsole

Repo

Purpose: Admin Console

Language: Typescript

Setup: Refer to the README

datacatalog

Repo | Code Reference

Purpose: Manage Input & Output Artifacts

Language: Go

flyteplugins

Repo | Code Reference

Purpose: Flyte Plugins

Language: Go

Setup:

  • Check for the Makefile in the root repo

  • Run the following commands:
    • make generate

    • make test_unit

    • make link

flytestdlib

Repo

Purpose: Standard Library for Shared Components

Language: Go

flytesnacks

Repo

Purpose: Examples, Tips, and Tricks to use Flytekit SDKs

Language: Python (In future, Java shall be added)

Setup:

  • If the Python code has to be tested, run it locally

  • If the Python code has to be tested in a cluster:
    • Run the make start command in the root directory of the flytesnacks repo

    • Visit https://localhost:30081 to view the Flyte console consisting of the examples present in flytesnacks/cookbook/core directory

    • To fetch the new dependencies and rebuild the image, run make register

flytectl

Repo

Purpose: A Standalone Flyte CLI

Language: Go

Setup:

  • Check for the Makefile in the root repo

  • Run the following commands:
    • make generate

    • make test_unit

    • make link

Issues

GitHub Issues is used for issue tracking. There are a variety of issue types available that you could use while filing an issue.

If none of the above fits your requirements, file a blank issue.

Documentation

Flyte uses Sphinx for documentation and godocs for Golang. godocs is quite simple – comment your code and you are good to go!

Sphinx spans across multiple repositories under the flyteorg repository. It uses reStructured Text (rst) files to store the documentation content. For both the API and code-related content, it extracts docstrings from the code files.

To get started, look into reStructuredText reference.

Docs Environment Setup

Install all the requirements from the docs-requirements.txt file present in the root of a repository.

pip install -r docs-requirements.txt

From the docs directory present in the repository root (for flytesnacks, docs is present in flytesnacks/cookbook), run the command:

make html

Note

For implicit targets, run make -C docs html.

You can then view the HTML pages in the docs/_build directory.

Note

For flyte repo, there is no docs directory. Instead, consider the rsts directory. To generate HTML files, run the following command in the root of the repo.

make -C rsts html

For minor edits that don’t require a local setup, you can edit the GitHub page in the documentation to propose the improvements.

The edit option is found at the bottom of a page, as shown below.

GitHub edit option for Documentation

Intersphinx

Intersphinx can generate automatic links to the documentation of objects in other projects.

To establish a reference to any other documentation from Flyte or within it, use intersphinx.

To do so, create an intersphinx_mapping in the conf.py file present in the docs/source directory.

For example:

intersphinx_mapping = {
    "python": ("https://docs.python.org/3", None),
    "flytekit": ("https://flyte.readthedocs.io/projects/flytekit/en/master/", None),
}

Note

docs/source is present in the repository root. Click here to view the intersphinx configuration.

The key refers to the name used to refer to the file (while referencing the documentation), and the URL denotes the precise location.

Here are a couple of examples that you can refer to:

Task: :std:doc:`generated/flytekit.task`

Output:

Task: flytekit.task

:std:doc:`Using custom words <generated/flytekit.task>`

Output:

Using custom words


Linking to Python elements changes based on what you’re linking to. Check out this section to learn more.


For instance, linking to the task decorator in flytekit uses the func role.

Link to flytekit code :py:func:`flytekit:flytekit.task`

Output:

Link to flytekit code flytekit.task()


Here are a couple more examples.

:py:mod:`Module <python:typing>`
:py:class:`Class <python:typing.Type>`
:py:data:`Data <python:typing.Callable>`
:py:func:`Function <python:typing.cast>`
:py:meth:`Method <python:pprint.PrettyPrinter.format>`

Output:

Module

Class

Data

Function

Method

For feedback at any point in the contribution process, feel free to reach out to us using the links on our Community page.