Contributing to Flyte

Thank you for taking the time to contribute to Flyte! Here are some guidelines for you to follow, which will make your first and follow-up contributions easier.

Note

Please read our Code of Conduct before contributing to Flyte.

Code

An issue tagged with good first issue is the best place to start for first-time contributors. Look into them here.

To take a step ahead, check out the repositories available under flyteorg.

Appetizer (for every repo): Fork and clone the concerned repository. Create a new branch on your fork and make the required changes. Create a pull request once your work is ready for review.

Note

Note: To open a pull request, follow this guide.

A piece of good news – You can be added as a committer to any ``flyteorg`` repo as you become more involved with the project.

Example PR for your reference: GitHub PR. A couple of checks are introduced to help in maintaining the robustness of the project.

  1. To get through DCO, sign off on every commit. (Reference)

  2. To improve code coverage, write unit tests to test your code.

Note

Format your Go code with golangci-lint followed by goimports (we used the same in the Makefile), and Python code with black (use make fmt command which contains both black and isort).

Environment Setup

Dependency Graph between various flyteorg repos

The dependency graph between various flyteorg repos

  1. flyte
    Purpose: Deployment, Documentation, and Issues
    Languages: Kustomize & RST
  2. flyteidl
    Purpose: The Flyte Workflow specification in protocol buffers which forms the core of Flyte
    Language: Protobuf
    Setup: Refer to the README
  3. flytepropeller
    Purpose: Kubernetes native execution engine for Flyte Workflows and Tasks
    Language: Go
    Setup:
    • Check for the Makefile in the root repo

    • Run the following commands:
      • make generate

      • make test_unit

      • make link

    • To compile, run make compile

  4. flyteadmin
    Purpose: Control Plane
    Language: Go
    Setup:
    • Check for the Makefile in the root repo

    • If the service code has to be tested, run it locally:
      • make compile

      • make server

    • To seed data locally:
      • make compile

      • make seed_projects

      • make migrate

    • To run integration tests locally:
      • make integration

      • (or, to run in containerized dockernetes): make k8s_integration

  5. flytekit
    Purpose: Python SDK & Tools
    Language: Python
    Setup: Refer to the Flytekit Contribution Guide
  6. flyteconsole
    Purpose: Admin Console
    Language: Typescript
    Setup: Refer to the README
  7. datacatalog
    Purpose: Manage Input & Output Artifacts
    Language: Go
  8. flyteplugins
    Purpose: Flyte Plugins
    Language: Go
    Setup:
    • Check for the Makefile in the root repo

    • Run the following commands:
      • make generate

      • make test_unit

      • make link

  9. flytestdlib
    Purpose: Standard Library for Shared Components
    Language: Go
  10. flytesnacks
    Purpose: Examples, Tips, and Tricks to use Flytekit SDKs
    Language: Python (In future, Java shall be added)
    Setup:
    • If the Python code has to be tested, run it locally

    • If the Python code has to be tested in a cluster:
      • Run the make start command in the root directory of the flytesnacks repo

      • Visit https://localhost:30081 to view the Flyte console consisting of the examples present in flytesnacks/cookbook/core directory

      • To fetch the new dependencies and rebuild the image, run make register

  11. flytectl
    Purpose: A Standalone Flyte CLI
    Language: Go
    Setup:
    • Check for the Makefile in the root repo

    • Run the following commands:
      • make generate

      • make test_unit

      • make link

Issues

GitHub Issues is used for issue tracking. There are a variety of issue types available that you could use while filing an issue.

If none of the above fits your requirements, file a blank issue.

Documentation

Flyte uses Sphinx for documentation and godocs for Golang. godocs is quite simple – comment your code and you are good to go!

Sphinx spans across multiple repositories under the flyteorg repository. It uses reStructured Text (rst) files to store the documentation content. For both the API and code-related content, it extracts docstrings from the code files.

To get started, look into reStructuredText reference.

Docs Environment Setup

Install all the requirements from the docs-requirements.txt file present in the root of a repository.

pip install -r docs-requirements.txt

From the docs directory present in the repository root (for flytesnacks, docs is present in flytesnacks/cookbook), run the command:

make html

Note

For implicit targets, run make -C docs html.

You can then view the HTML pages in the docs/_build directory.

Note

For flyte repo, there is no docs directory. Instead, consider the rsts directory. To generate HTML files, run the following command in the root of the repo.

make -C rsts html

For minor edits that don’t require a local setup, you can edit the GitHub page in the documentation to propose the improvements.

The edit option is found at the bottom of a page, as shown below.

GitHub edit option for Documentation

Intersphinx

Intersphinx can generate automatic links to the documentation of objects in other projects.

To establish a reference to any other documentation from Flyte or within it, use intersphinx.

To do so, create an intersphinx_mapping in the conf.py file present in the docs/source directory.

For example:

intersphinx_mapping = {
    "python": ("https://docs.python.org/3", None),
    "flytekit": ("https://flyte.readthedocs.io/projects/flytekit/en/master/", None),
}

Note

docs/source is present in the repository root. Click here to view the intersphinx configuration.

The key refers to the name used to refer to the file (while referencing the documentation), and the URL denotes the precise location.

Here are a couple of examples that you can refer to:

Task: :std:doc:`generated/flytekit.task`

Output:

Task: flytekit.task

:std:doc:`Using custom words <generated/flytekit.task>`

Output:

Using custom words


Linking to Python elements changes based on what you’re linking to. Check out this section to learn more.


For instance, linking to the task decorator in flytekit uses the func role.

Link to flytekit code :py:func:`flytekit:flytekit.task`

Output:

Link to flytekit code flytekit.task()


Here are a couple more examples.

:py:mod:`Module <python:typing>`
:py:class:`Class <python:typing.Type>`
:py:data:`Data <python:typing.Callable>`
:py:func:`Function <python:typing.cast>`
:py:meth:`Method <python:pprint.PrettyPrinter.format>`

Output:

Module

Class

Data

Function

Method