Authenticating in Flyte

Tags: Authentication, Infrastructure, Advanced

The Flyte platform consists of multiple components. Securing communication between each component is crucial to ensure the integrity of the overall system.

The following diagram summarizes the components and their interactions as part of Flyte’s auth implementation:

In summary, there are two main resources required for a complete auth flow in Flyte:

An identity layer

Using an implementation of the Open ID Connect (OIDC) specification, it enables clients to verify the identity of the end user based on the authentication performed by an Authorization Server. For this flow to work, you must first register Flyte as a new client (app) to the Identity Provider (IdP).

An authorization server

As defined by IETF’s RFC #6749, the authorization server’s role is to issue access tokens to the client after successfully authenticating the resource owner and obtaining authorization. In this context, the resource owner is the end user of Flyte; and the client is the tool or component that intends to interact with flyteadmin : flytepropeller, flyteconsole or any of the CLI tools.

There are two supported options to use an authorization server in Flyte:

• Internal authorization server: It comes pre-installed with Flyte and it is a suitable choice for quick start and testing purposes.

• External (custom) authorization server: This a service provided by one of the supported IdPs and is the recommended option if your organization needs to retain control over scope definitions and grants, token expiration policies and other advanced security controls.

Note

Regardless of the type of authorization server to use, you will still need an IdP to provide identity through OIDC.

Authentication Setup

Prerequisites

The following is required for non-sandbox (non flytectl demo) deployments:

• A public domain name (e.g. example.foobar.com)

• Routing of traffic from that domain name to the Kubernetes Flyte Ingress IP address

Note

Checkout this community-maintained guide for more information about setting up Flyte in production, including Ingress.

Identity Providers Support

Flyte supports OAuth2 and OpenId Connect to secure the various connections:

• OpenID Connect: used to secure user’s authentication to flyteadmin through the UI.

• OAuth2: used to secure connections from clients (i.e. pyflyte, flytectl and flytepropeller) to the flyteadmin service.

Support for these protocols varies per IdP. Checkout the following table to understand the available support level for your IdP:

Feature	Okta	Google free	GCP Identity Service	Azure AD	Auth0	KeyCloak	Github
OpenID Connect (OIDC)	Yes	Yes	Yes	Yes	Yes	Yes	No
Custom Auth Server	Yes	No	Yes	Yes	?	Yes	No

Identity Management layer : OIDC

In this section, you can find canonical examples of how to set up OIDC on some of the supported IdPs; enabling users to authenticate in the browser.

GoogleOktaKeycloakMicrosoft Azure AD

• Create an OAuth2 Client Credential following the official documentation and take note of the client_id and client_secret

• In the Authorized redirect URIs field, add http://localhost:30081/callback for sandbox deployments, or https://<your-deployment-URL>/callback for other methods of deployment.

Apply OIDC Configuration

flyte-binaryflyte-core

1. Generate a random password to be used internally by flytepropeller

2. Use the following command to generate a bcrypt hash for that password:

pip install bcrypt && python -c 'import bcrypt; import base64; print(base64.b64encode(bcrypt.hashpw("<your-random-password>".encode("utf-8"), bcrypt.gensalt(6))))'

3. Go to your values file and locate the auth section and replace values accordingly:

auth:
 enabled: true
 oidc:
   # baseUrl: https://accounts.google.com # Uncomment for Google
   # baseUrl: https://<keycloak-url>/auth/realms/<keycloak-realm> # Uncomment for Keycloak and update with your installation host and realm name
   # baseUrl: https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/authorize # Uncomment for Azure AD
   # For Okta use the Issuer URI from Okta's default auth server
   baseUrl: https://dev-<org-id>.okta.com/oauth2/default
   # Replace with the client ID and secret created for Flyte in your IdP
   clientId: <client_ID>
   clientSecret: <client_secret>
 internal:
   clientSecret: '<your-random-password>'
   # Use the output of step #2 (only the content inside of '')

   clientSecretHash: <your-hashed-password>

 authorizedUris:
 - https://<your-flyte-deployment-URL>

4. Save your changes

5. Upgrade your Helm release with the new values:

helm upgrade <release-name> flyteorg/flyte-binary -n <your-namespace> --values <your-values-file>.yaml

Where:

• <release-name> is the name of your Helm release, typically flyte-backend. You can find it using helm ls -n <your-namespace>

6. Verify that your Flyte deployment now requires successful login to your IdP to access the UI (https://<your domain>/console)

7. For flytectl / pyflyte, make sure that your local config file ($HOME/.flyte/config.yaml) includes the following option:

admin:
  ...
  authType: Pkce #change from the default `clientCred` to enable client auth without using shared secrets
  ...

Note

Congratulations!

It should now be possible to go to Flyte UI and be prompted for authentication. Flytectl should automatically pickup the change and start prompting for authentication as well. If you want to use an external OAuth2 provider for App authentication, please continue reading into the next section.

Custom Authorization Server

As mentioned previously, Flyte ships with an internal authorization server; hence setting up an external Authorization Server is optional and dependent on your organization’s security requirements.

In this section, you will find instructions on how to setup an OAuth2 Authorization Server in the different IdPs supported by Flyte:

Note

Google IdP

Google IdP does not offer an OAuth2 Authorization Server that could be used to protect external services (For example Flyte). In this case, Google offers a separate Cloud Product called Google Cloud Identity. Configuration for Cloud Identity is not included in this guide. If unavailable, setup can stop here and FlyteAdmin BuiltIn OAuth2 Authorization Server can be used instead.

OktaKeycloakAzure AD

Okta’s custom authorization servers are available through an add-on license. The free developer accounts do include access, which you can use to test before rolling out the configuration more broadly.

1. From the left-hand menu, go to Security > API

2. Click on Add Authorization Server.

3. Assign an informative name and set the audience to the public URL of FlyteAdmin (e.g. https://example.foobar.com).

Note

The audience must exactly match one of the URIs in the authorizedUris section above

4. Note down the Issuer URI; this will be used for all the baseUrl settings in the Flyte config.

5. Go to Scopes and click Add Scope.

6. Set the name to all (required) and check Required under the User consent option.

7. Uncheck the Block services from requesting this scope option and save your changes.

8. Add another scope, named offline. Check both the Required and Include in public metadata options.

9. Uncheck the Block services from requesting this scope option.

10. Click Save.

11. Go to Access Policies, click Add New Access Policy. Enter a name and description and enable Assign to - All clients.

12. Add a rule to the policy with the default settings (you can fine-tune these later).

13. Navigate back to the Applications section.

14. Create an integration for flytectl; it should be created with the OIDC - OpenID Connect sign-on method, and the Native Application type.

15. Add http://localhost:53593/callback to the sign-in redirect URIs. The other options can remain as default.

16. Assign this integration to any Okta users or groups who should be able to use the flytectl tool.

17. Note down the Client ID; there will not be a secret.

18. Create an integration for flytepropeller; it should be created with the OIDC - OpenID Connect sign-on method and Web Application type.

19. Check the Client Credentials option under Client acting on behalf of itself.

20. This app does not need a specific redirect URI; nor does it need to be assigned to any users.

21. Note down the Client ID and Client secret; you will need these later.

22. Take note of the Issuer URI for your Authorization Server. It will be used as the baseURL parameter in the Helm chart

You should have three integrations total - one for the web interface (flyteconsole), one for flytectl, and one for flytepropeller.

Apply external auth server configuration

Follow the steps in this section to configure flyteadmin to use an external auth server. This section assumes that you have already completed and applied the configuration for the OIDC Identity Layer.

flyte-binaryflyte-coreflyte-core with Azure AD

1. Go to the values YAML file you used to install Flyte using a Helm chart

2. Find the auth section and follow the inline comments to insert your configuration:

auth:
  enabled: true
  oidc:
  # baseUrl: https://<keycloak-url>/auth/realms/<keycloak-realm> # Uncomment for Keycloak and update with your installation host and realm name
  # baseUrl: https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/authorize # Uncomment for Azure AD
  # For Okta, use the Issuer URI of the custom auth server:
    baseUrl: https://dev-<org-id>.okta.com/oauth2/<auth-server-id>
  # Use the client ID and secret generated by your IdP for the first OIDC registration in the "Identity Management layer : OIDC" section of this guide
    clientId: <oidc-clientId>
    clientSecret: <oidc-clientSecret>
  internal:
  # Use the clientID generated by your IdP for the flytepropeller app registration
    clientId: <flytepropeller-client-id>
  #Use the secret generated by your IdP for flytepropeller
    clientSecret: '<flytepropeller-client-secret-non-encoded>'
  # Use the bcrypt hash for the clientSecret
    clientSecretHash: <-flytepropeller-secret-bcrypt-hash>
  authorizedUris:
  # Use here the exact same value used for 'audience' when the Authorization server was configured
  - https://<your-flyte-deployment-URL>

3. Find the inline section of the values file and add the following content, replacing where needed:

inline:
  auth:
    appAuth:
      authServerType: External
      externalAuthServer:
      # baseUrl: https://<keycloak-url>/auth/realms/<keycloak-realm> # Uncomment for Keycloak and update with your installation host and realm name
      # baseUrl: https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/authorize # Uncomment for Azure AD
      # For Okta, use the Issuer URI of the custom auth server:
        baseUrl: https://dev-<org-id>.okta.com/oauth2/<auth-server-id>
        metadataUrl: .well-known/oauth-authorization-server
      thirdPartyConfig:
        flyteClient:
          # Use the clientID generated by your IdP for the `flytectl` app registration
          clientId: <flytectl-client-id>
          redirectUri: http://localhost:53593/callback
          scopes:
          - offline
          - all
    userAuth:
      openId:
      # baseUrl: https://<keycloak-url>/auth/realms/<keycloak-realm> # Uncomment for Keycloak and update with your installation host and realm name
      # baseUrl: https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/authorize # Uncomment for Azure AD
      # For Okta, use the Issuer URI of the custom auth server:
        baseUrl: https://dev-<org-id>.okta.com/oauth2/<auth-server-id>
        scopes:
        - profile
        - openid
      # - offline_access # Uncomment if your IdP supports issuing refresh tokens (optional)
      # Use the client ID and secret generated by your IdP for the first OIDC registration in the "Identity Management layer : OIDC" section of this guide
        clientId: <oidc-clientId>

4. Save your changes

5. Upgrade your Helm release with the new configuration:

helm upgrade  <release-name> flyteorg/flyte-core -n <your-namespace> --values <your-updated-values-filel>.yaml

Note

Congratulations

At this point, every interaction with Flyte components -be it in the UI or CLI- should require a successful login to your IdP, where your security policies are maintained and enforced.

Disable Helm secret management

Alternatively, you can instruct Helm not to create and manage the secret for flytepropeller. In that case, you’ll have to create it following these steps:

1. Disable Helm secrets management in your values file

secrets:
  adminOauthClientCredentials:
    enabled: false #set to false
    # Replace with the client_id provided by provided by your IdP for flytepropeller.
    clientId: <client_id>

2. Create a secret declaratively:

 apiVersion: v1
 kind: Secret
 metadata:
  name: flyte-secret-auth
  namespace: flyte
 type: Opaque
 stringData:
# Replace with the client_secret provided by your IdP for flytepropeller.
   client_secret: <client_secret>

Continuous Integration - CI

If your organization does any automated registration, then you’ll need to authenticate with the client credentials flow. After retrieving an access token from the IDP, you can send it along to flyteadmin` as usual.

flytectlFlytekit / pyflyte

Flytectl’s config.yaml can be configured to use either PKCE (Proof key for code exchange) or Client Credentials (Client Credentials) flows.

1. Update config.yaml as follows:

admin:
    # Update with the Flyte's ingress endpoint (e.g. flyteIngressIP for sandbox or example.foobar.com)
    # You must keep the 3 forward-slashes after dns:
    endpoint: dns:///<Flyte ingress url>

    # Update auth type to `Pkce` or `ClientSecret`
    authType: Pkce

    # Set to the clientId (will be used for both Pkce and ClientSecret flows)
    # Leave empty to use the value discovered through flyteAdmin's Auth discovery endpoint.
    clientId: <Id>

    # Set to the location where the client secret is mounted.
    # Only needed/used for `ClientSecret` flow.
    clientSecretLocation: </some/path/to/key>

    # If required, set the scopes needed here. Otherwise, flytectl will discover scopes required for OpenID
    # Connect through flyteAdmin's Auth discovery endpoint.
    # scopes: [ "scope1", "scope2" ]

To read further about the available config options, please visit here

References

This collection of RFCs may be helpful to those who wish to investigate the implementation in more depth.

• OAuth2 RFC 6749

• OAuth Discovery RFC 8414

• PKCE RFC 7636

• JWT RFC 7519

There’s also more detailed information about the authentication flows in the Understanding Authentication.