Flyte Scheduler Configuration¶
Section: admin¶
endpoint (config.URL)¶
For admin types, specify where the uri of the service is located.
Default Value:
""
insecure (bool)¶
Use insecure connection.
Default Value:
"false"
insecureSkipVerify (bool)¶
InsecureSkipVerify controls whether a client verifies the server’s certificate chain and host name. Caution : shouldn’t be use for production usecases’
Default Value:
"false"
caCertFilePath (string)¶
Use specified certificate file to verify the admin server peer.
Default Value:
""
maxBackoffDelay (config.Duration)¶
Max delay for grpc backoff
Default Value:
8s
perRetryTimeout (config.Duration)¶
gRPC per retry timeout
Default Value:
15s
maxRetries (int)¶
Max number of gRPC retries
Default Value:
"4"
authType (uint8)¶
Type of OAuth2 flow used for communicating with admin.ClientSecret,Pkce,ExternalCommand are valid values
Default Value:
ClientSecret
tokenRefreshWindow (config.Duration)¶
Max duration between token refresh attempt and token expiry.
Default Value:
0s
useAuth (bool)¶
Deprecated: Auth will be enabled/disabled based on admin’s dynamically discovered information.
Default Value:
"false"
clientId (string)¶
Client ID
Default Value:
flytepropeller
clientSecretLocation (string)¶
File containing the client secret
Default Value:
/etc/secrets/client_secret
scopes ([]string)¶
List of scopes to request
Default Value:
[]
tokenUrl (string)¶
OPTIONAL: Your IdP’s token endpoint. It’ll be discovered from flyte admin’s OAuth Metadata endpoint if not provided.
Default Value:
""
pkceConfig (pkce.Config)¶
Config for Pkce authentication flow.
Default Value:
refreshTime: 5m0s
timeout: 15s
command ([]string)¶
Command for external authentication token generation
Default Value:
[]
config.Duration¶
Duration (int64)¶
Default Value:
8s
config.URL¶
URL (url.URL)¶
Default Value:
ForceQuery: false
Fragment: ""
Host: ""
Opaque: ""
Path: ""
RawFragment: ""
RawPath: ""
RawQuery: ""
Scheme: ""
User: null
url.URL¶
Scheme (string)¶
Default Value:
""
Opaque (string)¶
Default Value:
""
User (url.Userinfo)¶
Default Value:
null
Host (string)¶
Default Value:
""
Path (string)¶
Default Value:
""
RawPath (string)¶
Default Value:
""
ForceQuery (bool)¶
Default Value:
"false"
RawQuery (string)¶
Default Value:
""
Fragment (string)¶
Default Value:
""
RawFragment (string)¶
Default Value:
""
pkce.Config¶
timeout (config.Duration)¶
Default Value:
15s
refreshTime (config.Duration)¶
Default Value:
5m0s
Section: cloudevents¶
enable (bool)¶
Default Value:
"false"
type (string)¶
Default Value:
local
aws (interfaces.AWSConfig)¶
Default Value:
region: ""
gcp (interfaces.GCPConfig)¶
Default Value:
projectId: ""
kafka (interfaces.KafkaConfig)¶
Default Value:
Version: {}
brokers: null
eventsPublisher (interfaces.EventsPublisherConfig)¶
Default Value:
eventTypes: null
topicName: ""
reconnectAttempts (int)¶
Default Value:
"0"
reconnectDelaySeconds (int)¶
Default Value:
"0"
interfaces.AWSConfig¶
region (string)¶
Default Value:
""
interfaces.EventsPublisherConfig¶
topicName (string)¶
Default Value:
""
eventTypes ([]string)¶
Default Value:
null
interfaces.GCPConfig¶
projectId (string)¶
Default Value:
""
interfaces.KafkaConfig¶
Version (sarama.KafkaVersion)¶
Default Value:
{}
brokers ([]string)¶
Default Value:
null
sarama.KafkaVersion¶
version (array)¶
Default Value:
'[0 0 0 0]'
Section: cluster_resources¶
templatePath (string)¶
Default Value:
""
templateData (map[string]interfaces.DataSource)¶
Default Value:
{}
refreshInterval (config.Duration)¶
Default Value:
1m0s
customData (map[string]map[string]interfaces.DataSource)¶
Default Value:
{}
standaloneDeployment (bool)¶
Whether the cluster resource sync is running in a standalone deployment and should call flyteadmin service endpoints
Default Value:
"false"
Section: clusters¶
clusterConfigs ([]interfaces.ClusterConfig)¶
Default Value:
null
labelClusterMap (map[string][]interfaces.ClusterEntity)¶
Default Value:
null
Section: database¶
host (string)¶
Default Value:
""
port (int)¶
Default Value:
"0"
dbname (string)¶
Default Value:
""
username (string)¶
Default Value:
""
password (string)¶
Default Value:
""
passwordPath (string)¶
Default Value:
""
options (string)¶
Default Value:
""
debug (bool)¶
Default Value:
"false"
enableForeignKeyConstraintWhenMigrating (bool)¶
Whether to enable gorm foreign keys when migrating the db
Default Value:
"false"
maxIdleConnections (int)¶
maxIdleConnections sets the maximum number of connections in the idle connection pool.
Default Value:
"10"
maxOpenConnections (int)¶
maxOpenConnections sets the maximum number of open connections to the database.
Default Value:
"1000"
connMaxLifeTime (config.Duration)¶
sets the maximum amount of time a connection may be reused
Default Value:
1h0m0s
postgres (database.PostgresConfig)¶
Default Value:
dbname: postgres
debug: false
host: postgres
options: sslmode=disable
password: ""
passwordPath: ""
port: 5432
username: postgres
sqlite (database.SQLiteConfig)¶
Default Value:
file: ""
database.PostgresConfig¶
host (string)¶
The host name of the database server
Default Value:
postgres
port (int)¶
The port name of the database server
Default Value:
"5432"
dbname (string)¶
The database name
Default Value:
postgres
username (string)¶
The database user who is connecting to the server.
Default Value:
postgres
password (string)¶
The database password.
Default Value:
""
passwordPath (string)¶
Points to the file containing the database password.
Default Value:
""
options (string)¶
See http://gorm.io/docs/connecting_to_the_database.html for available options passed, in addition to the above.
Default Value:
sslmode=disable
debug (bool)¶
Whether or not to start the database connection with debug mode enabled.
Default Value:
"false"
database.SQLiteConfig¶
file (string)¶
The path to the file (existing or new) where the DB should be created / stored. If existing, then this will be re-used, else a new will be created
Default Value:
""
Section: domains¶
id (string)¶
Default Value:
development
name (string)¶
Default Value:
development
Section: externalevents¶
enable (bool)¶
Default Value:
"false"
type (string)¶
Default Value:
local
aws (interfaces.AWSConfig)¶
Default Value:
region: ""
gcp (interfaces.GCPConfig)¶
Default Value:
projectId: ""
eventsPublisher (interfaces.EventsPublisherConfig)¶
Default Value:
eventTypes: null
topicName: ""
reconnectAttempts (int)¶
Default Value:
"0"
reconnectDelaySeconds (int)¶
Default Value:
"0"
Section: flyteadmin¶
roleNameKey (string)¶
Default Value:
""
metricsScope (string)¶
Default Value:
'flyte:'
profilerPort (int)¶
Default Value:
"10254"
metadataStoragePrefix ([]string)¶
Default Value:
- metadata
- admin
eventVersion (int)¶
Default Value:
"2"
asyncEventsBufferSize (int)¶
Default Value:
"100"
maxParallelism (int32)¶
Default Value:
"25"
labels (map[string]string)¶
Default Value:
null
annotations (map[string]string)¶
Default Value:
null
interruptible (bool)¶
Default Value:
"false"
assumableIamRole (string)¶
Default Value:
""
k8sServiceAccount (string)¶
Default Value:
default
outputLocationPrefix (string)¶
Default Value:
""
Section: logger¶
show-source (bool)¶
Includes source code location in logs.
Default Value:
"false"
mute (bool)¶
Mutes all logs regardless of severity. Intended for benchmarks/tests only.
Default Value:
"false"
level (int)¶
Sets the minimum logging level.
Default Value:
"3"
formatter (logger.FormatterConfig)¶
Sets logging format.
Default Value:
type: json
logger.FormatterConfig¶
type (string)¶
Sets logging format type.
Default Value:
json
Section: namespace_mapping¶
mapping (string)¶
Default Value:
""
template (string)¶
Default Value:
'{{ project }}-{{ domain }}'
templateData (map[string]interfaces.DataSource)¶
Default Value:
null
Section: notifications¶
type (string)¶
Default Value:
local
region (string)¶
Default Value:
""
aws (interfaces.AWSConfig)¶
Default Value:
region: ""
gcp (interfaces.GCPConfig)¶
Default Value:
projectId: ""
publisher (interfaces.NotificationsPublisherConfig)¶
Default Value:
topicName: ""
processor (interfaces.NotificationsProcessorConfig)¶
Default Value:
accountId: ""
queueName: ""
emailer (interfaces.NotificationsEmailerConfig)¶
Default Value:
body: ""
emailServerConfig:
apiKeyEnvVar: ""
apiKeyFilePath: ""
serviceName: ""
sender: ""
subject: ""
reconnectAttempts (int)¶
Default Value:
"0"
reconnectDelaySeconds (int)¶
Default Value:
"0"
interfaces.NotificationsEmailerConfig¶
emailServerConfig (interfaces.EmailServerConfig)¶
Default Value:
apiKeyEnvVar: ""
apiKeyFilePath: ""
serviceName: ""
subject (string)¶
Default Value:
""
sender (string)¶
Default Value:
""
body (string)¶
Default Value:
""
interfaces.EmailServerConfig¶
serviceName (string)¶
Default Value:
""
apiKeyEnvVar (string)¶
Default Value:
""
apiKeyFilePath (string)¶
Default Value:
""
interfaces.NotificationsProcessorConfig¶
queueName (string)¶
Default Value:
""
accountId (string)¶
Default Value:
""
interfaces.NotificationsPublisherConfig¶
topicName (string)¶
Default Value:
""
Section: qualityofservice¶
tierExecutionValues (map[string]interfaces.QualityOfServiceSpec)¶
Default Value:
{}
defaultTiers (map[string]string)¶
Default Value:
{}
Section: queues¶
executionQueues (interfaces.ExecutionQueues)¶
Default Value:
[]
workflowConfigs (interfaces.WorkflowConfigs)¶
Default Value:
[]
Section: registration¶
maxWorkflowNodes (int)¶
Default Value:
"100"
maxLabelEntries (int)¶
Default Value:
"0"
maxAnnotationEntries (int)¶
Default Value:
"0"
workflowSizeLimit (string)¶
Default Value:
""
Section: remotedata¶
scheme (string)¶
Default Value:
none
region (string)¶
Default Value:
""
signedUrls (interfaces.SignedURL)¶
Default Value:
durationMinutes: 0
enabled: false
signingPrincipal: ""
maxSizeInBytes (int64)¶
Default Value:
"2097152"
inlineEventDataPolicy (int)¶
Specifies how inline execution event data should be saved in the backend
Default Value:
Offload
interfaces.SignedURL¶
enabled (bool)¶
Whether signed urls should even be returned with GetExecutionData, GetNodeExecutionData and GetTaskExecutionData response objects.
Default Value:
"false"
durationMinutes (int)¶
Default Value:
"0"
signingPrincipal (string)¶
Default Value:
""
Section: scheduler¶
profilerPort (config.Port)¶
Default Value:
10254
eventScheduler (interfaces.EventSchedulerConfig)¶
Default Value:
aws: null
local: {}
region: ""
scheduleNamePrefix: ""
scheduleRole: ""
scheme: local
targetName: ""
workflowExecutor (interfaces.WorkflowExecutorConfig)¶
Default Value:
accountId: ""
aws: null
local:
adminRateLimit:
burst: 10
tps: 100
region: ""
scheduleQueueName: ""
scheme: local
reconnectAttempts (int)¶
Default Value:
"0"
reconnectDelaySeconds (int)¶
Default Value:
"0"
config.Port¶
port (int)¶
Default Value:
"10254"
interfaces.EventSchedulerConfig¶
scheme (string)¶
Default Value:
local
region (string)¶
Default Value:
""
scheduleRole (string)¶
Default Value:
""
targetName (string)¶
Default Value:
""
scheduleNamePrefix (string)¶
Default Value:
""
aws (interfaces.AWSSchedulerConfig)¶
Default Value:
null
local (interfaces.FlyteSchedulerConfig)¶
Default Value:
{}
interfaces.FlyteSchedulerConfig¶
interfaces.WorkflowExecutorConfig¶
scheme (string)¶
Default Value:
local
region (string)¶
Default Value:
""
scheduleQueueName (string)¶
Default Value:
""
accountId (string)¶
Default Value:
""
aws (interfaces.AWSWorkflowExecutorConfig)¶
Default Value:
null
local (interfaces.FlyteWorkflowExecutorConfig)¶
Default Value:
adminRateLimit:
burst: 10
tps: 100
interfaces.FlyteWorkflowExecutorConfig¶
adminRateLimit (interfaces.AdminRateLimit)¶
Default Value:
burst: 10
tps: 100
interfaces.AdminRateLimit¶
tps (float64)¶
Default Value:
"100"
burst (int)¶
Default Value:
"10"
Section: storage¶
type (string)¶
Sets the type of storage to configure [s3/minio/local/mem/stow].
Default Value:
s3
connection (storage.ConnectionConfig)¶
Default Value:
access-key: ""
auth-type: iam
disable-ssl: false
endpoint: ""
region: us-east-1
secret-key: ""
stow (storage.StowConfig)¶
Storage config for stow backend.
Default Value:
{}
container (string)¶
Initial container (in s3 a bucket) to create -if it doesn’t exist-.’
Default Value:
""
enable-multicontainer (bool)¶
If this is true, then the container argument is overlooked and redundant. This config will automatically open new connections to new containers/buckets as they are encountered
Default Value:
"false"
cache (storage.CachingConfig)¶
Default Value:
max_size_mbs: 0
target_gc_percent: 0
limits (storage.LimitsConfig)¶
Sets limits for stores.
Default Value:
maxDownloadMBs: 2
defaultHttpClient (storage.HTTPClientConfig)¶
Sets the default http client config.
Default Value:
headers: null
timeout: 0s
signedUrl (storage.SignedURLConfig)¶
Sets config for SignedURL.
Default Value:
{}
storage.CachingConfig¶
max_size_mbs (int)¶
Maximum size of the cache where the Blob store data is cached in-memory. If not specified or set to 0, cache is not used
Default Value:
"0"
target_gc_percent (int)¶
Sets the garbage collection target percentage.
Default Value:
"0"
storage.ConnectionConfig¶
endpoint (config.URL)¶
URL for storage client to connect to.
Default Value:
""
auth-type (string)¶
Auth Type to use [iam,accesskey].
Default Value:
iam
access-key (string)¶
Access key to use. Only required when authtype is set to accesskey.
Default Value:
""
secret-key (string)¶
Secret to use when accesskey is set.
Default Value:
""
region (string)¶
Region to connect to.
Default Value:
us-east-1
disable-ssl (bool)¶
Disables SSL connection. Should only be used for development.
Default Value:
"false"
storage.HTTPClientConfig¶
headers (map[string][]string)¶
Default Value:
null
timeout (config.Duration)¶
Sets time out on the http client.
Default Value:
0s
storage.LimitsConfig¶
maxDownloadMBs (int64)¶
Maximum allowed download size (in MBs) per call.
Default Value:
"2"
storage.SignedURLConfig¶
stowConfigOverride (map[string]string)¶
Default Value:
null
storage.StowConfig¶
kind (string)¶
Kind of Stow backend to use. Refer to github/flyteorg/stow
Default Value:
""
config (map[string]string)¶
Configuration for stow backend. Refer to github/flyteorg/stow
Default Value:
{}
Section: task_resources¶
defaults (interfaces.TaskResourceSet)¶
Default Value:
cpu: "2"
ephemeralStorage: "0"
gpu: "0"
memory: 200Mi
storage: "0"
limits (interfaces.TaskResourceSet)¶
Default Value:
cpu: "2"
ephemeralStorage: "0"
gpu: "1"
memory: 1Gi
storage: "0"
interfaces.TaskResourceSet¶
cpu (resource.Quantity)¶
Default Value:
"2"
gpu (resource.Quantity)¶
Default Value:
"0"
memory (resource.Quantity)¶
Default Value:
200Mi
storage (resource.Quantity)¶
Default Value:
"0"
ephemeralStorage (resource.Quantity)¶
Default Value:
"0"
resource.Quantity¶
i (resource.int64Amount)¶
Default Value:
{}
d (resource.infDecAmount)¶
Default Value:
<nil>
s (string)¶
Default Value:
"2"
Format (string)¶
Default Value:
DecimalSI
resource.infDecAmount¶
Dec (inf.Dec)¶
Default Value:
null
resource.int64Amount¶
value (int64)¶
Default Value:
"2"
scale (int32)¶
Default Value:
"0"