Attribute of Lens manifest¶
Structure of Lens manifest¶
# RESOURCE META SECTION
name: saleslens # Lens Resource name (mandatory)
version: v1alpha # Resource manifest version (mandatory)
type: lens # Type of Resource (mandatory)
tags: # Tags (optional)
- sales
- lens
- resource
owner: iamgroot # Resource owner (optional)
layer: user # DataOS Layer (optional)
# LENS-SPECIFIC SECTION
lens:
compute: runnable-default # Compute Resource name (mandatory)
runAsApiKey: abcdefghijklmnopqrstuvwxyz # DataOS APIkey (optional)
runAsUser: iamgroot # DataOS UserID (optional)
secrets: # Referred Instance-secret configuration (mandatory for private repository)
- name: bitbucket-r # Instance-secret name (mandatory)
key: abcd # Key to be referred (optional)
keys: # List of keys to be referred (optional)
- abcd
- efgh
allKeys: true # Whether all keys are required or not (optional)
consumptionType: envVars # Secret consumption type (optional)
source: # mandatory
type: themis # mandatory
name: minithemis # mandatory
catalog: icebase
repo: # Code repository configuration (mandatory)
url: https://www.bitbucket.org/abcd/lens2 # Code Repository URL (mandatory)
lensBaseDir: lens2/sales/model # mandatory
secretId: bitbucket_r_r
syncFlags: # what are these?
- --ref=lens2test
api:
logLevel: INFO
replicas: 3
autoScaling:
enabled: true
minReplicas: 1
maxReplicas: 3
targetMemoryUtilizationPercentage: 60
targetCPUUtilizationPercentage: 60
resources:
requests:
cpu: 4Gi
memory: 1000m
limits:
cpu: 16Gi
memory: 2000m
router:
logLevel: INFO
resources:
requests:
cpu: 4Gi
memory: 1000m
limits:
cpu: 16Gi
memory: 2000m
worker:
logLevel: INFO
replicas: 2
resources:
requests:
cpu: 4Gi
memory: 1000m
limits:
cpu: 16Gi
memory: 2000m
iris:
logLevel: INFO
resources:
requests:
cpu: 4Gi
memory: 1000m
limits:
cpu: 16Gi
memory: 2000m
Configuration¶
Resource meta section¶
Click here to know Resource meta section configuration.
Lens-specific section¶
lens
¶
Description: The lens
attribute defines a mapping of Lens Resource-specific attributes.
Data Type | Requirement | Default Value | Possible Value |
---|---|---|---|
mapping | mandatory | none | valid Lens-specific attributes |
Example Usage:
lens:
compute: runnable-default
runAsApiKey: abcdefghijklmnopqrstuvwxyz
runAsUser: iamgroot
# additional Lens-specific attributes
compute
¶
Description: Defines the Compute Resource to be used by the Lens. The value should match a pre-defined Compute Resource within DataOS that suits the workload requirements of the Lens model.
Data Type | Requirement | Default Value | Possible Value |
---|---|---|---|
string | mandatory | none | runnable-default or any other valid runnable-type Compute Resource name |
Example Usage:
runAsApiKey
¶
Description: The runAsApiKey
attribute allows a user to assume the identity of another user by providing the latter's API key.
Data Type | Requirement | Default Value | Possible Value |
---|---|---|---|
string | optional | DataOS APIKEY of the owner | apikey of the use-case assignee |
Additional Details: The apikey can be obtained by executing the following command from the CLI:
In case no apikey is available, the below command can be run to create a new apikey:
This sample command below creates a new API key named myapikey
that will remain valid for 30 days.
Example Usage:
runAsUser
¶
Description: When the runAsUser
attribute is configured with the UserID of the use-case assignee, it grants the authority to perform operations on behalf of that user.
Data Type | Requirement | Default Value | Possible Value |
---|---|---|---|
string | optional | user-id of the owner applying the Resource | user-id of the use-case assignee |
Example Usage:
secrets
¶
Description: The secret
attribute is used to reference pre-defined Instance Secrets within the Lens manifest file. Each referred secret is a list of secret configurations.
Data Type | Requirement | Default Value | Possible Value |
---|---|---|---|
list of mappings | mandatory | none | list of secret configurations |
Attributes of secrets
name
: The name of the secret to be referred to. It should match the name of a pre-existing Instance Secret within DataOS. This is a mandatory attribute specified as a string.key
: String that identifies a specific key of the Secret. This is an optional attribute.keys
: List of key identifiers within the secret. This is optional.allKeys
: Boolean indicating whether all keys should be used. By default, it is false. This is also optional.consumptionType
: How the secret is consumed. By default its value isenvVars
, signifying secret are consumed as environment variables.
Example Usage: If the codebase for your Lens is stored in a private code repository, you can create a Secret for the same and refer to it within the manifest file of Lens in the following way:
name: github-r
version: v1
type: instance-secret
description: "bitbucket read secrets for lens repos"
layer: user
instance-secret:
type: key-value
acl: r
data:
GITSYNC_USERNAME: ${USERNAME}
GITSYNC_PASSWORD: ${PASSWORD}
source
¶
Description: Specifies the source configuration from which the Lens will be mapped.
Data Type | Requirement | Default Value | Possible Value |
---|---|---|---|
mapping | mandatory | none | valid source configuration |
Attributes of source
:
-
type
: The source type defines the type of data source to which your Lens will be mapped. This is a mandatory attribute, declared as a string. Possible types arethemis
,trino
, etc. -
name
: The identifier for the source to which Lens will be mapped. This is a mandatory attribute, declared as a string. - catalog: ()
Example Usage:
repo
¶
Description: Configures the code repository details for Lens.
Data Type | Requirement | Default Value | Possible Value |
---|---|---|---|
mapping | mandatory | none | valid repository configuration |
Attributes for repo
url
: URL of the repository containing the Lens codes and model. This is a mandatory attribute.lensBaseDir
: The path to the specific directory within the repository where the model files are stored. This is a mandatory field that tells the system where to look for model configurations within the repository.secretId
: Identifier for a secret. It's usually of the form${secret_name}_${acl}.
All hyphens within the secret name are also replaced by _.It is mandatory only in case of private repository.syncFlags
: Additional flags that control the synchronization behavior. Used to declare the specific branch in the repository using the--ref=${branch_name}
flag. The default branch will be the main or master branch.
Example Usage:
lens:
repo:
url: https://github.com/abcd/lens2
lensBaseDir: lens2/sales/model
secretId: github-r
syncFlags:
- --ref=lens2
api
¶
Description: Defines configurations for API instances of Lens, which handles incoming requests and executes business logic.
Data Type | Requirement | Default Value | Possible Value |
---|---|---|---|
mapping | optional | none | valid API instance configurations |
Attributes for api
-
logLevel
: The log level for the API instance classifies entries in logs in terms of urgency, which helps to filter logs during searches and helps control the amount of information in logs. By default, it's INFO. Possible values include INFO, WARN, DEBUG, and ERROR. Each of the possible values is used in a specific scenario as provided below:INFO
Designates informational messages that highlight the progress of the workloadWARN
Designates potentially harmful situationsDEBUG
Designates fine-grained informational events that are most useful while debuggingERROR
Designates error events that might still allow the workflow to continue running
-
replicas
: Number of API instances to deploy. By default, it's 1, and it could be any positive integer. -
autoScaling
: Auto-scaling configuration. Manage autoscaling to match changing workload levels. It comprises several attributes:enabled
: Indicates whether autoscaling is enabled in the form of a boolean. By default, it is false.minReplicas
: The minimum number of replicas for autoscaling. By default, it's 1, and could be any positive integermaxReplicas
: The maximum number of replicas for autoscaling. It could be any positive integer greater thanminReplicas
.targetMemoryUtilizationPercentage
: The target memory utilization percentage for autoscaling is the average memory usage of all pods in a deployment across the last minute divided by the requested CPU of this deployment. If the mean of the pods' CPU utilization is higher than the target you defined, then your replicas will be adjusted. Its value could be any integer between 0 to 100.targetCPUUtilizationPercentage
: The target CPU utilization percentage for autoscaling is the average CPU usage of all pods in a deployment across the last minute divided by the requested CPU of this deployment. If the mean of the pods' memory utilization is higher than the target you defined, then your replicas will be adjusted. Its value could be any integer between 0 to 100.
-
resources
: CPU and memory resource requests and limits. By default, the CPU and memory requests are 100m and 100Mi, while limits are 400m and 400Mi, respectively. The CPU units are declared in milliCPU(m) or CPU core, while memory is declared in Mebibytes (Mi) or Gibibytes (Gi). -
envs
: Environment variables for API instance.LENS2_SCHEDULED_REFRESH_TIMEZONES:
"UTC,America/Vancouver,America/Toronto"
Example Usage:
lens:
api:
replicas: 2
logLevel: info
envs:
LENS2_SCHEDULED_REFRESH_TIMEZONES: "UTC,America/Vancouver,America/Toronto"
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: 2000m
memory: 2048Mi
worker
¶
Description: Defines worker instance of Lens.
Data Type | Requirement | Default Value | Possible Value |
---|---|---|---|
mapping | optional | none | valid worker configurations |
Attributes for worker
-
logLevel
: The log level for the Worker classifies entries in logs in terms of urgency, which helps to filter logs during searches and helps control the amount of information in logs. By default, it's INFO. Possible values include INFO, WARN, DEBUG, and ERROR. Each of the possible values is used in a specific scenario as provided below:INFO
Designates informational messages that highlight the progress of the workloadWARN
: Designates potentially harmful situationsDEBUG
Designates fine-grained informational events that are most useful while debuggingERROR
Designates error events that might still allow the workflow to continue running
-
replicas
: Number of Worker to deploy. By default, it's 1, and it could be any positive integer. -
resources
: CPU and memory resource requests and limits. By default, the CPU and memory requests are 100m and 100Mi, while limits are 400m and 400Mi, respectively. The CPU units are declared in milliCPU(m) or CPU core, while memory is declared in Mebibytes (Mi) or Gibibytes (Gi). -
envs
: Environment variables for Worker.
Example Usage:
lens:
worker:
replicas: 1
logLevel: info
envs:
LENS2_SCHEDULED_REFRESH_TIMEZONES: "UTC,America/Vancouver,America/Toronto"
resources: # optional
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: 6000m
memory: 6048M
router
¶
Description: Manages interaction between API instance and workers.
Data Type | Requirement | Default Value | Possible Value |
---|---|---|---|
mapping | mandatory | none | valid router configuration |
Attributes for router
-
logLevel
: The log level for the Router classifies entries in logs in terms of urgency, which helps to filter logs during searches and helps control the amount of information in logs. By default, it's INFO. Possible values include INFO, WARN, DEBUG, and ERROR. Each of the possible values is used in a specific scenario as provided below:INFO
Designates informational messages that highlight the progress of the workloadWARN
Designates potentially harmful situationsDEBUG
Designates fine-grained informational events that are most useful while debuggingERROR
Designates error events that might still allow the workflow to continue running
-
replicas
Number of Router to deploy. By default, it's 1, and it could be any positive integer. -
resources
CPU and memory resource requests and limits. By default, the CPU and memory requests are 100m and 100Mi, while limits are 400m and 400Mi, respectively. The CPU units are declared in milliCPU(m) or CPU core, while memory is declared in Mebibytes (Mi) or Gibibytes (Gi). -
envs
: Environment variables for Router.
Example Usage:
lens:
router:
logLevel: info
envs:
LENS2_SCHEDULED_REFRESH_TIMEZONES: "UTC,America/Vancouver,America/Toronto"
resources: # optional
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: 6000m
memory: 6048Mi
iris
¶
Description: Manages interaction with Iris dashboards.
Data Type | Requirement | Default Value | Possible Value |
---|---|---|---|
mapping | mandatory | none | valid iris configuration |
Attributes for iris
-
logLevel
: The log level for Iris classifies entries in logs in terms of urgency, which helps to filter logs during searches and helps control the amount of information in logs. By default, it's INFO. Possible values include INFO, WARN, DEBUG, and ERROR. Each of the possible values is used in a specific scenario as provided below:INFO
Designates informational messages that highlight the progress of the workloadWARN
Designates potentially harmful situationsDEBUG
Designates fine-grained informational events that are most useful while debuggingERROR
Designates error events that might still allow the workflow to continue running
-
replicas
: Number of Iris replicas to deploy. By default, it's 1, and it could be any positive integer. -
resources
: CPU and memory resource requests and limits. By default, the CPU and memory requests are 100m and 100Mi, while limits are 400m and 400Mi, respectively. The CPU units are declared in milliCPU(m) or CPU core, while memory is declared in Mebibytes (Mi) or Gibibytes (Gi). -
envs
: Environment variables for Iris.
Example Usage: