Packages

o

io.dataos.spark.utils

SparkConfigUtils

object SparkConfigUtils

SparkConfigUtils - Contains helper method to deal with spark configuration e.g. 1. Load spark configuration from heimdall 2. Load spark configurations from depot config mounted in files named with depot.(key-value-properties secrets) 3. Load spark configurations from key-value secrets

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SparkConfigUtils
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. def loadSecrets(dataOsAddress: DataOsAddress): Map[String, String]

    read depot secrets - 1.

    read depot secrets - 1. look for key value properties by depot name 2. If not found - look into heimdall

  2. def loadSecretsFromHeimdall(): Map[String, String]

    Read depot secrets from the Heimdall service.

    Read depot secrets from the Heimdall service.

    returns

    A Scala Map containing the loaded secrets as key-value pairs. If no secrets are available, an empty Map is returned.

  3. def loadSecretsFromKVProperties(): Map[String, String]

    Load secrets from the key-value properties of the depot.

    Load secrets from the key-value properties of the depot.

    returns

    A Scala Map containing the loaded secrets as key-value pairs. If no secrets are available, an empty Map is returned.

  4. def loadSecretsFromKeyValue(): Map[String, String]

    Load secrets from the key-value secrets for the depot.

    Load secrets from the key-value secrets for the depot.

    returns

    A Scala Map containing the loaded secrets as key-value pairs. If no secrets are available, an empty Map is returned.