Packages

o

io.dataos.spark.utils

SparkConfigUtils

object SparkConfigUtils

SparkConfigUtils - Contains helper method to deal with spark configuration e.g. 1. Load spark configuration from heimdall 2. Load spark configurations from depot config mounted in files named with depot.(key-value-properties secrets) 3. Load spark configurations from key-value secrets

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SparkConfigUtils
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. def loadSecrets(dataOsAddress: DataOsAddress): Map[String, String]

    read depot secrets - 1.

    read depot secrets - 1. look for key value properties by depot name 2. If not found - look into heimdall

  13. def loadSecretsFromHeimdall(): Map[String, String]

    Read depot secrets from the Heimdall service.

    Read depot secrets from the Heimdall service.

    returns

    A Scala Map containing the loaded secrets as key-value pairs. If no secrets are available, an empty Map is returned.

  14. def loadSecretsFromKVProperties(): Map[String, String]

    Load secrets from the key-value properties of the depot.

    Load secrets from the key-value properties of the depot.

    returns

    A Scala Map containing the loaded secrets as key-value pairs. If no secrets are available, an empty Map is returned.

  15. def loadSecretsFromKeyValue(): Map[String, String]

    Load secrets from the key-value secrets for the depot.

    Load secrets from the key-value secrets for the depot.

    returns

    A Scala Map containing the loaded secrets as key-value pairs. If no secrets are available, an empty Map is returned.

  16. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  17. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  18. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  19. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  20. def toString(): String
    Definition Classes
    AnyRef → Any
  21. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  22. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  23. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from AnyRef

Inherited from Any

Ungrouped