Packages

case class SnowflakeInput(name: String, user: String, password: Option[String], token: Option[String], pemPrivateKey: Option[String], url: String, database: String, table: String, schema: String, snowflakeWarehouse: Option[String], options: Option[Map[String, String]], incremental: Option[Incremental]) extends IncrementalReader with DatasourceReader with Product with Serializable

Represents a Snowflake input configuration for data retrieval.

name

The name of the Snowflake input.

user

The username for authentication to Snowflake.

password

An optional password for authentication to Snowflake.

token

An optional authentication token for Snowflake (OAuth).

pemPrivateKey

An optional PEM private key path for OAuth authentication to Snowflake.

url

The Snowflake URL.

database

The name of the Snowflake database.

table

The name of the table to read data from in Snowflake.

schema

The name of the schema (collection) in Snowflake.

snowflakeWarehouse

An optional warehouse to use for the Snowflake input.

options

An optional map of additional options for the Snowflake input.

incremental

An optional incremental configuration for data processing.

Linear Supertypes
Serializable, Serializable, Product, Equals, DatasourceReader, Reader, IncrementalReader, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SnowflakeInput
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. DatasourceReader
  7. Reader
  8. IncrementalReader
  9. AnyRef
  10. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SnowflakeInput(name: String, user: String, password: Option[String], token: Option[String], pemPrivateKey: Option[String], url: String, database: String, table: String, schema: String, snowflakeWarehouse: Option[String], options: Option[Map[String, String]], incremental: Option[Incremental])

    name

    The name of the Snowflake input.

    user

    The username for authentication to Snowflake.

    password

    An optional password for authentication to Snowflake.

    token

    An optional authentication token for Snowflake (OAuth).

    pemPrivateKey

    An optional PEM private key path for OAuth authentication to Snowflake.

    url

    The Snowflake URL.

    database

    The name of the Snowflake database.

    table

    The name of the table to read data from in Snowflake.

    schema

    The name of the schema (collection) in Snowflake.

    snowflakeWarehouse

    An optional warehouse to use for the Snowflake input.

    options

    An optional map of additional options for the Snowflake input.

    incremental

    An optional incremental configuration for data processing.

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. val database: String
  7. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. val incremental: Option[Incremental]
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. val name: String

    The name of the data reader.

    The name of the data reader.

    Definition Classes
    SnowflakeInput → Reader
  13. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  14. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  16. val options: Option[Map[String, String]]
  17. val password: Option[String]
  18. val pemPrivateKey: Option[String]
  19. def persistIncrementalState(): Unit

    Persists the incremental state if any.

    Persists the incremental state if any.

    Definition Classes
    IncrementalReader
  20. def read(sparkSession: SparkSession): DataFrame

    Reads data from the Snowflake input based on the configuration.

    Reads data from the Snowflake input based on the configuration.

    sparkSession

    The SparkSession to use for reading data.

    returns

    A DataFrame containing the data read from Snowflake.

    Definition Classes
    SnowflakeInput → Reader
  21. def readIncremental(df: DataFrame, incremental: Option[Incremental]): DataFrame

    Reads data from a DataFrame with optional incremental settings.

    Reads data from a DataFrame with optional incremental settings.

    df

    The DataFrame to read data from.

    incremental

    Optional Incremental settings to apply.

    returns

    A new DataFrame after applying incremental settings if provided, otherwise the original DataFrame.

    Definition Classes
    IncrementalReader
  22. val schema: String
  23. val snowflakeWarehouse: Option[String]
  24. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  25. val table: String
  26. val token: Option[String]
  27. val url: String
  28. val user: String
  29. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  30. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  31. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from DatasourceReader

Inherited from Reader

Inherited from IncrementalReader

Inherited from AnyRef

Inherited from Any

Ungrouped