Packages

trait Reader extends IncrementalReader

The Reader trait represents a data reader that can read data and provide it as a DataFrame. It extends the IncrementalReader trait to support incremental reading capabilities.

Linear Supertypes
IncrementalReader, AnyRef, Any
Known Subclasses
DatasetReader, DatasourceReader, MinervaQueryReader, AutogenousInput, BigqueryInput, CassandraInput, ElasticsearchInput, EventHubInput, FileInput, FileStreamInput, JDBCInput, KafkaInput, MinervaInput, MongoDbInput, OpensearchInput, PulsarInput, RedisInput, RedshiftInput, SnowflakeInput
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. Reader
  2. IncrementalReader
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract val name: String

    The name of the data reader.

  2. abstract def read(sparkSession: SparkSession): DataFrame

    Reads data and returns it as a DataFrame using the provided SparkSession.

    Reads data and returns it as a DataFrame using the provided SparkSession.

    sparkSession

    The SparkSession to use for reading data.

    returns

    A DataFrame containing the data read by the reader.

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  13. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  14. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. def persistIncrementalState(): Unit

    Persists the incremental state if any.

    Persists the incremental state if any.

    Definition Classes
    IncrementalReader
  16. def readIncremental(df: DataFrame, incremental: Option[Incremental]): DataFrame

    Reads data from a DataFrame with optional incremental settings.

    Reads data from a DataFrame with optional incremental settings.

    df

    The DataFrame to read data from.

    incremental

    Optional Incremental settings to apply.

    returns

    A new DataFrame after applying incremental settings if provided, otherwise the original DataFrame.

    Definition Classes
    IncrementalReader
  17. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  18. def toString(): String
    Definition Classes
    AnyRef → Any
  19. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  20. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from IncrementalReader

Inherited from AnyRef

Inherited from Any

Ungrouped