package readers
Type Members
-
case class
DatasetReader(name: String, address: String, reader: DatasourceReader) extends IncrementalReader with Reader with Product with Serializable
Represents a Dataset reader that reads data from a specified data source using a given DatasourceReader.
Represents a Dataset reader that reads data from a specified data source using a given DatasourceReader.
- name
The name of the Dataset reader.
- address
The address of the data source to read from.
- reader
The DatasourceReader used to read data from the specified data source.
- trait DatasourceReader extends IncrementalReader with Reader
-
class
IncrementalReader extends AnyRef
Represents an Incremental Reader for reading data with incremental options.
-
case class
MinervaQueryReader(name: String, query: String, reader: DatasourceReader) extends IncrementalReader with Reader with Product with Serializable
Represents a MinervaQueryReader, a case class that extends the Reader trait.
Represents a MinervaQueryReader, a case class that extends the Reader trait.
- name
The name of the MinervaQueryReader.
- query
The query associated with this MinervaQueryReader.
- reader
The underlying DatasourceReader used to read data based on the MinervaQueryReader.
-
trait
Reader extends IncrementalReader
The Reader trait represents a data reader that can read data and provide it as a DataFrame.
The Reader trait represents a data reader that can read data and provide it as a DataFrame. It extends the IncrementalReader trait to support incremental reading capabilities.
-
class
ReaderFactory extends AnyRef
The ReaderFactory class is responsible for creating instances of different readers based on the provided InputConfig.
The ReaderFactory class is responsible for creating instances of different readers based on the provided InputConfig. It uses the AddressResolver and SparkSecretLoader to resolve addresses and load secrets if required.