Value Members

1.  final def !=(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

2.  final def ##(): Int

   Definition Classes

   AnyRef → Any

3.  final def ==(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

4.  final def asInstanceOf[T0]: T0

   Definition Classes

   Any

5.  def clone(): AnyRef

   Attributes

   protected[lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... ) @native()

6.  def ensureDBExistsInMetastore(dbName: String, locationUri: String, clients: HiveClientPool): Unit

   Ensures that the given database exists in the Hive Metastore.

   Ensures that the given database exists in the Hive Metastore.

   dbName

   The name of the database.

   locationUri

   The location URI of the database.

   clients

   The HiveClientPool to perform the operation.

7.  final def eq(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

8.  def equals(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

9.  def finalize(): Unit

   Attributes

   protected[lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( classOf[java.lang.Throwable] )

10.  final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

    Annotations

    @native()

11.  def getMetastoreUrl(addressInfo: Option[AddressInfo]): String

    Get the metastore URL from the provided AddressInfo.

    Get the metastore URL from the provided AddressInfo.

    addressInfo

    An optional AddressInfo that contains the connection information.

    returns

    The metastore URL as a String.

    Definition Classes

    Writer

12.  def hashCode(): Int

    Definition Classes

    AnyRef → Any

    Annotations

    @native()

13.  def initHiveMetastoreClientPool(uris: String, warehouseDir: String): HiveClientPool

    Initializes a Hive Metastore client pool with the given URIs and warehouse directory.

    Initializes a Hive Metastore client pool with the given URIs and warehouse directory.

    uris

    The Hive Metastore URIs.

    warehouseDir

    The Hive Metastore warehouse directory.

    returns

    The initialized HiveClientPool.

14.  final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

15.  val log: Logger
16.  def mergeStreamingConfig(streamingConfig: Option[Streaming], outputOpts: Map[String, AnyRef]): Streaming

    Merge the streaming configuration with the output options for streaming queries.

    Merge the streaming configuration with the output options for streaming queries. This method is used to combine options from both streamingConfig and outputOptions. If the streamingConfig is None, it uses options from the outputOptions and applies default values as needed.

    streamingConfig

    Optional streaming configuration for the streaming query.

    outputOpts

    Output options for the writer.

    returns

    A Streaming object containing the merged streaming configuration.

    Definition Classes

    Writer

17.  final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

18.  final def notify(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

19.  final def notifyAll(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

20.  def partitionBy(writer: DataFrameWriter[Row]): Unit

    Configures the DataFrame writer with partitioning options if provided.

    Configures the DataFrame writer with partitioning options if provided.

    writer

    The DataFrameWriter to configure.

21.  def setMandatoryArguments(mandatoryArgs: String*): Unit

    Set the mandatory arguments required by the writer implementation.

    Set the mandatory arguments required by the writer implementation.

    mandatoryArgs

    The names of mandatory arguments as varargs.

    Attributes

    protected

    Definition Classes

    Writer

22.  def sort(sortOptions: Option[SortOptions], dataFrame: DataFrame): DataFrame

    Sorts the DataFrame based on the given sort options.

    Sorts the DataFrame based on the given sort options.

    sortOptions

    The sorting configuration.

    dataFrame

    The DataFrame to be sorted.

    returns

    The sorted DataFrame.

23.  final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

24.  def toString(): String

    Definition Classes

    AnyRef → Any

25.  def validateMandatoryArguments(outputOptions: Map[String, Any]): Unit

    Validate if all mandatory arguments are present in the outputOptions map.

    Validate if all mandatory arguments are present in the outputOptions map. If any argument is missing, it throws a MissingWriterArgumentException.

    outputOptions

    The output options map containing the configuration for the writer.

    Definition Classes

    Writer

    Exceptions thrown

    MissingWriterArgumentException if any mandatory argument is missing.

26.  final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

27.  final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

28.  final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... ) @native()

29.  def write(dataFrame: DataFrame): Unit

    Writes the content of the DataFrame to files.

    Writes the content of the DataFrame to files.

    dataFrame

    The DataFrame to be written to files.

    Definition Classes

    FileOutputWriter → Writer

30.  def writeStream(dataFrame: DataFrame, streamingConfig: Option[Streaming]): StreamingQuery

    Optional method for writing streaming data from the DataFrame to the specified data source.

    Optional method for writing streaming data from the DataFrame to the specified data source.

    dataFrame

    The DataFrame containing the streaming data to be written.

    streamingConfig

    Optional configuration for the streaming query.

    returns

    A StreamingQuery representing the active streaming query. If streaming is not supported, this method throws an exception.

    Definition Classes

    Writer