class JDBCQueryWriter extends DatasourceWriter
A custom JDBCQueryWriter class that extends the DatasourceWriter. This class is responsible for writing data from a DataFrame to a JDBC data source using a custom query.
- Alphabetic
- By Inheritance
- JDBCQueryWriter
- DatasourceWriter
- Writer
- Serializable
- Serializable
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
-
new
JDBCQueryWriter(outputOptions: JDBCQueryWriterOptions)
- outputOptions
The JDBC query writer options containing the necessary parameters for writing to the JDBC data source.
Type Members
-
case class
MissingWriterArgumentException(message: String = "", cause: Throwable = None.orNull) extends Exception with Product with Serializable
- Definition Classes
- Writer
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
addValueToStatement(v: Any, stmt: PreparedStatement, i: Int): Unit
Adds a value to the prepared statement based on its data type.
Adds a value to the prepared statement based on its data type.
- v
The value to be added to the prepared statement.
- stmt
The prepared statement.
- i
The index of the parameter in the prepared statement.
-
def
alignPartitions(dataFrame: DataFrame, minPartitions: Option[Int], maxPartitions: Option[Int]): DataFrame
Aligns the partitions of the DataFrame based on the specified minimum and maximum partitions.
Aligns the partitions of the DataFrame based on the specified minimum and maximum partitions.
- dataFrame
The DataFrame to be aligned.
- minPartitions
The minimum number of partitions to use for repartitioning (optional).
- maxPartitions
The maximum number of partitions to use for coalescing (optional).
- returns
The aligned DataFrame.
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
getMetastoreUrl(addressInfo: Option[AddressInfo]): String
Get the metastore URL from the provided AddressInfo.
Get the metastore URL from the provided AddressInfo.
- addressInfo
An optional AddressInfo that contains the connection information.
- returns
The metastore URL as a String.
- Definition Classes
- Writer
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
def
mergeStreamingConfig(streamingConfig: Option[Streaming], outputOpts: Map[String, AnyRef]): Streaming
Merge the streaming configuration with the output options for streaming queries.
Merge the streaming configuration with the output options for streaming queries. This method is used to combine options from both streamingConfig and outputOptions. If the streamingConfig is None, it uses options from the outputOptions and applies default values as needed.
- streamingConfig
Optional streaming configuration for the streaming query.
- outputOpts
Output options for the writer.
- returns
A Streaming object containing the merged streaming configuration.
- Definition Classes
- Writer
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
def
setMandatoryArguments(mandatoryArgs: String*): Unit
Set the mandatory arguments required by the writer implementation.
Set the mandatory arguments required by the writer implementation.
- mandatoryArgs
The names of mandatory arguments as varargs.
- Attributes
- protected
- Definition Classes
- Writer
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
def
validateMandatoryArguments(outputOptions: Map[String, Any]): Unit
Validate if all mandatory arguments are present in the outputOptions map.
Validate if all mandatory arguments are present in the outputOptions map. If any argument is missing, it throws a MissingWriterArgumentException.
- outputOptions
The output options map containing the configuration for the writer.
- Definition Classes
- Writer
- Exceptions thrown
MissingWriterArgumentExceptionif any mandatory argument is missing.
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
def
write(dataFrame: DataFrame): Unit
Writes the data from the DataFrame to the specified JDBC data source using a custom query.
Writes the data from the DataFrame to the specified JDBC data source using a custom query.
- dataFrame
The DataFrame containing the data to be written to the JDBC data source.
- Definition Classes
- JDBCQueryWriter → Writer
-
def
writeStream(dataFrame: DataFrame, streamingConfig: Option[Streaming]): StreamingQuery
Optional method for writing streaming data from the DataFrame to the specified data source.
Optional method for writing streaming data from the DataFrame to the specified data source.
- dataFrame
The DataFrame containing the streaming data to be written.
- streamingConfig
Optional configuration for the streaming query.
- returns
A StreamingQuery representing the active streaming query. If streaming is not supported, this method throws an exception.
- Definition Classes
- Writer