class InstrumentationOutputWriter extends Writer
A custom InstrumentationOutputWriter class that extends the Writer. This class writes instrumentation data from a DataFrame to a specific metric using an instrumentation factory.
- Alphabetic
- By Inheritance
- InstrumentationOutputWriter
- Writer
- Serializable
- Serializable
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
-
new
InstrumentationOutputWriter(props: Map[String, String], dataFrameName: String, metricName: String, instrumentationFactory: InstrumentationFactory)
- props
The configuration properties for the instrumentation output writer.
- dataFrameName
The name of the DataFrame.
- metricName
The name of the metric to which the instrumentation data is written.
- instrumentationFactory
The instrumentation factory to create the client for writing the data.
Type Members
-
case class
MissingWriterArgumentException(message: String = "", cause: Throwable = None.orNull) extends Exception with Product with Serializable
- Definition Classes
- Writer
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
getMetastoreUrl(addressInfo: Option[AddressInfo]): String
Get the metastore URL from the provided AddressInfo.
Get the metastore URL from the provided AddressInfo.
- addressInfo
An optional AddressInfo that contains the connection information.
- returns
The metastore URL as a String.
- Definition Classes
- Writer
-
def
getTime(indexOfTimeCol: Option[Int], row: Row): Long
Retrieves the time value from the DataFrame row based on the index of the time column.
Retrieves the time value from the DataFrame row based on the index of the time column. If the index is not defined, the current system time is returned.
- indexOfTimeCol
The index of the time column in the DataFrame schema.
- row
The DataFrame row from which to retrieve the time value.
- returns
The time value as a long integer.
- Exceptions thrown
io.dataos.spark.exceptions.SparkWriteFailedExceptionif the defined time column is not a number or is null for the row.
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
lazy val
log: Logger
- Annotations
- @transient()
-
def
mergeStreamingConfig(streamingConfig: Option[Streaming], outputOpts: Map[String, AnyRef]): Streaming
Merge the streaming configuration with the output options for streaming queries.
Merge the streaming configuration with the output options for streaming queries. This method is used to combine options from both streamingConfig and outputOptions. If the streamingConfig is None, it uses options from the outputOptions and applies default values as needed.
- streamingConfig
Optional streaming configuration for the streaming query.
- outputOpts
Output options for the writer.
- returns
A Streaming object containing the merged streaming configuration.
- Definition Classes
- Writer
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
def
setMandatoryArguments(mandatoryArgs: String*): Unit
Set the mandatory arguments required by the writer implementation.
Set the mandatory arguments required by the writer implementation.
- mandatoryArgs
The names of mandatory arguments as varargs.
- Attributes
- protected
- Definition Classes
- Writer
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
- val timeColumnProperty: Option[String]
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
def
validateMandatoryArguments(outputOptions: Map[String, Any]): Unit
Validate if all mandatory arguments are present in the outputOptions map.
Validate if all mandatory arguments are present in the outputOptions map. If any argument is missing, it throws a MissingWriterArgumentException.
- outputOptions
The output options map containing the configuration for the writer.
- Definition Classes
- Writer
- Exceptions thrown
MissingWriterArgumentExceptionif any mandatory argument is missing.
- val valueColumnProperty: Option[String]
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
def
write(dataFrame: DataFrame): Unit
Writes the instrumentation data from the DataFrame to the specified metric.
Writes the instrumentation data from the DataFrame to the specified metric.
- dataFrame
The DataFrame containing the instrumentation data to be written.
- Definition Classes
- InstrumentationOutputWriter → Writer
-
def
writeStream(dataFrame: DataFrame, streamingConfig: Option[Streaming]): StreamingQuery
Optional method for writing streaming data from the DataFrame to the specified data source.
Optional method for writing streaming data from the DataFrame to the specified data source.
- dataFrame
The DataFrame containing the streaming data to be written.
- streamingConfig
Optional configuration for the streaming query.
- returns
A StreamingQuery representing the active streaming query. If streaming is not supported, this method throws an exception.
- Definition Classes
- Writer