package kafka
Type Members
-
case class
KafkaInput(name: String, brokers: String, topic: Option[String], topicPattern: Option[String], consumerGroup: Option[String], options: Option[Map[String, String]], schemaRegistryUrl: Option[String], schemaSubject: Option[String], schemaId: Option[Int], isBatch: Option[Boolean], format: Option[String], incremental: Option[Incremental]) extends IncrementalReader with DatasourceReader with Product with Serializable
Represents a Kafka data source input configuration.
Represents a Kafka data source input configuration.
- name
The name of the Kafka input.
- brokers
The Kafka brokers to connect to.
- topic
Optional Kafka topic to subscribe to.
- topicPattern
Optional Kafka topic pattern to subscribe to.
- consumerGroup
Optional Kafka consumer group ID.
- options
Optional additional Kafka options.
- schemaRegistryUrl
Optional URL for the Avro schema registry.
- schemaSubject
Optional subject name for the Avro schema.
- schemaId
Optional ID for the Avro schema.
- isBatch
Optional flag indicating batch processing.
- format
Optional data format for Kafka messages.
- incremental
Optional incremental configuration.
-
class
KafkaLagWriter extends StreamingQueryListener
A listener that tracks the progress of a streaming query and commits Kafka offsets.