Kafka Reader Activity

Continuously reads Kafka topics as a view.

Parameters

Name	Description	Type	Default
Title	Title of the activity to display on the designer.	text
Servers	List of host name and port pairs.	array
Topics	List of kafka topics.	array
Consumer Group	Consumer group name.	text
Offset Reset	Read the earliest or the latest offset.	enum	latest
Batch Duration	Batch duration in seconds.	int
Failure Threshold (%)	Stop processing stream when failed batches reaches this threshold (in percents). When set to 0, the stream will not stop even in case of batch failures.	int
Format	Message format	enum	text
Schema	Optional schema definition in Avro schema format.	text
As	Name of the DataRow view that is linked to the Kafka batch.	text
Options	Additional options.	key-value

Processing JSON messages

When processing json messages, the schema is defined as Avro schema:

{
    "type": "record",
    "name": "test",
    "fields": [
        {"name": "name", "type": "string"},
        {"name": "age", "type": "int"},
        {"name": "active", "type": "boolean"}]
}

Processing Avro messages

For avro format, regular Avro schema is expected:

{
    "type": "record",
    "name": "test",
    "fields": [
        {"name": "name", "type": "string"},
        {"name": "age", "type": "int"},
        {"name": "active", "type": "boolean"}]
}