Kafka Source Connector

Kafka

This page guides you through the process of setting up the Kafka source connector.

To use the Kafka source connector, you'll need:

A Kafka cluster 1.0 or above
Seek Connect user should be allowed to read messages from topics, and these topics should be created before reading from Kafka.

You'll need the following information to configure the Kafka source:

Group ID - The Group ID is how you distinguish different consumer groups. (e.g. group.id)
Protocol - The Protocol used to communicate with brokers.
Client ID - An ID string to pass to the server when making requests. The purpose of this is to be able to track the source of requests beyond just ip/port by allowing a logical application name to be included in server-side request logging. (e.g. Seek Connect-consumer)
Test Topic - The Topic to test in case the Seek Connect can consume messages. (e.g. test.topic)
Subscription Method - You can choose to manually assign a list of partitions, or subscribe to all topics matching specified pattern to get dynamically assigned partitions.
List of topic
Bootstrap Servers - A list of host/port pairs to use for establishing the initial connection to the Kafka cluster.
Schema Registry - Host/port to connect schema registry server. Note: It supports for AVRO format only.

Go to the UI and in the left navigation bar, click Sources. In the top-right corner, click +new source.
On the Set up the source page, enter the name for the Kafka connector and select Kafka from the Source type dropdown.
Follow the Setup the Kafka source

The Kafka source connector supports the following sync modes:

JSON - Json value messages. It does not support schema registry now.