Control how Lenses identifies your connectors in the Topology view. Catalog your connector types, set their icons, control how Lenses extracts the topics used by your connectors.
Lenses comes preconfigured for some of the popular connectors as well as the Stream Reactor connectors. If you see that Lenses doesn’t automatically identify your connector type then use the lenses.connectors.info setting to register it with Lenses.
lenses.connectors.info
Add a new HOCON object {} for every new Connector in your lenses.connectors.info list :
{}
lenses.connectors.info = [ { class.name = "The connector full classpath" name = "The name which will be presented in the UI" instance = "Details about the instance. Contains the connector configuration field which holds the information. If a database is involved it would be the DB connection details, if it is a file it would be the file path, etc" sink = true extractor.class = "The full classpath for the implementation knowing how to extract the Kafka topics involved. This is only required for a Source" icon = "file.png" description = "A description for the connector" author = "The connector author" } ]
This configuration allows the connector to work with the topology graph, and also have the RBAC rules applied to it.
To extract the topics information from the connector configuration, source connectors require an extra configuration. The extractor class should be: io.lenses.config.kafka.connect.SimpleTopicsExtractor. Using this extractor requires an extra property configuration. It specifies the field in the connector configuration which determines the topics data is sent to.
io.lenses.config.kafka.connect.SimpleTopicsExtractor
Here is an example for the file source:
lenses.connectors.info = [ { class.name = "org.apache.kafka.connect.file.FileStreamSource" name = "File" instance = "file" sink = false property = "topic" extractor.class = "io.lenses.config.kafka.connect.SimpleTopicsExtractor" } ]
And an example of a splunk sink connector and a debezium sql server connector
lenses.connectors.info = [ { class.name = "com.splunk.kafka.connect.SplunkSinkConnector" name = "Splunk Sink", instance = "splunk.hec.uri" sink = true, extractor.class = "io.lenses.config.kafka.connect.SimpleTopicsExtractor" icon = "splunk.png", description = "Stores Kafka data in Splunk" docs = "https://github.com/splunk/kafka-connect-splunk", author = "Splunk" }, { class.name = "io.debezium.connector.sqlserver.SqlServerConnector" name = "CDC MySQL" instance = "database.hostname" sink = false, property = "database.history.kafka.topic" extractor.class = "io.lenses.config.kafka.connect.SimpleTopicsExtractor" icon = "debezium.png" description = "CDC data from RDBMS into Kafka" docs = "//debezium.io/docs/connectors/mysql/", author = "Debezium" } ]
Follow the configuration below for a Lenses setup via Docker compose. Here is an example for SplunkSinkConnector and debezium sql server connector. The two connectors are already covered, this is used for illustration purpose.
version: '3' services: lenses: image: lensesio/lenses:latest container_name: lenses ... environment: ... LENSES_CONNECTORS_INFO: | [ { class.name = "com.splunk.kafka.connect.SplunkSinkConnector" name = "Splunk Sink", instance = "splunk.hec.uri" sink = true, extractor.class = "io.lenses.config.kafka.connect.SimpleTopicsExtractor" icon = "splunk.png", description = "Stores Kafka data in Splunk" docs = "https://github.com/splunk/kafka-connect-splunk", author = "Splunk" }, { class.name = "io.debezium.connector.sqlserver.SqlServerConnector" name = "CDC MySQL" instance = "database.hostname" sink = false, property = "database.history.kafka.topic" extractor.class = "io.lenses.config.kafka.connect.SimpleTopicsExtractor" icon = "debezium.png" description = "CDC data from RDBMS into Kafka" docs = "//debezium.io/docs/connectors/mysql/", author = "Debezium" } ] ...
On this page
