Managing Topics¶
With Lenses, you can create, delete and change the configuration of topics by navigating to the Topics
view from the side menu.
This view displays:
- A chart of the data flowing across the Kafka cluster being monitored
- A topic list and details for each topic
- Functionality to create and delete topics.
Create a topic¶
To create a topic select the New Topic
button and enter the required details to create the topic.
The minimum requirements are:
name
,partitions
replication
.
Additionally, you can supply extra configuration to override the broker defaults by selecting the Add Config
link.
This also allows setting common overrides such as retention time
, retention size
and clean up policy
.
Delete a topic¶
To delete a topic select the trash can button for the topic and confirm if you wish Lenses to remove the topic.
Note
A topic will be marked for deletion and Kafka will remove it once the consumers have stopped.
Modify a topic¶
To alter the configuration of an existing topic, select the topic and then the Config
tab.
Important
You might want to have control over who makes topic changes.By default topic creation, deletion and editing, is only allowed to users with WRITE access, such as admin and write.
View topic partitions¶
Detailed information and metrics about a topic can be seen in the topic Partitions
tab.
It displays the distribution of messages across all the topic partitions,
provides detailed information for each partition, but also monitors and alerts on in-sync replicas.
Consumers¶
Lenses tracks consumers in real time; to view the consumer groups for a given topic select the Consumers
tab once you have navigated to the topic.
This will display:
- Consumer groups active on this topic
- The clients in the consumer group
- The lead broker
- The consumer lag across the group updated in real time
- The stability of the consumer group
Viewing live data¶
Lenses provides the ability to visualize live data in topics by utilizing the Lenses SQL engine, simply select
the topic and you are dropped into the Live Stream
tab. This view provides:
- A SQL window to select which data you want to see from the topic
- Ability to inspect the data
- Pause and restart the stream.
The SQL window allows you to select and filter fields from both the key and value of the records, for a full description of the capabilities please refer to the Lenses SQL section.
Hovering over the records will automatically pause the stream to allow you to inspect records.
Viewing historical data¶
Lenses provides the ability to visualize and download historical data in topics by utilizing the Lenses SQL engine, simply select
the topic and select the Data
tab. This view provides:
- A SQL window to select which data you want to see from the topic
- An offset slider to scroll through data
- A tree view of the data
- A grid-like view of the data
- A raw JSON view of the data.
The SQL window allows you to select and filter fields from both the key and value of the records, for a full description of the capabilities please refer to the Lenses SQL section.
Inserting new data¶
Lenses allows you to insert new data into a topic, provided your account has write or admin permissions to that topic.
You can insert new Avro or JSON payload, by dropping new records onto the UI, and you can even sample, or auto-generated messages.
This is usually helpful for your QA team when testing micro-services architectures, or for pushing new scheduling messages.