Infrastructure Monitoring

Lenses works with any version of Apache Kafka 1.0 or later. All services: Kafka Brokers, Zookeepers, Schema Registries, Kafka Connect Clusters are automatically monitored.

Kafka services

From the Header Bar Menu, go to the Dashboard panel.
On the side navigation, select Services under the Monitor section.

Enabling JMX is recommended for advanced monitoring.

Kafka monitoring

Lenses monitors (by default every 10 seconds) your entire streaming data platform infrastructure and has the following alert rules built-in:

Rule	This rule fires when
Kafka broker is down	A Kafka broker from the cluster is not healthy
Zookeeper node is down	A Zookeeper node is not healthy
Connect Worker is down	A Kafka Connect worker node is not healthy
Schema Registry is down	A Schema Registry instance is not healthy
Under replicated partitions	The Kafka cluster has 1 or more under-replicated partitions
Partitions offline	The Kafka cluster has 1 or more partitions offline (partitions without an active leader)
Active Controller	The Kafka cluster has 0 or more than 1 active controllers
Multiple Broker versions	The Kafka cluster is under a version upgrade, and not all brokers have been upgraded
File-open descriptors on Brokers	A Kafka broker has an alarming number of file-open descriptors. When the operating system is exceeding 90% of the available open file descriptors
Average % the request handler is idle	The average fraction of time the request handler threads are idle is dangerously low. The alert is HIGH when the value is smaller than 10%, and CRITICAL when it is smaller than 2%.
Fetch requests failure	Fetch requests are failing. If the rate of failures per second is > 10% the alert level is set to CRITICAL, otherwise it is set to HIGH.
Produce requests failure	Producer requests are failing. When the value is > 10% the alert level is set to CRITICAL, otherwise it is set to HIGH.
Broker disk usage	A Kafka broker’s disk usage is greater than the cluster average. The build-in threshold is 1 GByte.
Leader imbalance	A Kafka broker has more leader replicas than the average broker in the cluster.

Broker decommission

If you change your Kafka cluster size, or replace an existing Kafka broker with another, Lenses will raise an active alert as it will detect that a broker of your Kafka cluster is no longer available. If the Kafka broker has been intentionally removed, then decommission it:

Navigate to Services.
Select the broker and click on the actions in the options menu and click on the Decommission option.

Other services

Lenses monitors the current metrics for each service providing like this a current health check state. If you wish to have historical data about the services, see the Grafana and Prometheus for Kafka section.