Azure HDInsight

We recommend you install Lenses with HDInsight Kafka cluster. You can install Lenses as an HDInsight application in two ways:

  1. When creating HDInsight Kafka cluster, or
  2. Lenses can be added to an existing cluster via a template or the HDInsight dashboard (recommended).
HDInsight Kafka New HDInsight Cluster
Existing HDInsight Cluster

Prerequisites 

  • Azure subscription

Installing when creating a cluster 

Create a HDInsight cluster in Azure.

1) Select Kafka as the Cluster Type, enter the requested details, click next and configure the storage.

Create HDInsight Kafka

2) On the Configuration & Pricing tab, click Add Application.

Add Lenses on HDInsight Kafka

3) Select Lenses in the Application blade, enter a valid license and accept the terms.

HDInsight Application blade

4) Select the VM size for Lenses:

Lenses VM size

Lenses will be installed and auto configured on an edge node of the HDInsight cluster. Read how to configure Zookeeper metrics.

Once the deployment is complete, in the Applications section on the left hand side menu, select Lenses, click the URI to go to Lenses and log in with admin/admin.

Log in to HDInsight with Lenses

Install on existing cluster 

Lenses can be install against an existing cluster and will be installed and auto configured on an edge node of the HDInsight cluster. Read how to configure Zookeeper metrics.

You can either install Lenses as an application from the HDInsight dashboard or via a template.

Install via the HDInsight dashboard 

1) Select the cluster in the Azure portal

2) Select Application from the left hand side menu

Adding Lenses app to HDInsight

3) Add a new application

HDInsight add new application

4) Select Lenses from the Application blade, provide the license and accept the terms.

HDInsight Lenses license

In the Applications section on the left hand side menu, select Lenses, click the URI to go to Lenses and log in with admin/admin.

How to log in to HDInsight Lenses

Installing via a template with a HDInsight cluster 

Enter your resource group, location, HDInsight cluster name, license and accept the terms and conditions and click create:


HDInsight Azure marketplace

Once the deployment is complete, in the Applications section on the left hand side menu, select Lenses, click the URI to go to Lenses and log in with admin/admin.

HDInsight Lenses installation complete

VNET-to-VNET with Peering 

Azure reference networking

You can deploy Lenses in the different VNETs to your HDInsight Apache Kafka but you cannot use the provided FQDNs by HDInsight.

Visit the Kafka Service in the HDInsight admin panel find the kafka-env configuration:


Azure Kafka environment

To configure Kafka to advertise IP addresses, add the following text to the bottom of the kafka-env-template field:

Configure Kafka to advertise IP addresses instead of FQDN 

IP_ADDRESS=$(hostname -i)
echo advertised.listeners=$IP_ADDRESS
sed -i.bak -e '/advertised/{/advertised@/!d;}' /usr/hdp/current/kafka-broker/conf/server.properties
echo "advertised.listeners=PLAINTEXT://$IP_ADDRESS:9092" >> /usr/hdp/current/kafka-broker/conf/server.properties

After you have configured the Kafka Service turn on the maintenance mode in actions dropdown and restart all the brokers.


Azure Kafka maintenance


Azure Kafka restart