Elasticsearch is one of the most popular integrations with Kafka. Lenses integrates with Elasticsearch to create a seamless data journey by providing a 360 view to the streaming flow. You can preview or query data present on Elasticsearch indices and assign metadata to surface relevant information faster, all while keeping your data secure and compliant.
Integration summary
transactions*
Data sources integrating with the catalog, are governed by granular permissions subject to namespaces. Like this you can shape the data catalog based on your teams or access constraints.
Access Management & permissions
In the data catalog, a user is able to search for terms such as index name, schema fields or descriptions and filter by tags.
Choose the index and navigate to the detailed view. There we can see Data [1], Schema, Shards and Metadata information [2, 3]. We can specify, the number of items, we want to include in our query, view the results in Tree or Table View and see the Schema for each individual field.
The data catalog refreshes its cache every 30s by default, but it’s subject to configuration.
Navigate to the SQL Studio to run SQL queries.
SQL Studio
The Lenses SQL Studio provides a familiar query editor that allows writing Lenses SQL queries to retrieve results from individual data sources. You can also Download the results and preview them in either Tree or Table View.
Lenses SQL
Download
To run a query for Elasticsearch data use the connection name:
USE myElasticsearch; SELECT * FROM myIndex LIMIT 100;
If you have any question, please refer to the following list. If you still have more, we are more than welcome to answer any question at our community channel.
How can I view the status of my connection?
Currently, Lenses UI provides a visual indication of the connection health status. Unfortunately, that is only visible in the Explore screen.However, uncovering connection problems should be fairly easy. You can inspect the Lenses Logs and search for ERROR entries related to your connection. More on Logs
ERROR
What versions of Elasticsearch does Lenses support?
We offically support Elasticsearh versions ranging from 6 to 7 with all minor and patch releases for each version.
How many Elasticsearch connections can Lenses handle?
A Lenses on a Kubernetes cluster with 4GB of RAM can handle up to 10 Elasticsearch connections with approximetaly 10K Indices with good responsiveness.
Why is my Elasticsearch Node ‘Yellow’, even when it’s healthy?
Note that in a single node cluster, even a healthy index will always have a Yellow status. That doesn’t mean everything is NOT OK, however your replica rules are not satisfied. The status will update to green once your cluster contains enough nodes to allocate the replicas, even if Lenss is connected to just one node.
What Elasticsearch features are not supported currently?
The main limitation of Lenses SQL for Elasticsearch, is its ability to convert SQL Statements to the query language expected by the Elasticsearch REST query interface.
Does Lenses support write operations for Elasticsearch indices?
Support for write operations (e.g. inserting/updating records, creating/altering indices, etc) is not planned for the immediate future.
On this page
