Release Notes

Note

London, UK - December 10, 2018 Lenses 2.2 is now generally available

What’s New in 2.2

  • Data Policies with field-level security for a Data Officer to protect Confidential or Personal information data. Data is sensitive and should not be in plain text. Fields like credit card or social security number and personal identification data can now be tracked and protected.
  • Intelligent SQL with auto completion that is context-aware to easily query and aggregate your data.
  • User-defined function support.
  • Custom authentication based on the incoming HTTP request. The details on how to use it can be found here
  • New SQL-engine for Tabled SQL queries (or bound queries) which support aggregation amongst other great things. You can now run queries like this SELECT count(*) as total FROM iot_data WHERE device_id='id01' for data at rest.
  • The Lenses CLI now has extra commands
    • shell for interactive queries, both browsing and continuous with autocompletion
    • export to export resources such as topics, processor and connectors as request enabling GitOps
    • import to import Lenses landscapes, for example, created by the export command, into Lenses enabling GitOps.
    • policies to view and manage data policies

Improvements

  • Topology View improvements for consumers, producers and micro-services. Inactive or idle instances are tracked better.
  • Time travelling support for exploring data on Kafka topics based on the event time in the metadata.
  • Micro-Service application landscape topology support.
  • CQRS event-sourcing support for querying Avro Unions with WHERE TYPEOF(fieldC) = 'io.lenses.domain.LensesIsGreat'
  • Lenses API performance improvements.
  • Basic auth for Kafka Connect API and Schema Registry.
  • Kafka consumers screen now supports thousands of entries and now displays the application type.
  • No Lenses restarts are required when users/groups are changed. The security.conf file now supports hot reload.
  • When a Kafka broker is decommissioned, the UI now allows the user to instruct Lenses the broker has been decommissioned.
  • When a Kafka topic is deleted, if there is any associated Avro schema then that will be deleted as well.
  • New role introduced to allow specific users to set a Kafka topic storage format.
  • New role introduced to allow specific users to set Kafka consumer groups alerts.
  • Better handling of Apache Kafka timeouts when setting a consumer group offset.
  • Improved the Kubernetes pod labels. Labels now are in the form lenses.io/<type>.
  • Improvements in the Lenses way identifies new or removed Kafka topics.
  • Logging improvements in Lenses for better debugging.
  • SQL Processors for Kubernetes no longer require the Lenses SQL runner Docker image to be rebuilt with JKS, jaas.config, and keytabs embedded. Lenses leverages Kubernetes secrets to deliver the settings to the running pod.
  • Binding to Kafka 2.0.1 client libraries.
  • Lenses CLI
    • machine-friendly JSON output deprecated and replaced by –output flag which accepts json|yaml|table
    • sql live command is now deprecated and replace by a single sql command with a --live-stream flag to have continuous queries. New flags added to return key and metadata or keys only.
    • sql running and sql cancel query command have been dropped in favor of SHOW QUERIES and ``KILL QUERY` see manage queries. Use the sql or shell command instead.

Fixes

  • Kerberos support for HortonWorks Schema registry (#256).

  • Lenses continuous SQL queries

    • Fixed joins referencing SQL code defined via WITH and referencing nested fields.
    • Using nested or array items in the GROUP BY clause is now possible.
    • IF function now supports nested functions for the SQL parameter.
    • Avro deserialization returning primitive not wrapped in NonRecordContainer is now handled.

  • Better error message when inserting records via JSON into a topic when the topic storage payload includes Bytes.

  • The custom microservices metrics reflect the correct value when there are multiple instances of the same application running.

  • Lenses topic for custom application metrics is now automatically created.

  • _connect-configs, _connect-status, _connect-offsets are now considered system topics automatically.

  • Viewing Kafka Consumer Groups now takes into account the user groups whitelisting/blacklisting.

  • When a Kafka consumer alert setting is removed, it now stops continuously raising the alert if it has already triggered.

Deprecated API

  • The endpoints for retrieving data via SQL, as explained here, have been deprecated and they will be removed in a future version

2.2.1 Patch Release

Note

London, UK - December, 2018 Lenses v2.2.1 is now generally available

The patch is focused on fixes and performance improvements.

Fixes

  • Data protection policies were not applied when the default policies were set up

2.2.2 Patch Release

Note

London, UK - December, 2018 Lenses v2.2.2 is now generally available

The patch is focused on fixes and performance improvements.

Fixes

  • Handle timestamp data browsing on topics where the records do not have a timestamp set
  • Invalid License message displays the reason

2.2.3 Patch Release

Note

London, UK - December, 2018 Lenses v2.2.3 is now generally available

The patch is focused on fixes and performance improvements.

Fixes

  • Topology rendering was overlapping nodes in certain scenarios
  • Dashboard inconsistency for Zookeeper health status
  • UI is not querying anymore the topics endpoint while in dashboard view
  • Support Schema Registry 5.1

Improvements

  • The landoop/lenses docker image now supports SSL keys and certificates in PEM format

2.2.4 Patch Release

Note

London, UK - January, 2019 Lenses v2.2.4 is now generally available

The patch is focused on fixes and performance improvements.

Fixes

  • Table query error when the Avro schema contains an Avro MAP field
  • UI not rendering the SQL results in obscure scenarios
  • Topic screen for live data now shows the number of messages fetched

Improvements

  • Topic data browsing first data batch load performance has been improved considerably
  • Topic data export now contains the record metadata information (offset, partition, topic, timestamp)

2.2.5 Patch Release

Note

London, UK - January, 2019 Lenses v2.2.5 is now generally available

Fixes

  • In some occasions the TableStorageRead and TableStorageWrite roles were ignored

2.2.6 Patch Release

Note

London, UK - January, 2019 Lenses v2.2.6 is now generally available

Fixes

  • The Save button for the topic storage format in some scenarios was not working
  • Deserialization issues for Avro payloads with MAP fields
  • SQL processors error when Schema Registry was not configured

Improvements

  • Small performance gain on reading Avro payloads with a MAP field

2.2.7 Patch Release

Note

London, UK - February, 2019 Lenses v2.2.7 is now generally available

Fixes

  • When starting Lenses, Data Policies sometimes aren’t loaded
  • In the SQL screen, when using the raw data output, continuous scrolling doesn’t work
  • A topic without messages can crash the browser if the user sets timestamp browsing mode

2.2.8 Patch Release

Note

London, UK - February, 2019 Lenses v2.2.8 is now generally available

Fixes

  • Lenses always binds to all interfaces despite lenses.ip setting
  • In the Topics page, the metrics bar occasionally disappears for a few seconds
  • In the SQL page, in Grid view, after a SELECT query, the result of an INSERT query isn’t shown

2.2.9 Patch Release

Note

London, UK - February, 2019 Lenses v2.2.9 is now generally available

Fixes

  • Regression in kubernetes SQL processors sets wrong env var for schema registry
  • Typo in delete records confirmation dialog