Upgrade notes

Lenses 5.0 is a major release which brings in new features to core components. Some of these are breaking changes, which means extra care should be taken before upgrading and following best practices: first upgrade development and staging environments, before moving to production.

Breaking Changes and Caution Items 

Here is a shortlist of the changes that require actions before the upgrade. The most significant change is the dynamic concept of managing core Lenses connections ( Kafka , Schema Registry , Kafka Connect , Zookeeper , Kerberos ) and Wizard mode for new Lenses installations.

• The first time Lenses 5.0 is run, it will upgrade the internal database. The process is not reversible. It is recommended to backup the internal database before upgrading, so it is possible to downgrade if needed. The database holds the user and service accounts, groups information, audits, alerts, and more.
• Breaking changes in the configuration: certain keys must be removed from lenses.conf or Lenses will not start. Check Configuration Changes .

Update Process 

Please backup your database (lenses.storage.directory or PostgreSQL database) before updating to Lenses 5.0. The only way of updating Lenses in any older version is to go through version 4.3.3 or newer. Any other attempt (i.e. using version 4.3.0, 4.2.1 etc.) will fail with proper error message in the logs, but it will not break the database. Rollback will be possible using the same database —still we strongly recommend making a backup before the upgrade.

Using the Lenses Archive 

Download the latest 5.0 archive and extract it in a new directory on your server. It is important to avoid extracting an archive over an older installation to avoid having multiple versions of libraries. Instead, you should remove (or rename) the old directory, then move the new into its place. Copy if needed and update your lenses.conf and security.conf files as described later — Configuration Changes . Make sure Lenses Storage Directory is kept intact. The folder is where persistent data is stored, such as users, groups, audits, data policies, connections, and more.

Using the Lenses Docker 

The docker image uses tags to distinguish between versions. The latest tag (lensesio/lenses:latest) brings the latest stable version of Lenses. There are minor tags to help users get the latest patch in a minor version (e.g 5.0, 4.3) and patch tags to help users pin to a specific patch (e.g 5.0.0, 4.3.6). The best practice advice is to use the minor tag (lensesio/lenses:5.0), which ensures that your installation will always get compatible updates until you made a conscious decision to upgrade the minor version.

To get Lenses 5.0, make sure you keep at least your /data/storage volume to not lose your data. Other volumes supported by the docker are /data/kafka-streams-state (SQL Processors state), /data/log (log files on disk), /data/plugins (custom UDFs).

Pull the 5.0 docker:

docker pull lensesio/lenses:5.0

Follow the Configuration Changes section to update your settings if needed. Stop your current container and restart with the 5.0 image, mounting any volumes you might need.

Lenses Box 

If you are a Box user, pull the latest version, preserve your /data volume and restart Lenses:

docker pull lensesio/box:5.0
docker stop [CURRENT BOX NAME or ID]
docker run -v /path/to/box/data:/data -e EULA="..." -p 3030 lensesio/box:5.0

Helm 

Download the latest charts and update your values.yaml as described in Helm Section . Remember that Lenses Storage Directory should be stored in a persistent volume and be kept intact between updates. To support a potential downgrade, make sure this volume is backed-up before installing a newer version of Lenses.

helm repo add lensesio https://helm.repo.lenses.io/
helm repo update

From version 5.0 there are also fields that are deprecated and moved under lenses.provision configuration. Below is a detailed list of deprecated fields .

1. Existing Helm Release will have to first be upgraded to the latest 4.3 version: https://github.com/lensesio/lenses-helm-charts/blob/release/4.3/charts/lenses/ before upgrading to version 5.0

2. Backup you Lenses database first, either Postgresql or H2 Help Center article, on how to backup the H2 database on various scenarios

3. If no lenses.storage field is set (no Postgresql database), you will have to have Lenses Persistence enabled.

4. Upgrade the Release to 5.0 using lenses.provision.enabled: false in values as follows:

lenses:
  provision:
    enabled: false

5. By default if no provision sidecar container is enabled, Lenses will retrieve all existing configuration from Lenses database, at start-up. Even if no existing configuration is found, Lenses will default to Wizard Mode

6. Migrate existing configuration, that have been deprecated to lenses.provision.yaml & configure any existing secrets with lenses.provision.sidecar.additionalVolumeMounts as shown in Lenses Provision Sidecar container configuration

Helm deprecated fields from previous versions 

These fields are deprecated from version 5.0 onwards and the configuration is moved to lenses.provision.yaml as described in Helm section Lenses Sidecar Provision

License field

• lenses.license

Brokers fields

• lenses.kafka
• lenses.kafka.ssl
• lenses.kafka.sasl
• lenses.kafka.metrics
• lenses.kafka.bootstrapServers

Zookeepers fields

• lenses.zookeepers
• lenses.zookeepers.enabled
• lenses.zookeepers.chroot
• lenses.zookeepers.hosts

Schema Registries fields

• lenses.schemaRegistries
• lenses.schemaRegistries.enabled
• lenses.schemaRegistries.topic
• lenses.schemaRegistries.hosts
• lenses.schemaRegistries.ssl
• lenses.schemaRegistries.security

Kafka Connect Clusters fields

• lenses.connectClusters
• lenses.connectClusters.enabled
• lenses.connectClusters.external
• lenses.connectClusters.clusters

Cloud Installations 

Use the latest version available in the marketplaces. Remember that Lenses Storage Directory should be provided as a persistent volume and be kept intact between updates. If a new image does not exist, you may be able to update Lenses in-place. Our support team will be happy to go through the available options with you.

Configuration Changes 

Lenses 5.0 configuration is NOT compatible with older versions. The configuration of “core” connections (Kafka Brokers, Schema Registry, Zookeeper, Kerberos, Connect) and license are now API based and must be removed from the configuration file (lenses.conf).

Coming from an older version of Lenses (4.3.3 or later), these settings have been automatically migrated for you in the database. You only have to remove them from the configuration file. If you are starting Lenses for the first time, you will get a wizard in your browser to setup connections.

Below you will find the list of configuration keys that have changed. They are rarely used, and most users will not have to take any action.

Changes 

The following configuration keys have been renamed or removed:

Removed

Removed

Removed

Removed

Removed

Removed

Removed

Removed

Kafka changes 

Configuration keys for Kafka (listed below) has been removed. Lenses will not start with that configuration.

lenses.kafka.brokers*
lenses.kafka.settings*
lenses.kafka.metrics*

For more information please check Kafka connection page here .

Schema Registry changes 

Configuration keys for Schema Registry (listed below) has been removed. Lenses will not start with that configuration.

lenses.schema.registry.urls*
lenses.schema.registry.auth*
lenses.schema.registry.username*
lenses.schema.registry.password*
lenses.schema.registry.ssl*
lenses.schema.registry.settings*

For more information please check Schema Registry connection page here .

Kafka Connect changes 

Configuration keys for Kafka Connect (listed below) has been removed. Lenses will not start with that configuration.

lenses.kafka.connect.clusters*
lenses.kafka.connect.ssl*

For more information please check Kafka Connect connection page here .

Kerberos changes 

Configuration keys for Kerberos (listed below) has been removed. Lenses will not start with that configuration.

lenses.kubernetes.processor.krb5

For more information please check Kerberos connection page here .

Zookeeper changes 

Configuration keys for Zookeeper (listed below) has been removed. Lenses will not start with that configuration.

lenses.zookeeper.hosts*
lenses.zookeeper.chroot*
lenses.zookeeper.security.enabled*
lenses.zookeeper.max.session.ms*
lenses.zookeeper.max.connection.ms*

For more information please check Zookeeper connection page here .

License changes 

Configuration keys for License (listed below) has been removed. Lenses will not start with that configuration.

lenses.license.file

For more information please check License page here .

APIs 

See the API documentation

Special Notes for 5.0.3 

Lenses 5.0.3 includes a breaking change that might affect teams running SQL Processors inside Kubernetes.

Existing SQL processors are not affected, but newly created processors require the current (at the time of writing 5.0.3) or later SQL Processor docker image.

This is handled automatically unless you use a custom SQL Processor docker image or a private registry. In such case, please rebuild your docker image or re-pull into your registry using the latest SQL Processor image on Docker Hub (lensesioextra/sql-processor:5.0).