Helm


View on GitHub

A Helm chart is available and is the recommended way to install in Kubernetes. Only Helm v3 is supported.


helm repo add lensesio https://helm.repo.lenses.io
helm repo update

Use helm to install Lenses, with default values:

helm install lenses lensesio/lenses --namespace lenses --create-namespace

The official Helm chart is available on GitHub. Use charts/lenses/values.yaml as a reference for the full list of available parameters and features.

Lenses Provision 

Starting from version 5.3 Lenses provision operation has 3 primary methods:

For a detailed list of deprecated fields visit: Helm Deprecated Fields

Lenses Wizard Only Mode 

To avoid automatic provisioning via the filesystem method, we can simply disable it using:

lenses:
  provision:
    enabled: false

Then we can follow the Lenses UI Wizard Mode, after Lenses is deployed. Lenses UI Wizard Guide, or use the Provisioning API

Lenses Provision 

The Lenses provisioning method employs a dual input mechanism, allowing you to configure connections and licenses either via the API or by locating the necessary files—namely, provisioning.yaml, license.json, and the required connection certificates—within the filesystem. This flexibility enables users to choose the most convenient and suitable approach for their provisioning needs.


All connection URLs and additional options have been consolidated within the lenses.provision field. This includes various configuration options such as License, Kafka Brokers, Zookeepers, Schema Registries, Kafka Connect Clusters, and their corresponding security and metrics settings. This streamlined organization makes it easier to manage and configure these essential components within your environment.


Both Lenses API provisioning and the lenses.provision Helm values field, depend on a common provisioning.yaml structure. You can find details in Lenses Provision Yaml which is required reading to help with configuring the field lenses.provision. Field lenses.provision can accept either normal yaml syntax, or stringified yaml input.


Some important sections of lenses.provision.yaml 


Example:

lenses:
  provision:
    enabled: true
    version: "2"

    license: "[LICENSE FILE CONTENTS]"

    connections:
      kafka: # Constant connection type name
        - name: kafka
          tags: [ "production", "cluster-1" ] # required, even if empty array
          configuration:
            protocol:
              value: SASL_SSL
            kafkaBootstrapServers:
              value:
                - SASL_SSL://my-cluster-1:9096
                - SASL_SSL://my-cluster-2:9096
                - SASL_SSL://my-cluster-3:9096

      connect:
        - name: custom-connect-cluster
          tags: [ "production", "cluster-1", "lenses-processors" ] # required, even if empty array
          configuration:
            workers:
              value:
                - https://my-custom-connect-cluster:8083
            metricsPort:
              value: 8084
            metricsSsl:
              value: true
            metricsType:
              value: JMX

Default admin user 

The security user contains the default username and password for the super administrator. See how to secure your admin account. For more details visit values.yaml

Example:

lenses:
  security:
    defaultUser:
      username: admin
      password: admin

Lenses Storage & Persistence 

There are two ways to configure a Lenses Database Storage

  • An H2 embedded database on the local filesystem - (quick start and default option)
  • Postgresql database - (production recommended option)

For more information, on how to configure Lenses database, see Persistent Storage

When opting for the local filesystem approach or when utilizing the Data Policies module, it becomes necessary to ensure the persistence of H2 data as well as your data policy rules. To achieve this, you must configure Lenses with a Kubernetes Persistent Volume using the persistence field within the Helm values. You can find more details in Persistence

Example:

persistence:
  enabled: true
  accessModes:
    - ReadWriteOnce
  size: 5Gi

To enable the PostgreSQL option, you can activate it by setting the lenses.storage.postgres.enabled field to true. Additionally, you can fine-tune and specify the database connection details by utilizing the configuration fields nested under lenses.storage.postgres. For more information, on how to configure Postgresql as Lenses database, see Postgresql

Example:

lenses:
  storage:
    postgres:
      enabled: true
      host:
      port: # optional, defaults to 5432
      username: # use "external" to manage it using secrets
      password: # use "external" to manage it using secrets
      database:
      schema: # optional, defaults to public schema


      # Set the username for postgres connection
      # Use the value "external" to bypass the Helm validation and handle the username externally.
      # The common use case for using "external" is to take the actual value from a Kubernetes Secret resource already deployed.
      # To do so, add the following definition under 'lenses.additionalEnv' key:
      # - name: LENSES_STORAGE_POSTGRES_USERNAME
      #   valueFrom:
      #     secretKeyRef:
      #       name: [SECRET_RESOURCE_NAME]
      #       key: [SECRET_RESOURCE_KEY]

      # Set the password for postgres connection
      # Use the value "external" to bypass the Helm validation and handle the password externally.
      # The common use case for using "external" is to take the actual value from a Kubernetes Secret resource already deployed.
      # To do so, add the following definition under 'lenses.additionalEnv' key:
      # - name: LENSES_STORAGE_POSTGRES_PASSWORD
      #   valueFrom:
      #     secretKeyRef:
      #       name: [SECRET_RESOURCE_NAME]
      #       key: [SECRET_RESOURCE_KEY]

Lenses Provision Secrets 

Beginning with Lenses version 5.3, a streamlined approach to managing secrets via Helm is introduced as Provisioning version 2. For reference to the previous method, refer to the Lenses 5.5 documentation.

Now, secrets intended for connection details, such as Kafka broker certificates, can be conveniently specified under lenses.provision.secrets. These secrets are automatically mapped to the files directory within the provisioning path you specify. To include them directly within your values.yaml file, simply add the required secret entries under lenses.provision.secrets.data.

If you wish to define a custom provisioning directory, you can easily do so by modifying the lenses.provision.path field within the values.yaml file. For a more comprehensive understanding of how this provisioning feature operates, you can refer to the Provisioning documentation for detailed insights and guidance.

To reference these secrets within the provisioning.yaml manifest, you can utilize the file paths provided by Lenses. Lenses offers two built-in environment variables: LENSES_PROVISIONING_PATH and LENSES_PROVISIONING_FILE_PATH, which respectively represent the provisioning path and file path. By default, all secrets defined in lenses.provision.secrets can be accessed at the path /mnt/provision-secrets.

Example:

lenses:
  provision:
    enabled: true
    version: "2"

    license: { "license": "json" }
    connections:
      kafka: # Constant connection type name
        - name: kafka
          tags: [ "production", "cluster-1" ] # required, even if empty array
          configuration:
            kafkaBootstrapServers:
              value:
                - SSL://your.kafka.broker.0:9093
                - SSL://your.kafka.broker.1:9093
                - SSL://your.kafka.broker.2:9093
            protocol:
              value: SSL
            sslTruststore:
              file: ${LENSES_PROVISIONING_FILES_PATH}/kafka.truststore.jks
            sslTruststorePassword:
              value: "truststorePassword"
            sslKeystore:
              file: ${LENSES_PROVISIONING_FILES_PATH}/kafka.keystore.jks
            sslKeystorePassword:
              value: "keystorePassword"
            sslKeyPassword:
              value: "keyPassword"

    secrets:
      # Base64 encoded
      data:
        kafka.keystore.jks: |-
                    /u3+7QAAAAIAAAACAAAAAgAGY2Fyb290...==
        kafka.truststore.jks: |-
                    /u3+7QAAAAIAAAABAAAAAgAGY2Fyb290...==

Streaming SQL options 

To configure Streaming SQL options for Lenses use the lenses.sql field. For more details visit values.yaml

Example:

lenses:
  sql:
    mode: KUBERNETES
    heap: 900M
    memLimit: 1152M
    memRequest: 128M
    ssl:
      trustStoreFileData: |-

      keyStoreFileData: |-

      trustStorePassword:
      keyStorePassword:
      keyPassword:

Lenses HTTPS (TLS) 

lenses.tls contains tls options for configuring Lenses with TLS termination. For more details visit values.yaml

Example:

lenses:
  tls:
    enabled: true
    # base64 encoded keystore data
    # openssl base64 < keystore.jks | tr -d '\n'
    keyStoreFileData: |-
            /u3+7QAAAAIAAAACAAAAAgAGY2Fyb290...==
    # base64 keystore password
    # echo "$password" | tr -d '\n' | base64
    keyStorePassword: |-
            YWRtaW4xMjM0
    # base64 encoded truststore data
    # openssl base64 < truststore.jks | tr -d '\n'
    trustStoreFileData: |-
            /u3+7QAAAAIAAAABAAAAAgAGY2Fyb290...==
    # base64 truststore password
    # echo "$password" | tr -d '\n' | base64
    trustStorePassword: |-
            YWRtaW4xMjM0
    # base64 key password
    # echo "$password" | tr -d '\n' | base64
    keyPassword: |-
            YWRtaW4xMjM0
    clientAuth: false

Single Sign On ( SAML 2.0 ) 

Users can authenticate seamlessly via Single Sign-On (SSO) using the SAML 2.0 protocol, leveraging one of the supported integrations. It’s important to note that when using SSO, enabling TLS is also necessary for secure communication. For more details visit values.yaml

Example:

lenses:
  security:
    saml:
      enabled: true
      baseUrl: "https://lenses-prod.eastus2.cloudapp.azure.com"
      provider: "azure"
      keyStoreFileData: |-
                YmFzZTY0IG9mIGtleXN0b3JlIA==
      keyStorePassword: "password"
      keyPassword: "password"
      metadataFileData: |-
                LUlkUCBiYXNlNjQgWE1MIGZpbGUgY29udGVudC0=

Additional Sidecar containers 

With sidecar container configuration, you have the flexibility to deploy supplementary containers, such as data extractors or generators, alongside the Lenses container and provision sidecar. These additional containers can serve various purposes to enhance the functionality of your deployment. You can read more about sidecars. For more details visit values.yaml

# Simple example
sidecarContainers:
  - name: sidecar-example
    image: alpine
    command: [ "sh", "-c", "watch datetime" ]