Persistent Storage

Lenses configuration can be stored:

• on the local filesystem - (quick start and default option)
• in a PostgreSQL database - (recommended) and takes preference when configured

Local storage 

By default, Lenses will store its internal state in the storage folder. We advise to explicitly set this location, ensure the Lenses process has permission to read and write files in this directory and have an upgrade and backup policy in place.

lenses.storage.directory = "/path/to/persistent/data/directory"

PostgreSQL 

Lenses can persist its internal state to a remote PostgreSQL database server.

Current minimum requirements:

• Postgres server running version 9.6 or higher

The recommended configuration is to create a dedicated login role and database for Lenses, setting the Lenses role as the owner of the database. This will mean Lenses will only be able to manage that database and requires no superuser privileges.

Example psql command for initial setup:

# login as superuser and add Lenses role and database
psql -U postgres -d postgres <<EOF
CREATE ROLE lenses WITH LOGIN PASSWORD 'changeme';
CREATE DATABASE lenses OWNER lenses;
EOF

You can then configure Lenses as so:

# in lenses.conf
lenses.storage.postgres.host="my-postgres-server"
lenses.storage.postgres.port=5431 # optional, defaults to 5432
lenses.storage.postgres.username="lenses"
lenses.storage.postgres.database="lenses"
lenses.storage.postgres.password="changeme"

Additional configuration for the PostgreSQL database connection can be passed under the lenses.storage.postgres.properties configuration prefix. The supported parameters can be found in the PostgreSQL documentation. For example:

# require SSL encryption with full host verification
lenses.storage.postgres.properties.ssl=true
lenses.storage.postgres.properties.sslmode="verify-full"
lenses.storage.postgres.properties.sslcert="/path/to/certs/lenses.crt.pem"
lenses.storage.postgres.properties.sslkey="/path/to/certs/lenses.key.pk8"
lenses.storage.postgres.properties.sslpassword="mypassword"
lenses.storage.postgres.properties.sslrootcert="/path/to/certs/CA.crt.pem"

Migration of local storage to PostgreSQL 

Enabling PostgreSQL storage for an existing Lenses installation means the data will be automatically migrated to the PostgreSQL schema on the first run.

After this process has succeeded, a lensesdb.postgresql.migration file will be created in the local storage directory to flag that the migration has already been run. You can then delete the local storage directory and remove the lenses.storage.directory configuration.

If, for whatever reason, you want to re-run the migration to PostgreSQL, deleting the lensesdb.postgresql.migration file will cause Lenses to re-attempt migration on the next restart. The migration process will fail if it encounters any data that can’t be migrated into PostgreSQL, so re-running the migration should only be done on an empty PostgreSQL schema to avoid duplicate record failures.

Connection pooling 

Lenses uses HikariCP library for high-performance database connection pooling.

The default settings should generally perform well, but can be overridden via the lenses.storage.hikaricp configuration prefix. The supported parameters can be found in the HikariCP documentation.

For example:

# set maximumPoolSize to 25
lenses.storage.hikaricp.maximum.pool.size=25