Data Governance

With the new regulations around data privacy such as GDPR, HIPAA, AML and companies handling private data making efforts to minimise their risks, we make sure that we keep Data Governance at high standards for your data in motion.

  • Right to request a copy of personal data ✔
  • Keep detailed records of data activities ✔
  • Protect Sensitive Data ✔
  • Role-based Data Access ✔
  • Access Control Lists ✔
  • SSL Encryption and Kerberos ✔
  • LDAP and Active Directory ✔

Right to request a copy of personal data

Lenses SQL engine allows thorough examination and retrieval of records. Collect and store in CSV or JSON format any records you want to retrieve. Automate any requests for personal data via APIs and CLI tool.

SELECT * FROM topicA WHERE customer.id = XXX LIMIT 1000000

Keep detailed records on data activities

All data activities are automatically captured and the full audit trail is preserved.

  • Any user access to data, is audited into an immutable topic, so that we know WHO accessed WHAT data and WHEN.
  • Any Kafka Connect data integration, is audited into an immutable topic so that we know WHICH connect process accessed data and WHEN
  • Any SQL processor is also audited into an immutable topic so that we know WHAT data activities have been performed on data and WHEN
  • Any System (infrastructure level) action like creating a topic, or changing a configuration, or deleting a topic, is audited into an immutable topic

Note: ACLs on audit topics to restrict write level access

Protect Sensitive Data

If leveraging Apache Avro as a data container, you can annotate with "obfuscate":"true" any sensitive field. Once a field is “protected” Lenses will always obfuscate the contents. Use this to make sensitive fields invisible to any user via any means: Web UI, REST, WS, CLI, Python, JDBC.

{
  "type" : "record",
  "name" : "Customer",
  "namespace" : "com.acme.streaming",
  "fields" : [
    {
      "name" : "ssn",
      "type" : "int"
      "obfuscate": "true"
    }, {
      "name" : "posId",
      "type" : "string",
    }
  ]
}

Role Based Data Access

Lenses provides fine-grained level access to users and groups of your organization to your data. You can also map the role based access to your existing LDAP or AD (Active Directory).

Apart from different read, write and admin levels, you can control via whitelists or blacklist rules WHO can access WHAT data:

WHITELIST example of a user in a group with READ and WRITE access to topics starting with equity_

lenses.security.groups=[
  {"name": "equityUK", "roles": ["read", "write"], topic: { whitelist: ["equity_.*"] } }
]
lenses.security.users = [
  {"username": "traderUK", "password": "pass", "displayname": "Trader", "groups": ["equityUK"]}
]

BLACKLIST example of a user in a group with READ access to all topics but those starting with private_

lenses.security.groups=[
  {"name": "viewPublic", "roles": ["read"], topic: { blacklist: ["private_.*"] } }
]
lenses.security.users = [
  {"username": "publicA", "password": "pass", "displayname": "Public", "groups": ["viewPublic"]}
]

Access Control Lists

When authentication is enabled (via Kerberos or TLS certificates), you can utilize the Kafka ACLs infrastructure.

SSL Encryption and Kerberos

All the components of Lenses fully support Kafka clusters with SASL_SSL or SSL. Once the components used ( Kafka Connect clusters and/or Kubernetes) and Lenses are configured to operate over SSL you can have 100% network-level encryption.

LDAP and Active Directory

Pluggable mapping of existing users and groups in LDAP or AD into various access levels to streaming data.

Note

Lenses is leveraged across multiple FTSE-100 companies, Financial institutes and other highly regulated industries, thus Data Governance is a first-class citizen. For additional information contact us