Data Governance¶

With the new regulations around data privacy such as GDPR, HIPAA, AML, companies handling private data put a lot of effort to minimize their risks. We make sure we keep Data Governance at high standards for your data in motion.

✔ Right to request a copy of personal data

✔ Keep detailed records of data activities

✔ Protect Sensitive Data

✔ Role-based Data Access

✔ Access Control Lists

✔ SSL Encryption and Kerberos

✔ LDAP and Active Directory

Right to request a copy of personal data

Lenses SQL engine allows thorough examination and retrieval of records. Collect and export the records required as CSV or JSON format. Automate any requests for personal data via APIs and CLI tool.

SELECT * FROM topicA WHERE customer.id = XXX LIMIT 1000000

Keep detailed records on data activities

All actions impacting your data are automatically captured and the full audit trail is preserved.

Any user access to data. The WHO accessed WHAT and WHEN report is available.

Any Kafka Connect data integration. You know WHICH connect process accessed data and WHEN.

Any SQL processor. You know WHAT data activities have been performed on data and WHEN.

Any System (infrastructure level) action like creating a topic, or changing a configuration, or deleting a topic is tracked and stored.

Note: ACLs on audit topics to restrict write level access

Protect Sensitive Data

If leveraging Apache AVRO as a data container, you can annotate with "obfuscate":"true" any sensitive field. Once a field is “protected” Lenses will always obfuscate the contents. Use this to make sensitive fields invisible to any user via any means Web UI, REST, WS, CLI, Python, Go, JDBC.

{
  "type" : "record",
  "name" : "Customer",
  "namespace" : "com.acme.streaming",
  "fields" : [
    {
      "name" : "ssn",
      "type" : "int"
      "obfuscate": "true"
    }, {
      "name" : "posId",
      "type" : "string",
    }
  ]
}

Role Based Data Access

Lenses provides fine-grained data level access for your organization users and groups. You can map the role-based access to your existing LDAP or AD (Active Directory).

Apart from different read, write and admin levels, you can control via whitelists or blacklist rules WHO can access WHAT data:

WHITELIST example of a user in a group with READ and WRITE access to topics starting with equity_

lenses.security.groups=[
  {"name": "equityUK", "roles": ["read", "write"], topic: { whitelist: ["equity_.*"] } }
]
lenses.security.users = [
  {"username": "traderUK", "password": "pass", "displayname": "Trader", "groups": ["equityUK"]}
]

BLACKLIST example of a user in a group with READ access to any topics but those starting with private_

lenses.security.groups=[
  {"name": "viewPublic", "roles": ["read"], topic: { blacklist: ["private_.*"] } }
]
lenses.security.users = [
  {"username": "publicA", "password": "pass", "displayname": "Public", "groups": ["viewPublic"]}
]

Access Control Lists

When authentication is enabled (via Kerberos or TLS certificates), you can utilize the Kafka ACLs infrastructure.

SSL Encryption and Kerberos

All the components of Lenses fully support Kafka clusters with SASL_SSL or SSL providing 100% network-level encryption.

LDAP and Active Directory

You can plug your own custom code for mapping users and groups when using LDAP or AD .

Note

Lenses is used across multiple FTSE-100 companies, Financial institutes and other highly regulated industries, thus Data Governance is a first-class citizen. For additional information contact us