¶ Kafka SASL ACL Management
CTO2B platform uses kafka-gitops based ACL management model managable by CTO2B kafka-gitops operator (Figure 1)
Figure 1. High level implementation architecture
¶ Kacl custom resource definition
Kacl - define Kafka SASL user and custom ACLs. (Figure 1.1)
Figure 1.1 Kacl deployment resource creation tree
Kacl can be deployed as separate deployment or as dependency bundle with application deployment (Figure 2)
Figure 2. Deployment Options
¶ Kacl values
Lets drill down in to dev-andre-app-k8s-kacl-vote-vote example Kacl deployment values (Table 1)
belongsTo: shared
cluster: kafka-bootstrap.kafka.svc.cluster.local
tenant: vote
# capability user name in full username vote-api-user
user: api
idempotent: false
acls: {}
# - topic: api.commands.
# role: multiwriter
# user: core
global: {}
metadata: {}
| kacl values | Table 1 |
|---|---|
belongsTo: |
its kafka instance name you have deployed. |
cluster: |
kafka cluster bootstrap service name. It never change if you deploy from CTO2B catalog. just namespace matters kafka-bootstrap.<namespace>.svc.cluster.local instance name auto added full cluster expected in current example kafka-shared-kafka-bootstrap.kafka.svc.cluster.local |
tenant: |
tenant name where tenant will be auto added to etch topic policy and kafka user. usaly tenant = namespace |
user: |
capability/application user name. full user looks like <tenant>-<user>-user. In this example username would be vote-api-user Is part of ACL policy |
idempotent: |
enable IdempotentWrite and TransactionalDescribe for cluster resource if needed |
acls: [optional] |
custom additional ACL's for other users. In this example vote-api-user getting all privileges on topics starting with <stage>.<tenant>.<user>. like dev.vote.api.* and same for consumer group dev.vote.api.*, but if we like to grant another application to access some topic like dev.vote.api.events we need define it in acls: |
topic: |
part of acls: require to define topic in LITERAL (dev.vote.api.events) or PREFIX (dev.vote.api.) way. NOTE: <stage>.<tenant>. not need to be defined in topic: name |
role: |
role definition refer for selection to (Table 2) |
acls.user: |
user you grant access to your capability topic you manage |
global |
defining acls: as global if we join with application deployment |
metadata |
system value to pass stage, environment, cloud and other system vars. usualy --> Do not modify! <-- |
¶ Kacl access roles
simple roles that we define as additional custom ACL for another user which is not part of this Kacl
| Kacl roles | | Table 2 |
| Role | Rights | Description |
|---|---|---|
reader |
READ |
only LITERAL to specific topic read |
multireader |
READ |
to PREFIXed topics read all aftersome. |
writer |
WRITE |
only LITERAL to specific topic write. Write allow read as well |
multiwriter |
WRITE |
to PREFIXed topics write to all aftersome. Write allow read as well |
admin |
ALL |
only LITERAL to specific topic admin |
multiadmin |
ALL |
to PREFIXed topics all rights aftersome. |
¶ TLS Certificates
TLS Certificate is not part of Kacl resource and should be deployed as Certificate apiVersion: cert-manager.io/v1 custom certificate request for Tenant issuer
¶ Example use
Voting application tenant (Figure 3) where we have UI, API and Bootstrap demo app pods. Bootstrap application uses demo user and creates and controlls all topics required for API application. This allow us to clone vote tenant and ensure multi-tenancy in shared kafka cluster.
Figure 3. Voting tenant architecture
In demo debug application deployment Chart.yaml we link kacl chart
apiVersion: v2
name: kafka-debug
description: Kafka Debug
version: 0.1.0
dependencies:
- name: kacl
version: 0.1.0
repository: "file://../app-k8s-kacl/"
then update default values by adding kacl: part. This required just once
Values
¶ debug demo app chart
- Update values for debug demo app
- Add kacl default values
- update kacl values
- define topics you will create and manage
- define demo user and custom ACLs for api user
- update kacl values
¶ Some Code
resources:
limits:
memory: "1024Mi"
cpu: 1500m
requests:
memory: "512Mi"
cpu: 100m
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: cloud.google.com/gke-nodepool
operator: In
values:
- pool
- matchExpressions:
- key: pool-type
operator: In
values:
- main
enVars:
- name: KAFKA_BOOTSTRAP_SERVERS
value: "kafka-shared-kafka-bootstrap.kafka.svc.cluster.local:9093"
features:
strimzi: true
tag: latest
serviceAccount:
annotations: {}
kacl:
belongsTo: shared
cluster: kafka-bootstrap.kafka.svc.cluster.local
tenant: vote
# capability user name in full username vote-api-user
user: demo
idempotent: false
global:
acls:
- topic: demo.team.red
role: writer
user: api
- topic: demo.team.green
role: writer
user: api
- topic: demo.team.blue
role: writer
user: api
metadata: {}
so we have grant literal write role to x3 topics . this can be dome with prefixed single entry as example :
global:
acls:
- topic: demo.team.
role: multiwriter
user: api
create kacl deployment for user api . It provide default rights for vote-api-user to dev.vote.api. topics
belongsTo: shared
cluster: kafka-bootstrap.kafka.svc.cluster.local
tenant: vote
user: api
idempotent: false
acls: {}
global: {}
metadata: {}
the debug demo app has buil din deployment variable for SASL injection from secret that is auto created by kacl deployment. same for API application
- name: KAFKA_JAAS_CONFIG
valueFrom:
secretKeyRef:
name: sview-kafka-{{ $stage }}-{{.Values.kacl.tenant }}-{{ .Values.kacl.user }}-user
key: sasl.jaas.config