github.com/argoproj/argo-events@v1.9.1/docs/eventbus/kafka.md (about)

     1  Kafka is a widely used event streaming platform. We recommend using Kafka if
     2  you have a lot of events and want to horizontally scale your Sensors. If you
     3  are looking to get started quickly with Argo Events we recommend using
     4  Jetstream instead.
     5  
     6  When using a Kafka EventBus you must already have a Kafka cluster set up and
     7  topics created (unless you have auto create enabled, see [topics](#topics)
     8  below).
     9  
    10  ## Example
    11  ```yaml
    12  kind: EventBus
    13  metadata:
    14    name: default
    15  spec:
    16    kafka:
    17      url: kafka:9092   # must be managed independently
    18      topic: "example"  # optional
    19  ```
    20  
    21  See [here](https://github.com/argoproj/argo-events/blob/master/api/event-bus.md#kafkabus)
    22  for the full specification.
    23  
    24  ## Properties
    25  ### url
    26  Comma seperated list of kafka broker urls, the kafka broker must be managed
    27  independently of Argo Events.
    28  
    29  ### topic
    30  The topic name, defaults to `{namespace-name}-{eventbus-name}`. Two additional
    31  topics per Sensor are also required, see see [topics](#topics) below for more
    32  information.
    33  
    34  ### version
    35  Kafka version, we recommend not manually setting this field in most
    36  circumstances. Defaults to the oldest supported stable version.
    37  
    38  ### tls
    39  Enables TLS on the kafka connection.
    40  ```
    41  tls:
    42    caCertSecret:
    43      name: my-secret
    44      key: ca-cert-key
    45    clientCertSecret:
    46      name: my-secret
    47      key: client-cert-key
    48    clientKeySecret:
    49      name: my-secret
    50      key: client-key-key
    51  ```
    52  
    53  ### sasl
    54  Enables SASL authentication on the kafka connection.
    55  ```
    56  sasl:
    57    mechanism: PLAIN
    58    passwordSecret:
    59      key: password
    60      name: my-user
    61    userSecret:
    62      key: user
    63      name: my-user
    64  ```
    65  
    66  ### consumerGroup.groupName
    67  Consumer group name, defaults to `{namespace-name}-{sensor-name}`.
    68  
    69  ### consumerGroup.rebalanceStrategy
    70  The kafka rebalance strategy, can be one of: sticky, roundrobin, range.
    71  Defaults to range.
    72  
    73  ### consumerGroup.startOldest
    74  When starting up a new group do we want to start from the oldest event
    75  (true) or the newest event (false). Defaults to false
    76  
    77  ## Security
    78  You can enable TLS or SASL authentication, see above for configuration
    79  details. You must enable these features in your Kafka Cluster and make
    80  the certifactes/credentials available in a Kubernetes secret.
    81  
    82  ## Topics
    83  The Kafka EventBus requires one event topic and two additional topics (trigger
    84  and action) per Sensor. These topics will not be created automatically unless
    85  the Kafka `auto.create.topics.enable` cluster configuration is set to true,
    86  otherwise it is your responsibility to create these topics. If a topic does
    87  not exist and cannot be automatically created, the EventSource and/or Sensor
    88  will exit with an error.
    89  
    90  If you want to take advantage of the horizontal scaling enabled by the Kafka
    91  EventBus be sure to create topics with more than one partition.
    92  
    93  By default the topics are named as follows.
    94  
    95  | topic | name |
    96  | ----- | ---- |
    97  | event | `{namespace}-{eventbus-name}` |
    98  | trigger | `{namespace}-{eventbus-name}-{sensor-name}-trigger` |
    99  | action | `{namespace}-{eventbus-name}-{sensor-name}-action` |
   100  
   101  If a topic name is specified in the EventBus specification, then the topics are
   102  named as follows.
   103  
   104  | topic | name |
   105  | ----- | ---- |
   106  | event | `{spec.kafka.topic}` |
   107  | trigger | `{spec.kafka.topic}-{sensor-name}-trigger` |
   108  | action | `{spec.kafka.topic}-{sensor-name}-action` |
   109  
   110  ## Horizontal Scaling and Leader Election
   111  
   112  Sensors that use a Kafka EventBus can scale horizontally. Specifiying replicas
   113  greater than one will result in all Sensor pods actively processing events.
   114  However, an EventSource that uses a Kafka EventBus cannot necessarily be
   115  horizontally scaled in an active-active manner, see [EventSource HA](../eventsources/ha.md)
   116  for more details. In an active-passive scenario a [Kubernetes leader election](../eventsources/ha.md#kubernetes-leader-election)
   117  is used.