Configuring Apache Kafka® target endpoints

• Managed Service for Kafka cluster: Select the cluster to connect to.

  • Username: Specify the username that Data Transfer will use to connect to the database.

  • Password: Enter the user's password to the database.

• Broker URLs: Specify the IP addresses or FQDNs of the broker hosts.

  If the Apache Kafka® port number differs from the standard one, specify it with a colon after the host name:

  <broker host IP or FQDN>:<port number>
  

• SSL: Use encryption to protect the connection.

• PEM Certificate: If encryption of transmitted data is required, for example, to meet the PCI DSS requirements, upload the certificate file or add its contents as text.

• Endpoint network interface: Select or create a subnet in the desired availability zone.

  If the value in this field is specified for both endpoints, both subnets must be hosted in the same availability zone.

• Authentication: Select the connection type (SASL or No authentication).

  If you select SASL:

  • Username: Specify the name of the account, under which Data Transfer will connect to the topic.
  • Password: Enter the account password.
  • Mechanism: Select the hashing mechanism (SHA 256 or SHA 512).

• Topic:

  • Topic full name: Specify the name of the topic to send messages to.

  • Topic prefix: Specify the topic prefix, similar to the Debezium database.server.name setting. Messages will be sent to a topic named <topic_prefix>.<schema>.<table_name>.

Data Transfer supports CDC for transfers from PostgreSQL, MySQL, and YDB databases to Apache Kafka®. Data is sent to the target in Debezium format. For more information about CDC mode, see Change data capture.

• Save transactions order: Do not split an event stream into independent queues by table.

• Serializing settings: Select the serialization type (Auto or Debezium).

  • Debezium serializer settings: Specify the Debezium serialization parameters.