Configuring Apache Kafka® target endpoints
- Managed Service for Apache Kafka® cluster connection or custom installation settings and serialization settings, including those based on Compute Cloud VMs. These are required parameters.
- Apache Kafka topic settings.
Managed Service for Apache Kafka® cluster
Connecting to the database with the cluster ID specified in Nebius Israel. Available only for clusters deployed in Managed Service for Apache Kafka®.
Managed Service for Kafka cluster: Select the cluster to connect to.
Username: Specify the username that Data Transfer will use to connect to the database.
Password: Enter the user's password to the database.
Connecting to the database with explicitly specified network addresses.
Broker URLs: Specify the IP addresses or FQDNs of the broker hosts.
If the Apache Kafka® port number differs from the standard one, specify it with a colon after the host name:
<broker host IP or FQDN>:<port number>
SSL: Use encryption to protect the connection.
PEM Certificate: If encryption of transmitted data is required, for example, to meet the PCI DSS requirements, upload the certificate file or add its contents as text.
If the value in this field is specified for both endpoints, both subnets must be hosted in the same availability zone.
Authentication: Select the connection type (
If you select
- Username: Specify the name of the account, under which Data Transfer will connect to the topic.
- Password: Enter the account password.
- Mechanism: Select the hashing mechanism (SHA 256 or SHA 512).
Topic settings Apache Kafka®
Topic full name: Specify the name of the topic to send messages to.
Topic prefix: Specify the topic prefix, similar to the
Debezium database.server.namesetting. Messages will be sent to a topic named
Data Transfer supports CDC for transfers from PostgreSQL, MySQL, and YDB databases to Apache Kafka®. Data is sent to the target in Debezium format. For more information about CDC mode, see Change data capture.
In YDB, CDC mode is supported starting from version 22.5.
- Save transactions order: Do not split an event stream into independent queues by table.