Configuring Greenplum® source endpoints
When creating or editing an endpoint, you can define:
- custom installation settings, including those based on Compute Cloud VMs. These are required parameters.
- Additional parameters.
Custom installation
Connecting to the database with explicitly specified network addresses and ports.
-
Coordinator host: Specify the IP or FQDN of the primary master host to connect to.
-
Coordinator port: Specify the port for Data Transfer to use to connect to the primary master host.
-
Coordinator mirror host: Specify the IP address or FQDN of the standby master host to connect to (leave the field empty if your cluster only has one master host).
-
Coordinator mirror port: Specify the port for Data Transfer to use to connect to the standby master host (leave the field empty if there is only one master host in your cluster).
-
Greenplum cluster segments: Specify segment host connection information. If you omit these, segment host addresses will be retrieved automatically from the master host housekeeping table.
-
CA certificate: Upload the certificate file or add its contents as text if transmitted data must be encrypted, for example, to meet PCI DSS requirements.
-
Subnet ID: Select or create a subnet in the desired availability zone.
If the value in this field is specified for both endpoints, both subnets must be hosted in the same availability zone.
-
Database: Specify the name of the database in the selected cluster.
-
User: Specify the username that Data Transfer will use to connect to the database.
-
Password: Enter the user's password to the database.
- Security groups: Select the cloud network to host the endpoint and security groups for network traffic.
This will let you apply the specified security group rules to the VMs and clusters in the selected network without changing the settings of these VMs and clusters. For more information, see Network in Data Transfer.
Additional settings
-
Included tables: Data is only transferred from listed tables.
If a table is partitioned, you can use this field to specify both the entire table and individual partitions.
Make sure that, for tables to be included in the list, all the necessary privileges are granted to the user on whose behalf data will be transferred.
When you add new tables when editing an endpoint used in Snapshot and increment or Replication transfers with the Replicating status, the data history for these tables will not get uploaded. To add a table with its historical data, use the List of objects to be transferred (Preview) field in the transfer settings.
-
Excluded tables: Data from the listed tables is not transferred.
If a table is partitioned, to exclude it from the list, make sure to list all of its partitions.
The lists include the name of the schema (description of DB contents, structure, and integrity constraints) and the table name. Both lists support expressions in the following format:
<schema name>.<table name>: Fully qualified table name.<schema name>.*: All tables in the specified schema.<table name>: Table in the default schema.
-
Snapshot consistency: When enabled, Data Transfer will apply additional steps to the source to assure snapshot consistency.
-
Service object schema: Schema for placing auxiliary objects of the transfer.
Specifics of working with the Greenplum source
Data Transfer only supports Greenplum® version 6. Greenplum® versions 4 and 5 are not supported.
The service performs operations on a Greenplum® cluster with the READ COMMITTED level of isolation.
Data Transfer supports operation with parallel copy enabled for a Greenplum® source.
During operation with parallel copy enabled, Data Transfer maintains an open transaction on the Greenplum® master host. If this transaction is interrupted, a transfer will return an error.
With parallel copy disabled, a transfer will move data from Greenplum® objects such as TABLE, VIEW, FOREIGN TABLE, and EXTERNAL TABLE. Data from these objects will be treated as data from ordinary tables and processed by the target accordingly. With parallel copy enabled, a transfer will only move tables (TABLE objects). Tables with the DISTRIBUTED REPLICATED(https://gpdb.docs.pivotal.io/6-19/admin_guide/distribution.html) [allocation policy] will not be transferred.
Snapshot consistency
When starting a transfer with parallel copy disabled (by default), the service copies data only interacting with the Greenplum® cluster's master host. The copied tables are accessed in ACCESS SHARE lock mode. Snapshot consistency is achieved through Greenplum® mechanisms.
When starting a transfer with parallel copy enabled, the service copies data interacting with both the Greenplum® cluster's master host and segment hosts in utility mode. Access to the tables to be copied locks the tables in ACCESS SHARE or SHARE mode depending on the Snapshot consistency setting.
To guarantee snapshot consistency, transfers with parallel copy enabled need to ensure that data in the tables being transferred remains static. For ACCESS SHARE locks (by default), the service does not guarantee that the data will remain static: this must be assured externally. For SHARE locks, the Greenplum® mechanisms guarantee that data in the source tables remains static.
Greenplum® and Greenplum Database® are registered trademarks or trademarks of VMware, Inc. in the United States and/or other countries.