Using DSE Advanced Replication
Operations including starting, stopping, and configuring DSE Advanced Replication.
dse.yaml
The location of the dse.yaml file depends on the type of installation:| Package installations | /etc/dse/dse.yaml |
| Tarball installations | installation_location/resources/dse/conf/dse.yaml |
cassandra.yaml
The location of the cassandra.yaml file depends on the type of installation:| Package installations | /etc/dse/cassandra/cassandra.yaml |
| Tarball installations | installation_location/resources/cassandra/conf/cassandra.yaml |
Operations including starting, stopping, and configuring DSE Advanced Replication.
- Starting DSE Advanced Replication
- Stopping DSE Advanced Replication
- Configuring global configuration settings
- Configuring destination settings
- Configuring channel settings
- Security
- Data insert methods
- Monitoring operations
Starting DSE Advanced Replication
Attention: Prerequisite: If you are
using Advanced Replication V1 from DSE 5.0, you must upgrade to DSE 5.1 and migrate to Advanced Replication
V2.
Before you can start and use DSE Advanced Replication, you must create the user keyspaces and tables on the source cluster and the destination cluster.
On all nodes in the source cluster:
- Enable replication in the dse.yaml
file.Uncomment all
advanced_replication_optionsentries, setenabled: true, and specify a directory to hold advanced replication log files withadvanced_replication_directory:# Advanced Replication configuration settings advanced_replication_options: enabled: true advanced_replication_directory: /var/lib/cassandra/advrep - Enable Capture-Data-Change (CDC) in the
cassandra.yaml file on a per-node basis for each
source:
cdc_enabled: true cdc_raw_directory: /var/lib/cassandra/cdc_rawNote: Advanced Replication will not start if CDC is not enabled. Either use the default directory or change it to a preferred location. - Consider increasing the default CDC disk space, depending on the load (default: 4096
MB or 1/8 of the total space where
cdc_raw_directoryresides):cdc_total_space_in_mb: 16384 - Commitlog compression is turned off by default. To avoid problems with advanced
replication, this option should NOT be
used:
# commitlog_compression: # - class_name: LZ4Compressor - Do a rolling restart: restart the nodes in the source cluster one at a time while the other nodes continue to operate online.
Disabling DSE Advanced Replication
When replication is not enabled, data is not written to the replication log. On all nodes
in the source cluster:
- To disable replication, edit the dse.yaml file.In the
advanced_replication_optionssection, setenabled: false.# Advanced Replication configuration settings advanced_replication_options: enabled: false - Do a rolling restart: restart the nodes in the source cluster one at a time while the other nodes continue to operate online.
- To clean out the data that was used for DSE Advanced Replication, use
cqlshto remove these keyspaces:DROP TABLE dse_system.advrep_source_config; DROP TABLE dse_system.advrep_destination_config; DROP TABLE dse_system.advrep_repl_channel_config; DROP KEYSPACE dse_advrep;
Configuring global configuration settings
Global settings apply to the entire source cluster. These global settings are stored in the CQL table dse_system.advrep_source_config that is automatically created.
Change global settings by using the dse advrep command
line tool with this syntax:
dse advrep conf ...To view the source node configuration settings:
dse advrep conf listThe result is:
-----------------------------------
|name |value |
-----------------------------------
|audit_log_file |/tmp/myaudit.gz|
-----------------------------------
|audit_log_enabled|true |
-----------------------------------The following table describes the configuration keys, their default values, and identifies when a restart of the source node is required for the change to be recognized.
The
dse advrep command line tool uses these configuration keys as command
arguments to the dse advrep command line
tool. | Configuration key | Default value | Description | Restart required |
|---|---|---|---|
| permits | 30,000 | Maximum number of messages that can be replicated in parallel over all destinations. | No |
| source-id | N/A | Identifies this source cluster and all inserts from this cluster. The source-id must also exist in the primary key on the destination for population of the source-id to occur. | No |
| collection-expire-after-write | N/A | ||
| collection-time-slice-count | 5 | The number of files which are open in the ingestor simultaneously. | Yes |
| collection-time-slice-width | 60 seconds | The time period in seconds for each data block ingested. Smaller time widths => more files. Larger timer widths => larger files but more data to resend on CRC mismatches. | Yes |
| invalid-message-log | SYSTEM_LOG | Select one of these logging strategies to adopt when an invalid message is
discarded: SYSTEM_LOG: Log the CQL query and the error message in the system log on the destination. CHANNEL_LOG: Store the CQL query and the error message in files in /var/lib/cassandra/advrep/invalid_queries on the destination. NONE: Perform no logging. See Managing invalid messages. Requires node restart. |
No |
| audit-log-enable | false | Specifies whether to store the audit log. | Yes |
| audit-log-file | /tmp/advrep_rl_audit.log | Specifies the file name prefix template for the audit log file. The file name is appended with .gz if compressed using gzip. | Yes |
| audit-log-max-life-span-mins | 0 |
Specifies the maximum lifetime of audit log files.
Periodically, when log files are rotated, audit log files are purged when
they:
|
Yes |
| audit-log-rotate-time-mins | 60 | Specifies the time interval to rotate the audit log file. On rotation, the rotated file is appended with the log counter .[logcounter], incrementing from [0]. To disable rotation, set to 0. | Yes |
Configuring destination settings
A destination is a location to which source data will be written. Destinations are stored in the CQL table dse_system.advrep_destination_config that is automatically created.
Change destination settings by using the dse advrep
command line tool with this syntax:
dse advrep destination ...
You can verify the channel configuration before you change it. For
example:
dse advrep destination list-confThe result is:
--------------------------------------------------------------------------------------------
|destination|name |value |
--------------------------------------------------------------------------------------------
|mydest |driver_ssl_enabled |false |
--------------------------------------------------------------------------------------------
|mydest |addresses |10.200.182.251 |
--------------------------------------------------------------------------------------------
|mydest |driver_read_timeout |15000 |
--------------------------------------------------------------------------------------------
|mydest |driver_connections_max |8 |
--------------------------------------------------------------------------------------------
|mydest |source_id_column |source_id |
--------------------------------------------------------------------------------------------
|mydest |driver_connect_timeout |15000 |
--------------------------------------------------------------------------------------------
|mydest |driver_ssl_protocol |TLS |
--------------------------------------------------------------------------------------------
|mydest |driver_consistency_level |QUORUM |
--------------------------------------------------------------------------------------------
|mydest |driver_used_hosts_per_remote_dc |0 |
--------------------------------------------------------------------------------------------
|mydest |driver_allow_remote_dcs_for_local_cl|false |
--------------------------------------------------------------------------------------------
|mydest |driver_compression |lz4 |
--------------------------------------------------------------------------------------------
|mydest |driver_connections |1 |
--------------------------------------------------------------------------------------------
|mydest |driver_ssl_cipher_suites |[TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,|
| | |, |
| | |TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, |
| | |, |
| | |TLS_RSA_WITH_AES_256_CBC_SHA256, |
| | |, |
| | |TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, |
| | |, |
| | |TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, |
| | |, |
| | |TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, |
| | |, |
| | |TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, |
| | |, |
| | |TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, |
| | |, |
| | |TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, |
| | |, |
| | |TLS_RSA_WITH_AES_256_CBC_SHA, |
| | |, |
| | |TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, |
| | |, |
| | |TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, |
| | |, |
| | |TLS_DHE_RSA_WITH_AES_256_CBC_SHA, |
| | |, |
| | |TLS_DHE_DSS_WITH_AES_256_CBC_SHA, |
| | |, |
| | |TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, |
| | |, |
| | |TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, |
| | |, |
| | |TLS_RSA_WITH_AES_128_CBC_SHA256, |
| | |, |
| | |TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, |
| | |, |
| | |TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, |
| | |, |
| | |TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, |
| | |, |
| | |TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, |
| | |, |
| | |TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, |
| | |, |
| | |TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, |
| | |, |
| | |TLS_RSA_WITH_AES_128_CBC_SHA, |
| | |, |
| | |TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, |
| | |, |
| | |TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, |
| | |, |
| | |TLS_DHE_RSA_WITH_AES_128_CBC_SHA, |
| | |, |
| | |TLS_DHE_DSS_WITH_AES_128_CBC_SHA, |
| | |, |
| | |TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, |
| | |, |
| | |TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, |
| | |, |
| | |TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, |
| | |, |
| | |TLS_RSA_WITH_AES_256_GCM_SHA384, |
| | |, |
| | |TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, |
| | |, |
| | |TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, |
| | |, |
| | |TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, |
| | |, |
| | |TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, |
| | |, |
| | |TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, |
| | |, |
| | |TLS_RSA_WITH_AES_128_GCM_SHA256, |
| | |, |
| | |TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, |
| | |, |
| | |TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, |
| | |, |
| | |TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, |
| | |, |
| | |TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, |
| | |, |
| | |TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, |
| | |, |
| | |TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, |
| | |, |
| | |SSL_RSA_WITH_3DES_EDE_CBC_SHA, |
| | |, |
| | |TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, |
| | |, |
| | |TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, |
| | |, |
| | |SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, |
| | |, |
| | |SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, |
| | |, |
| | |TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, |
| | |, |
| | |TLS_ECDHE_RSA_WITH_RC4_128_SHA, |
| | |, |
| | |SSL_RSA_WITH_RC4_128_SHA, |
| | |, |
| | |TLS_ECDH_ECDSA_WITH_RC4_128_SHA, |
| | |, |
| | |TLS_ECDH_RSA_WITH_RC4_128_SHA, |
| | |, |
| | |SSL_RSA_WITH_RC4_128_MD5, |
| | |, |
| | |TLS_EMPTY_RENEGOTIATION_INFO_SCSV] |
--------------------------------------------------------------------------------------------
|mydest |source_id |source1 |
--------------------------------------------------------------------------------------------
|mydest |transmission_enabled |true |
--------------------------------------------------------------------------------------------
|llpdest |driver_ssl_enabled |false |
--------------------------------------------------------------------------------------------
|llpdest |addresses |10.200.177.184 |
--------------------------------------------------------------------------------------------
|llpdest |driver_read_timeout |15000 |
--------------------------------------------------------------------------------------------
|llpdest |driver_connections_max |8 |
--------------------------------------------------------------------------------------------
|llpdest |source_id_column |source_id |
--------------------------------------------------------------------------------------------
|llpdest |driver_connect_timeout |15000 |
--------------------------------------------------------------------------------------------
|llpdest |driver_ssl_protocol |TLS |
--------------------------------------------------------------------------------------------
|llpdest |driver_consistency_level |ONE |
--------------------------------------------------------------------------------------------
|llpdest |driver_used_hosts_per_remote_dc |0 |
--------------------------------------------------------------------------------------------
|llpdest |driver_allow_remote_dcs_for_local_cl|false |
--------------------------------------------------------------------------------------------
|llpdest |driver_compression |lz4 |
--------------------------------------------------------------------------------------------
|llpdest |driver_connections |1 |
--------------------------------------------------------------------------------------------
|llpdest |driver_ssl_cipher_suites |[TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,|
| | |, |
| | |TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, |
| | |, |
| | |TLS_RSA_WITH_AES_256_CBC_SHA256, |
| | |, |
| | |TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, |
| | |, |
| | |TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, |
| | |, |
| | |TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, |
| | |, |
| | |TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, |
| | |, |
| | |TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, |
| | |, |
| | |TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, |
| | |, |
| | |TLS_RSA_WITH_AES_256_CBC_SHA, |
| | |, |
| | |TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, |
| | |, |
| | |TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, |
| | |, |
| | |TLS_DHE_RSA_WITH_AES_256_CBC_SHA, |
| | |, |
| | |TLS_DHE_DSS_WITH_AES_256_CBC_SHA, |
| | |, |
| | |TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, |
| | |, |
| | |TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, |
| | |, |
| | |TLS_RSA_WITH_AES_128_CBC_SHA256, |
| | |, |
| | |TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, |
| | |, |
| | |TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, |
| | |, |
| | |TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, |
| | |, |
| | |TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, |
| | |, |
| | |TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, |
| | |, |
| | |TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, |
| | |, |
| | |TLS_RSA_WITH_AES_128_CBC_SHA, |
| | |, |
| | |TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, |
| | |, |
| | |TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, |
| | |, |
| | |TLS_DHE_RSA_WITH_AES_128_CBC_SHA, |
| | |, |
| | |TLS_DHE_DSS_WITH_AES_128_CBC_SHA, |
| | |, |
| | |TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, |
| | |, |
| | |TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, |
| | |, |
| | |TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, |
| | |, |
| | |TLS_RSA_WITH_AES_256_GCM_SHA384, |
| | |, |
| | |TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, |
| | |, |
| | |TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, |
| | |, |
| | |TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, |
| | |, |
| | |TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, |
| | |, |
| | |TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, |
| | |, |
| | |TLS_RSA_WITH_AES_128_GCM_SHA256, |
| | |, |
| | |TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, |
| | |, |
| | |TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, |
| | |, |
| | |TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, |
| | |, |
| | |TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, |
| | |, |
| | |TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, |
| | |, |
| | |TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, |
| | |, |
| | |SSL_RSA_WITH_3DES_EDE_CBC_SHA, |
| | |, |
| | |TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, |
| | |, |
| | |TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, |
| | |, |
| | |SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, |
| | |, |
| | |SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, |
| | |, |
| | |TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, |
| | |, |
| | |TLS_ECDHE_RSA_WITH_RC4_128_SHA, |
| | |, |
| | |SSL_RSA_WITH_RC4_128_SHA, |
| | |, |
| | |TLS_ECDH_ECDSA_WITH_RC4_128_SHA, |
| | |, |
| | |TLS_ECDH_RSA_WITH_RC4_128_SHA, |
| | |, |
| | |SSL_RSA_WITH_RC4_128_MD5, |
| | |, |
| | |TLS_EMPTY_RENEGOTIATION_INFO_SCSV] |
--------------------------------------------------------------------------------------------
|llpdest |source_id |source1 |
--------------------------------------------------------------------------------------------
|llpdest |transmission_enabled |false |
--------------------------------------------------------------------------------------------The following table describes the configuration keys, their default values, and identifies when a restart of the source node is required for the change to be recognized.
| Configuration key | Default value | Description | Restart required |
|---|---|---|---|
| separator | N/A | Field separator. | No |
| name | N/A | Name for destination (required). | No |
| addresses | none | REQUIRED. A comma separated list of IP addresses that are used to connect to the destination cluster using the DataStax Java driver. | No |
| driver-allow-remote-dcs-for-local-cl | false | Set to true to enable automatic failover for destination clusters with multiple datacenters. The value of the driver-consistency-level parameter must be LOCAL_ONE or LOCAL_QUORUM. | Yes |
| driver-compression | lz4 | The compression algorithm the DataStax Java driver uses to send data from the source to the destination. Supported values are lz4 and snappy. | Yes |
| driver-connect-timeout | 15000 | Time in milliseconds the DataStax Java driver waits to connect to a server. | No |
| driver-connections | 32 | The number of connections the DataStax Java driver will create. | Yes |
| driver-connections-max | 256 | The maximum number of connections the DataStax Java driver will create. | Yes |
| driver-max-requests-per-connection | 1024 | The maximum number of requests per connection the DataStax Java driver will create. | |
| driver-consistency-level | ONE | The consistency level used by the DataStax Java driver when executing statements for replicating data to the destination. Specify a valid DSE level: ANY, ONE, TWO, THREE, QUORUM, ALL, LOCAL_QUORUM, EACH_QUORUM, SERIAL, LOCAL_SERIAL, or LOCAL_ONE. | No |
| driver-local-dc | N/A | For destination clusters with multiple datacenters, you can explicitly define the name of the datacenter that you consider local. Typically, this is the datacenter that is closest to the source cluster. This value is used only for clusters with multiple data enters. | Yes |
| driver-pwd | none | Driver password if the destination requires a user and password to connect. Changing
the driver-pwd value for connection to a destination will automatically connect, but
with a slight delay. Note: By default, driver user names and passwords are plain
text. DataStax recommends encrypting the driver passwords before you add them to
the CQL table. |
Yes |
| driver-read-timeout | 15000 | Time in milliseconds the DataStax Java driver waits to read responses from a server. | No |
| driver-ssl-enabled | false | Whether SSL is enabled for connection to the destination. | Yes |
| driver-ssl-disabled | Disable SSL for connection to the destination. | ||
| driver_ssl_keystore_path | none | The path to the keystore for connection to DSE when SSL client authentication is enabled. | Yes |
| driver_ssl_keystore_password | none | The keystore password for connection to DSE when SSL client authentication is enabled. | Yes |
| driver_ssl_keystore_type | none | The keystore type for connection to DSE when SSL client authentication is enabled. | Yes |
| driver_ssl_truststore_path | none | The path to the truststore for connection to DSE when SSL is enabled. | Yes |
| driver-ssl-truststore-password | none | The truststore password for connection to DSE when SSL is enabled. | Yes |
| driver-ssl-truststore-type | none | The keystore type for connection to DSE when SSL client authentication is enabled. | Yes |
| driver-ssl-protocol | TLS | The SSL protocol for connection to DSE when SSL is enabled. | Yes |
| driver-ssl-cipher-suites | none | A comma-separated list of SSL cipher suites for connection to DSE when SSL is enabled. Cipher suites must be supported by the source machine. | Yes |
| driver-used-hosts-per-remote-dc | 0 | To use automatic failover for destination clusters with multiple datacenters, you must define the number of hosts per remote datacenter that the datacenter aware round robin policy (DCAwareRoundRobinPolicy) considers available. | Yes |
| driver-user | none | Driver username if the destination requires a user and password to connect. Changing the driver-user value for connection to a destination will automatically connect, but with a slight delay. | Yes |
| source-id | N/A | Identifies this source cluster and all inserts from this cluster. The source-id must also exist in the primary key on the destination for population of the source-id to occur. | No |
| source-id-column | source-id | The column to use on remote tables to insert the source id as part of the update. If this column is not present on the table that is being updated, the source id value is ignored. | No |
| transmission-enabled | false | Specify if data collector for the table should be replicated to the destination using boolean value. | No |
Configuring channel settings
A replication channel is a defined channel of change data between source clusters and destination clusters.
A replication channel is defined by the source cluster, source keyspace, source table name, destination cluster, destination keyspace, and destination table name. Replications for each channel (unique keyspace and table) are stored in the CQL table dse_system.advrep_repl_channel_config that is automatically created.
Change the settings using the dse advrep command
line tool with this syntax:
dse advrep channel ...
You can verify the channel configuration before you change it. For
example:
dse advrep channel statusThe result is:
------------------------------------------------------------------------------------------------------------------------------------------------------
|dc |keyspace|table |collecting|transmitting|replication order|priority|dest ks|dest table |src id |src id col|dest |dest enabled|
------------------------------------------------------------------------------------------------------------------------------------------------------
|Cassandra|foo |bar |true |true |FIFO |2 |foo |bar |source1|source_id |mydest|true |
------------------------------------------------------------------------------------------------------------------------------------------------------Properties are continuously read from the metadata, so a restart is not required after
configuration changes are made. The following table describes the configuration
settings.
| Column name | Description |
|---|---|
| separator | Field separator. |
| keyspace | The keyspace on the source for the table to replicate. |
| table | The table name on the source to replicate. |
| source-id | Placeholder to override the source-id that is defined in the advrep_conf metadata |
| source-id-column | Placeholder to override the source-id-column that is defined in advrep_conf metadata. |
| enabled | If true, replication will start for this table. If false, no more messages from this table will be saved to the replication log. |
| data-center-id | Datacenter this replication channel is meant for, if none specified the
replication will happen in all specified dc1. |
| destination | Destination to which data is written. |
| destination-keyspace | The keyspace on the destination for the replicated table. |
| destination-table | The table name on the destination for the replicated table. |
| priority | Messages are marked by priority in descending order (DESC). |
| transmission-enabled | Specify if the data collector for the table should be replicated to the destination. |
| fifo-order | Specify if the channel should be replicated in FIFO order (default). |
| lifo-order | Specify if the channel should be replicated in LIFO order. |
Security
Authentication credentials can be provided in several ways, see .
The user who is doing the replicating with DSE Advanced Replication requires table and keyspace level authorization. If the same user access is required, then ensure that the authorization is the same on the source and destination clusters.
Advanced Replication also supports setting row-level permissions on the destination cluster.
The user which connects to the destination cluster must have permission to write to the
specified destination table at the row level replicated from the source, according to the
RLAC restrictions. The user is specified with the --driver-user
destination setting. Row-level access
control (RLAC) on the source cluster does not impact Advanced Replication. Because Advanced
Replication reads the source data at the raw CDC file layer, it essentially reads as a
superuser and has access to all configured data tables.
Advanced Replication supports encrypting the driver passwords. Driver passwords are stored
in a CQL table. By default, driver passwords are plain text. DataStax recommends encrypting
the driver passwords before you add them to the CQL table. Create a global encryption key,
called a system_key for SSTable encryption. Each node in
the source cluster must have the same system key. The destination does not require this
key.
- In the dse.yaml file:
- Verify that the config_encryption_active property is
false:
config_encryption_active: false - Enable driver password encryption with the
conf_driver_password_encryption_enabled
property:
conf_driver_password_encryption_enabled: true - Define where system keys are stored on disk. The location of the key is specified on the command line with the -d option or with system_key_directory in dse.yaml. The default filepath is /etc/dse/conf.
- To configure the filename of the generated encryption key, set the config_encryption_key_name option in
dse.yaml. The default name is
system_key.
- Verify that the config_encryption_active property is
false:
- Generate a system key:On-server:
dsetool createsystemkey cipher_algorithm strength system_key_file
Off-serverdsetool createsystemkey cipher_algorithm strength system_key_file -kmip=kmip_groupname
For example:dsetool createsystemkey 'AES/ECB/PKCS5Padding' 128 system_key_file
where system_key_file is a unique file name for the generated system key file. See createsystemkey.Result: Configure transparent data encryption (TDE) on a per table basis. You can configure encryption with or without compression. You can create a global encryption key in the location that is specified by system_key_directory in the dse.yaml file. This default global encryption key is used when the system_key_file subproperty is not specified.
- Copy the returned value.
- On any node in the source cluster, use the dse command to set the
encrypted password in the DSE Advanced Replication
environment:
dse advrep destination --driver-pwd "Sa9xOVaym7bddjXUT/eeOQ==" --driver-user "username"
- Start dse.
SSL configuration and ports
For details about SSL configuration with DSE Advanced Replication, refer to .
Note: Enabling client encryption will encrypt all traffic on
the
native_transport_port (default: 9042). If both
encrypted and unencrypted traffic is required, an additional
cassandra.yaml setting must be enabled. The
native_transport_port_ssl (default: 9142) sets an
additional dedicated port to carry encrypted transmissions, while
native_transport_port carries unencrypted transmissions.Data insert methods
There are several ways to get data into a DataStax Enterprise cluster. Any normal paths used will result in data replication using DSE Advanced Replication.
Supported data insert methods:
- CQL insert, including cqlsh and applications that use the standard DSE drivers
- from a CSV file
- Solr HTTP or CQL
- Spark saveToCassandra
- sstableloader (Cassandra bulk loader)
- OpsCenter restore from backup
- Spark bulkSaveToCassandra
Monitoring operations
Advanced replication can be monitored with JMX metrics. The outgoing replication queue size is a key factor to watch. See Metrics for more details.