DSE Advanced Replication command line tool

DSE Advanced Replication commands

These DSE Advanced Replication commands are available:

DSE Advanced Replication command options

Synopsis

dse  advrep [--v1] [connection_options] [command] [sub_command] [sub_command_options]

To show the command line help for

dse
    advrep

:

dse advrep help

Authentication credentials can be provided in several ways, see Connecting to authentication enabled clusters.

The optional flag, --v1, can be used to access advanced replication commands for DSE 5.0; advanced replication commands changed with DSE 5.1. See DSE 5.0 Advanced Replication documentation for commands compatible with the --v1 flag. An example of using the --v1 flag is:

dse advrep --v1 edge conf --edge-id "edge1" --edge-id-col-name "edge_id" --hub-ip-addresses "10.200.182.148"

Client to DSE connection commands

The default port for DSE Advanced Replication is 9042. Connection options are specified as:

dse advrep [connection_options]


Connection options	Description
`--separator field_separator`	Specify the field separator if `--no-pretty-print` is used for printing. Default: comma
`--no-pretty-print`	If specified, data is printed as a comma delimited list. If not specified, data is printed as tabular output.
`-u username`	DSE username, or alternatively, DSE_USERNAME environment variable can be used.
`--jmx-user`	JMX metrics user
`--jmx-port`	JMX port (default: 7199)
`--jmx-pwd`	JMX metrics password
`-p password`	DSE password, or alternatively, DSE_PASSWORD environment variable can be used.
`--use-server-config`	Read connection configuration from server YAML files (dse.yaml and cassandra.yaml) instead of reading them from configuration files. Use only when the DSE installation against which the command run is a running node.
`--cipher-suites ssl_cipher_suites`	A comma-separated list of SSL cipher suites for connection to DSE when SSL is enabled. For example, `--cipher-suites=c1,c2,c3`.
`--host address`	The DSE host RPC broadcast address. The default value is localhost.
`--port port`	The DSE native protocol RPC connection port.
`--kerberos-enabled true\|false`	true\|false - If true, enable Kerberos for connection to DSE. For example, `--kerberos-enabled true`.
`--keystore-password ssl_keystore_password`	Keystore password for connection to DSE when SSL client authentication is enabled.
`--keystore-path ssl_keystore_path`	Set path to the keystore for connection to DSE when SSL client authentication is enabled.
`--keystore-type ssl_keystore_type`	Set keystore type for connection to DSE when SSL client authentication is enabled. JKS is the type for keys generated by the Java keytool binary, but other types are possible, depending on user environment.
`--truststore-password ssl_truststore_password`	Set the truststore password for connection to DSE when SSL is enabled.
`--truststore-path ssl_truststore_path`	Set path to truststore for connection to DSE when SSL is enabled.
`--truststore-type ssl_truststore_type`	Set truststore type for connection to DSE when SSL is enabled. JKS is the type for keys generated by the Java keytool binary, but other types are possible, depending on user environment.
`--sasl-protocol-name dse_service_principal`	The SASL protocol name must match the username of the Kerberos service principal used by the DSE server.
`--ssl`	Specify whether SSL is enabled for connection to DSE. `--ssl-enabled true` is the same as `--ssl`.
`--ssl-protocol ssl_protocol`	SSL protocol for connection to DSE when SSL is enabled.
`-t`	Delegation token which can be used to login, or alternatively, DSE_TOKEN environment variable can be used.

Connection options examples:

dse advrep --host ip-10-200-300-138.example.lan --kerberos-enabled=true conf list

and

dse advrep --use-server-config conf list

Printing option example:

dse advrep --no-pretty-print destination list-conf --separator "|"

will result in output:

destination|name|value
 mydest|addresses|192.168.200.100
 mydest|transmission-enabled|true
 mydest|driver-ssl-cipher-suites|TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256,TLS_DHE_RSA_WITH_AES_128_CBC_SHA256,TLS_DHE_DSS_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDH_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_DSS_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_RC4_128_SHA,TLS_ECDHE_RSA_WITH_RC4_128_SHA,SSL_RSA_WITH_RC4_128_SHA,TLS_ECDH_ECDSA_WITH_RC4_128_SHA,TLS_ECDH_RSA_WITH_RC4_128_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_DSS_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,SSL_RSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA,SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA,SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA,SSL_RSA_WITH_RC4_128_MD5,TLS_EMPTY_RENEGOTIATION_INFO_SCSV
 mydest|driver-ssl-enabled|false
 mydest|driver-ssl-protocol|TLS
 mydest|name|mydest
 mydest|driver-connect-timeout|15000
 mydest|driver-max-requests-per-connection|1024
 mydest|driver-connections-max|8
 mydest|driver-connections|1
 mydest|driver-compression|lz4
 mydest|driver-consistency-level|ONE
 mydest|driver-allow-remote-dcs-for-local-cl|false
 mydest|driver-used-hosts-per-remote-dc|0
 mydest|driver-read-timeout|15000

If --no-pretty-print is not used, the output is in tabular format by default.

Replication channel commands

Synopsis

dse advrep channel command [sub_command]  [sub_command_options]

To show the command line help for

dse advrep
    channel

:

dse advrep help channel

Command and command arguments for:

dse advrep channel sub_command [sub_command_options]


Sub-command	Sub-command options
`create`	Create a replication channel for a keyspace: `--source-keyspace keyspace_name` (required) `--destination-table destination_table_name` `--source-id-column` `--fifo-order` Specify if the channel should be replicated in FIFO order (default). `--lifo-order` Specify if the channel should be replicated in LIFO order. `--destination destination` (required) Destination where the replication will be sent. `--source-table source_table_name` (required) Source table to replicate. `--collection-enabled true \| false` specify if the source table should be enabled for replication collection on creation. `--priority channel_priority` `--transmission-enabled true \| false` Specify if data collector for the table should be replicated to the destination.
`update`	Update a replication channel for a keyspace: `--source-keyspace keyspace_name` `--destination-table destination_table_name` `--source-id-column` `--fifo-order` Specify if the channel should be replicated in FIFO order (default). `--lifo-order` Specify if the channel should be replicated in LIFO order. `--destination destination` Destination where the replication will be sent. `--source-table source_table_name` Source table to replicate. `--collection-enabled true \| false` specify if the source table should be enabled for replication collection on creation. `--priority channel_priority` `--transmission-enabled true \| false` Specify if data collector for the table should be replicated to the destination.
`delete`	Delete replication channel: `--source-keyspace keyspace_name` (required) `--destination destination` (required) Destination where the replication will be sent. `--source-table source_table_name` (required) Source table to replicate. `--data-center-id data-center-id Datacenter for this channel`
`pause`	Pause replication channel for a keyspace: `--source-keyspace keyspace_name` `--data-center-ids data-center-id(s) Comma-separated list of data-center-ids on which to filter` `--destinations destination(s)` Comma-separated list of destinations `--transmission If specified, no data for the source table is sent to the specified destination.` `--source-table source_table_name` `--collection` If specified, no data for the source table is collected.
`resume`	Resume replication for a paused channel: `--source-keyspace keyspace_name` `--data-center-ids data-center-id(s) Comma-separated list of data-center-ids on which to filter` `--destinations destination(s)` Comma-separated list of destinations `--transmission If specified, no data for the source table is sent to the specified destination.` `--source-table source_table_name` `--collection` If specified, no data for the source table is collected.
`status`	Verify status of replication channel: `--keyspace keyspace_name` `--destination destination` Destination where the replication will be sent. `--table source_table_name` Source table to replicate. `--data-center-id data-center-id Datacenter for this channel`
`truncate`	Truncate a channel, all messages currently in the replication log for that channel will not be replicated: `--source-keyspace keyspace_name` `--data-center-ids data-center-id(s) Comma-separated list of data-center-ids on which to filter` `--destinations destination(s)` Comma-separated list of destinations `--source-table source_table_name`

Destination commands

Synopsis

dse advrep destination [sub_command] [sub_command_options]

To show the command line help for

dse advrep
    destination

:

dse advrep help destination

Commands and command arguments for edge configuration and replication:

dse advrep destination sub_command [sub_command_options]


Command	Command arguments
`create`	Create a channel from a keyspace source table to a destination cluster: `--name` (required) `--addresses` (required) `--transmission-enabled true\|false` `--driver-user password` `--driver-pwd username` `--driver-connect-timeout ms_driver_waits_to_connect_server` `--driver-read-timeout ms_driver_waits_to_read_server_responses` `--driver-compression lz4_or_snappy_algorithm` `--driver-connections num_connections_to_create` `--driver-connections-max max_num_connections_to_create` `--driver-max-requests-per-connections max_num_requests_per_connection` `--driver-local-dc dc_name` `--driver-consistency-level consistency` [ANY\|ONE\|TWO\|THREE\|QUORUM\|ALL\|LOCAL_QUORUM\|EACH_QUORUM\|SERIAL\|LOCAL_SERIAL\|LOCAL_ONE] `--driver-used-hosts-per-remote-dc number_of_hosts` `--driver-allow-remote-dcs-for-local-cl true\|false` `--driver-ssl-enabled true\|false` `--driver-ssl-keystore-path ssl_keystore_path` `--driver_ssl_keystore_password ssl_keystore_password` `--driver_ssl_keystore_type ssl_keystore_type` `--driver_ssl_truststore_path ssl_truststore_path` `--driver-ssl-truststore-password ssl_truststore_password` `--driver-ssl-truststore-type ssl_keystore_type` `--driver-ssl-protocol ssl_protocol` `--driver-ssl-cipher-suites ssl_cipher_suites`
`update`	Update a channel from a keyspace source table to a destination cluster: `--name` (required) `--addresses` `--transmission-enabled true\|false` `--driver-user password` `--driver-pwd username` `--driver-connect-timeout ms_driver_waits_to_connect_server` `--driver-read-timeout ms_driver_waits_to_read_server_responses` `--driver-compression lz4_or_snappy_algorithm` `--driver-connections num_connections_to_create` `--driver-connections-max max_num_connections_to_create` `--driver-max-requests-per-connections max_num_requests_per_connection` `--driver-local-dc dc_name` `--driver-consistency-level consistency` [ANY\|ONE\|TWO\|THREE\|QUORUM\|ALL\|LOCAL_QUORUM\|EACH_QUORUM\|SERIAL\|LOCAL_SERIAL\|LOCAL_ONE] `--driver-used-hosts-per-remote-dc number_of_hosts` `--driver-allow-remote-dcs-for-local-cl true\|false` `--driver-ssl-enabled true\|false` `--driver-ssl-keystore-path ssl_keystore_path` `--driver_ssl_keystore_password ssl_keystore_password` `--driver_ssl_keystore_type ssl_keystore_type` `--driver_ssl_truststore_path ssl_truststore_path` `--driver-ssl-truststore-password ssl_truststore_password` `--driver-ssl-truststore-type ssl_keystore_type` `--driver-ssl-protocol ssl_protocol` `--driver-ssl-cipher-suites ssl_cipher_suites`
`delete`	Delete destination: `--name` (required)
`list-conf`	List the destination configuration: `--name` (required)
`remove-conf`	Remove configuration for a channel from a keyspace source table to a destination cluster: `--name` (required) `--transmission-enabled true\|false` `--driver-user password` `--driver-pwd username` `--driver-connect-timeout ms_driver_waits_to_connect_server` `--driver-read-timeout ms_driver_waits_to_read_server_responses` `--driver-compression lz4_or_snappy_algorithm` `--driver-connections num_connections_to_create` `--driver-connections-max max_num_connections_to_create` `--driver-local-dc dc_name` `--driver-consistency-level consistency` [ANY\|ONE\|TWO\|THREE\|QUORUM\|ALL\|LOCAL_QUORUM\|EACH_QUORUM\|SERIAL\|LOCAL_SERIAL\|LOCAL_ONE] `--driver-used-hosts-per-remote-dc number_of_hosts` `--driver-allow-remote-dcs-for-local-cl true\|false` `--driver-ssl-enabled true\|false` `--driver-ssl-keystore-path ssl_keystore_path` `--driver_ssl_keystore_password ssl_keystore_password` `--driver_ssl_keystore_type ssl_keystore_type` `--driver_ssl_truststore_path ssl_truststore_path` `--driver-ssl-truststore-password ssl_truststore_password` `--driver-ssl-truststore-type ssl_keystore_type` `--driver-ssl-protocol ssl_protocol` `--driver-ssl-cipher-suites ssl_cipher_suites`
`list`	List the destination - no options

Configuration commands

Synopsis

dse advrep conf [sub_command] [sub_command_options]

To show the command line help for

dse advrep
    conf

:

dse advrep help conf

Commands and command arguments for configuration:

dse advrep conf sub_command [sub_command_options]


Command	Command arguments
`update`	Update the configuration: `--collection-max-open-files value Number of option files kept` `--audit-log-file audit_log_name`Audit log file name `--audit-log-compression none \| gzip`Enable audit log compression. `--audit-log-enabled true \| false`Enable audit logging. `--audit-log-max-life-span-mins minutes`Audit log max lifespan in minutes before discarding. `--audit-log-rotate-mins minutes`Number of minutes before audit log rotates files. `--permits` Maximum number of messages that can be replicated in parallel over all destinations (default: 30,000). `--invalid-message-log` Error information for messages that could not be replicated are saved to system_log, channel_log (default), or none. `--collection-time-slice-count count` The number of files which are open in the ingestor simultaneously. Default: 5 `--collection-time-slice-width` The time period in seconds for each data block ingested. Smaller time widths equal more files. Larger timer widths equal larger files but more data to resend on CRC mismatches. Default: 60 seconds `--collection-expire-after-write`
`remove`	Remove configuration: `--collection-max-open-files value Number of option files kept` `--audit-log-file audit_log_name`Audit log file name `--audit-log-compression` Disable audit log compression. `--audit-log-enabled true \| false`Enable audit logging. `--audit-log-max-life-span-mins minutes`Audit log max lifespan in minutes before discarding. `--audit-log-rotate-mins minutes`Number of minutes before audit log rotates files. `--permits` Maximum number of messages that can be replicated in parallel over all destinations (default: 30,000). `--invalid-message-log` Error information for messages that could not be replicated are saved to system_log, channel_log (default), or none. `--collection-time-slice-count count` The number of files which are open in the ingestor simultaneously. `--collection-time-slice-width` The time period in seconds for each data block ingested. Smaller time widths equal more files. Larger timer widths equal larger files but more data to resend on CRC mismatches. `--collection-expire-after-write`
`list`	List the configuration - no options

Metrics

Display metrics from the command line once JMX access is configured:

dse advrep metrics

To show the command line help for dse advrep metrics:

dse advrep help metrics

Command options for:

dse advrep metrics list [sub_command_options]


Command	Command options
`list`	`[ --metric-group filter_by_group ]` `[ --metric-type filter_by_type ]`

Replication log commands

Display replication log information from the command line.

dse advrep replog

To show the command line help for

dse advrep
     replog

:

dse advrep help replog

Command options for:

dse advrep replog sub_command [sub_command_options]


Command	Command options
count	`--data-center-id data-center-id` `--destination destination` (required) Destination where the replication will be sent. `--source-keyspace keyspace_name` (required) `--source-table table_name` (required)
`analyze-audit-log`	`--file filename`Audit log file.

Replication command examples

The dse advrep channel command line tool examples and results:

Creating a destination

dse advrep --verbose destination create --name mydest --addresses 10.200.182.148 --transmission-enabled true
Destination mydest created

Creating a replication source channel

dse advrep channel create --source-keyspace foo --source-table bar --source-id source1 --source-id-column source_id --destination mydest --destination-keyspace foo --destination-table bar --collection-enabled true --priority 1  
Created channel dc=Cassandra keyspace=foo table=bar to mydest

Viewing a replication source channel status

dse advrep channel status
------------------------------------------------------------------------------------------------------------------------------------------------------
|dc       |keyspace|table          |collecting|transmitting|replication order|priority|dest ks|dest table     |src id |src id col|dest  |dest enabled|
------------------------------------------------------------------------------------------------------------------------------------------------------
|Cassandra|foo     |bar            |true      |false       |FIFO             |1       |foo    |bar            |source1|source_id |mydest|true        |
------------------------------------------------------------------------------------------------------------------------------------------------------

Removing a destination channel

dse advrep delete --name mydest

Removing a replication source channel

dse advrep delete --source-keyspace foo --source-table bar --destination mydest

Resuming collection from a source to a destination

dse advrep channel resume --source-keyspace foo --source-table bar --destinations mydest --collection
Channel dc=Cassandra keyspace=foo table=bar  collection to mydest was resumed

Resuming transmission from a source to a destination

dse advrep channel resume --source-keyspace foo --source-table bar --destinations mydest --transmission
Channel dc=Cassandra keyspace=foo table=bar transmission to mydest was resumed

Pausing transmission from a source to a destination

The replication is stopped for the entire edge cluster. Specify the IP address of any node on the edge.

dse advrep channel pause --source-keyspace foo --source-table bar --destinations mydest --transmission
Channel dc=Cassandra keyspace=foo table=bar transmission to mydest was paused

Verifying the record count held in the replication log

dse advrep replog count --destination mydest --source-keyspace foo --source-table bar

 2

Metrics command examples

The dse advrep metrics command line tool examples and results.

Show all metrics

dse advrep --host localhost --port 7199 metrics list
-----------------------------------------------
|Group         |Type                    |Count|
-----------------------------------------------
|ReplicationLog|CommitLogsToConsume     |1    |
-----------------------------------------------
|ReplicationLog|MessageAddErrors        |0    |
-----------------------------------------------
|Tables        |MessagesInReplicationLog|0    |
-----------------------------------------------
|ReplicationLog|CommitLogsDeleted       |2    |
-----------------------------------------------

-----------------------------------------------------------------------------------------------------------------------------------------------------
|Group         |Type                 |Count|RateUnit     |MeanRate             |FifteenMinuteRate     |OneMinuteRate          |FiveMinuteRate       |
-----------------------------------------------------------------------------------------------------------------------------------------------------
|ReplicationLog|MessagesAdded        |4    |events/second|2.640169784463287E-4 |3.5260944599250134E-10|1.5526670553919196E-107|7.437289948631685E-24|
-----------------------------------------------------------------------------------------------------------------------------------------------------
|ReplicationLog|MessagesDeleted      |0    |events/second|0.0                  |0.0                   |0.0                    |0.0                  |
-----------------------------------------------------------------------------------------------------------------------------------------------------
|ReplicationLog|MessagesAcknowledged |4    |events/second|2.6401646866982826E-4|3.802761115744516E-10 |4.967788987385306E-107 |9.354595245515022E-24|
-----------------------------------------------------------------------------------------------------------------------------------------------------
|ReplicationLog|CommitLogMessagesRead|3551 |events/second|0.23438054858983395  |0.28481365017306315   |0.2547432175113809     |0.30189617888098824  |
-----------------------------------------------------------------------------------------------------------------------------------------------------

Filter by group

dse advrep --host localhost --port 7199 metrics list --metric-group Tables
---------------------------------------
|Group |Type                    |Count|
---------------------------------------
|Tables|MessagesInReplicationLog|0    |
---------------------------------------

Filter by group and type

dse advrep --host localhost --port 7199 metrics list --metric-type MessagesAdded
--------------------------------------------------------------------------------------------------------------------------------------------
|Group         |Type         |Count|RateUnit     |MeanRate             |FifteenMinuteRate     |OneMinuteRate         |FiveMinuteRate       |
--------------------------------------------------------------------------------------------------------------------------------------------
|ReplicationLog|MessagesAdded|4    |events/second|2.6221009715537605E-4|3.1552879575422323E-10|2.932609260931711E-108|5.329051114010331E-24|
--------------------------------------------------------------------------------------------------------------------------------------------