DSE Advanced Replication command line tool

These DSE Advanced Replication commands are available:

• Command options for all commands
• Client to DSE connection commands
• Replication channel commands
• Replication destination commands
• Replication configuration commands
• Replication log commands
• Metrics commands

Synopsis

dse  advrep [--v1] [connection_options] [command] [sub_command] [sub_command_options]

To show the command line help for dse advrep:

dse advrep help

The optional flag, --v1, can be used to access advanced replication commands for DSE 5.0; advanced replication commands changed with DSE 5.1. See DSE 5.0 Advanced Replication documentation for commands compatible with the --v1 flag. An example of using the --v1 flag is:

dse advrep --v1 edge conf --edge-id "edge1" --edge-id-col-name "edge_id" --hub-ip-addresses "10.200.182.148"

The default port for DSE Advanced Replication is 9042. Connection options are specified as:

dse advrep [connection_options]

Connection options	Description
--separator field_separator	Specify the field separator if --no-pretty-print is used for printing. Default: comma
--no-pretty-print	If specified, data is printed as a comma delimited list. If not specified, data is printed as tabular output.
-u username	DSE username, or alternatively, DSE_USERNAME environment variable can be used.
--jmx-user	JMX metrics user
--jmx-port	JMX port (default: 7199)
--jmx-pwd	JMX metrics password
-p password	DSE password, or alternatively, DSE_PASSWORD environment variable can be used.
--use-server-config	Read connection configuration from server YAML files (dse.yaml and cassandra.yaml) instead of reading them from configuration files. Use only when the DSE installation against which the command run is a running node.
--cipher-suites ssl_cipher_suites	A comma-separated list of SSL cipher suites for connection to DSE when SSL is enabled. For example, --cipher-suites=c1,c2,c3.
--host address	The DSE host RPC broadcast address. The default value is localhost.
--port port	The DSE native protocol RPC connection port.
--kerberos-enabled true|false	true|false - If true, enable Kerberos for connection to DSE. For example, --kerberos-enabled true.
--keystore-password ssl_keystore_password	Keystore password for connection to DSE when SSL client authentication is enabled.
--keystore-path ssl_keystore_path	Set path to the keystore for connection to DSE when SSL client authentication is enabled.
--keystore-type ssl_keystore_type	Set keystore type for connection to DSE when SSL client authentication is enabled. JKS is the type for keys generated by the Java keytool binary, but other types are possible, depending on user environment.
--truststore-password ssl_truststore_password	Set the truststore password for connection to DSE when SSL is enabled.
--truststore-path ssl_truststore_path	Set path to truststore for connection to DSE when SSL is enabled.
--truststore-type ssl_truststore_type	Set truststore type for connection to DSE when SSL is enabled. JKS is the type for keys generated by the Java keytool binary, but other types are possible, depending on user environment.
--sasl-protocol-name dse_service_principal	The SASL protocol name must match the username of the Kerberos service principal used by the DSE server.
--ssl	Specify whether SSL is enabled for connection to DSE. --ssl-enabled true is the same as --ssl.
--ssl-protocol ssl_protocol	SSL protocol for connection to DSE when SSL is enabled.
-t	Delegation token which can be used to login, or alternatively, DSE_TOKEN environment variable can be used.

Connection options examples:

dse advrep --host ip-10-200-300-138.example.lan --kerberos-enabled=true conf list

and

dse advrep --use-server-config conf list

Printing option example:

dse advrep --no-pretty-print destination list-conf --separator "|"

will result in output:

destination|name|value
 mydest|addresses|192.168.200.100
 mydest|transmission-enabled|true
 mydest|driver-ssl-cipher-suites|TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256,TLS_DHE_RSA_WITH_AES_128_CBC_SHA256,TLS_DHE_DSS_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDH_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_DSS_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_RC4_128_SHA,TLS_ECDHE_RSA_WITH_RC4_128_SHA,SSL_RSA_WITH_RC4_128_SHA,TLS_ECDH_ECDSA_WITH_RC4_128_SHA,TLS_ECDH_RSA_WITH_RC4_128_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_DSS_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,SSL_RSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA,SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA,SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA,SSL_RSA_WITH_RC4_128_MD5,TLS_EMPTY_RENEGOTIATION_INFO_SCSV
 mydest|driver-ssl-enabled|false
 mydest|driver-ssl-protocol|TLS
 mydest|name|mydest
 mydest|driver-connect-timeout|15000
 mydest|driver-max-requests-per-connection|1024
 mydest|driver-connections-max|8
 mydest|driver-connections|1
 mydest|driver-compression|lz4
 mydest|driver-consistency-level|ONE
 mydest|driver-allow-remote-dcs-for-local-cl|false
 mydest|driver-used-hosts-per-remote-dc|0
 mydest|driver-read-timeout|15000

If --no-pretty-print is not used, the output is in tabular format by default.

Synopsis

dse advrep channel command [sub_command]  [sub_command_options]

To show the command line help for dse advrep channel:

dse advrep help channel

Command and command arguments for:

dse advrep channel sub_command [sub_command_options] 

Sub-command	Sub-command options
create	Create a replication channel for a keyspace: --source-keyspace keyspace_name (required) --destination-table destination_table_name --source-id-column --fifo-order Specify if the channel should be replicated in FIFO order (default). --lifo-order Specify if the channel should be replicated in LIFO order. --destination destination (required) Destination where the replication will be sent. --source-table source_table_name (required) Source table to replicate. --collection-enabled true | false  specify if the source table should be enabled for replication collection on creation. --priority channel_priority --transmission-enabled true | false Specify if data collector for the table should be replicated to the destination.
update	Update a replication channel for a keyspace: --source-keyspace keyspace_name --destination-table destination_table_name --source-id-column --fifo-order Specify if the channel should be replicated in FIFO order (default). --lifo-order Specify if the channel should be replicated in LIFO order. --destination destination Destination where the replication will be sent. --source-table source_table_name Source table to replicate. --collection-enabled true | false  specify if the source table should be enabled for replication collection on creation. --priority channel_priority --transmission-enabled true | false Specify if data collector for the table should be replicated to the destination.
delete	Delete replication channel: --source-keyspace keyspace_name (required) --destination destination (required) Destination where the replication will be sent. --source-table source_table_name (required) Source table to replicate. --data-center-id data-center-id Datacenter for this channel
pause	Pause replication channel for a keyspace: --source-keyspace keyspace_name --data-center-ids data-center-id(s) Comma-separated list of data-center-ids on which to filter --destinations destination(s) Comma-separated list of destinations --transmission If specified, no data for the source table is sent to the specified destination. --source-table source_table_name --collection If specified, no data for the source table is collected.
resume	Resume replication for a paused channel: --source-keyspace keyspace_name --data-center-ids data-center-id(s) Comma-separated list of data-center-ids on which to filter --destinations destination(s) Comma-separated list of destinations --transmission If specified, no data for the source table is sent to the specified destination. --source-table source_table_name --collection If specified, no data for the source table is collected.
status	Verify status of replication channel: --keyspace keyspace_name --destination destination Destination where the replication will be sent. --table source_table_name Source table to replicate. --data-center-id data-center-id Datacenter for this channel
truncate	Truncate a channel, all messages currently in the replication log for that channel will not be replicated: --source-keyspace keyspace_name --data-center-ids data-center-id(s) Comma-separated list of data-center-ids on which to filter --destinations destination(s) Comma-separated list of destinations --source-table source_table_name

Synopsis

dse advrep destination [sub_command] [sub_command_options]

To show the command line help for dse advrep destination:

dse advrep help destination

Commands and command arguments for edge configuration and replication:

dse advrep destination sub_command [sub_command_options]

Command	Command arguments
create	Create a channel from a keyspace source table to a destination cluster: --name (required) --addresses (required) --transmission-enabled true|false --driver-user password --driver-pwd username --driver-connect-timeout ms_driver_waits_to_connect_server --driver-read-timeout ms_driver_waits_to_read_server_responses --driver-compression lz4_or_snappy_algorithm --driver-connections num_connections_to_create --driver-connections-max max_num_connections_to_create --driver-max-requests-per-connections max_num_requests_per_connection --driver-local-dc dc_name --driver-consistency-level consistency [ANY|ONE|TWO|THREE|QUORUM|ALL|LOCAL_QUORUM|EACH_QUORUM|SERIAL|LOCAL_SERIAL|LOCAL_ONE] --driver-used-hosts-per-remote-dc number_of_hosts --driver-allow-remote-dcs-for-local-cl true|false --driver-ssl-enabled true|false --driver-ssl-keystore-path ssl_keystore_path --driver_ssl_keystore_password ssl_keystore_password --driver_ssl_keystore_type ssl_keystore_type --driver_ssl_truststore_path ssl_truststore_path --driver-ssl-truststore-password ssl_truststore_password --driver-ssl-truststore-type ssl_keystore_type --driver-ssl-protocol ssl_protocol --driver-ssl-cipher-suites ssl_cipher_suites
update	Update a channel from a keyspace source table to a destination cluster: --name (required) --addresses --transmission-enabled true|false --driver-user password --driver-pwd username --driver-connect-timeout ms_driver_waits_to_connect_server --driver-read-timeout ms_driver_waits_to_read_server_responses --driver-compression lz4_or_snappy_algorithm --driver-connections num_connections_to_create --driver-connections-max max_num_connections_to_create --driver-max-requests-per-connections max_num_requests_per_connection --driver-local-dc dc_name --driver-consistency-level consistency [ANY|ONE|TWO|THREE|QUORUM|ALL|LOCAL_QUORUM|EACH_QUORUM|SERIAL|LOCAL_SERIAL|LOCAL_ONE] --driver-used-hosts-per-remote-dc number_of_hosts --driver-allow-remote-dcs-for-local-cl true|false --driver-ssl-enabled true|false --driver-ssl-keystore-path ssl_keystore_path --driver_ssl_keystore_password ssl_keystore_password --driver_ssl_keystore_type ssl_keystore_type --driver_ssl_truststore_path ssl_truststore_path --driver-ssl-truststore-password ssl_truststore_password --driver-ssl-truststore-type ssl_keystore_type --driver-ssl-protocol ssl_protocol --driver-ssl-cipher-suites ssl_cipher_suites
delete	Delete destination: --name (required)
list-conf	List the destination configuration: --name (required)
remove-conf	Remove configuration for a channel from a keyspace source table to a destination cluster: --name (required) --transmission-enabled true|false --driver-user password --driver-pwd username --driver-connect-timeout ms_driver_waits_to_connect_server --driver-read-timeout ms_driver_waits_to_read_server_responses --driver-compression lz4_or_snappy_algorithm --driver-connections num_connections_to_create --driver-connections-max max_num_connections_to_create --driver-local-dc dc_name --driver-consistency-level consistency [ANY|ONE|TWO|THREE|QUORUM|ALL|LOCAL_QUORUM|EACH_QUORUM|SERIAL|LOCAL_SERIAL|LOCAL_ONE] --driver-used-hosts-per-remote-dc number_of_hosts --driver-allow-remote-dcs-for-local-cl true|false --driver-ssl-enabled true|false --driver-ssl-keystore-path ssl_keystore_path --driver_ssl_keystore_password ssl_keystore_password --driver_ssl_keystore_type ssl_keystore_type --driver_ssl_truststore_path ssl_truststore_path --driver-ssl-truststore-password ssl_truststore_password --driver-ssl-truststore-type ssl_keystore_type --driver-ssl-protocol ssl_protocol --driver-ssl-cipher-suites ssl_cipher_suites
list	List the destination - no options

Synopsis

dse advrep conf [sub_command] [sub_command_options]

To show the command line help for dse advrep conf:

dse advrep help conf

Commands and command arguments for configuration:

dse advrep conf sub_command [sub_command_options]

Command	Command arguments
update	Update the configuration: --collection-max-open-files value Number of option files kept --audit-log-file audit_log_nameAudit log file name --audit-log-compression none | gzipEnable audit log compression. --audit-log-enabled true | falseEnable audit logging. --audit-log-max-life-span-mins minutesAudit log max lifespan in minutes before discarding. --audit-log-rotate-mins minutesNumber of minutes before audit log rotates files. --permits Maximum number of messages that can be replicated in parallel over all destinations (default: 30,000). --invalid-message-log Error information for messages that could not be replicated are saved to system_log, channel_log (default), or none. --collection-time-slice-count count The number of files which are open in the ingestor simultaneously. Default: 5 --collection-time-slice-width The time period in seconds for each data block ingested. Smaller time widths equal more files. Larger timer widths equal larger files but more data to resend on CRC mismatches. Default: 60 seconds --collection-expire-after-write
remove	Remove configuration: --collection-max-open-files value Number of option files kept --audit-log-file audit_log_nameAudit log file name --audit-log-compression Disable audit log compression. --audit-log-enabled true | falseEnable audit logging. --audit-log-max-life-span-mins minutesAudit log max lifespan in minutes before discarding. --audit-log-rotate-mins minutesNumber of minutes before audit log rotates files. --permits Maximum number of messages that can be replicated in parallel over all destinations (default: 30,000). --invalid-message-log Error information for messages that could not be replicated are saved to system_log, channel_log (default), or none. --collection-time-slice-count count The number of files which are open in the ingestor simultaneously. --collection-time-slice-width The time period in seconds for each data block ingested. Smaller time widths equal more files. Larger timer widths equal larger files but more data to resend on CRC mismatches. --collection-expire-after-write
list	List the configuration - no options

Display metrics from the command line once JMX access is configured:

dse advrep metrics

To show the command line help for dse advrep metrics:

dse advrep help metrics

Command options for:

dse advrep metrics list [sub_command_options]

Command	Command options
list	[ --metric-group filter_by_group ] [ --metric-type filter_by_type ]

Display replication log information from the command line.

dse advrep replog

To show the command line help for dse advrep replog:

dse advrep help replog

Command options for:

dse advrep replog sub_command [sub_command_options]

Command	Command options
count	--data-center-id data-center-id --destination destination (required) Destination where the replication will be sent. --source-keyspace keyspace_name (required) --source-table table_name (required)
analyze-audit-log	--file filenameAudit log file.

The dse advrep channel command line tool examples and results:

Creating a destination

dse advrep --verbose destination create --name mydest --addresses 10.200.182.148 --transmission-enabled true
Destination mydest created

Creating a replication source channel

dse advrep channel create --source-keyspace foo --source-table bar --source-id source1 --source-id-column source_id --destination mydest --destination-keyspace foo --destination-table bar --collection-enabled true --priority 1  
Created channel dc=Cassandra keyspace=foo table=bar to mydest

Viewing a replication source channel status

dse advrep channel status
------------------------------------------------------------------------------------------------------------------------------------------------------
|dc       |keyspace|table          |collecting|transmitting|replication order|priority|dest ks|dest table     |src id |src id col|dest  |dest enabled|
------------------------------------------------------------------------------------------------------------------------------------------------------
|Cassandra|foo     |bar            |true      |false       |FIFO             |1       |foo    |bar            |source1|source_id |mydest|true        |
------------------------------------------------------------------------------------------------------------------------------------------------------

Removing a destination channel

dse advrep delete --name mydest

Removing a replication source channel

dse advrep delete --source-keyspace foo --source-table bar --destination mydest

Resuming collection from a source to a destination

dse advrep channel resume --source-keyspace foo --source-table bar --destinations mydest --collection
Channel dc=Cassandra keyspace=foo table=bar  collection to mydest was resumed

Resuming transmission from a source to a destination

dse advrep channel resume --source-keyspace foo --source-table bar --destinations mydest --transmission
Channel dc=Cassandra keyspace=foo table=bar transmission to mydest was resumed

Pausing transmission from a source to a destination

The replication is stopped for the entire edge cluster. Specify the IP address of any node on the edge.

dse advrep channel pause --source-keyspace foo --source-table bar --destinations mydest --transmission
Channel dc=Cassandra keyspace=foo table=bar transmission to mydest was paused

Verifying the record count held in the replication log

dse advrep replog count --destination mydest --source-keyspace foo --source-table bar

 2

The dse advrep metrics command line tool examples and results.

Show all metrics

dse advrep --host localhost --port 7199 metrics list
-----------------------------------------------
|Group         |Type                    |Count|
-----------------------------------------------
|ReplicationLog|CommitLogsToConsume     |1    |
-----------------------------------------------
|ReplicationLog|MessageAddErrors        |0    |
-----------------------------------------------
|Tables        |MessagesInReplicationLog|0    |
-----------------------------------------------
|ReplicationLog|CommitLogsDeleted       |2    |
-----------------------------------------------

-----------------------------------------------------------------------------------------------------------------------------------------------------
|Group         |Type                 |Count|RateUnit     |MeanRate             |FifteenMinuteRate     |OneMinuteRate          |FiveMinuteRate       |
-----------------------------------------------------------------------------------------------------------------------------------------------------
|ReplicationLog|MessagesAdded        |4    |events/second|2.640169784463287E-4 |3.5260944599250134E-10|1.5526670553919196E-107|7.437289948631685E-24|
-----------------------------------------------------------------------------------------------------------------------------------------------------
|ReplicationLog|MessagesDeleted      |0    |events/second|0.0                  |0.0                   |0.0                    |0.0                  |
-----------------------------------------------------------------------------------------------------------------------------------------------------
|ReplicationLog|MessagesAcknowledged |4    |events/second|2.6401646866982826E-4|3.802761115744516E-10 |4.967788987385306E-107 |9.354595245515022E-24|
-----------------------------------------------------------------------------------------------------------------------------------------------------
|ReplicationLog|CommitLogMessagesRead|3551 |events/second|0.23438054858983395  |0.28481365017306315   |0.2547432175113809     |0.30189617888098824  |
-----------------------------------------------------------------------------------------------------------------------------------------------------

Filter by group

dse advrep --host localhost --port 7199 metrics list --metric-group Tables
---------------------------------------
|Group |Type                    |Count|
---------------------------------------
|Tables|MessagesInReplicationLog|0    |
---------------------------------------

Filter by group and type

dse advrep --host localhost --port 7199 metrics list --metric-type MessagesAdded
--------------------------------------------------------------------------------------------------------------------------------------------
|Group         |Type         |Count|RateUnit     |MeanRate             |FifteenMinuteRate     |OneMinuteRate         |FiveMinuteRate       |
--------------------------------------------------------------------------------------------------------------------------------------------
|ReplicationLog|MessagesAdded|4    |events/second|2.6221009715537605E-4|3.1552879575422323E-10|2.932609260931711E-108|5.329051114010331E-24|
--------------------------------------------------------------------------------------------------------------------------------------------