Managing invalid messages
DSE Advanced Replication strategies for managing invalid messages when replication fails.
During message replication, DSE Advanced Replicates attempts to manipulate the message to ensure successful replication. In some cases, replication might occur with only a subset of the data.
In other cases, replication fails when there are too many differences between the schema on the edge cluster and the schema on the hub cluster. For example, schema incompatibilities occur when a column in the hub has a different type than the same column in the edge, or a table in the edge doesn’t contain all the columns that form the primary key of the same table in the hub.
When a message cannot be replicated, you can configure the maximum number of retries. If replication still fails after that maximum is reached, the message is discarded and removed from the replication log. The replication log on the source cluster stores data in preparation for transmission to the destination cluster.
- SYSTEM_LOG: Log the CQL query and the error message in the system log on the hub. This is the default value.
- TABLE_LOG: Store the CQL query and the error message in the dse_advrep.advrep_invalid_messages_log Cassandra table on the hub.
- NONE: Perform no logging.
- keyspace_name: keyspace name of the invalid query
- table_name: table name of the invalid query
- time_bucket: an hourly time bucket to prevent the Cassandra partition from getting too wide
- id: a time based id (timeuuid)
- cql_string: the CQL query string, explicitly specifies the original timestamp by including the USING TIMESTAMP option.
- error_msg: the error message
Global settings apply to the entire edge cluster. These global settings are stored in the CQL table dse_system.advrep_conf that is automatically created. To define configuration keys to change global settings, you can the dse advrep command line tool or write directly to the CQL table. No action is required to keep using the default system log strategy.
Procedure
To manage invalid messages using the table logging strategy: