nodetool repair

Repairs one or more tables in a cluster when all involved replicas are up and accessible.

Repairs tables on one or more nodes in a cluster when all involved replicas are up and accessible.

Before using this command, make sure you understand how node repair works. See Repairing nodes.

All nodetool repair command options are optional. When optional command arguments are not specified, the defaults are:

Full repair runs on all keyspaces and all tables.
Repair runs in parallel on all nodes with the same replica data at the same time.
The number of job threads is 1.
No tracing. No validation.

Important: Tables with NodeSync enabled will be skipped for repair operations run against all or specific keyspaces. For individual tables, running the repair command will be rejected when NodeSync is enabled.

Important: If repair encounters a down replica, an error occurs and the repair process halts. After bringing all replicas online, re-run

nodetool
                    repair

Tip: OpsCenter provides a repair option for running a manual repair.

Synopsis

nodetool [connection_options] repair 
[-dcpar | -seq] 
[-full | -inc]
[-hosts ip_address [ip_address ...]]
[-local | -dc datacenter_name[,datacenter_name,...]] 
[-pl] [-pr] [-prv] 
[-pull -hosts local_ip_address [remote_ip_address]
[-j job_threads]
[-st start_token -et end_token] 
[-tr] [--] 
[keyspace_name table_name [table_name ...]]

Table 1. Legend
Syntax conventions	Description
UPPERCASE	Literal keyword.
Lowercase	Not literal.
`Italics`	Variable value. Replace with a valid option or user-defined value.
`[ ]`	Optional. Square brackets ( `[ ]` ) surround optional command arguments. Do not type the square brackets.
`( )`	Group. Parentheses ( `( )` ) identify a group to choose from. Do not type the parentheses.
`\|`	Or. A vertical bar ( `\|` ) separates alternative elements. Type any one of the elements. Do not type the vertical bar.
`...`	Repeatable. An ellipsis ( `...` ) indicates that you can repeat the syntax element as often as required.
`'Literal string'`	Single quotation ( `'` ) marks must surround literal strings in CQL statements. Use single quotation marks to preserve upper case.
`{ key:value }`	Map collection. Braces ( `{ }` ) enclose map collections or key value pairs. A colon separates the key and the value.
`<datatype1,datatype2>`	Set, list, map, or tuple. Angle brackets ( `< >` ) enclose data types in a set, list, map, or tuple. Separate the data types with a comma.
`cql_statement;`	End CQL statement. A semicolon ( `;` ) terminates all CQL statements.
`[ -- ]`	Separate the command line options from the command arguments with two hyphens ( `--` ). This syntax is useful when arguments might be mistaken for command line options.
`' <schema> ... </schema> '`	Search CQL only: Single quotation marks ( `'` ) surround an entire XML schema declaration.
`@xml_entity='xml_entity_type'`	Search CQL only: Identify the entity and literal value to overwrite the XML element in the schema and solrconfig files.

The short form and long form parameters are comma-separated.

Connection options

-h, --host hostname: The hostname or IP address of a remote node or nodes. When omitted, the default is the local machine.
-p, --port jmx_port: The JMX port number.
-pw, --password jmxpassword: The JMX password for authenticating with secure JMX. If a password is not provided, you are prompted to enter one.
-pwf, --password-file jmx_password_filepath: The filepath to the file that stores JMX authentication credentials.
-u, --username jmx_username: The username for authenticating with secure JMX.

Command arguments

--: Separates an option from an argument that could be mistaken for a option.
-dc datacenter_name, --in-dc datacenter_name: Comma-separated list of datacenters to limit repairs to. Datacenter names are case sensitive. Decreases network traffic while repairing more nodes than the local option. When this option is not specified, repair is run cluster-wide on all nodes that contain replicas.
-dcpar, --dc-parallel: Run repairs on all nodes with the same replica data at the same time, recommended for repairs across datacenters. A single node in each datacenter runs repair, one after another until the repair is complete. This option combines sequential and parallel repair by simultaneously running a sequential repair in all datacenters. Use with the -local option only when the datacenter nodes have all the data for all ranges.
-et, --end-token end_token: The token at which the range ends. Requires start token (-st).
-force, --force: Filter out down endpoints.
-full, --full: Issue a full repair of all data ranges on the node where the command is issued and stream data to all nodes that have replicas for any of the token ranges held by the node where the command is run. A full repair keeps unrepaired and repaired data together. A full repair of all SSTables on a node takes a lot of time and is resource-intensive. Full repair is the default repair strategy.
-hosts, --in-hosts host_name: Repair specific hosts.
-inc, --inc: Issue an incremental repair.
Incremental repair splits the data into repaired and unrepaired SSTables and marks the repaired data with a RepairedAt timestamp. Incremental repair consumes less time and resources because it skips SSTables that are already marked as repaired. However, because unrepaired and repaired data is split, all subsequent repairs must also be run in incremental mode. To migrate to full repair strategy, see /en/dse/6.7/dse-admin/datastax_enterprise/operations/changeRepairStrategies.html. The percent repaired output of nodetool tablestats shows the percentage of data (uncompressed) that is marked as repaired across all non-system tables on a node.
-j, --job-threads num_threads: Number of threads to run repair jobs. Usually this means number of tables to repair concurrently. Default: 1. Max: 4.
Warning: Increasing job threads puts more load on repairing nodes.
keyspace_name table_name [table_name ...]: The keyspace to repair followed by one or many tables. If not specified, all tables will be repaired.
-local, --in-local-dc: Repair only against nodes in the same datacenter.
-pl, --pull: Runs a one-way repair directly from another node that has a replica in the same token range. This option minimizes performance impact when cross-datacenter repairs are required.
-pr, --partitioner-range: Repair only the first range returned by the partitioner.
-prv, --preview: Determine ranges and amount of data to be streamed, but doesn't perform repair.
-seq, --sequential: Perform a sequential repair operation. If not specified, then run parallel repair operations.
Important: Parallel repair operations cause a significant impact on performance.
-st, --start-token start_token: The token at which the range starts. Requires end token (-et).
table_name: One or more table names, separated by a space.
-tr, --trace: Trace the repair. Traces are logged to system_traces.events.
-vd, --validate: Checks that repaired data is in sync between nodes.
Note: Out of sync repaired data indicates a full repair should be run.

Examples

Run sequential repair of all keyspaces

nodetool repair -seq

Run partitioner range repair

nodetool repair -pr

Run start-point-to-end-point repair of all nodes between two nodes on the ring

nodetool repair -st -9223372036854775808 -et -3074457345618258603

Restrict repair to local datacenter

nodetool repair -dc DC1

Results

Results in output for repairs:

[2019-12-18 18:40:16,449] Starting repair command #1 (d59dbd60-21c5-11ea-8e0e-cf718f0bee2a), repairing keyspace system_traces with repair options (parallelism: sequential, primary range: false, incremental: false, job threads: 1, ColumnFamilies: {}, dataCenters: {}, hosts: {}, previewKind: NONE, # of ranges: 2, pull repair: false, force repair: false)
[2019-12-18 18:40:16,560] Repair session d5a49b30-21c5-11ea-8e0e-cf718f0bee2a for range [(-3074457345618258603,3074457345618258602]] finished (progress: 16%)
[2019-12-18 18:40:16,601] Repair session d5a69700-21c5-11ea-8e0e-cf718f0bee2a for range [(3074457345618258602,-9223372036854775808]] finished (progress: 33%)
[2019-12-18 18:40:16,603] Repair completed successfully
[2019-12-18 18:40:16,607] Repair command #1 finished in 0 seconds
...

Results in output for keyspaces that don't require repairs:

[...
[2019-12-18 18:40:16,855] Replication factor is 1. No repair is needed for keyspace 'cycling'

The system.log shows repair runs only on IP addresses in DC1:

. . .
                    INFO  [AntiEntropyStage:1] 2014-07-24 22:23:10,708 RepairSession.java:171 
                    - [repair #16499ef0-1381-11e4-88e3-c972e09793ca] Received merkle tree 
                    for sessions from /192.168.2.101
                    INFO  [RepairJobTask:1] 2014-07-24 22:23:10,740 RepairJob.java:145 
                    - [repair #16499ef0-1381-11e4-88e3-c972e09793ca] requesting merkle trees 
                    for events (to [/192.168.2.103, /192.168.2.101])
                    . . .