Trace reads at different consistency levels

Running and tracing queries that read data at different consistency levels.

After performing the setup steps, run and trace queries that read data at different consistency levels. The tracing output shows that using three replicas on a five-node cluster, a consistency level of ONE processes responses from one of three replicas, QUORUM from two of three replicas, and ALL from three of three replicas.

Procedure

  1. Connect cqlsh to the first node in the ring.
    ccm node1 cqlsh
  2. On the cqlsh command line, create a keyspace that specifies using three replica for data distribution in the cluster.
    cqlsh> CREATE KEYSPACE demo_cl WITH replication = {'class':'SimpleStrategy', 'replication_factor':3};
  3. In the three-replica keyspace, create a table, and insert some values:
    cqlsh> USE demo_cl;
    cqlsh:demo_cl> CREATE TABLE demo_table ( id int PRIMARY KEY, col1 int, col2 int );
    cqlsh:demo_cl> INSERT INTO demo_table (id, col1, col2) VALUES (0, 0, 0);
  4. Turn on tracing and use the CONSISTENCY command to check that the consistency level is ONE, the default.
    cqlsh:demo_cl> TRACING on;
    cqlsh:demo_cl> CONSISTENCY;
    
    The output should be:
    Current consistency level is 1.
  5. Query the table to read the value of the primary key.
    cqlsh:demo_cl> SELECT * FROM demo_table WHERE id = 0;
    The output includes tracing information:
     id | col1 | col2
    ----+------+------
      0 |    0 |    0
    
    (1 rows)
    
    Tracing session: 0f5058d0-6761-11e4-96a3-fd07420471ed
    
     activity                                                                                        | timestamp                  | source    | source_elapsed
    -------------------------------------------------------------------------------------------------+----------------------------+-----------+----------------
                                                                                  Execute CQL3 query | 2014-11-08 08:05:29.437000 | 127.0.0.1 |              0
                    Parsing SELECT * FROM demo_table WHERE id = 0 LIMIT 10000; [SharedPool-Worker-1] | 2014-11-08 08:05:29.438000 | 127.0.0.1 |            820
                                                           Preparing statement [SharedPool-Worker-1] | 2014-11-08 08:05:29.438000 | 127.0.0.1 |           1637
                                                    Sending message to /127.0.0.3 [WRITE-/127.0.0.3] | 2014-11-08 08:05:29.439000 | 127.0.0.1 |           2211
                                                    Sending message to /127.0.0.4 [WRITE-/127.0.0.4] | 2014-11-08 08:05:29.439000 | 127.0.0.1 |           2237
                                                        Message received from /127.0.0.1 [Thread-10] | 2014-11-08 08:05:29.441000 | 127.0.0.3 |             75
                                Executing single-partition query on demo_table [SharedPool-Worker-1] | 2014-11-08 08:05:29.441000 | 127.0.0.4 |            818
                                                  Acquiring sstable references [SharedPool-Worker-1] | 2014-11-08 08:05:29.441000 | 127.0.0.4 |            861
                                                   Merging memtable tombstones [SharedPool-Worker-1] | 2014-11-08 08:05:29.441000 | 127.0.0.4 |            915
     Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones [SharedPool-Worker-1] | 2014-11-08 08:05:29.442000 | 127.0.0.4 |            999
                                    Merging data from memtables and 0 sstables [SharedPool-Worker-1] | 2014-11-08 08:05:29.442000 | 127.0.0.4 |           1018
                                                   Merging memtable tombstones [SharedPool-Worker-1] | 2014-11-08 08:05:29.442000 | 127.0.0.3 |           1058
     Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones [SharedPool-Worker-1] | 2014-11-08 08:05:29.442000 | 127.0.0.3 |           1146
                                    Merging data from memtables and 0 sstables [SharedPool-Worker-1] | 2014-11-08 08:05:29.442000 | 127.0.0.3 |           1165
                                            Read 1 live and 0 tombstoned cells [SharedPool-Worker-1] | 2014-11-08 08:05:29.442000 | 127.0.0.3 |           1223
                                              Enqueuing response to /127.0.0.1 [SharedPool-Worker-1] | 2014-11-08 08:05:29.442001 | 127.0.0.3 |           1504
                                                         Message received from /127.0.0.4 [Thread-7] | 2014-11-08 08:05:29.443000 | 127.0.0.1 |           6399
                                                    Sending message to /127.0.0.1 [WRITE-/127.0.0.1] | 2014-11-08 08:05:29.443000 | 127.0.0.3 |           1835
                                                         Message received from /127.0.0.3 [Thread-8] | 2014-11-08 08:05:29.443000 | 127.0.0.1 |           6449
                                           Processing response from /127.0.0.4 [SharedPool-Worker-2] | 2014-11-08 08:05:29.443000 | 127.0.0.1 |           6623
                                           Processing response from /127.0.0.3 [SharedPool-Worker-3] | 2014-11-08 08:05:29.443000 | 127.0.0.1 |           6635
                                                                                    Request complete | 2014-11-08 08:05:29.443897 | 127.0.0.1 |           6897
  6. Change the consistency level to QUORUM and run the SELECT statement again.
    cqlsh:demo_cl> CONSISTENCY quorum;
    cqlsh:demo_cl> SELECT * FROM demo_table WHERE id = 0;
     id | col1 | col2
    ----+------+------
      0 |    0 |    0
    
    (1 rows)
    
    
    Tracing session: 3bbae430-6761-11e4-96a3-fd07420471ed
    
     activity                                                                                        | timestamp                  | source    | source_elapsed
    -------------------------------------------------------------------------------------------------+----------------------------+-----------+----------------
                                                                                  Execute CQL3 query | 2014-11-08 08:06:43.955000 | 127.0.0.1 |              0
                    Parsing SELECT * FROM demo_table WHERE id = 0 LIMIT 10000; [SharedPool-Worker-1] | 2014-11-08 08:06:43.955000 | 127.0.0.1 |             71
                                                           Preparing statement [SharedPool-Worker-1] | 2014-11-08 08:06:43.955000 | 127.0.0.1 |            267
                                                    Sending message to /127.0.0.4 [WRITE-/127.0.0.4] | 2014-11-08 08:06:43.956000 | 127.0.0.1 |           1628
                                                    Sending message to /127.0.0.5 [WRITE-/127.0.0.5] | 2014-11-08 08:06:43.956000 | 127.0.0.1 |           1690
                                                         Message received from /127.0.0.1 [Thread-9] | 2014-11-08 08:06:43.957000 | 127.0.0.5 |             95
                                Executing single-partition query on demo_table [SharedPool-Worker-2] | 2014-11-08 08:06:43.957000 | 127.0.0.4 |            229
                                                  Acquiring sstable references [SharedPool-Worker-2] | 2014-11-08 08:06:43.957000 | 127.0.0.4 |            249
                                                   Merging memtable tombstones [SharedPool-Worker-2] | 2014-11-08 08:06:43.957000 | 127.0.0.4 |            299
     Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones [SharedPool-Worker-2] | 2014-11-08 08:06:43.957000 | 127.0.0.4 |            387
                                    Merging data from memtables and 0 sstables [SharedPool-Worker-2] | 2014-11-08 08:06:43.957000 | 127.0.0.4 |            408
                                            Read 1 live and 0 tombstoned cells [SharedPool-Worker-2] | 2014-11-08 08:06:43.957000 | 127.0.0.4 |            469
                                              Enqueuing response to /127.0.0.1 [SharedPool-Worker-2] | 2014-11-08 08:06:43.957001 | 127.0.0.4 |            734
                                                    Sending message to /127.0.0.1 [WRITE-/127.0.0.1] | 2014-11-08 08:06:43.957001 | 127.0.0.4 |            894
                                                         Message received from /127.0.0.4 [Thread-7] | 2014-11-08 08:06:43.958000 | 127.0.0.1 |           3383
                                           Processing response from /127.0.0.4 [SharedPool-Worker-2] | 2014-11-08 08:06:43.958000 | 127.0.0.1 |           3612
                                Executing single-partition query on demo_table [SharedPool-Worker-1] | 2014-11-08 08:06:43.959000 | 127.0.0.5 |           1462
                                                  Acquiring sstable references [SharedPool-Worker-1] | 2014-11-08 08:06:43.959000 | 127.0.0.5 |           1509
                                                   Merging memtable tombstones [SharedPool-Worker-1] | 2014-11-08 08:06:43.959000 | 127.0.0.5 |           1569
     Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones [SharedPool-Worker-1] | 2014-11-08 08:06:43.959000 | 127.0.0.5 |           1662
                                    Merging data from memtables and 0 sstables [SharedPool-Worker-1] | 2014-11-08 08:06:43.959000 | 127.0.0.5 |           1681
                                            Read 1 live and 0 tombstoned cells [SharedPool-Worker-1] | 2014-11-08 08:06:43.959000 | 127.0.0.5 |           1760
                                              Enqueuing response to /127.0.0.1 [SharedPool-Worker-1] | 2014-11-08 08:06:43.959001 | 127.0.0.5 |           2104
                                                        Message received from /127.0.0.5 [Thread-10] | 2014-11-08 08:06:43.960000 | 127.0.0.1 |           5330
                                                    Sending message to /127.0.0.1 [WRITE-/127.0.0.1] | 2014-11-08 08:06:43.960000 | 127.0.0.5 |           2423
                                           Processing response from /127.0.0.5 [SharedPool-Worker-2] | 2014-11-08 08:06:43.960000 | 127.0.0.1 |           5519
                                                                                    Request complete | 2014-11-08 08:06:43.960947 | 127.0.0.1 |           5947
  7. Change the consistency level to ALL and run the SELECT statement again.
    cqlsh:demo_cl> CONSISTENCY ALL;
    cqlsh:demo_cl> SELECT * FROM demo_table WHERE id = 0;
     id | col1 | col2
    ----+------+------
      0 |    0 |    0
    
    (1 rows)
    
    
    Tracing session: 4da75ca0-6761-11e4-96a3-fd07420471ed
    
     activity                                                                                        | timestamp                  | source    | source_elapsed
    -------------------------------------------------------------------------------------------------+----------------------------+-----------+----------------
                                                                                  Execute CQL3 query | 2014-11-08 08:07:14.026000 | 127.0.0.1 |              0
                    Parsing SELECT * FROM demo_table WHERE id = 0 LIMIT 10000; [SharedPool-Worker-1] | 2014-11-08 08:07:14.026000 | 127.0.0.1 |             73
                                                           Preparing statement [SharedPool-Worker-1] | 2014-11-08 08:07:14.026000 | 127.0.0.1 |            271
                                                    Sending message to /127.0.0.4 [WRITE-/127.0.0.4] | 2014-11-08 08:07:14.027000 | 127.0.0.1 |            978
                                                         Message received from /127.0.0.1 [Thread-9] | 2014-11-08 08:07:14.027000 | 127.0.0.5 |             56
                                                    Sending message to /127.0.0.5 [WRITE-/127.0.0.5] | 2014-11-08 08:07:14.027000 | 127.0.0.1 |           1012
                                Executing single-partition query on demo_table [SharedPool-Worker-2] | 2014-11-08 08:07:14.027000 | 127.0.0.3 |            253
                                                    Sending message to /127.0.0.3 [WRITE-/127.0.0.3] | 2014-11-08 08:07:14.027000 | 127.0.0.1 |           1054
                                                  Acquiring sstable references [SharedPool-Worker-2] | 2014-11-08 08:07:14.027000 | 127.0.0.3 |            275
                                                   Merging memtable tombstones [SharedPool-Worker-2] | 2014-11-08 08:07:14.027000 | 127.0.0.3 |            344
     Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones [SharedPool-Worker-2] | 2014-11-08 08:07:14.028000 | 127.0.0.3 |            438
                                                  Acquiring sstable references [SharedPool-Worker-1] | 2014-11-08 08:07:14.028000 | 127.0.0.5 |            461
                                                   Merging memtable tombstones [SharedPool-Worker-1] | 2014-11-08 08:07:14.028000 | 127.0.0.5 |            525
     Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones [SharedPool-Worker-1] | 2014-11-08 08:07:14.028000 | 127.0.0.5 |            622
                                    Merging data from memtables and 0 sstables [SharedPool-Worker-1] | 2014-11-08 08:07:14.028000 | 127.0.0.5 |            645
                                            Read 1 live and 0 tombstoned cells [SharedPool-Worker-1] | 2014-11-08 08:07:14.028000 | 127.0.0.5 |            606
                                              Enqueuing response to /127.0.0.1 [SharedPool-Worker-1] | 2014-11-08 08:07:14.028001 | 127.0.0.5 |           1125
                                                         Message received from /127.0.0.3 [Thread-8] | 2014-11-08 08:07:14.029000 | 127.0.0.1 |           3224
                                                    Sending message to /127.0.0.1 [WRITE-/127.0.0.1] | 2014-11-08 08:07:14.029000 | 127.0.0.5 |           1616
                                           Processing response from /127.0.0.3 [SharedPool-Worker-3] | 2014-11-08 08:07:14.029000 | 127.0.0.1 |           3417
                                                        Message received from /127.0.0.5 [Thread-10] | 2014-11-08 08:07:14.029000 | 127.0.0.1 |           3454
                                                         Message received from /127.0.0.4 [Thread-7] | 2014-11-08 08:07:14.029000 | 127.0.0.1 |           3516
                                           Processing response from /127.0.0.5 [SharedPool-Worker-2] | 2014-11-08 08:07:14.029000 | 127.0.0.1 |           3627
                                           Processing response from /127.0.0.4 [SharedPool-Worker-2] | 2014-11-08 08:07:14.030000 | 127.0.0.1 |           3688
                                                                                    Request complete | 2014-11-08 08:07:14.030347 | 127.0.0.1 |           4347