Expiring a DSE Search column

You can update a DSE Search column to set a time when data expires.

You can update a DSE Search column to set a time when data expires in these ways:

If you configure TTL in the solrconfig.xml file, and then use the Solr HTTP API or CQL to configure a different TTL, the Solr HTTP API or CQL takes precedence.

Configuring expiration using the Solr HTTP API

Use the Solr HTTP API update command to set a time-to-live (TTL). You can construct a URL to update data that includes the TTL per-document or per-field parameter:
  • Using the ttl parameter

    Specifies per-document TTL. For example:

    curl http://host:port/solr/mykeyspace.mytable/update?ttl=86400
  • Using the ttl.field name parameter

    Specifies per-field TTL. For example:

    curl http://host:port/solr/mykeyspace.mytable/update?ttl.myfield=86400

Configuring expiration using CQL

Using a CQL INSERT or UPDATE operation, you can set the TTL property. For example, continuing with the example of using a collection set, insert a 5 minute (300 seconds) TTL property on the all columns of the Einstein data:
INSERT INTO mysolr (id, name, title, body)
  VALUES ('126', 'Albert Einstein', 'Success', 'If A is success
  in life, then A equals x plus y plus z. Work is x; y is play;
  and z is keeping your mouth shut.')
  USING TTL 300;

After a few seconds, check the remaining time-to-live on the data:

SELECT TTL (name) FROM mykeyspace.mysolr WHERE id = '126';

The output after 9 seconds expire is:

 ttl(name)
-----------
        291

After the remaining time has passed, the data expires, and querying the data returns no results. If you refresh the Solr Admin console, the number of documents is 3 instead of 4.

Configuring expiration scope

You can configure the solrconfig.xml to include the TTL per-document or per-field on data added to the Solr index or Cassandra database. You construct a Solr HTTP API query to search the Solr index using a ttl component. Depending on the configuration, TTL then applies to the entire document or just to a named field.

To configure per-document or per-field TTL in the update handler:

  1. Configure the high-performance update handler section of the solrconfig.xml.
    • For per-document TTL, add these lines to the high-performance updateHandler section:
      <!-- The default high-performance update handler -->
       <updateHandler class="solr.DirectUpdateHandler2">
      . . .
      
      <lst name="defaults">
       <int name="ttl">1</int>
      </lst>
      
    • For per-field TTL, add these lines to the updateHandler section:
      <lst name = "defaults">
       <int  name = "ttl.<column/field name1>">1</int>
       <int  name = "ttl.<column/field name2>">1</int>
       <int  name = "ttl.<column/field name3>">1</int>
       <int  name = "ttl.<column/field name4>">1</int>
       . . .
      </lst>
      
  2. Re-index the data by uploading the schema.xml and solrconfig.xml and reloading the Solr core.

Managing expired columns

After Cassandra expires a column using the time-to-live (TTL) mechanism, DSE Search can still find the expired column. The column data remains in the index until one of the following conditions is met:

Setting the ttl rebuild timeout properties is the recommended method for managing expired columns.