Expiring a DSE Search column

Set a time when data expires to update a DSE Search column.

You can update a DSE Search column to set a time when data expires in these ways:

Configuring the high-performance update handler
Configuring per-document TTL causes removal of the entire document. Configuring per-field TTL causes removal of the field only.
Using the Solr HTTP API
Using CQL to set TTL

If you configure TTL in the solrconfig.xml, and then use the Solr HTTP API to set TTL, the latter takes precedence. If you configure TTL in the solrconfig.xml and then use CQL to set TTL, the latter also takes precedence.

Configuring expiration using the Solr HTTP API

Use the Solr HTTP API update command to set a time-to-live (TTL). You can construct a URL to update data that includes the TTL per-document or per-field parameter:

Using the ttl parameter
Specifies per-document TTL. For example:
```
curl http://host:port/solr/mykeyspace.mytable/update?ttl=86400
```
Using the ttl.field name parameter
Specifies per-field TTL. For example:
```
curl http://host:port/solr/mykeyspace.mytable/update?ttl.myfield=86400
```

Configuring expiration using CQL

Using a CQL INSERT or UPDATE operation, you can set the TTL property. For example, continuing with the example of using a collection set, insert a 5 minute (300 seconds) TTL property on the all columns of the Einstein data:

INSERT INTO mysolr (id, name, title, body)
  VALUES ('126', 'Albert Einstein', 'Success', 'If A is success
  in life, then A equals x plus y plus z. Work is x; y is play;
  and z is keeping your mouth shut.')
  USING TTL 300;

After a few seconds, check the remaining time-to-live on the data:

SELECT TTL (name) FROM mykeyspace.mysolr WHERE id = '126';

The output after 9 seconds expire is:

 ttl(name)
-----------
        291

After the remaining time has passed, the data expires, and querying the data returns no results. If you refresh the Solr Admin console, the number of documents is 3 instead of 4.

Configuring expiration scope

You can configure the solrconfig.xml to include the TTL per-document or per-field on data added to the Solr index or Cassandra database. You construct a Solr HTTP API query to search the Solr index using a ttl component. Depending on the configuration, TTL then applies to the entire document or just to a named field.

To configure per-document or per-field TTL in the update handler:

Configure the high-performance update handler section of the solrconfig.xml.

For per-document TTL, add these lines to the high-performance updateHandler section:

<!-- The default high-performance update handler -->
 <updateHandler class="solr.DirectUpdateHandler2">
. . .

<lst name="defaults">
 <int name="ttl">1</int>
</lst>

For per-field TTL, add these lines to the updateHandler section:

<lst name = "defaults">
 <int  name = "ttl.<column/field name1>">1</int>
 <int  name = "ttl.<column/field name2>">1</int>
 <int  name = "ttl.<column/field name3>">1</int>
 <int  name = "ttl.<column/field name4>">1</int>
 . . .
</lst>

Re-index the data by uploading the schema.xml and solrconfig.xml and reloading the Solr core.

Managing expired columns

After Cassandra expires a column using the time-to-live (TTL) mechanism, DSE Search/Solr can still find the expired column. The column data remains in the index until one of the following conditions is met:

Re-indexing occurs due to a DSE Search ttl rebuild timeout.
Set the ttl rebuild timeout properties in the dse.yaml file.
All columns in a row expire due to the Cassandra time-to-live (TTL) mechanism, triggering removal of the entire row/Solr document from the index.

Setting the ttl rebuild timeout properties is the recommended method for managing expired columns.