Updating the index after data expires (TTL)
Configure purge of expired and deleted data.
Time-To-Live (TTL) set on a CQL field also applies to the indexed values. The DSE Search engine searches the index for expired documents and then deletes them from the index in batches of a configurable size, as defined by the ttl_index_rebuild_options in the dse.yaml file. The default interval is 300 seconds (5 minutes).
Setting data expiration in CQL
- Using CQL INSERT or UPDATE, set the TTL
property.For example, insert a row with a life of 60 seconds into the health_data demo:
INSERT INTO demo.health_data (id, age, gender) VALUES (9999,88,'female') USING TTL 60;
- To force the index to update with the new
data:
COMMIT SEARCH INDEX ON demo.health_data ;
- In this configuration example, after 60 seconds, the row is removed from the CQL table and the search index.
Configuring expiration scope
You can configure the solrconfig.xml to include the TTL per-document or per-field on data added to the search index or the DSE database. You construct a Solr HTTP API query to query the search index using a ttl component. Depending on the configuration, TTL then applies to the entire document or just to a named field.
Managing expired columns
- Reindexing occurs due to a DSE Search TTL rebuild timeout.
Set the ttl rebuild timeout properties in the dse.yaml file.
- All columns in a row expire due to the time-to-live (TTL) mechanism, triggering removal of the entire row from the index.
Setting the TTL timeout properties is the recommended method for managing expired columns.