Using dynamic fields

Using dynamic fields, you can index content in fields that are not explicitly defined by the schema. A common use case for dynamic fields is to identify fields that should not be indexed or to implement a schema-less index.

Search schema fields that are dynamic and multiValued are not supported in CQL-based search indexes.

Best practices

  • Avoid or limit the use of dynamic fields.

    Apache Lucene® allocates memory for each unique field (column) name. For example, for a row with columns A, B, C, and another row with B, D, E, Lucene allocates 5 chunks of memory. For millions of rows, the heap is unwieldy.

  • Instead of using dynamic fields, use a default query field, and then perform queries against the combined field.

  • Use the FieldInputTransformer (FIT) API.

Spatial subfields prefix naming conventions

Dynamic fields for spatial subfields use prefix naming conventions to enable using map types to store geospatial data:

<types>
    <fieldType class="solr.LatLonType" multiValued="false" name="LatLonType" subFieldPrefix="llt_"/>
    <fieldType name="tdouble" class="solr.TrieDoubleField" precisionStep="8" positionIncrementGap="0"/>
</types>
<fields>
    <dynamicField indexed="true" name="latmap*" stored="true" type="LatLonType"/>
    <dynamicField name="llt_*"  type="tdouble"  indexed="true"  stored="true"/>
</fields>

To use a dynamic field

  • Include an Apache Solr™ dynamic field in the search index schema.

    Name the field using a wildcard at the beginning or end of the field. For example, an asterisk prefix or suffix in the field name in the schema designates a dynamic field:

    • dyna_*

    • *_s

  • To define the map collection column in CQL, use the same base name (no asterisk) that you used for the field in search index schema.

    For example, use dyna_* in the search index schema and dyna_ for the name of the CQL map collection.

  • Use type text for the map key:

    CREATE TABLE my_dynamic_table (
      . . .
      dyna_ map<text, int>,
      . . .
    );
  • Using CQL, insert data into the map using the base name as a prefix or suffix in the first component of each map pair:

    { <prefix_literal : literal, prefix_literal : literal, . . .> }

    The CQL map looks like:

    'dyna_' : {dyna_1 : 1, dyna_2 : 2, dyn_3 : 3}

    DSE Search maps the dynamic field to a map collection column.

Was this helpful?

Give Feedback

How can we improve the documentation?

© 2024 DataStax | Privacy policy | Terms of use

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: +1 (650) 389-6000, info@datastax.com