Using dynamic fields

How to use dynamic fields to index content in fields that are not explicitly defined by the schema.

Using dynamic fields, you can index content in fields that are not explicitly defined by the schema. A common use case for dynamic fields is to identify fields that should not be indexed or to implement a schema-less index.

Search schema fields that are dynamic and multiValued are not supported in CQL-based search indexes.

Spatial subfields prefix naming conventions

Dynamic fields for spatial subfields use prefix naming conventions to enable using map types to store geospatial data:
    <fieldType class="solr.LatLonType" multiValued="false" name="LatLonType" subFieldPrefix="llt_"/>
    <fieldType name="tdouble" class="solr.TrieDoubleField" precisionStep="8" positionIncrementGap="0"/>
    <dynamicField indexed="true" name="latmap*" stored="true" type="LatLonType"/>
    <dynamicField name="llt_*"  type="tdouble"  indexed="true"  stored="true"/>

Best practices

  • Avoid or limit the use of dynamic fields.

    Apache Lucene® allocates memory for each unique field (column) name. For example, for a row with columns A, B, C, and another row with B, D, E, Lucene allocates 5 chunks of memory. For millions of rows, the heap is unwieldy.

  • Instead of using dynamic fields, use a default query field, and then perform queries against the combined field.
  • Use the FieldInputTransformer (FIT) API.

To use a dynamic field

  • Include an Apache Solr dynamic field in the search index schema.

    Name the field using a wildcard at the beginning or end of the field. For example, an asterisk prefix or suffix in the field name in the schema designates a dynamic field.

    • dyna_*
    • *_s
  • To define the map collection column in CQL, use the same base name (no asterisk) that you used for the field in search index schema.

    For example, use dyna_* in the search index schema and dyna_ for the name of the CQL map collection.

  • Use type text for the map key:
    CREATE TABLE my_dynamic_table (
      . . .
      dyna_ map<text, int>,
      . . .
  • Using CQL, insert data into the map using the base name as a prefix or suffix in the first component of each map pair:

    { prefix_literal : literal, prefix_literal : literal, . . . }

    The CQL map looks like:
    'dyna_' : {dyna_1 : 1, dyna_2 : 2, dyn_3 : 3}
    DSE Search maps the dynamic field to a map collection column.