Using dynamic fields

How to use dynamic fields to index content in fields that are not explicitly defined by the schema and process multiple Solr fields.

Using dynamic fields, you can index content in fields that are not explicitly defined by the schema. Dynamic fields allow you to process multiple Solr fields the same way. A common use case for dynamic fields is to identify fields that should not be indexed or to implement a schema-less index.

In CQL-based Solr cores, the Solr schema fields that are dynamic and multivalued are not supported.

Note: To set up separate dynamic fields for each key, set the dynamic field name to the key. Only keys that match a dynamic field name are indexed.

Spatial subfields prefix naming conventions

Dynamic fields for spatial subfields use prefix naming conventions to enable using Cassandra map types to store geospatial data:
    <fieldType class="solr.LatLonType" multiValued="false" name="LatLonType" subFieldPrefix="llt_"/>
    <fieldType name="tdouble" class="solr.TrieDoubleField" precisionStep="8" positionIncrementGap="0"/>
    <dynamicField indexed="true" name="latmap*" stored="true" type="LatLonType"/>
    <dynamicField name="llt_*"  type="tdouble"  indexed="true"  stored="true"/>

Best practices

  • Avoid or limit the use of dynamic fields.

    Lucene allocates memory for each unique field (column) name, so if you have a row with columns A, B, C, and another row with B, D, E, Lucene allocates 5 chunks of memory. For millions of rows, the heap is unwieldy.

  • Instead of using dynamic fields, use Copy fields instead and then perform queries against the combined field.
  • Use the FieldInputTransformer (FIT) API as an option.

To use a dynamic field

  • Include a Solr dynamic field in schema.xml.

    Name the field using a wildcard at the beginning or end of the field. For example, an asterisk prefix or suffix in the field name in the schema designates a dynamic field.

    • dyna_*
    • *_s
  • In CQL, to define the map collection column, use the same base name (no asterisk) that you used for the field in schema.xml.

    For example, use dyna_* in schema.xml and dyna_ for the name of the CQL map collection.

  • Use type text for the map key. For example:
    CREATE TABLE my_dynamic_table (
      . . .
      dyna_ map<text, int>,
      . . .
  • Using CQL, insert data into the map using the base name as a prefix or suffix in the first component of each map pair. The format of the map using a prefix is:

    { prefix_literal : literal, prefix_literal : literal, . . . }

    For example, the CQL map looks like this:
    'dyn_' : {dyn_1 : 1, dyn_2 : 2, dyn_3 : 3}
    DSE Search maps the Solr dynamic field to a Cassandra map collection column, as shown in the advanced tutorial.