CQL 3 pushdown filter
Optimize the processing of the data by moving filtering expressions in Pig as close to the data source as possible.
DataStax 3.1.4 includes a CqlStorage URL option, use_secondary. Setting the option to true
optimizes the processing of the data by moving filtering expressions in Pig as close to the data
source as possible. To use this capability:
-
Create a secondary index for the Cassandra table.
For Pig pushdown filtering, the secondary index must have the same name as the column it is indexing.
-
Include the use_secondary option with a value of true in the url format for CqlStorage. For example:
newdata = LOAD 'cql://ks/cf_300000_keys_50_cols?use_secondary=true' USING CqlStorage();