Creating a configuration file for DataStax Bulk Loader

Use a configuration file to set values for dsbulk.

The configuration file for setting DataStax Bulk Loader and related DataStax Java Driver values are written in a simple format, one option per line.

Load a configuration from a local file

############ my-application.conf ############

dsbulk {
   # The name of the connector to use
   connector.name = "csv"
   # CSV field delimiter
   connector.csv.delimiter = "|"
   # The keyspace to connect to
   schema.keyspace = "myKeyspace"
   # The table to connect to
   schema.table = "myTable"
   # The field-to-column mapping
   schema.mapping = "0=name, 1=age, 2=email" 
}

datastax-java-driver {
   advanced {
      auth-provider {
         class = "PlainTextAuthProvider"
         username = "myAccount4"
         password = "dse#r0cks!"
      }
   }
}

For information about using SSL with DataStax Bulk Loader, refer to Using SSL with dsbulk.

Tip: Starting in DataStax Bulk Loader 1.4.0, all driver settings need to be prefixed with datastax-java-driver. On the command line, you can shorten this prefix to driver. All other DataStax Bulk Loader settings (that are not driver related) must use the dsbulk prefix in configuration files. However, on the command line, you can omit the dsbulk prefix. To avoid confusion, configuration files are formatted with the following equivalent Human-Optimized Config Object Notation (HOCON) syntax:

dsbulk { connector.name = "csv" ...
            }

. For information about HOCON, refer to this specification.

Important: DataStax recommends specifying username and password credentials in a configuration file instead of on the command line. When done in a configuration file, the credentials are not displayed in subsequent commands such as ps -ef | grep cqlsh.

To use the configuration file, specify -f filename, where filename is the configuration file. Example:

dsbulk load -f my-application.conf -url export.csv -k ks1 -t table1