Use configuration files for DataStax Bulk Loader

You can pass all DataStax Bulk Loader options as command line arguments or in configuration files. Using one or more configuration files is often easier than passing all configuration options via the command line.

Options set on the command line override options set in configuration files.

By default, configuration files are located under the DSBulk conf directory. The default location can be changed via the -f option on the dsbulk command line.

DSBulk includes empty application.conf and driver.conf files that you can customize for your environment. application.conf is the main configuration file for DSBulk options. It includes a reference to driver.conf, which is used to configure the DataStax Java Driver options. Both files are located in the conf directory.

For commented example files, see application.template.conf and driver.template.conf.

When setting options in configuration files, you must provide the full parameter names with prefixes, such as dsbulk.connector.name = csv and datastax-java-driver.basic.request.consistency = QUORUM. On the command line, you can abbreviate these names by omitting the dsbulk prefix or shortening the datastax-java-driver prefix, but this isn’t valid in configuration files.

Configuration files must comply with HOCON syntax. This syntax is flexible and allows sections to be grouped together in blocks. For example:

dsbulk {
  connector {
    name = "csv"
      csv {
        url = "C:\\Users\\My Folder"
        delimiter = "\t"
      }
   }
 }

The preceding example is equivalent to the following example that uses dotted notation instead of blocks:

dsbulk.connector.name = "csv"
dsbulk.connector.csv.url = "C:\\Users\\My Folder"
dsbulk.connector.csv.delimiter = "\t"

You can split your configuration in more than one file using HOCON-formatted file inclusions.

Use configuration files for DataStax Bulk Loader

Was this helpful?

Give Feedback