Steps to enable DSEFS on DataStax Enterprise nodes.
DSEFS is disabled by default. You can enable DSEFS on any node in a DataStax
Enterprise cluster. Plan your deployment with one DSEFS server per DataStax
Enterprise node. A single DSEFS must exist in a single datacenter, because DSEFS
uses LOCAL_QUORUM data consistency.
The location of
the
dse.yaml file depends
on the type of installation:
Installer-Services |
/etc/dse/dse.yaml |
Package installations |
/etc/dse/dse.yaml |
Installer-No Services |
install_location/resources/dse/conf/dse.yaml |
Tarball installations |
install_location/resources/dse/conf/dse.yaml |
Prerequisites
DataStax Enterprise 5.0 or later is
required..
Procedure
On each node:
-
In the file, set the properties for
the DSE File System options:
dsefs_options:
enabled: false
keyspace_name: dsefs
work_dir: /var/lib/dsefs
public_port: 5598
private_port: 5599
data_directories:
- dir: /var/lib/dsefs/data
storage_weight: 1.0
min_free_space: 5368709120
-
Enable DSEFS:
-
Define the keyspace for storing the DSEFS metadata:
-
Define the work directory for storing the DSEFS metadata for the local
node. The work directory must not be shared with other DSEFS
nodes:
-
Define the public port on which DSEFS listens for clients:
public_port: 5598
DataStax
recommends that all nodes in the cluster have the same value. Firewalls must open this port to trusted clients. The service
on this port is bound to the RPC address.
-
Define the private port for DSEFS inter-node communication:
private_port: 5599
Do not open this
port to firewalls; this private port must be not visible from outside of the
cluster.
-
Set the data directories where the file data blocks are stored locally
on each node.
data_directories:
- dir: /var/lib/dsefs/data
If
you use the default
/var/lib/dsefs/data data
directory, verify that the directory exists and that you have root
access. Otherwise, you can define your own directory location, change
the ownership of the directory, or both:
sudo mkdir -p /var/lib/dsefs/data; sudo chown -R $USER:$GROUP /var/lib/dsefs/data
Ensure
that the data directory is writeable by the DataStax Enterprise user.
Put the data directories on different physical devices than the
Cassandra database. Using multiple data directories on JBOD improves
performance and capacity.
-
For each data directory, set the weighting factor to specify how much data to
place in this directory, relative to other directories in the cluster. This soft
constraint determines how DSEFS distributes the data. For example, a directory with a
value of 3.0 receives about three times more data than a directory with a value of
1.0.
data_directories:
- dir: /var/lib/dsefs/data
storage_weight: 1.0
-
For each data directory, define the reserved space, in bytes, to not use for storing file data
blocks. You can use a unit of measure suffix to specify other size units. For example:
terabyte (1tb), gigabyte (10g), and megabyte (5000mb).
data_directories:
- dir: /var/lib/dsefs/data
storage_weight: 1.0
min_free_space: 5368709120
-
Restart the node.
-
Repeat steps for the remaining nodes.
- Optional:
With guidance from DataStax Support, you can tune advanced DSEFS
properties:
# Advanced properties for DSEFS
# service_startup_timeout_ms: 30000
# service_close_timeout_ms: 600000
# server_close_timeout_ms: 600000
# gossip_options:
# round_delay_ms: 5000
# startup_delay_ms: 5000
# shutdown_delay_ms: 30000
# rest_options:
# request_timeout_ms: 330000
# connection_open_timeout_ms: 55000
# client_close_timeout_ms: 60000
# server_request_timeout_ms: 300000
# idle_connection_timeout_ms: 0
# transaction_options:
# transaction_timeout_ms: 3000
# conflict_retry_delay_ms: 200
# execution_retry_delay_ms: 1000
# execution_retry_count: 3
- service_startup_timeout_ms
- Wait time, in milliseconds, before the DSEFS server times out while waiting for
services to bootstrap. Default: 30000
-
Continue with using
DSEFS.