DSEFS command line tool
Options and command arguments for the DSE File System (DSEFS).
The DSEFS functionality supports operations including uploading, downloading, moving, and deleting files, creating directories, and verifying the DSEFS status.
DSEFS commands are available only in the logical datacenter. DSEFS works with secured and unsecured clusters, see DSEFS authentication.
You can interact with the DSEFS file system in several modes: interactive command line shell, as part of dse commands, or with a REST API.
Interactive DSEFS command line shell
Action | Command line |
---|---|
Launch DSEFS shell | dse fs dsefs / >The DSEFS prompt shows the current working directory on DSEFS. The current local working directory that you launch DSEFS from is the default directory that is used for searching local files. |
Launch DSEFS shell with precedence given to the specified hosts |
dse fs --prefer-contact-points -h 10.0.0.2,10.0.0.5 The Without the |
View entire DSEFS command list |
dsefs / > help |
View help for any DSEFS command |
dsefs / > help dsefs_command |
Add a comment to a DSEFS shell command | Use the # character. Everything after the #
character will be
ignored.dsefs / > get archive.tgz local_archive.tgz #retrieve the archive |
Exit DSEFS shell | Press Ctrl+D or type exit |
Configuring DSEFS shell logging
The default location of the DSEFS shell log file .dsefs-shell.log is the user home directory. The default log level is INFO. To configure DSEFS shell logging, edit the installation_location/resources/dse/conf/logback-dsefs-shell.xml file.
Using with the dse command line
dse
:dse [dse_auth_credentials] fs dsefs_command [options]
dse -u user1 -p mypassword fs "df -h"
Optional command arguments are enclosed in square brackets. For example, [dse_auth_credentials] and [-R]
Variable values are italicized. For example, directory and [subcommand].
Working with the local file system in the DSEFS shell
You can refer to files in the local file system by prefixing paths with
file:
. For example the following command will list files in the system
root directory:
dsefs dsefs://127.0.0.1:5598/ > ls file:/
bin cdrom dev home lib32 lost+found mnt proc run srv tmp var initrd.img.old vmlinuz.old boot data etc lib lib64 media opt root sbin sys usr initrd.img vmlinuz
If you need to perform many subsequent operations on the local file system, first change
the current working directory to file:
or any local file system path:
dsefs dsefs://127.0.0.1:5598/ > cd file: dsefs file:/home/user1/path/to/local/files > ls conf src target build.sbt dsefs file:/home/user1/path/to/local/files > cd .. dsefs file:/home/user1/path/to/local >
DSEFS shell remembers the last working directory of each file system separately. To go back to the previous DSEFS directory, enter:
dsefs file:/home/user1/path/to/local/files > cd dsefs: dsefs dsefs://127.0.0.1:5598/ >
To go back again to the previous local directory:
dsefs dsefs://127.0.0.1:5598/ > cd file: dsefs file:/home/user1/path/to/local/files >
To refer to a path relative to the last working directory of the file system, prefix a
relative path with either dsefs:
or file:
. The following
session will create a directory new_directory
in the directory
/home/user1
:
dsefs dsefs://127.0.0.1:5598/ > cd file:/home/user1 dsefs file:/home/user1 > cd dsefs: dsefs dsefs://127.0.0.1:5598/ > mkdir file:new_directory dsefs dsefs://127.0.0.1:5598/ > realpath file:new_directory file:/home/user1/new_directory dsefs dsefs://127.0.0.1:5598/ > stat file:new_directory DIRECTORY file:/home/user1/new_directory: Owner user1 Group user1 Permission rwxr-xr-x Created 2017-01-15 13:10:06+0200 Modified 2017-01-15 13:10:06+0200 Accessed 2017-01-15 13:10:06+0200 Size 4096
To copy a file between two different file systems, you can also use the cp
command with explicit file system prefixes in the paths:
dsefs file:/home/user1/test > cp dsefs:archive.tgz another-archive-copy.tgz dsefs file:/home/user1/test > ls another-archive-copy.tgz archive-copy.tgz archive.tgz
Authentication
For $ dse dse_auth_credentials
you can provide user
credentials in several ways, see Connecting to authentication enabled clusters. For authentication with DSEFS, see DSEFS authentication.
Executing multiple commands
DSEFS can execute multiple commands on one line. Use quotes around the commands and arguments. Each command will be executed separately by DSEFS.
dse fs 'cat file1 file2 file3 file4' 'ls dir1'
DSEFS command options
DSEFS command | Description and command arguments |
---|---|
append source
destination |
Append a local file to a remote file.
|
cat file_or_files |
Concatenate files and print on the standard output.
|
cd directory |
Change the remote working directory in DSEFS.
|
chgrp [options] group
path |
Change file or directory group ownership.
|
chmod [options] octal permission
mode
path |
Change the permissions of a file or directory.
|
chown [options]
path |
Change files or directories ownership and/or group ownership.
|
cp [options] source
destination |
Copies a file within a file system or between two file systems. If the
destination path points to a different file system than DSEFS, the block size and
redundancy options are ignored.
|
df [options] |
List the DSEFS file system status and disk space usage.
|
exit |
Exit the DSEFS shell client. You can also type Ctrl+D to exit the shell. |
fsck |
Perform a file system consistency check and repair file system errors. Run fsck after running umount, or if you encounter file write errors (for example, timeouts). |
get source
destination |
Get a file from the DSEFS remote file system and copy the file to the local
file system.
|
ls [options]
[file_system_entry_or_entries] |
List the DSEFS file system entries (files or directories) in the current
working directory.
|
mkdir [options]
dir_or_dirs |
Make a new directory or directories.
|
mv source
destination |
Move or rename a file or directory.
|
put [options] source
destination |
Copy a local file to the DSEFS.
|
pwd [path] |
Print the working directory of the current file system or specified path.
|
realpath [options]
path |
Print the resolved absolute path for a specified file or directory.
|
rename path
name |
Rename a file or directory in the current location.
|
rm [-r, -R] path
|
Remove files or directories.
|
rmdir path
|
Remove an empty directory or directories.
|
stat file_or_dir [-v] |
Display the file system entry status.
|
truncate file |
Truncate a file to 0 bytes. Useful for retaining the metadata for the
file.
|
umount [-f] locations |
Unmount file system storage locations.
After running umount, run fsck to add missing block replicas taken away by the unmounted location. |
Removing a DSEFS node
When removing a node running DSEFS from a DSE cluster, additional steps are needed to ensure proper correctness within the DSEFS data set.
- From a node in the same datacenter as the node to be removed, start the DSEFS shell.
dse fs
- Show the current DSEFS nodes with the
df
command.dsefs > df
Location Status DC Rack Host Address Port Directory Used Free Reserved 144e587c-11b1-4d74-80f7-dc5e0c744aca up GraphAnalytics rack1 node1.example.com 10.200.179.38 5598 /var/lib/dsefs/data 0 29289783296 5368709120 98ca0435-fb36-4344-b5b1-8d776d35c7d6 up GraphAnalytics rack1 node2.example.com 10.200.179.39 5598 /var/lib/dsefs/data 0 29302099968 5368709120
- Find the node to be removed in the list and note the UUID value for it under the Location column.
- If the node is up, unmount it from DSEFS with the command
umount UUID
. - If the node is not up (for example, after a hardware failure), force unmount it from
DSEFS with the command
umount -f UUID
. - Continue with the normal steps for removing a node.
Examples
dsefs / > ls -l
dsefs / > put file:/bluefile greenfileTo view the new file in the DSEFS directory:
dsefs / > ls -l
Type Permission Owner Group Length Modified Name file rwxrwxrwx none none 17 2016-05-11 09:34:26+0000 greenfile
dse fs "mkdir /test2" && dse fs "put README.md /test2/README.md"To view the new directory listing:
dse fs "ls -l /test2"
Type Permission Owner Group Length Modified Name
file rwxrwxrwx none none 3382 2016-03-07 23:20:34+0000 README.md
dse fs "mkdir / test2" "put README.md /test/README.md"