Enable automatic failover

Follow these steps to enable automatic failover from the primary OpsCenter instance to the designated backup OpsCenter instance. Enabling failover requires minimal initial set up on the backup OpsCenter.

When configuring automatic failover, both the primary and secondary OpsCenter instances must be running the same OpsCenter version. The OpsCenter daemon (opscenterd) and the DataStax Agents must be running the same version before configuring high availability. DataStax cannot guarantee results if the primary and secondary OpsCenter instances are running different versions.

Locate the opscenterd.conf and the address.yaml files:
- Package installations: /etc/opscenter/opscenterd.conf and /var/lib/datastax-agent/conf/address.yaml
- Tarball installations: INSTALL_DIRECTORY/conf/opscenterd.conf and INSTALL_DIRECTORY/conf/address.yaml

Ensure that address.yaml isn’t being managed by a third-party configuration management system.

During failover, OpsCenter automatically changes stomp_interface in address.yaml to point to the backup OpsCenter instance. If a separate configuration management system is managing address.yaml, that change might be undone when the configuration management system pushes its next update.

Set up a hostname/IP that can switch between primary and backup OpsCenter instances to avoid changing the browser URL for OpsCenter if a failover occurs. If you do not set up a hostname or IP for seamless URL switching post-failover, inform your OpsCenter users of any alternate URL to access OpsCenter.

To override the primary OpsCenter URL generated by OpsCenter during failover, set the override_primary_redirect_url option in opscenterd.conf.
Mirror the configuration directories stored on the primary OpsCenter instance to the backup OpsCenter instance using the method you prefer, such as NFS mount or rysnc. See Backing up critical configuration data for instructions on backing up the active OpsCenter instance to the standby instance.

Create and run an automated script to keep the mirrored directories in sync.

The following example cron scripts run rsync to synchronize the configuration directories every 5 minutes.

Package installation
Tarball installation

*/5 * * * * /usr/bin/rsync -az /etc/opscenter USER@BACKUP_HOST:/etc/opscenter

*/5 * * * * /usr/bin/rsync -az /var/lib/opscenter/ssl USER@BACKUP_HOST:/var/lib/opscenter/ssl

*/5 * * * * /usr/bin/rsync -az install_location/conf USER@BACKUP_HOST:install_location/conf

*/5 * * * * /usr/bin/rsync -az install_location/ssl USER@BACKUP_HOST:install_location/ssl

When a failover occurs, you must manually stop the sync scripts on the former primary OpsCenter instance and start the sync scripts on the new primary OpsCenter instance. Failure to do so will result in configuration changes on the new primary OpsCenter instance being overwritten by stale files from the former primary OpsCenter instance.

If you want to override the default values, edit the failover section of the opscenterd.conf file.

If you make any changes to the opscenterd.conf file, you must restart OpsCenter.

Option Description Default

heartbeat_period

Frequency (in seconds) that the primary OpsCenter instance sends a heartbeat to the backup OpsCenter instance.

heartbeat_reply_period

Frequency (in seconds) with which the OpsCenter backup instance sends a heartbeat to the primary OpsCenter instance.

300

heartbeat_fail_window

Amount of time (in seconds) that must elapse before the lack of a heartbeat triggers a failover.

failover_configuration_directory

Directory location where failover-specific configuration is stored. The failover_id file is also located in the failover directory.

Don’t mirror the failover_configuration_directory across OpsCenter installs when configuring OpsCenter to support failover.

Package installation: /var/lib/opscenter/failover/

Tarball installation: INSTALL_DIRECTORY/failover/

override_primary_redirect_url

If set, overrides the primary OpsCenter URL automatically generated during failover. Example: https://opscenter:80. This option provides more flexibility for setting a port in the redirect URL.

Empty. Allows OpsCenter to generate the redirect URL for the newly primary OpsCenter instance switched from backup OpsCenter instance status.

On the backup OpsCenter instance, in the failover directory, create a primary_opscenter_location configuration file that indicates the IP address of the primary OpsCenter daemon to monitor.

Create the file at the appropriate location for your installation type:
- Package installation: /var/lib/opscenter/failover/primary_opscenter_location
- Tarball installation: INSTALL_DIRECTORY/failover/primary_opscenter_location
Make sure the primary_opscenter_location file only contains the IP address of the primary OpsCenter instance and nothing more:
```
cat primary_opscenter_location
```
```
55.100.200.300
```
Ensure the user running OpsCenter has at least read permission for the primary_opscenter_location file.

Before the backup OpsCenter instance can take over as the primary OpsCenter instance, the backup OpsCenter instance deletes the primary_opscenter_location file in the event of a failover. After a failover, recreate the primary_opscenter_location file on the newly designated backup OpsCenter instance.

Enable automatic failover

Was this helpful?

Give Feedback