Data Node Cluster Configuration

Encryption 

Cipher configuration for the Network Probe, including the java.properties defaults and the procedure for releases older than 7.6.1, is described in Network Probe Installation.

Browser layer encryption 

Browser layer encryption is available in the TLS and Encryption section.

For the step-by-step procedure to install and join a new node to an existing cluster, see Multi-node Cluster Installation in the Installation chapter. The sections below cover node role definitions, configuration parameters, and a reference example.

Node roles 

Every instance of the Data Node server is called a node. A collection of connected nodes is called a cluster. All nodes know about all the other nodes in the cluster and can forward client requests to the appropriate node.

Besides that, each node serves one or more purposes:

Master-eligible node - A node that has a node.master set to true (default), which makes it eligible to be elected as the master node, which controls the cluster
Data node - A node that has a node.data set to true (default). Data nodes hold data and perform data-related operations such as CRUD, search, and aggregations
Client node - A client node has both node.master and node.data set to false. It can neither hold data nor become the master node. It behaves as a “smart router” and is used to forward cluster-level requests to the master node and data-related requests (such as search) to the appropriate data nodes

Naming convention 

Data Node requires little configuration before going to work.

The following settings must be considered before going to production:

path.data and path.logs - default locations of the data.
cluster.name - A node can only join a cluster when it shares its cluster.name with all the other nodes in the cluster. The default name is “logserver”, but you should change it to an appropriate name that describes the purpose of the cluster. You can do this in the /etc/logserver/logserver.yml file.
node.name - By default, Data Node will use the first seven characters of the randomly generated UUID as the node ID. Node ID is persisted and does not change when a node restarts. It is worth configuring a more human-readable name: node.name: prod-data-2 in file /etc/logserver/logserver.yml
network.host - parameter specifying network interfaces to which Data Node can bind. The default is network.host: [“_local_”, “_site_”].
discovery - configuration for cluster node discovery:
- discovery.seed_hosts - specify a list of other nodes in the cluster that are likely to be live and contactable;
- cluster.initial_master_nodes - list of master-eligible nodes that should participate in the initial cluster bootstrap.
heap size - By default, Data Node tells the JVM to use a heap with a minimum (Xms) and maximum (Xmx) size of 1 GB. When moving to production, it is important to configure heap size to ensure that Data Node has enough heap available

Config files 

To configure the Data Node cluster you must specify some parameters in the Data Node main .yml configuration file on every node that will be connected to the cluster:

/etc/logserver/:
- cluster.name:name_of_the_cluster - same for every node;
- node.name:name_of_the_node - uniq for every node;
- node.master:true_or_false
- node.data:true_or_false
- network.host:["_local_","_site_"]
- discovery.seed_hosts
- cluster.initial_master_nodes
/etc/logserver/log4j2.properties:
- logger: action: DEBUG - for easier debugging.

TLS Certificates 

To generate TLS certificates for each node of the cluster, you can check the logserver-cluster.yml config that is provided with the tlstool.sh.

cd /usr/share/logserver/utils/tlstool
bash tlstool.sh -c config/logserver-cluster.yml -ca -crt

Example setup 

Example of the Data Node cluster configuration:

File /etc/logserver/logserver.yml:

cluster.name: tm-lab
node.name: "logserver"
node.master: true
node.data: true
network.host: 127.0.0.1,10.0.0.4
http.port: 9200
discovery.seed_hosts: ["10.0.0.4:9300","10.0.0.5:9300","10.0.0.6:9300"]
cluster.initial_master_nodes: ["logserver"]

Apply the configuration on the node:

systemctl restart logserver

Then check the cluster over the API:

curl -XGET '127.0.0.1:9200/_cat/nodes?v'

host            ip   heap.percent ram.percent load node.role master name
10.0.0.4   10.0.0.4     18           91      0.00 -        -       els01
10.0.0.5   10.0.0.5     66           91      0.00 d        *       els02
10.0.0.6   10.0.0.6     43           86      0.65 d        m       els03
10.0.0.7   10.0.0.7     45           77      0.26 d        m       els04

Or follow the cluster log:

tail -f /var/log/logserver/tm-lab.log

The log file is named after cluster.name, so replace tm-lab with the name used on the node.

Adding a new node to the existing cluster 

For the full procedure, see Multi-node Cluster Installation in the Installation chapter.

In brief:

Run ./install.sh -i on the new node. When asked “Is this node a part of an existing cluster?” answer Y.
Update /etc/logserver/logserver.yml with the cluster parameters: cluster.name, node.name, node.master, node.data, discovery.seed_hosts.
If the node has the data role, clear the data directory: rm -rf /var/lib/logserver/*
Finalize: ./install.sh configure