Troubleshooting

Network Probe Troubleshooting

Network Probe registration and management is handled by the license-service. The commands below operate on this service.

Debugging

To check probe’s logs:

journalctl -fu license-service

By default, only errors and warnings are logged. To change that, and enable for example debug level, set log_level: DEBUG in the /opt/license-service/license-service.conf file.

Another operation in progress

The message shown below may be encountered during actions performed on both files and pipelines.

It means that the system is already performing some actions, and in order to ensure the consistency of files, it is necessary to wait until previous actions are completed. After waiting a while, try again to perform the desired action and it should succeed without any problem.

Restarting the probe

To restart the service enter:

systemctl restart license-service

Removing already registered probe

If the probe is running stop the service:
```
systemctl stop license-service
```
Open the “Network Probes” tab in the GUI and go into the details of the probe you are interested in.
After opening the details, copy the opened url and cut out the id located after the /app/network_probe/probes/ section and the /services section. For example id extracted from url below is f7fdb48bf5252cb41ab4162d96144a0d463b7ae5330bae6f37f501cb97fb272d199a59cc97bfdc1d2fd46e981ae42a8a59c66a40932c4bce27d786efe1f2dcc3.
```
https://127.0.0.1:5601/app/network_probe/probes/f7fdb48bf5252cb41ab4162d96144a0d463b7ae5330bae6f37f501cb97fb272d199a59cc97bfdc1d2fd46e981ae42a8a59c66a40932c4bce27d786efe1f2dcc3/services
```
Execute the following query, replacing $ID with the value obtained in the previous step, as well as $USER and $PASSWORD to Data Node:
```
curl -u$USER:$PASSWORD -XDELETE '127.0.0.1:9200/.networkprobes/_doc/$ID'
```

Re-registering probe

If you have a problem with the probe and would like to re-register it, you have to:

Follow the steps in the previous section - Removing already registered probe.
Remove the following file: /opt/license-service/hashKeyKS.p12.network_probe.
Start the probe:
```
systemctl start license-service
```

The probe will be registered as a brand new one, if the license allows it and free slots are available.

Authentication Troubleshooting

System accounts returning 401 / GUI authentication errors

Symptoms: The GUI shows “Error while fetching user details”, API calls with valid credentials return HTTP 401, or license-service enters a crash loop.

The most common cause is a desynchronization between the .auth index and the keystore or service configuration files. This can happen after an upgrade, a partial restore, or manual intervention.

Step 1 — Re-sync credentials

Run the password utility on the client node:

/usr/share/elasticsearch/utils/logserver-password-util.sh update_services --yes

This propagates the keystore credentials to all service configuration files and restarts the affected services. If credentials in the keystore are correct but service configs drifted, this resolves the issue without any index changes.

Step 2 — Verify cluster health

curl -u $USER:$PASSWORD http://127.0.0.1:9200/_cluster/health?pretty

Check that the cluster is green and the .auth index is present:

curl -u $USER:$PASSWORD http://127.0.0.1:9200/_cat/indices/.auth*?v

Step 3 — If the above does not resolve the issue

Temporarily disable the logserver_auth plugin to regain API access, following the procedure in Plugins Management. After disabling, run the password utility again to ensure keystore and configs are consistent, then re-enable the plugin.

Warning

Disabling logserver_auth removes authentication enforcement on the Data Node API for the duration. Isolate the node from external network access before proceeding.