Quick Start - Prometheus

On Linux

1. Download the latest package metricshub-linux-0.9.08.tar.gz using wget and save it under /tmp:

   sudo wget -O /tmp/metricshub-linux-0.9.08.tar.gz https://github.com/sentrysoftware/metricshub/releases/download/v0.9.08/metricshub-linux-0.9.08.tar.gz
   

2. Run the below command to unzip /tmp/metricshub-linux-0.9.08.tar.gz under /opt:

   sudo tar -xzvf /tmp/metricshub-linux-0.9.08.tar.gz -C /opt/
   

There is no need to create a specific subdirectory for metricshub as the archive already contains a metricshub directory.

On Windows

1. Download the latest package, metricshub-windows-0.9.08.zip, from the MetricsHub Releases^[1] page

2. Right-click on the archive, select Extract All…, enter C:\Program Files\, and click Extract. This will place the MetricsHub directory in C:\Program Files\.

Note: You will need administrative privileges to unzip into C:\Program Files.

On Linux

1. Run the below command to download Prometheus:

   sudo wget -O /tmp/prometheus-{version}.linux-{architecture}.tar.gz https://github.com/prometheus/prometheus/releases/download/v{version}/prometheus-{version}.linux-{architecture}.tar.gz
   

   where {version} and {architecture} should be replaced by the prometheus version and processor architecture.

2. Run the below command to extract the package into /opt/prometheus:

   sudo mkdir -p /opt/prometheus && sudo tar -xzvf /tmp/prometheus-{version}.linux-{architecture}.tar.gz -C /opt/prometheus --strip-components=1
   

Note: Make sure to use the corresponding Prometheus version and CPU architecture for {version} and {architecture}. For example, prometheus-2.52.0.linux-amd64 for version 2.52.0 and amd64 architecture. Refer to the Prometheus download site^[2] to find the right Prometheus package.

On Windows

1. Download prometheus-{version}.windows-{architecture}.zip^[2]

2. Right-click the archive, select Extract All…", enter C:\Program Files\, and click Extract. This will place the prometheus-{version}.windows-{architecture} directory in C:\Program Files\

3. Under C:\Program Files\, rename the prometheus-{version}.windows-{architecture} directory to Prometheus.

Note: Make sure to use the corresponding Prometheus version and CPU architecture for {version} and {architecture}. For example, prometheus-2.52.0.windows-amd64 for version 2.52.0 and amd64 architecture.

Create your configuration file

sudo cp /opt/metricshub/lib/config/metricshub-example.yaml /opt/metricshub/lib/config/metricshub.yaml

Configure localhost monitoring

The metricshub-example.yaml file you copied already contains the necessary configuration to monitor your localhost through WMI. The relevant section should look like this:

resources:
  localhost:
    attributes:
      host.name: localhost
      host.type: linux
    protocols:
      osCommand:
        timeout: 120

resources:
  localhost:
    attributes:
      host.name: localhost
      host.type: windows
    protocols:
      wmi:
        timeout: 120

Configure Metrics Exporter

Add the below configuration under the otel section to push metrics to Prometheus:

otel:
  otel.exporter.otlp.metrics.endpoint: http://localhost:9090/api/v1/otlp/v1/metrics
  otel.exporter.otlp.metrics.protocol: http/protobuf

Start Prometheus

Start the MetricsHub Agent

cd /opt/metricsHub/bin
sudo ./service

cd "C:\Program Files\MetricsHub"
MetricsHubServiceManager.exe

Verify that metrics are sent to Prometheus

In Prometheus^[3], search for any metrics starting with metricshub_ or hw_ to confirm that data is actually received.

Check Logs

Several log files are created as soon as the MetricsHub Agent is started:

• a global MetricsHub log file
• one log file per configured host.

They are stored under:

• makefile /opt/metricshub/lib/logs (Linux environments)
• or C:\Users\{Username}\AppData\Local\metricshub\logs (Windows environments).

You can configure the log level in the metricshub.yaml file by setting the loggerLevel parameter to:

• info for high level information
• warn for logging warning messages that indicate potential issues which are not immediately critical
• all, trace, or debug for more comprehensive details
• error or fatal for identifying critical issues.

metricshub.yaml is stored in:

• /opt/metricsHub/lib/config/(Linux environments)
• or C:\ProgramData\MetricsHub\config\ (Windows environments).

The most common errors you may encounter are:

1. Incorrect Indentation

   An incorrect indentation in the metricshub.yaml file prevents the MetricsHub Agent from starting and generates the following exception in the metricshub-agent-global-error-{timestamp}.log file:

   [2024-04-30T15:56:16,944][ERROR][o.s.m.a.MetricsHubAgentApplication] Failed to start MetricsHub Agent.
   com.fasterxml.jackson.dataformat.yaml.snakeyaml.error.MarkedYAMLException: mapping values are not allowed here in 'reader', line 29, column 16:
       host.type:windows
   

2. Wrong Host Configuration

   The following entry will be created in the metricshub-agent-{hostname}-{timestamp}.log file if the host configured cannot be reached:

   [o.s.m.e.c.h.NetworkHelper] Hostname {hostname} - Could not resolve the hostname to a valid IP address. The host is considered remote.
   

   If the host is correctly configured, ensure it is reachable by pinging it and testing your network.

3. Failure to export metrics

The following error occurs if a local OTLP receiver is unavailable to collect MetricsHub logs:

Feb 27, 2024 1:24:26 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog WARNING: Failed to export metrics. 
Server responded with gRPC status code 2. Error message: Failed to connect to localhost/[0:0:0:0:0:0:0:1]:4317

To solve this problem, ensure that the OTLP receiver and more specifically the otel.exporter.otlp.metrics.endpoint and otel.exporter.otlp.logs.endpoint parameters are correctly set in the metricshub.yaml configuration file.