📊 Story Grafana Dashboard

Monitoring Overview

Server monitoring with Node Exporter, Prometheus, and Grafana provides detailed insights into the real-time performance and health of your server and nodes. By collecting metrics such as CPU usage, memory consumption, disk space, and network traffic, it helps identify and resolve issues promptly, such as SSD shortages or abnormal CPU usage that might indicate node inactivity.

With a Grafana dashboard, you can also visualize network-specific metrics. This includes monitoring block heights, peer counts, validator status, transaction rates, and network performance, ensuring comprehensive monitoring for both server and blockchain node performance in a unified interface.

Node Exporter: This is a program for collecting system metrics from servers. It provides the ability to get detailed information about system resources such as CPU, memory, disk space usage, and more. Node Exporter makes this data available for monitoring through other systems such as Prometheus.
Prometheus: This is a metrics monitoring and collection system. It uses an HTTP-based data collection model to collect metrics from a variety of sources, including Node Exporter. Prometheus allows you to collect, visualize, and analyze this data.
Grafana: This is a tool for data visualization and creating dashboards for monitoring. It integrates with monitoring systems such as Prometheus and allows you to create convenient and informative graphs, tables, and dashboards for analyzing metrics.

System Requirements

CPU - 2 cores
RAM - 2 GB
SSD - At least 20GB free

Installation guide

📌 Node Exporter

Download and install Node Exporter:

wget https://github.com/prometheus/node_exporter/releases/download/v1.3.1/node_exporter-1.3.1.linux-amd64.tar.gz
tar -xvf node_exporter-1.3.1.linux-amd64.tar.gz
sudo mv node_exporter-1.3.1.linux-amd64/node_exporter /usr/local/bin/

Create a systemd service:

sudo tee /etc/systemd/system/node_exporter.service > /dev/null <<EOF
[Unit]
Description=Node Exporter
After=network-online.target

[Service]
User=$USER
ExecStart=/usr/local/bin/node_exporter --web.listen-address=:9222
Restart=always
RestartSec=3
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target
EOF

Start and enable the service

sudo systemctl daemon-reload
sudo systemctl start node_exporter
sudo systemctl enable node_exporter
sudo systemctl status node_exporter

📌 Prometheus

Install Prometheus

wget https://github.com/prometheus/prometheus/releases/download/v2.41.0/prometheus-2.41.0.linux-amd64.tar.gz
tar -xvf prometheus-2.41.0.linux-amd64.tar.gz
mv prometheus-2.41.0.linux-amd64 /usr/local/bin/prometheus

Create a systemd service:

sudo tee /etc/systemd/system/prometheus.service > /dev/null <<EOF
[Unit]
Description=Prometheus
After=network-online.target

[Service]
User=$USER
ExecStart=/usr/local/bin/prometheus/prometheus --config.file=/usr/local/bin/prometheus/prometheus.yml --web.listen-address=:9999
Restart=always

[Install]
WantedBy=multi-user.target
EOF

Start and enable the service

sudo systemctl daemon-reload
sudo systemctl enable prometheus
sudo systemctl start prometheus
sudo systemctl status prometheus

Enable Prometheus in config.toml and look at the prometheus port of your node

sed -i 's/prometheus = false/prometheus = true/' $HOME/.story/story/config/config.toml
sudo systemctl restart story
grep -i "prometheus_listen_addr" $HOME/.story/story/config/config.toml

Configure the `prometheus.yml` file

Set the value of your IP Server and PORT

SERVER_IP=""
PORT=""

Generate the configuration file content

cat <<EOF | sudo tee $HOME/prometheus/prometheus.yml > /dev/null
global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "node_exporter"
    static_configs:
      - targets: ['$SERVER_IP:9100']
        labels:
          label: "Service"

  - job_name: 'prometheus'
    scrape_interval: 5s
    scrape_timeout: 5s
    static_configs:
      - targets: ['$SERVER_IP:9999']

  - job_name: 'StoryMetrics'
    static_configs:
      - targets: ['$SERVER_IP:$PORT']
EOF

Restart prometheus

sudo systemctl restart prometheusd
sudo systemctl status prometheusd

📌 Grafana

Install Grafana

sudo apt-get install -y adduser libfontconfig1 && \
    wget https://dl.grafana.com/oss/release/grafana_8.0.6_amd64.deb && \
    sudo dpkg -i grafana_8.0.6_amd64.deb

    sudo systemctl daemon-reload && \
    sudo systemctl enable grafana-server && \
    sudo systemctl restart grafana-server
    sudo systemctl status grafana