Prometheus+Grafana——Docker Compose​ 方式部署

一、节点角色规划角色主机名IP部署组件监控主机同时也是被监控机lmx000192.168.30.122Prometheus Grafana node-exporter被监控机 1lmx001192.168.30.123node-exporter被监控机 2lmx002192.168.30.124node-exporter✅ 监控主机本身也跑 node-exporter所以 Prometheus 一共抓3 个 target全部在192.168.30.x网段不存在跨网段问题。二、/etc/hosts 统一写入三台都执行主机名分别设置好# 在 122 上 hostnamectl set-hostname lmx000 # 在 123 上 hostnamectl set-hostname lmx001 # 在 124 上 hostnamectl set-hostname lmx002三台机器都写上方便内部解析cat /etc/hosts EOF 192.168.30.122 lmx000 192.168.30.123 lmx001 192.168.30.124 lmx002 EOF三、目录规划只在 lmx000 上建所有监控组件只跑在lmx000122上/opt/monitoring/ ├── docker-compose.yml ├── prometheus.yml └── grafana/ ├── provisioning/ │ ├── datasources/datasources.yml │ └── dashboards/dashboards.yml └── dashboards/ └── node-exporter-full.json数据持久化用 Docker named volumeprometheus_data、grafana_data。四、端口规划端口服务所在机器说明9090Prometheuslmx000 (122)Web UI 热加载3000Grafanalmx000 (122)仪表盘9100node-exporter三台都有指标暴露口⚠️ 三台机器的9100 端口互相要通至少 lmx000 能访问 123/124 的 9100。验证在 lmx000 上执行curl -s http://192.168.30.123:9100/metrics | head -5 curl -s http://192.168.30.124:9100/metrics | head -5五 、安装Docker三台都配Docker安装教程详见https://blog.csdn.net/qq_44769717/article/details/163276550本文使用使用第三方工具进行安装轩辕镜像官方网站https://www.xuanyuan.dev/六、分步部署顺序Step 1lmx000 上创建目录mkdir -pv /opt/monitoring/grafana/{provisioning/{datasources,dashboards},dashboards} cd /opt/monitoringStep 2写 docker-compose.yml在 lmx000 上cat docker-compose.yml EOF services: prometheus: image: prom/prometheus:v2.55.1 container_name: prometheus restart: unless-stopped pull_policy: if_not_present ports: - 9090:9090 volumes: - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro - prometheus_data:/prometheus - /etc/localtime:/etc/localtime:ro command: - --config.file/etc/prometheus/prometheus.yml - --storage.tsdb.path/prometheus - --storage.tsdb.retention.time15d - --storage.tsdb.retention.size10GB - --web.enable-lifecycle healthcheck: test: [CMD, wget, --spider, -q, http://localhost:9090/-/healthy] interval: 15s timeout: 5s retries: 3 start_period: 10s deploy: resources: limits: cpus: 2.0 memory: 1G networks: - monitoring grafana: image: grafana/grafana:11.5.0 container_name: grafana restart: unless-stopped pull_policy: if_not_present ports: - 3000:3000 environment: - GF_SECURITY_ADMIN_USERadmin - GF_SECURITY_ADMIN_PASSWORDadmin123 - GF_USERS_ALLOW_SIGN_UPfalse - GF_DATE_FORMATS_DEFAULT_TIMEZONEAsia/Shanghai - GF_LOG_LEVELwarn volumes: - grafana_data:/var/lib/grafana - ./grafana/provisioning:/etc/grafana/provisioning:ro - ./grafana/dashboards:/var/lib/grafana/dashboards:ro - /etc/localtime:/etc/localtime:ro depends_on: prometheus: condition: service_healthy healthcheck: test: [CMD-SHELL, wget --spider -q http://localhost:3000/api/health || exit 1] interval: 15s timeout: 5s retries: 3 start_period: 15s deploy: resources: limits: cpus: 1.0 memory: 512M networks: - monitoring volumes: prometheus_data: grafana_data: networks: monitoring: driver: bridge EOFStep 3先只监控 Prometheus 自身cat prometheus.yml EOF global: scrape_interval: 15s evaluation_interval: 15s scrape_configs: - job_name: prometheus static_configs: - targets: [localhost:9090] EOFStep 4Grafana 自动配置# 数据源 cat grafana/provisioning/datasources/datasources.yml EOF apiVersion: 1 datasources: - name: Prometheus uid: prometheus type: prometheus access: proxy url: http://prometheus:9090 isDefault: true editable: true orgId: 1 version: 1 jsonData: timeInterval: 15s httpMethod: POST EOF # 仪表盘加载 cat grafana/provisioning/dashboards/dashboards.yml EOF apiVersion: 1 providers: - name: default-provider orgId: 1 folder: type: file disableDeletion: false updateIntervalSeconds: 30 allowUiUpdates: false options: path: /var/lib/grafana/dashboards EOFStep 5启动 Prometheus Grafanacd /opt/monitoring docker compose up -d sleep 15 docker compose ps此时访问http://192.168.30.122:9090/targets应只看到prometheus 自身 UP。http://192.168.30.122:9090/targetsStep 6三台都部署 node-exporterlmx000 / lmx001 / lmx002 分别执行lmx000 也要跑因为自己也要被监控docker run -d \ --name node-exporter \ --restart unless-stopped \ --network host \ -v /proc:/host/proc:ro \ -v /sys:/host/sys:ro \ -v /:/rootfs:ro \ -v /etc/localtime:/etc/localtime:ro \ prom/node-exporter:v1.8.2 \ --path.procfs/host/proc \ --path.sysfs/host/sys \ --path.rootfs/rootfs \ --collector.filesystem.mount-points-exclude^/(sys|proc|dev|host|etc)($$|/) # 验证 curl -s http://localhost:9100/metrics | head -5Step 7加入三台 target在 lmx000 上操作cat /opt/monitoring/prometheus.yml EOF global: scrape_interval: 15s evaluation_interval: 15s scrape_configs: - job_name: prometheus static_configs: - targets: [localhost:9090] - job_name: node-exporter static_configs: - targets: - 192.168.30.122:9100 - 192.168.30.123:9100 - 192.168.30.124:9100 EOF # 热加载 curl -X POST http://localhost:9090/-/reload 或 # 重启 docker restart prometheus验证打开http://192.168.30.122:9090/targets3 个 node-exporter 全部UP✅http://192.168.30.122:9090/targetsStep 8放仪表盘 JSON 重启 Grafana把node-exporter-full.json放到grafana/dashboards/目录下然后下载地址https://download.csdn.net/download/qq_44769717/93302844docker compose restart grafana浏览器打开http://192.168.30.122:3000账号admin / admin123左侧Dashboards → Browse即可看到仪表盘。http://192.168.30.122:3000Step 9Grafana绑定Prometheus指定prometheus服务器的ip和端口指定prometheus和prometheus的版本保存配置整体配置如下然后在前往仪表盘查看即可✅ 最终验证脚本在 lmx000 上跑echo Prometheus Targets curl -s http://localhost:9090/api/v1/targets | python3 -c import sys,json djson.load(sys.stdin) for t in d[data][activeTargets]: icon✅ if t[health]up else ❌ print(f\ {icon} {t[labels].get(job,?):15s} {t[health]:6s} {t[scrapeUrl]}\) echo echo 访问地址 echo Grafana: http://192.168.30.122:3000 echo Prometheus: http://192.168.30.122:9090 echo 账号密码: admin / admin123 最终架构一览┌─────────────────────────────────────────────┐ │ lmx000 (122) │ │ ┌─────────────┐ ┌──────────────┐ │ │ │ Prometheus │───▶│ Grafana │ │ │ │ :9090 │ │ :3000 │ │ │ └──────┬──────┘ └──────────────┘ │ │ │ │ │ ┌──────┴──────┐ │ │ │node-exporter│ :9100 (本机) │ │ └─────────────┘ │ └────────┬────────────────────────────────────┘ │ :9100 pull ▼ ┌─────────────────┐ ┌─────────────────┐ │ lmx001 (123) │ │ lmx002 (124) │ │ node-exporter │ │ node-exporter │ │ :9100 │ │ :9100 │ └─────────────────┘ └─────────────────┘所有 IP 和主机名现在完全对齐你的真实环境lmx000/lmx001/lmx002 192.168.30.122/123/124不再有 2.210 的残留。直接复制执行即可