freeleaps-ops/docs/prometheus-metrics-intergration-guideline.md

2.5 KiB
Raw Blame History

1. 服务提供Metrics接口

webapi/providers目录下创建一个metrics.py文件用于提供Metrics接口。

内容如下

def register(app):
  instrumentator = (
      Instrumentator().instrument(
        app,
        metric_namespace="freeleaps-mertics",
        metric_subsystem=app_settings.APP_NAME)
  )

  @app.on_event("startup")
  async def startup():
    instrumentator.expose(app, endpoint="/api/_/metrics", should_gzip=True)
    logging.info("Metrics endpoint exposed at /api/_/metrics")

参考https://gitea.freeleaps.mathmast.com/freeleaps/freeleaps-service-hub/src/branch/master/apps/metrics的接入方式

2. 配置Metrics采集

2.1. 找到helm pkg目录

对于freeleaps内部服务而言,你需要在freeleaps-ops仓库下找到该服务部署的helm-package配置

比如metrics服务他的配置在https://gitea.freeleaps.mathmast.com/freeleaps/freeleaps-ops/src/branch/master/freeleaps/helm-pkg/metrics

对于freeleaps租户服务而言(通过one-click部署),你需要在对应服务仓库下找到.freeleas/devops/helm-pkg

比如magicleaps服务他的配置在https://gitea.freeleaps.mathmast.com/products/magicleaps/src/branch/main/.freeleaps/devops/helm-pkg

2.2. 新增新增servicemonitor配置

在templates文件夹下新增servicemonitor.yaml文件,内容如下,内容参考https://gitea.freeleaps.mathmast.com/freeleaps/freeleaps-ops/src/branch/master/freeleaps/helm-pkg/notification/templates/notification/servicemonitor.yaml

根据value.yaml中的配置更新必要内容比如第6行从notification修改为metrics

2.3. 修改value.{alpha|prod}.yaml

新增serviceMonitor配置参考https://gitea.freeleaps.mathmast.com/freeleaps/freeleaps-ops/src/branch/master/freeleaps/helm-pkg/notification/values.prod.yaml

  - name: notification-service
    type: ClusterIP
    port: 8003
    targetPort: 8003
    serviceMonitor:
      enabled: true # alpha环境设置为false
      labels:
        release: kube-prometheus-stack
      namespace: freeleaps-monitoring-system
      internal: 30s
      scrapeTimeout: ''

3. 配置Grafanana Dashboard

2.1. 在helm-pkg下新增dashboard.yaml

内容参考https://gitea.freeleaps.mathmast.com/freeleaps/freeleaps-ops/src/branch/master/freeleaps/helm-pkg/notification/templates/notification/dashboard.yaml

2.2. 修改value.{alpha|prod}.yaml

dashboard:
  enabled: true # alphah环境设置为false
  name: freeleaps-prod-metrics-dashboard
  title: Freeleaps Metrics Dashboard (PROD/ALPHA)
  metricsPrefix: freeleaps_metrics