# ding-talk

**Repository Path**: yunwe/ding-talk

## Basic Information

- **Project Name**: ding-talk
- **Description**: 实现Prometheus环境下的自定监控告警推送，支持【钉钉|飞书|邮箱】
- **Primary Language**: Python
- **License**: MulanPSL-2.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 3
- **Forks**: 0
- **Created**: 2022-05-16
- **Last Updated**: 2025-05-23

## Categories & Tags

**Categories**: Uncategorized

**Tags**: 监控, 钉钉, 分析, Python, prometheus

## README

## ding-talk

该项目主要实现接收alertmanager推送的json，再推送至指定平台，在此基础上新增了几点功能：

1. 历史告警存储，使用es实现。

2. 告警字段过滤。默认的字段太多，几条就能刷屏，通过在配置文件拦截字段。

3. 支持字段中文自定义，可以根据个人需要修改通知字段的中文。

   ![image-20230421090712145](https://huang-images.oss-cn-beijing.aliyuncs.com/img/image-20230421090712145.png)

   Title调整为黄色需要修改prometheus rule标签，比如：

   ![image-20230421090956075](https://huang-images.oss-cn-beijing.aliyuncs.com/img/image-20230421090956075.png)

   如果有 `severity: warning` 标签则字体为黄色，其他value或者不设置则为红色，恢复统一为绿色。

4. 推送目的支持 钉钉/飞书/邮件，需要在配置文件修改。

5. 周报推送，每周一上午9点推送上一周告警情况，需配置ES。

## 安装教程

### 自行构建

```shell
git clone -b master https://gitee.com/yunwe/ding-talk.git
docker build -t harbor.com/ding-talk:1.4.6 .
```

部分环境可能出现以下报错：

> ```
> Traceback (most recent call last):
>   File "/ding-talk/app.py", line 7, in <module>
>     server.serve_forever()
>   File "/usr/local/lib/python3.12/site-packages/gevent/baseserver.py", line 398, in serve_forever
>     self.start()
>   File "/usr/local/lib/python3.12/site-packages/gevent/baseserver.py", line 336, in start
>     self.init_socket()
>   File "/usr/local/lib/python3.12/site-packages/gevent/pywsgi.py", line 1669, in init_socket
>     self.update_environ()
>   File "/usr/local/lib/python3.12/site-packages/gevent/pywsgi.py", line 1681, in update_environ
>     name = socket.getfqdn(address[0])
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "/usr/local/lib/python3.12/site-packages/gevent/_socketcommon.py", line 286, in getfqdn
>     hostname, aliases, _ = gethostbyaddr(name)
>                            ^^^^^^^^^^^^^^^^^^^
>   File "/usr/local/lib/python3.12/site-packages/gevent/_socketcommon.py", line 252, in gethostbyaddr
>     return get_hub().resolver.gethostbyaddr(ip_address)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "/usr/local/lib/python3.12/site-packages/gevent/resolver/thread.py", line 66, in gethostbyaddr
>     return self.pool.apply(_socket.gethostbyaddr, args, kwargs)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "/usr/local/lib/python3.12/site-packages/gevent/pool.py", line 161, in apply
>     return self.spawn(func, *args, **kwds).get()
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "/usr/local/lib/python3.12/site-packages/gevent/threadpool.py", line 546, in spawn
>     self.adjust()
>   File "/usr/local/lib/python3.12/site-packages/gevent/threadpool.py", line 492, in adjust
>     self._adjust_step()
>   File "/usr/local/lib/python3.12/site-packages/gevent/threadpool.py", line 472, in _adjust_step
>     self._add_thread()
>   File "/usr/local/lib/python3.12/site-packages/gevent/threadpool.py", line 499, in _add_thread
>     self._WorkerGreenlet(self)
>   File "/usr/local/lib/python3.12/site-packages/gevent/threadpool.py", line 106, in __init__
>     start_new_thread(self._begin, ())
> RuntimeError: can't start new thread
> ```

可以使用另一个分支解决

```shell
git clone -b python38 https://gitee.com/yunwe/ding-talk.git
cd ding-talk
docker build -t harbor.com/ding-talk:python38-1.4.6 .
```

### 直接使用镜像

二选一

#### k8s部署

复制other目录下的ding-talk.yaml文件

```
kubectl -f https://gitee.com/yunwe/ding-talk/raw/master/other/ding-talk.yaml
```

#### docker部署
挂载配置文件
```shell
docker run -d --restart=always --name ding-talk \
-p 5000:5000 \
-v /mnt/ding-talk/config.ini:/ding-talk/config.ini \
registry.cn-hangzhou.aliyuncs.com/huang-image/ding-talk:1.4.9
```
或者使用docker-compose

```yaml
version: '3'
services:
  ding-talk:
    container_name: dingtalk
    image: registry.cn-hangzhou.aliyuncs.com/huang-image/ding-talk:1.4.9
    ports:
      - 5000:5000
    volumes:
      - /data/prometheus/dingtalk/config.ini:/ding-talk/config.ini
```
```shell
docker-compose -f  docker-compose.yaml up -d
```

`config.ini`文件可从项目根目录下载，根据自己需求修改！

## 关于配置文件

### 周报功能配置

如果开启周报，需要在对应推送的平台配置需要推送的群组，此处以钉钉为例：

![image-20230426103128837](https://huang-images.oss-cn-beijing.aliyuncs.com/img/image-20230426103128837.png)

下面是推送示例

![image-20230421085903172](https://huang-images.oss-cn-beijing.aliyuncs.com/img/image-20230421085903172.png)

如果不需要开启，则在此处配置disable（默认enable)

![image-20230426103250863](https://huang-images.oss-cn-beijing.aliyuncs.com/img/image-20230426103250863.png)

### 推送平台配置

需在`config.ini` 先配置推送方式，[settings] --> push_dest，默认配置为钉钉

#### 钉钉

1、钉钉告警支持给多个群发送告警，在`config.ini`定义n个组，格式是：组名（随意）=钉钉自定义机器人Token,手机号1,手机号2，用英文逗号分隔

![image-20230314150450085](https://huang-images.oss-cn-beijing.aliyuncs.com/img/image-20230314150450085.png)

在alertmanager.yml的receivers配置url，如http://192.168.1.2:5000/webhook/send/group1, 请求地址尾部的group1即上面`config.ini`定义的组名，**必须一致**。

根据需要可以写多个group，例如下图的 http://webhook-dingtalk:5000/webhook_mention_all/send 可以改成 http://webhook-dingtalk:5000/webhook/send/group2，即推送到第二个群

![image-20220902144340785](https://huang-images.oss-cn-beijing.aliyuncs.com/img/image-20220902144340785.png)

#### 邮件推送
发件服务器password 需要进行加密，加密功能使用方法查看 【历史告警保存】↓

![image-20230206111219178](https://huang-images.oss-cn-beijing.aliyuncs.com/img/image-20230206111219178.png)

#### 飞书推送
跟钉钉告警配置类似，支持多个群组

![image-20230314150546706](https://huang-images.oss-cn-beijing.aliyuncs.com/img/image-20230314150546706.png)



### 历史告警保存

> 注意：历史告警功能依赖于ElasticSearch，ES版本必须≥7.x ！

1、开启历史告警保存功能需要先在`config.ini` [settings] 中把字段 db 设置为 disable(默认)

2、ding-talk部署后进行es账号、密码加密

> 如果没有配置账号密码，不需要填写！

  ```shell
  # 进入容器执行命令，即可获取加密后的账号密码
  curl -H "Content-Type:application/json" -H "Data_Type:msg" -X POST -d '{"username": "elastic", "password": "my_password"}' http://127.0.0.1:5000/encrypt/
  ```

在`config.ini`填写加密后的es账号密码

![image-20220902145241529](https://huang-images.oss-cn-beijing.aliyuncs.com/img/image-20220902145241529.png)

3、将config.ini里的 `db=disable` 改为 enable，重启服务

### 推送字段拦截

支持自定义字段拦截，`config.ini` 在以下位置写入，将不会推送以下字段

![image-20220902145615806](https://huang-images.oss-cn-beijing.aliyuncs.com/img/image-20220902145615806.png)

### 中文自定义配置

支持自定义转义，在`config.ini`以下位置写入

![image-20230406124244638](https://huang-images.oss-cn-beijing.aliyuncs.com/img/image-20230406124244638.png)

## 懒人专用

> 如果嫌配置麻烦可直接使用自带配置，默认钉钉，修改下ding-talk配置中的token为你自己群的即可。
>
> alertmanager可直接复制：

```yaml
global:
  resolve_timeout: 5m
route:
  # 这里的标签列表是接收到报警信息后的重新分组标签，例如，接收到的报警信息里面有许多具有 cluster=A 和 alertname=LatncyHigh 这样的标签的报警信息将会批量被聚合到一个分组里面
  group_by: ["alertname"]
  # 当一个新的报警分组被创建后，需要等待至少group_wait时间来初始化通知，这种方式可以确保您能有足够的时间为同一分组来获取多个警报，然后一起触发这个报警信息。
  group_wait: 10s
  # 当第一个报警发送后，等待'group_interval'时间来发送新的一组报警信息。
  group_interval: 1m
  # 如果一个报警信息已经发送成功了，等待'repeat_interval'时间来重新发送他们
  repeat_interval: 8h
  # 默认的receiver如果一个报警没有被一个route匹配，则发送给默认的接收器
  receiver: "dingding.webhook1"
  # 上面所有的属性都由所有子路由继承，并且可以在每个子路由上进行覆盖。
  routes:
    - receiver: dingding.webhook2
      match_re:
        project: openData
      group_wait: 10s
      group_interval: 1m
      repeat_interval: 8h
      continue: false
    - receiver: dingding.webhook1
      match_re:
        severity: info|warning|error
      group_wait: 10s
      group_interval: 1m
      repeat_interval: 8h
receivers:
  - name: "dingding.webhook1"
    webhook_configs:
      - url: "http://webhook-dingtalk:5000/webhook/send/group1" #改成service name
        send_resolved: true
  - name: "dingding.webhook2"
    webhook_configs:
      - url: "http://webhook-dingtalk:5000/webhook/send/group2"
        send_resolved: true
```