日志收集系统PLG（Promtail+Loki+Grafana）先容及部署
一、简 介Loki是受Prometheus开导由Grafana Labs团队开源的程度可扩展，高度可用的多租户日志聚合系统。 开发语言: Google Go。它的设计具有很高的成本效益，并且易于操作。使用标签来作为索引，而不是对全文进行检索，也就是说，你通过这些标签既可以查询日志的内容也可以查询到监控的数据签，极大地低落了日志索引的存储。系统架构十分简朴，由以下3个部门组成 ：
Promtail 是署理，负责收集日志并将其发送给 loki 。
Loki 是主服务器，负责存储日志和处理查询 。
Grafana 用于 UI 展示。
只要在应用程序服务器上安装promtail来收集日志然后发送给Loki存储，就可以在Grafana UI界面通过添加Loki为数据源进行日志查询（假如Loki服务器性能不敷，可以部署多个Loki进行存储及查询）。作为一个日志系统不光只有查询分析日志的本领，还能对日志进行监控和报警。
https://img2024.cnblogs.com/blog/1235769/202404/1235769-20240410180427985-431905210.png
二、原理分析
promtail收集并将日志发送给loki的 Distributor 组件 Distributor会对接收到的日志流进行正确性校验，并将验证后的日志分批并行发送到Ingester Ingester 接受日志流并构建数据块，压缩后存放到所连接的存储后端 Querier 收到HTTP查询哀求，并将哀求发送至Ingester 用以获取内存数据 ，Ingester 收到哀求后返回符合条件的数据 ； 假如 Ingester 没有返回数据，Querier 会从后端存储加载数据并遍历去重实行查询 ，通过HTTP返回查询结果
三、对比
PLG与ELK比力优势ELK虽然功能丰富，但规模复杂，资源占用高，操作苦难，很多功能往往用不上，有点杀鸡用牛刀的感觉。 Loki 不对日志进行全文索引。通过存储压缩非结构化日志和索引元数据，Loki 操作起来会更简朴，更省成本。 通过使用与 Prometheus 相同的标签记录流对日志进行索引和分组，这使得日志的扩展和操作效率更高。 安装部署简朴快速，且受 Grafana 原生支持。
四、下载
curl -O -L "https://github.com/grafana/loki/releases/download/v1.5.0/loki-linux-amd64.zip"
curl -O -L "https://github.com/grafana/loki/releases/download/v1.5.0/promtail-linux-amd64.zip"
wget https://dl.grafana.com/enterprise/release/grafana-enterprise-8.3.3.linux-amd64.tar.gz五、 安装loki、grafana、promtail
安装loki
先将loki-linux-amd64.zip和grafana-enterprise-8.3.3.linux-amd64.tar.gz上传到日志服务器（10.60.134.55）；再将promtail-linux-amd64.zip上传到应用服务器（10.60.134.60、10.60.134.48、10.60.134.56）。
在日志服务器 上创建目次 /usr/local/logsCollect/loki用于安装loki，创建目次/usr/local/logsCollect/grafana用于安装grafana，如下：
https://img2024.cnblogs.com/blog/1235769/202404/1235769-20240411082208543-1368946498.png
mkdir/data/loki
mkdir/data/loki/{chunks,index}在日志服务器 上创建目次，用于文件存储和索引存储。
解压unzip loki-linux-amd64.zip，并设置config.yaml
auth_enabled: false
server:
http_listen_port: 8094
ingester:
lifecycler:
address: 10.60.134.55 #日志服务器ip，即本机地址
ring:
kvstore:
store: inmemory
replication_factor: 1
final_sleep: 0s
chunk_idle_period: 5m
chunk_retain_period: 30s
schema_config:
configs:
- from: 2024-04-01
store: boltdb
object_store: filesystem
schema: v11
index:
prefix: index_
period: 168h #每张表的时间范围7天
storage_config:
boltdb:
directory: /data/loki/index #索引文件存储地址
filesystem:
directory: /data/loki/chunks#块存储地址
limits_config:
enforce_metric_name: false
reject_old_samples: true
reject_old_samples_max_age: 168h
chunk_store_config:
# 最大可查询历史日期 7天,这个时间必须是schema_config中的period的倍数，否则报错。
max_look_back_period: 168h
# 表的保留期7天
table_manager:
retention_deletes_enabled: true
retention_period: 168h并创建启动脚本start.sh和关闭脚本shutdown.sh
#!/bin/bash
nohup ./loki-linux-amd64 -config.file=./config.yaml >./server.log 2>&1 &
echo "$!" > pid#!/bin/bash
kill -9 `cat pid`
echo "关闭成功!"启动loki下令：sh start.sh。如下日志表示启动成功。
https://img2024.cnblogs.com/blog/1235769/202404/1235769-20240411094444559-423518817.png
安装 grafana
起首解压grafana-enterprise-8.3.3.linux-amd64.tar.gz，创建启动脚本start.sh和关闭脚本shutdown.sh。注意conf/defaults.ini为grafana的设置文件，其中http_port为浏览器访问端口。
https://img2024.cnblogs.com/blog/1235769/202404/1235769-20240411083442319-381381805.png
https://img2024.cnblogs.com/blog/1235769/202404/1235769-20240411083621206-1935131442.png
启动grafana下令：sh start.sh。如下日志表示启动成功。
https://img2024.cnblogs.com/blog/1235769/202404/1235769-20240411083825374-850935683.png
在应用服务器安装 promtail
在应用服务器 上创建目次 /usr/local/logsCollect/promtail用于安装promtail。
解压promtail-linux-amd64.zip，并设置promtail.yaml
server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: ./positions.yaml
clients:
- url: http://10.60.134.55:8094/loki/api/v1/push #日志服务器loki地址和端口
scrape_configs:
#ucenter1
- job_name: zcbackend-172.29.21.22-1
static_configs:
- targets:
- 10.60.134.60
- labels:
job: zcbachend-172.29.21.22-1
host: 10.60.134.60
__path__: /jiuqi/zichan/zichanyitihuazhenghexiangmu-8084/backend/server.log#本机日志路径并创建启动脚本start.sh和关闭脚本shutdown.sh
https://img2024.cnblogs.com/blog/1235769/202404/1235769-20240411084423846-1944405605.png
启动promtail下令：sh start.sh。如下日志表示启动成功。
https://img2024.cnblogs.com/blog/1235769/202404/1235769-20240411084653218-1163271220.png
六、 测试
打开grafana。默认密码admin/admin。
https://img2024.cnblogs.com/blog/1235769/202404/1235769-20240411091033622-926202864.png
添加数据源
https://img2024.cnblogs.com/blog/1235769/202404/1235769-20240411094629292-870940695.png
测试连接
https://img2024.cnblogs.com/blog/1235769/202404/1235769-20240411094804694-880276184.png
查询日志
https://img2024.cnblogs.com/blog/1235769/202404/1235769-20240411095104115-337192498.png
也能通过导入的方式添加面板，json文件如下：
https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gifhttps://images.cnblogs.com/OutliningIndicators/ExpandedBlockStart.gif{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": {
"type": "datasource",
"uid": "grafana"
},
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"target": {
"limit": 100,
"matchAny": false,
"tags": [],
"type": "dashboard"
},
"type": "dashboard"
}
]
},
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"id": 3,
"iteration": 1671275851848,
"links": [],
"liveNow": false,
"panels": [
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": {
"type": "loki",
"uid": "${ENV}"
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 5,
"w": 24,
"x": 0,
"y": 0
},
"hiddenSeries": false,
"id": 4,
"legend": {
"alignAsTable": true,
"avg": false,
"current": false,
"max": false,
"min": false,
"rightSide": true,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "8.5.10",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"datasource": {
"type": "loki",
"uid": "hJ7iGN-Sz"
},
"expr": "sum (count_over_time({filename=~\"$log\",host=~\"$host\"} )) by (host,filename)",
"hide": false,
"legendFormat": "{{host}}=={{filename}}",
"refId": "B"
}
],
"thresholds": [],
"timeRegions": [],
"title": "日志量统计",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"mode": "time",
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:319",
"format": "short",
"logBase": 1,
"show": true
},
{
"$$hashKey": "object:320",
"format": "short",
"logBase": 1,
"show": true
}
],
"yaxis": {
"align": false
}
},
{
"datasource": {
"type": "loki",
"uid": "hJ7iGN-Sz"
},
"description": "",
"gridPos": {
"h": 21,
"w": 24,
"x": 0,
"y": 5
},
"id": 2,
"options": {
"dedupStrategy": "exact",
"enableLogDetails": false,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": true,
"sortOrder": "Descending",
"wrapLogMessage": true
},
"pluginVersion": "7.4.3",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "hJ7iGN-Sz"
},
"expr": "{filename=~\"$log\",host=~\"$host\"}|~ \"(?i)$log_level\"",
"hide": false,
"maxLines": 500,
"refId": "C"
}
],
"title": "日志",
"transparent": true,
"type": "logs"
}
],
"refresh": "",
"schemaVersion": 36,
"style": "dark",
"tags": [
"PLG"
],
"templating": {
"list": [
{
"current": {
"selected": false,
"text": "Loki",
"value": "Loki"
},
"hide": 0,
"includeAll": false,
"label": "选择环境",
"multi": false,
"name": "ENV",
"options": [],
"query": "loki",
"queryValue": "",
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"type": "datasource"
},
{
"current": {
"selected": true,
"text": "资产server",
"value": "资产server"
},
"datasource": {
"type": "loki",
"uid": "hJ7iGN-Sz"
},
"definition": "label_values({},filename)",
"hide": 0,
"includeAll": false,
"label": "日志",
"multi": false,
"name": "log",
"options": [
{
"selected": true,
"text": "资产server-8084",
"value": "/jiuqi/zichan/zichanyitihuazhenghexiangmu-8084/backend/server.log"
},
{
"selected": true,
"text": "资产server-8085",
"value": "/jiuqi/zichan/zichanyitihuazhenghexiangmu-8085/backend/server.log"
},
{
"selected": true,
"text": "资产server-8086",
"value": "/jiuqi/zichan/zichanyitihuazhenghexiangmu-8086/backend/server.log"
},
{
"selected": true,
"text": "公车server-8087",
"value": "/jiuqi/gongche/yitihuagongchehouduan-8087/backend/server.log"
},
{
"selected": true,
"text": "公车server-8088",
"value": "/jiuqi/gongche/yitihuagongchehouduan-8088/backend/server.log"
}
],
"query": "label_values({},filename)",
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"sort": 1,
"type": "query"
},
{
"current": {
"selected": true,
"text": "172.29.21.22",
"value": "10.60.134.60"
},
"datasource": {
"type": "loki",
"uid": "hJ7iGN-Sz"
},
"definition": "label_values({},host)",
"hide": 0,
"includeAll": false,
"label": "主机IP",
"multi": false,
"name": "host",
"options": [
{
"selected": true,
"text": "资产-172.29.21.22",
"value": "10.60.134.60"
},
{
"selected": true,
"text": "资产-172.29.21.164",
"value": "10.60.134.48"
},
{
"selected": true,
"text": "公车-172.29.21.36",
"value": "10.60.134.56"
}
],
"query": "label_values({},host)",
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"sort": 1,
"type": "query"
},
{
"allValue": "(^\\\\S|^\\\\s)",
"current": {
"selected": false,
"text": "All",
"value": "$__all"
},
"description": "可以直接输入搜索的关键字进行过滤",
"hide": 0,
"includeAll": true,
"label": "关键字过滤",
"multi": false,
"name": "log_level",
"options": [
{
"selected": true,
"text": "All",
"value": "$__all"
},
{
"selected": false,
"text": "INFO",
"value": "INFO"
},
{
"selected": false,
"text": "DEBUG",
"value": "DEBUG"
},
{
"selected": false,
"text": "WARN",
"value": "WARN"
},
{
"selected": false,
"text": "ERROR",
"value": "ERROR"
},
{
"selected": false,
"text": "UNKNOWN",
"value": "UNKNOWN"
},
{
"selected": false,
"text": "直接输入关键字搜索",
"value": "直接输入关键字搜索"
}
],
"query": "INFO,DEBUG,WARN,ERROR,UNKNOWN,直接输入关键字搜索",
"queryValue": "",
"skipUrlSync": false,
"type": "custom"
}
]
},
"time": {
"from": "now-1h",
"to": "now"
},
"timepicker": {},
"timezone": "",
"title": "Loki日志收集",
"uid": "NlV_8QD7k",
"version": 19,
"weekStart": ""
}logcollect.jsonhttps://img2024.cnblogs.com/blog/1235769/202404/1235769-20240411123558934-728186213.png
免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！更多信息从访问主页：qidao123.com:ToB企服之家，中国第一个企服评测及商务社交产业平台。
页:
[1]