ELK

elk介绍

运维职员需要对系统和业务日志举行精准把控，便于分析系统和业务状态。日志分布在差别的服务器上，传统的使用传统的方法依次登录每台服务器查看日志，既繁琐又效率低下。所以我们需要集中化的日志管理工具将位于差别服务器上的日志网络到一起, 然后举行分析,展示。

前期准备

1、修改主机名

1. [root@node1 ~]# hostnamectl hostname vm1.example.com
2. [root@node1 ~]# bash
4. [root@node1 ~]# hostnamectl hostname vm2.example.com
5. [root@node1 ~]# bash
6. [root@vm2 ~]#
8. [root@node1 ~]# hostnamectl hostname v3.example.com
9. [root@node1 ~]# bash
10. [root@v3 ~]#

复制代码

2、配置/ect/hosts

1. [root@vm1 ~]# vim /etc/hosts
2. [root@vm1 ~]# cat /etc/hosts
3. 127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
4. ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
5. 192.168.100.30        vm1.example.com        kibana
6. 192.168.100.80        vm2.example.com        elasticsearch
7. 192.168.100.90        vm3.example.com        logstash
8. [root@vm1 ~]# scp /etc/hosts root@192.168.100.80:/etc/hosts
9. The authenticity of host '192.168.100.80 (192.168.100.80)' can't be established.
10. ED25519 key fingerprint is SHA256:Ci2qzv2Hvt2jld5Q8LBu35qRbAnKzC3EaGZRV6Htsw0.
11. This key is not known by any other names
12. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
13. Warning: Permanently added '192.168.100.80' (ED25519) to the list of known hosts.
14. root@192.168.100.80's password:
15. hosts                                                    100%  281   249.2KB/s   00:00   
16. [root@vm1 ~]# scp /etc/hosts root@192.168.100.90:/etc/hosts
17. The authenticity of host '192.168.100.90 (192.168.100.90)' can't be established.
18. ED25519 key fingerprint is SHA256:Ci2qzv2Hvt2jld5Q8LBu35qRbAnKzC3EaGZRV6Htsw0.
19. This host key is known by the following other names/addresses:
20.     ~/.ssh/known_hosts:1: 192.168.100.80
21. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
22. Warning: Permanently added '192.168.100.90' (ED25519) to the list of known hosts.
23. root@192.168.100.90's password:
24. hosts                                                    100%  281   681.0KB/s   00:00   
25. [root@vm1 ~]#

复制代码

3、检查防火墙selinux是否关闭

1. [root@vm1 ~]# systemctl status firewalld
2. ○ firewalld.service - firewalld - dynamic firewall daemon
3.      Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; preset: enabled)
4.      Active: inactive (dead)
5.        Docs: man:firewalld(1)   
6. [root@vm1 ~]# getenforce
7. Disabled
8. [root@vm1 ~]#
10. [root@vm2 ~]# yum -y install lrzsz tar net-tools wget

复制代码

4、时钟同步

1. [root@vm1 ~]# yum -y install chrony
2. [root@vm1 ~]# systemctl restart chronyd
3. [root@vm1 ~]# systemctl enable chronyd
4. [root@vm1 ~]# timedatectl
5.                Local time: Mon 2024-08-19 16:02:19 CST
6.            Universal time: Mon 2024-08-19 08:02:19 UTC
7.                  RTC time: Mon 2024-08-19 08:02:19
8.                 Time zone: Asia/Shanghai (CST, +0800)
9. System clock synchronized: yes
10.               NTP service: active
11.           RTC in local TZ: no
12. [root@vm1 ~]# hwclock -w

复制代码

elasticsearch摆设

介绍

Elasticsearch(简称ES)是一个开源的分布式搜刮引擎,Elasticsearch还是一个分布式文档数据库。所以它
提供了大量数据的存储功能,快速的搜刮与分析功能。
1、安装JAVA包

1. [root@vm1 ~]# yum -y install java-1.8.0*
2. [root@vm2 ~]# yum -y install java-1.8.0*
3. [root@vm3 ~]# yum -y install java-1.8.0*
5. [root@vm1 ~]# java -version
6. openjdk version "1.8.0_422"
7. OpenJDK Runtime Environment (build 1.8.0_422-b05)
8. OpenJDK 64-Bit Server VM (build 25.422-b05, mixed mode)

复制代码

2、解压安装包，修改配置文件

1. [root@vm2 ~]# ls
2. anaconda-ks.cfg  -e  elasticsearch-6.5.2.rpm  -i.bak
3. [root@vm2 ~]# rpm -ivh elasticsearch-6.5.2.rpm
5. [root@vm2 ~]#
6. [root@vm2 ~]# cd /etc/elasticsearch/
7. [root@vm2 elasticsearch]# ls
8. elasticsearch.keystore  jvm.options        role_mapping.yml  users
9. elasticsearch.yml       log4j2.properties  roles.yml         users_roles
10. [root@vm2 elasticsearch]# vim elasticsearch.yml
11. cluster.name: elk-cluster
12. path.data: /var/lib/elasticsearch
13. path.logs: /var/log/elasticsearch
14. network.host: 0.0.0.0
15. http.port: 9200
16. [root@vm2 elasticsearch]# systemctl restart elasticsearch
17. [root@vm2 elasticsearch]# systemctl enable elasticsearch
18. Created symlink /etc/systemd/system/multi-user.target.wants/elasticsearch.service → /usr/lib/systemd/system/elasticsearch.service.
19. [root@vm2 elasticsearch]# ss -anlt
20. State    Recv-Q   Send-Q       Local Address:Port       Peer Address:Port   Process   
21. LISTEN   0        128                0.0.0.0:22              0.0.0.0:*               
22. LISTEN   0        4096                     *:9300                  *:*               
23. LISTEN   0        128                   [::]:22                 [::]:*               
24. LISTEN   0        4096                     *:9200                  *:*   
25. [root@vm2 elasticsearch]# curl http://192.168.100.80:9200/_cluster/health?pretty
26. {
27.   "cluster_name" : "elk-cluster",
28.   "status" : "green",
29.   "timed_out" : false,
30.   "number_of_nodes" : 1,
31.   "number_of_data_nodes" : 1,
32.   "active_primary_shards" : 0,
33.   "active_shards" : 0,
34.   "relocating_shards" : 0,
35.   "initializing_shards" : 0,
36.   "unassigned_shards" : 0,
37.   "delayed_unassigned_shards" : 0,
38.   "number_of_pending_tasks" : 0,
39.   "number_of_in_flight_fetch" : 0,
40.   "task_max_waiting_in_queue_millis" : 0,
41.   "active_shards_percent_as_number" : 100.0
42. }
43. [root@vm2 elasticsearch]#

复制代码

elasticsearch集群摆设

1. vm1：
2. [root@vm1 ~]# ls
3. anaconda-ks.cfg  -e  elasticsearch-6.5.2.rpm  -i.bak
4. [root@vm1 ~]# rpm -ivh elasticsearch-6.5.2.rpm
5. warning: elasticsearch-6.5.2.rpm: Header V4 RSA/SHA512 Signature, key ID d88e42b4: NOKEY
6. Verifying...                          ################################# [100%]
7. Preparing...                          ################################# [100%]
8. Creating elasticsearch group... OK
9. Creating elasticsearch user... OK
10. Updating / installing...
11.    1:elasticsearch-0:6.5.2-1          ################################# [100%]
12. ### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
13. sudo systemctl daemon-reload
14. sudo systemctl enable elasticsearch.service
15. ### You can start elasticsearch service by executing
16. sudo systemctl start elasticsearch.service
17. Created elasticsearch keystore in /etc/elasticsearch
18. /usr/lib/tmpfiles.d/elasticsearch.conf:1: Line references path below legacy directory /var/run/, updating /var/run/elasticsearch → /run/elasticsearch; please update the tmpfiles.d/ drop-in file accordingly.
19. ------------------------------------------------------------------------
20. [root@vm1 ~]# vim /etc/elasticsearch/elasticsearch.yml
21. ------------------------------------------------------------------------
22. cluster.name: elk-cluster
23. node.name: 192.168.100.30                本机IP或主机名
24. node.master: false                                 指定不为master节点
25. path.data: /var/lib/elasticsearch
26. path.logs: /var/log/elasticsearch
27. network.host: 0.0.0.0
28. http.port: 9200
29. discovery.zen.ping.unicast.hosts: ["192.168.100.30", "192.168.100.80"] 集群所有节点IP
30. ------------------------------------------------------------------------
31. [root@vm1 ~]# systemctl restart elasticsearch
32. [root@vm1 ~]# systemctl enable elasticsearch
33. Created symlink /etc/systemd/system/multi-user.target.wants/elasticsearch.service → /usr/lib/systemd/system/elasticsearch.service.
34. [root@vm1 ~]#
36. vm2：
37. [root@vm2 elasticsearch]# vim elasticsearch.yml
38. -------------------------------------------------------------------
39. cluster.name: elk-cluster
40. node.name: 192.168.100.80                                 本机IP或主机名
41. node.master: true 指定为master节点
42. path.data: /var/lib/elasticsearch
43. path.logs: /var/log/elasticsearch
44. network.host: 0.0.0.0
45. http.port: 9200
46. discovery.zen.ping.unicast.hosts: ["192.168.100.30", "192.168.100.80"] 集群所有节点IP
47. -----------------------------------------------------------------
48. [root@vm2 elasticsearch]# systemctl restart elasticsearch

复制代码

elaticsearch基础API利用

1、RestFul API 格式

1. RestFul API 格式:curl -X<verb> ‘<protocol>://<host>:<port>/<path>?<query_string>’-d ‘<body>’

复制代码

参数描述verbHTTP方法，比如GET、POST、PUT、HEAD、DELETEhostES集群中的任意节点主机名portES HTTP服务端口，默认9200path索引路径query_string可选的查询请求参数。比方?pretty参数将返回JSON格式数据-d里面放一个GET的JSON格式请求主体body本身写的 JSON格式的请求主体 2、查看节点信息

1. [root@vm2 elasticsearch]# curl http://192.168.100.80:9200/_cat/nodes?v
2. ip             heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
3. 192.168.100.30           13          95   0    0.00    0.00     0.00 di        -      192.168.100.30
4. 192.168.100.80           11          96   0    0.00    0.00     0.00 mdi       *      192.168.100.80

复制代码

3、查看索引信息和新增索引

1. [root@vm2 elasticsearch]# curl http://192.168.100.80:9200/_cat/indices?v
2. health status index uuid pri rep docs.count docs.deleted store.size pri.store.size  //没有索引
3. [root@vm2 elasticsearch]# curl -X PUT http://192.168.100.80:9200/nginx_access_log
4. {"acknowledged":true,"shards_acknowledged":true,"index":"nginx_access_log"}[root@vm2 elasticsearch]# curl -X PUT http:/
5. [root@vm2 elasticsearch]# curl http://192.168.100.80:9200/_cat/indices?v
6. health status index            uuid                   pri rep docs.count docs.deleted store.size pri.store.size
7. green  open   nginx_access_log PGrIVaIERO2IizDOKL9b9A   5   1          0            0      2.2kb          1.1kb

复制代码

4、删除索引

5、导入数据

1. [root@vm2 ~]# ls
2. accounts.json  anaconda-ks.cfg  -e  elasticsearch-6.5.2.rpm  -i.bak
3. [root@vm2 ~]#  curl -H "Content-Type: application/json" -XPOST "192.168.100.80:9200/bank/_doc/_bulk?pretty&refresh" --data-binary "@accounts.json"
4. [root@vm2 ~]# curl "192.168.100.80:9200/_cat/indices?v"
5. health status index            uuid                   pri rep docs.count docs.deleted store.size pri.store.size
6. green  open   nginx_access_log PGrIVaIERO2IizDOKL9b9A   5   1          0            0      2.5kb          1.2kb
7. green  open   bank             RZH-6IBNSOmQpduyCHSRKA   5   1       1000            0    965.6kb        482.5kb

复制代码

6、查询bank索引的数据(使用查询字符串举行查询)

1. [root@vm2 ~]# curl -X GET "192.168.100.80:9200/bank/_search?q=*&sort=account_number:asc&pretty"
2. 、
3. 默认结果为10条
4. _search 属于一类API，用于执行查询操作
5. q=* ES批量索引中的所有文档
6. sort=account_number:asc 表示根据account_number按升序对结果排序
7. pretty调整显示格式

复制代码

7、查询bank索引的数据 (使用json格式举行查询)

1. [root@vm2 ~]# curl -X GET "192.168.100.80:9200/bank/_search" -H 'content-Type:application/json' -d'
2. > {
3. > “query”: { "match_all": {} },
4. > "sort": [
5. > { "account_number": "asc"}
6. > ]
7. > }
8. > '
9. {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unexpected character ('“' (code 8220 / 0x201c)): was expecting double-quote to start field name\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@6738f56b; line: 3, column: 4]"}],"type":"json_parse_exception","reason":"Unexpected character ('“' (code 8220 / 0x201c)): was expecting double-quote to start field name\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@6738f56b; line: 3, column: 4]"},"status":500}[root@vm2 ~]#

复制代码

8、match_all 查询

匹配所有文档。默认查询

1. [root@vm2 ~]# curl -X GET "192.168.100.80:9200/bank/_search?pretty" -H "content-Type:application/json" d'
2. > {
3. > "query": { "match_all": {} }
4. > }
5. > '
6. # query告诉我们查询什么
7. # match_all是我们查询的类型
8. # match_all查询仅仅在指定的索引的所有文件进行搜索

复制代码

9、from，size 查询

除了query参数外，还可以传递其他参数影响查询效果，比如前面提到的sort，接下来使用的size

1. [root@vm2 ~]# curl -X GET "192.168.100.80:9200/bank/_search?pretty" -H 'content-Type:application/json' -d'
2. {
3. "query":{ "match_all": {} },
4. "size":1
5. }
6. '

复制代码

10、指定位置与查询条数

1. [root@vm2 ~]# curl -X GET "192.168.100.80:9200/bank/_search?pretty" -H 'content-Type:application/json' -d'
2. > {
3. > "query": { "match_all": {} }
4. > "from": 0
5. > "size": 2
6. > }
7. > '

复制代码

11、匹配查询字段

返回_source字段中的片段字段

1. [root@vm2 ~]# curl -X GET "192.168.100.80:9200/bank/_search?pretty" -H 'content-Type:application/json' -d'
2. {
3. "query": { "match_all": {} },
4. > "_source": ["account_number","balance"]
5. > }
6. > '

复制代码

12、match 查询

1. [root@vm2 ~]# curl -X GET "192.168.100.80:9200/bank/_search?pretty" -H 'content-Type:application/json' -d'
2. {
3. "query": { "match": {"account_number": 20} }
4. > }
5. > '

复制代码

13、基本搜刮查询，针对特定字段或字段集合举行搜刮

1. [root@vm2 ~]# curl -X GET "192.168.100.80:9200/bank/_search?pretty" -H >'content-Type:application/json' -d'
2. >{
3. >"query": { "match": {"account_number": "mill"} }
4. >}
5. >'

复制代码

14、bool 查询

bool must 查询的字段必须同时存在
查询包含mill和lane的所有账户

1. [root@vm2 ~]# curl -X GET "192.168.100.80:9200/bank/_search?pretty" -H 'content-Type:application/json' -d'
2. > {
3. > "query": {
4. > "bool": {
5. > "must": [
6. > { "match": {"address": "mill"} },
7. > { "match": {"address": "lane"} }
8. > ]
9. > }
10. > }
11. > }
12. > '

复制代码

15、range 查询

指定区间内的数字大概时间
利用符：gt大于，gte大于等于，lt小于，lte小于等于

1. [root@vm2 ~]# curl -X GET "10.1.1.12:9200/bank/_search?pretty" -H 'Content-Type:
2. application/json' -d'
3. >{
4. >"query": {
5. >"bool": {
6. >"must": { "match_all": {} },
7. >"filter": {
8. >"range": {
9. >"balance": {
10. >"gte": 20000,
11. >"lte": 30000
12. >}
13. >}
14. >}
15. >}
16. >}
17. >}
18. >'

复制代码

elasticsearch-head

elasticsearch-head是集群管理、数据可视化、增编削查、查询语句可视化工具。从ES5版本后安装方式
和ES2以上的版本有很大的差别,在ES2中可以直接在bin目录下实行plugin install xxxx 来举行安装,但是
在ES5中这种安装方式变了,要想在ES5中安装Elasticsearch Head必须要安装NodeJs,然后通过NodeJS来
启动Head。
安装nodejs

1. [root@vm1 ~]# ls
2. anaconda-ks.cfg  -e  elasticsearch-6.5.2.rpm  -i.bak  node-v10.24.1-linux-x64.tar.xz
3. [root@vm1 ~]# tar xf node-v10.24.1-linux-x64.tar.xz -C /usr/local/
4. [root@vm1 ~]# ls /usr/local/
5. bin  etc  games  include  lib  lib64  libexec  node-v10.24.1-linux-x64  sbin  share  src
6. [root@vm1 ~]# mv /usr/local/node-v10.24.1-linux-x64/  /usr/local/nodejs
7. [root@vm1 ~]# ls /usr/local/
8. bin  etc  games  include  lib  lib64  libexec  nodejs  sbin  share  src
9. [root@vm1 ~]# ln -s /usr/local/nodejs/bin/npm /bin/npm
10. [root@vm1 ~]# ln -s /usr/local/nodejs/bin/node /bin/node
11. [root@vm1 ~]#

复制代码

安装es-head

1. [root@vm2 bin]# yum -y install unzip
2. [root@vm2 ~]# ls
3. accounts.json    -e                       elasticsearch-head-master.zip  node-v10.24.1-linux-x64.tar.xz
4. anaconda-ks.cfg  elasticsearch-6.5.2.rpm  -i.bak
5. [root@vm2 ~]# unzip elasticsearch-head-master.zip
6. [root@vm2 ~]# cd elasticsearch-head-master/
7. [root@vm2 elasticsearch-head-master]# npm install -g grunt-cli --registry=http://registry.npm.taobao.org
8. ##  --registry=http://registry.npm.taobao.org 网络不好就添加，网络好就不需要添加
11. [root@vm2 elasticsearch-head-master]# npm install -g grunt-cli --registry=http://registry.npm.taobao.org
13. added 56 packages in 5s
15. 5 packages are looking for funding
16.   run `npm fund` for details
17. npm notice
18. npm notice New major version of npm available! 8.19.4 -> 10.8.2
19. npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.2
20. npm notice Run npm install -g npm@10.8.2 to update!
21. npm notice
22. [root@vm2 elasticsearch-head-master]# npm install --registry=http://registry.npm.taobao.org

复制代码

1. 解决报错
2. [root@vm2 elasticsearch-head-master]# npm install phantomjs-prebuilt@2.1.16 --ignore-script
3. npm WARN EBADENGINE Unsupported engine {
4. npm WARN EBADENGINE   package: 'karma@1.3.0',
5. npm WARN EBADENGINE   required: { node: '0.10 || 0.12 || 4 || 5 || 6' },
6. npm WARN EBADENGINE   current: { node: 'v16.20.2', npm: '8.19.4' }
7. npm WARN EBADENGINE }
8. npm WARN EBADENGINE Unsupported engine {
9. npm WARN EBADENGINE   package: 'http2@3.3.7',
10. npm WARN EBADENGINE   required: { node: '>=0.12.0 <9.0.0' },
11. npm WARN EBADENGINE   current: { node: 'v16.20.2', npm: '8.19.4' }
12. npm WARN EBADENGINE }
13. npm WARN deprecated inflight@1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
14. npm WARN deprecated source-map-url@0.4.1: See https://github.com/lydell/source-map-url#deprecated
15. npm WARN deprecated rimraf@2.7.1: Rimraf versions prior to v4 are no longer supported
16. npm WARN deprecated rimraf@2.7.1: Rimraf versions prior to v4 are no longer supported
17. npm WARN deprecated rimraf@2.7.1: Rimraf versions prior to v4 are no longer supported
18. npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
19. npm WARN deprecated har-validator@5.1.5: this library is no longer supported
20. npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
21. npm WARN deprecated json3@3.3.2: Please use the native JSON object instead of JSON 3
22. npm WARN deprecated rimraf@2.2.8: Rimraf versions prior to v4 are no longer supported
23. npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
24. npm WARN deprecated glob@5.0.15: Glob versions prior to v9 are no longer supported
25. npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
26. npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
27. npm WARN deprecated source-map-resolve@0.5.3: See https://github.com/lydell/source-map-resolve#deprecated
28. npm WARN deprecated chokidar@1.7.0: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
29. npm WARN deprecated glob@7.1.7: Glob versions prior to v9 are no longer supported
30. npm WARN deprecated glob@7.0.6: Glob versions prior to v9 are no longer supported
31. npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
32. npm WARN deprecated phantomjs-prebuilt@2.1.16: this package is now deprecated
33. npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
34. npm WARN deprecated http2@3.3.7: Use the built-in module in node 9.0.0 or newer, instead
35. npm WARN deprecated json3@3.2.6: Please use the native JSON object instead of JSON 3
36. npm WARN deprecated coffee-script@1.10.0: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
37. npm WARN deprecated log4js@0.6.38: 0.x is no longer supported. Please upgrade to 6.x or higher.
38. npm WARN deprecated core-js@2.6.12: core-js@<3.23.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Some versions have web compatibility issues. Please, upgrade your dependencies to the actual version of core-js.
40. added 528 packages, and audited 529 packages in 33s
42. 22 packages are looking for funding
43.   run `npm fund` for details
45. 45 vulnerabilities (3 low, 7 moderate, 27 high, 8 critical)
47. To address issues that do not require attention, run:
48.   npm audit fix
50. To address all issues possible (including breaking changes), run:
51.   npm audit fix --force
53. Some issues need review, and may require choosing
54. a different dependency.
56. Run `npm audit` for details.
57. [root@vm2 elasticsearch-head-master]#

复制代码

1. [root@vm2 elasticsearch-head-master]# npm install  --registry=http://registry.npm.taobao.org
2. [root@vm2 elasticsearch-head-master]# nohup npm run start &
3. [root@vm2 elasticsearch-head-master]# ss -anlt
4. State       Recv-Q       Send-Q             Local Address:Port             Peer Address:Port      Process      
5. LISTEN      0            511                      0.0.0.0:9100                  0.0.0.0:*                     
6. LISTEN      0            128                      0.0.0.0:22                    0.0.0.0:*                     
7. LISTEN      0            4096                           *:9300                        *:*                     
8. LISTEN      0            4096                           *:9200                        *:*                     
9. LISTEN      0            128                         [::]:22                       [::]:*                     
10. [root@vm2 elasticsearch-head-master]#

复制代码

修改ES集群配置文件,并重启服务

1. [root@vm1 ~]# vim /etc/elasticsearch/elasticsearch.yml
2. [root@vm1 ~]# vim /etc/elasticsearch/elasticsearch.yml
3. http.cors.enabled: true
4. http.cors.allow-origin: "*"  
5. #添加两行
6. [root@vm1 ~]# systemctl restart elasticsearch
7. [root@vm2 ~]# systemctl restart elasticsearch
8. [root@vm1 ~]# ss -anlt
9. State       Recv-Q       Send-Q             Local Address:Port             Peer Address:Port      Process      
10. LISTEN      0            128                      0.0.0.0:22                    0.0.0.0:*                     
11. LISTEN      0            4096                           *:9300                        *:*                     
12. LISTEN      0            4096                           *:9200                        *:*                     
13. LISTEN      0            128                         [::]:22                       [::]:*                     
14. [root@vm1 ~]#
15. [root@vm2 ~]# ss -anlt
16. State       Recv-Q       Send-Q             Local Address:Port             Peer Address:Port      Process      
17. LISTEN      0            511                      0.0.0.0:9100                  0.0.0.0:*                     
18. LISTEN      0            128                      0.0.0.0:22                    0.0.0.0:*                     
19. LISTEN      0            4096                           *:9300                        *:*                     
20. LISTEN      0            4096                           *:9200                        *:*                     
21. LISTEN      0            128                         [::]:22                       [::]:*                     
22. [root@vm2 ~]#

复制代码

logstash摆设

摆设

1. [root@v3 ~]# ls
2. anaconda-ks.cfg  -e  -i.bak  logstash-6.5.2.rpm
3. [root@v3 ~]# rpm -ivh logstash-6.5.2.rpm
4. [root@v3 ~]# cd /etc/logstash/
5. [root@v3 logstash]# ls
6. conf.d       log4j2.properties     logstash.yml   startup.options
7. jvm.options  logstash-sample.conf  pipelines.yml
8. [root@v3 logstash]# vim logstash.yml
9. -------------------------------------------------------------
10. path.data: /var/lib/logstash
11. path.config: /etc/logstash/conf.d/
12. http.host: "0.0.0.0"
13. path.logs: /var/log/logstash
14. -------------------------------------------------------------

复制代码

验证方式一：

1. [root@v3 logstash]# cd /usr/share/logstash/bin/
2. [root@v3 bin]# ./logstash -e 'input {stdout {}} output {stdout {}}'

复制代码

末尾出现：

验证方式二：

1. [root@v3 ~]# vim /etc/logstash/conf.d/test.conf
2. [root@v3 ~]# cat /etc/logstash/conf.d/test.conf
3. input {
4.         stdin {
5.         }
6. }
8. filter {
9. }
11. output {
12.         stdout {
13.                 codec => rubydebug
14.         }
15. }
16. [root@v3 ~]#
18. [root@v3 ~]# cd /usr/share/logstash/bin/
19. [root@v3 bin]# ./logstash --path.settings /etc/logstash -f /etc/logstash/conf.d/test.conf -t
20. --path.settings 指定logstash主配置文件目录
21. -f 指定片段配置文件
22. -t 测试配置文件是否正确
23. -r参数很强大,会动态装载配置文件,也就是说启动后,可以不用重启修改配置文件
24. codec => rubydebug这句可写可不定,默认就是这种输出方式

复制代码

出现：

1. [root@v3 bin]# ./logstash --path.settings /etc/logstash -r -f /etc/logstash/conf.d/test.conf
2. Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
3. [2024-08-20T14:35:14,083][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/var/lib/logstash/queue"}
4. [2024-08-20T14:35:14,106][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/var/lib/logstash/dead_letter_queue"}
5. [2024-08-20T14:35:14,542][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
6. Configuration OK
7. [2024-08-20T14:35:16,347][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash
8. [root@v3 bin]#
9. [root@v3 bin]# ./logstash --path.settings /etc/logstash -r -f /etc/logstash/conf.d/test.conf
10. Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
11. [2024-08-20T14:38:00,603][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
12. [2024-08-20T14:38:00,615][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.5.2"}
13. [2024-08-20T14:38:00,645][INFO ][logstash.agent           ] No persistent UUID file found. Generating new UUID {:uuid=>"8843a144-df1e-45d7-a38b-c67a4758c30e", :path=>"/var/lib/logstash/uuid"}
14. [2024-08-20T14:38:02,829][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
15. [2024-08-20T14:38:03,016][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0xce3fefd sleep>"}
16. The stdin plugin is now waiting for input:
17. [2024-08-20T14:38:03,059][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}

复制代码

日志收罗

收罗messages日志

1. [root@v3 bin]# vim /etc/logstash/conf.d/test.conf
2. [root@v3 bin]# cat /etc/logstash/conf.d/test.conf
3. input {
4. file {
5. path => "/var/log/messages"
6. start_position => "beginning"
7. }
8. }
9. output {
10. elasticsearch{
11. hosts => ["192.168.100.80:9200"]
12. index => "test-%{+YYYY.MM.dd}"
13. }
14. }
15. [root@v3 bin]# ps -ef | grep java   #停止服务

复制代码

收罗多日志源

1. [root@v3 bin]# vim /etc/logstash/conf.d/test.conf
2. [root@v3 bin]# cat /etc/logstash/conf.d/test.conf
3. input {
4.         file {
5.                 path => "/var/log/messages"
6.                 start_position => "beginning"
7.                 type => "messages"
8.         }
9.         file {
10.                 path => "/var/log/dnf.log"
11.                 start_position => "beginning"
12.                 type => "dnf"
13.         }
14. }
16. filter{
18. }
23. output{
24.         if [type] == "messages" {
25.                 elasticsearch {
26.                         hosts => ["192.168.100.30:9200","192.168.100.80:9200"]
27.                         index => "messages-%{+YYYY-MM-dd}"
28.                         }
29.                 }
30.         if [type] == "dnf" {
31.                 elasticsearch {
32.                         hosts => ["192.168.100.30:9200","192.168.100.80:9200"]
33.                         index => "yum-%{+YYYY-MM-dd}"
34.                         }
35.                 }
36. }
38. [root@v3 bin]# ./logstash --path.settings /etc/logstash -r -f /etc/logstash/conf.d/test.conf &
39. [root@v3 bin]# ss -anlt
40. State        Recv-Q       Send-Q             Local Address:Port             Peer Address:Port       Process      
41. LISTEN       0            128                      0.0.0.0:22                    0.0.0.0:*                        
42. LISTEN       0            50                             *:9600                        *:*                        
43. LISTEN       0            128                         [::]:22                       [::]:*   

复制代码

kibana摆设

摆设

1. [root@vm1 ~]# ls
2. 04-ELK2.pdf      -e                       -i.bak                   node-v10.24.1-linux-x64.tar.xz
3. anaconda-ks.cfg  elasticsearch-6.5.2.rpm  kibana-6.5.2-x86_64.rpm
4. [root@vm1 ~]# rpm -ivh kibana-6.5.2-x86_64.rpm
5. warning: kibana-6.5.2-x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID d88e42b4: NOKEY
6. Verifying...                          ################################# [100%]
7. Preparing...                          ################################# [100%]
8. Updating / installing...
9.    1:kibana-6.5.2-1                   ################################# [100%]
10. [root@vm1 ~]#
11. [root@vm1 ~]# cd /etc/kibana/
12. [root@vm1 kibana]# ls
13. kibana.yml
14. [root@vm1 kibana]# vim kibana.yml
15. ---------------------------------------------------------------
16. server.port: 5601 端口
17. server.host: "0.0.0.0" 监听所有,允许所有人能访问
18. elasticsearch.url: "http://192.168.100.30:9200" ES集群的路径
19. logging.dest: /var/log/kibana.log 我这里加了kibana日志,方便排错与调试
20. ---------------------------------------------------------------
21. [root@vm1 kibana]# cd /var/log/
22. [root@vm1 log]# ls
23. anaconda  cron             dnf.rpm.log    hawkey.log-20240819  messages           secure            sssd
24. audit     cron-20240819    elasticsearch  lastlog              messages-20240819  secure-20240819   tallylog
25. btmp      dnf.librepo.log  firewalld      maillog              private            spooler           wtmp
26. chrony    dnf.log          hawkey.log     maillog-20240819     README             spooler-20240819
27. [root@vm1 log]# touch kibana.log
28. [root@vm1 log]# chown kibana.kibana kibana.log
29. [root@vm1 log]# systemctl restart kibana
30. [root@vm1 log]# systemctl enable kibana
31. [root@vm1 log]#

复制代码

汉化

1. [root@vm1 ~]# unzip kibana-6.5.4_hanization-master.zip -d /usr/local/
2. [root@vm1 ~]# cd /usr/local/kibana-6.5.4_hanization-master
3. 这里要注意:1,要安装python; 2,rpm版的kibana安装目录为/usr/share/kibana/
4. [root@vm1 kibana-6.5.4_hanization-master]# python main.py  /usr/share/kibana/
6. 汉化完后需要重启
7. [root@vm1 Kibana_Hanization-master]# systemctl stop kibana
8. [root@vm1 Kibana_Hanization-master]# systemctl start kibana

复制代码

免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！更多信息从访问主页：qidao123.com:ToB企服之家，中国第一个企服评测及商务社交产业平台。