es数据备份和迁徙Elasticsearch
Elasticsearch数据备份与规复
条件
# 注意：
1.在进行本地备份时使用--type需要备份索引和数据（mapping,data）
2.在将数据备份到另外一台ES节点时需要比本地备份多备份一种数据类型（analyzer,mapping,data,template） 一.当地备份与规复
# 前提：
· 必须要有Node环境和npm软件：nodejs，npm
1.:下载包
wget https://nodejs.org/dist/v14.17.1/node-v14.17.1-linux-x64.tar.xz
2.:安装包（如果本身有node包但是不是新版本建议卸载重装）
xz -d node-v14.17.1-linux-x64.tar.xz
tar -xvf node-v14.17.1-linux-x64.tar
echo "export NODE_HOME=/home/workspaces/node-v14.17.1-linux-x64
export PATH=$PATH:$NODE_HOME/bin
export NODE_PATH=$NODE_HOME/lib/node_modules" >>/etc/profile
source /etc/profile
3.:验证安装是否成功
node -v
4.:设置淘宝镜像
npm config set registry http://registry.npm.taobao.org
5.：安装npm（只需要在一个节点安装即可，如果前端还有nginx做反向代理可以每个节点都装）
# yum install -y npm
6.：进入下载head插件代码目录
# cd /usr/local/
7.：从GitHub上克隆代码到本地（或者直接从找es-head的包）
# git clone git://github.com/mobz/elasticsearch-head.git
8.：克隆完成后，进入elasticsearch插件目录
# cd elasticsearch-head/
9.：清除缓存
# npm cache clean -f
#使用npm安装n模块（不同的项目js脚本所需的node版本可能不同，所以就需要node版本管理工具）
10.安装elasticdump
# npm install elasticdump -g
11.本地目录备份
## 第一次进行的是索引的备份
## 第二次进行的是数据的备份
# ./elasticdump --input=http://10.0.0.54:9200/student/user --output=/tmp/student_mapping.json --type=mapping
Wed, 12 Aug 2020 07:41:59 GMT | starting dump
Wed, 12 Aug 2020 07:41:59 GMT | got 1 objects from source elasticsearch (offset: 0)
Wed, 12 Aug 2020 07:41:59 GMT | sent 1 objects to destination file, wrote 1
Wed, 12 Aug 2020 07:41:59 GMT | got 0 objects from source elasticsearch (offset: 1)
Wed, 12 Aug 2020 07:41:59 GMT | Total Writes: 1
Wed, 12 Aug 2020 07:41:59 GMT | dump complete
# ./elasticdump --input=http://10.0.0.54:9200/student/user --output=/tmp/student_data.json --type=data
Wed, 12 Aug 2020 07:42:21 GMT | starting dump
Wed, 12 Aug 2020 07:42:21 GMT | got 8 objects from source elasticsearch (offset: 0)
Wed, 12 Aug 2020 07:42:21 GMT | sent 8 objects to destination file, wrote 8
Wed, 12 Aug 2020 07:42:21 GMT | got 0 objects from source elasticsearch (offset: 8)
Wed, 12 Aug 2020 07:42:21 GMT | Total Writes: 8
Wed, 12 Aug 2020 07:42:21 GMT | dump complete
##备份之后用Elasticsearch Head插件模拟数据丢失，误删除student索引和数据并用elasticdump恢复
# ./elasticdump --output=http://10.0.0.54:9200/student --input=/tmp/student_mapping.json --type=mapping
Wed, 12 Aug 2020 07:46:21 GMT | starting dump
Wed, 12 Aug 2020 07:46:21 GMT | got 1 objects from source file (offset: 0)
Wed, 12 Aug 2020 07:46:21 GMT | sent 1 objects to destination elasticsearch, wrote 1
Wed, 12 Aug 2020 07:46:21 GMT | got 0 objects from source file (offset: 1)
Wed, 12 Aug 2020 07:46:21 GMT | Total Writes: 1
Wed, 12 Aug 2020 07:46:21 GMT | dump complete
# ./elasticdump --output=http://10.0.0.54:9200/student --input=/tmp/student_data.json --type=data
Wed, 12 Aug 2020 07:46:40 GMT | starting dump
Wed, 12 Aug 2020 07:46:40 GMT | got 8 objects from source file (offset: 0)
Wed, 12 Aug 2020 07:46:40 GMT | sent 8 objects to destination elasticsearch, wrote 8
Wed, 12 Aug 2020 07:46:40 GMT | got 0 objects from source file (offset: 8)
Wed, 12 Aug 2020 07:46:40 GMT | Total Writes: 8
Wed, 12 Aug 2020 07:46:40 GMT | dump complete #根据索引名和类型进行备份
https://img-blog.csdnimg.cn/img_convert/f65d08bcbb3ce9d02ab8331563a25d54.png
#备份成功。数据恢复成功
https://img-blog.csdnimg.cn/img_convert/5035bec244d8a9a4393b0e1fe4915794.png
二.备份到另一台ES节点（如果是同一集群的节点索引不能重名）
# 备份到另一台ES节点（同一集群索引不能重名）
# ./elasticdump --input=http://10.0.0.54:9200/student --output=http://10.0.0.55:9200/students --type=analyzer
Wed, 12 Aug 2020 08:14:00 GMT | starting dump
Wed, 12 Aug 2020 08:14:00 GMT | got 1 objects from source elasticsearch (offset: 0)
Wed, 12 Aug 2020 08:14:01 GMT | sent 1 objects to destination elasticsearch, wrote 1
Wed, 12 Aug 2020 08:14:01 GMT | got 0 objects from source elasticsearch (offset: 1)
Wed, 12 Aug 2020 08:14:01 GMT | Total Writes: 1
Wed, 12 Aug 2020 08:14:01 GMT | dump complete
# ./elasticdump --input=http://10.0.0.54:9200/student --output=http://10.0.0.55:9200/students --type=mapping
Wed, 12 Aug 2020 08:14:10 GMT | starting dump
Wed, 12 Aug 2020 08:14:10 GMT | got 1 objects from source elasticsearch (offset: 0)
Wed, 12 Aug 2020 08:14:10 GMT | sent 1 objects to destination elasticsearch, wrote 1
Wed, 12 Aug 2020 08:14:10 GMT | got 0 objects from source elasticsearch (offset: 1)
Wed, 12 Aug 2020 08:14:10 GMT | Total Writes: 1
Wed, 12 Aug 2020 08:14:10 GMT | dump complete
# ./elasticdump --input=http://10.0.0.54:9200/student --output=http://10.0.0.55:9200/students --type=data
Wed, 12 Aug 2020 08:14:15 GMT | starting dump
Wed, 12 Aug 2020 08:14:15 GMT | got 8 objects from source elasticsearch (offset: 0)
Wed, 12 Aug 2020 08:14:15 GMT | sent 8 objects to destination elasticsearch, wrote 8
Wed, 12 Aug 2020 08:14:15 GMT | got 0 objects from source elasticsearch (offset: 8)
Wed, 12 Aug 2020 08:14:15 GMT | Total Writes: 8
Wed, 12 Aug 2020 08:14:15 GMT | dump complete
#elasticdump --input=http://10.0.0.54:9200/student --output=http://100.10.0.55:9200/students --type=template
https://img-blog.csdnimg.cn/img_convert/2fda9d8bdcbf026cf947c5f610393f21.png
https://img-blog.csdnimg.cn/img_convert/7ffdc6fb98caafd11da4a06aff080474.png
https://img-blog.csdnimg.cn/img_convert/0508b5b00fa39a1eb6f8eb3cea0eb877.png
三.当地备份与规复脚本
# 1.本地备份脚本
# vim output_Es.sh
#!/bin/bash
read -p '要备份的机器是：'${1}
#要导出的索引名
index_name='
student
'
for index in `echo $index_name`
do
echo "start input index ${index}"
elasticdump --input=http://${1}:9200/${index} --output=/data/${index}_alias.json --type=alias &> /dev/null
elasticdump --input=http://${1}:9200/${index} --output=/data/${index}_analyzer.json --type=analyzer &> /dev/null
elasticdump --input=http://${1}:9200/${index} --output=/data/${index}_data.json --type=data &> /dev/null
elasticdump --input=http://${1}:9200/${index} --output=/data/${index}_alias.json --type=alias &> /dev/null
elasticdump --input=http://${1}:9200/${index} --output=/data/${index}_template.json --type=template &> /dev/null
done
# 2.恢复脚本
# vim input_Es.sh
#!/bin/bash
read -p '要导入的机器是：'${1}
#要导入的索引名
index_name='
student
'
for index in `echo $index_name`
do
echo "start input index ${index}"
elasticdump --input=/data/${index}_alias.json --output=http://${1}:9200/${index} --type=alias &> /dev/null
elasticdump --input=/data/${index}_analyzer.json --output=http://${1}:9200/${index} --type=analyzer &> /dev/null
elasticdump --input=/data/${index}_data.json --output=http://${1}:9200/${index} --type=data &> /dev/null
elasticdump --input=/data/${index}_template.json --output=http://${1}:9200/${index} --type=template &> /dev/null
done 测试脚本
# 0.配置环境变量
# vim /etc/profile.d/elasticdump.sh
export PATH=/root/node-v10.13.0-linux-x64/lib/node_modules/elasticdump/bin:$PATH
# source /etc/profile
# 1.创建备份目录
# mkdir /data
# 2.执行备份脚本
# sh output_Es.sh
要备份的机器是：10.0.0.54
start input index student
# 3.查看备份目录
# ll /data/
total 16
-rw-r--r-- 1 root root 5 Aug 12 16:59 student_alias.json
-rw-r--r-- 1 root root101 Aug 12 16:59 student_analyzer.json
-rw-r--r-- 1 root root 1284 Aug 12 16:59 student_data.json
-rw-r--r-- 1 root root 5 Aug 12 16:59 student_template.json
https://img-blog.csdnimg.cn/img_convert/fc4e261a28b45f8999669b8af30b462b.png
# 模拟误删除操作
https://img-blog.csdnimg.cn/img_convert/ff045c3d6f0df35deac48a417d98b0ec.png
# 执行恢复脚本
#sh input_Es.sh
要导入的机器是：10.0.0.54
start input index student
## 数据恢复
https://img-blog.csdnimg.cn/img_convert/63ebcac882a25b7a37744377aa37465d.png
https://img-blog.csdnimg.cn/img_convert/ca22742744224a6c272f536d3e82567b.png
免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！更多信息从访问主页：qidao123.com:ToB企服之家，中国第一个企服评测及商务社交产业平台。
页:
[1]