openEuler24.03 LTS下安装Hive3
目次前提条件
安装MySQL
卸载原有mysql及mariadb
下载mysql
解压mysql
安装mysql
启动mysql服务
开机自启动mysql服务
登录mysql
修改mysql暗码
远程连接mysql
安装Hive
下载安装包
解压
设置环境变量
解决日记包辩论
将mysql驱动拷贝到lib目次
设置Hive
创建hive-site.xml
调整输出日记级别
初始化Hive元数据库
修改元数据库字符集
进入Hive命令行
简单使用Hive
退出Hive命令行
前提条件
Hive依赖于Hadoop，所以必要先安装好Hadoop，可参考：openEuler24.03 LTS下安装Hadoop3完全分布式
安装MySQL
Hive的元数据存储在关系型数据库中，这里使用的关系型数据库为MySQL，所以必要先安装好MySQL。
在node2安装MySQL。
卸载原有mysql及mariadb
sudo systemctl stop mysql mysqld 2>/dev/null
sudo rpm -qa | grep -i 'mysql\|mariadb' | xargs -n1 sudo rpm -e --nodeps 2>/dev/null
sudo rm -rf /var/lib/mysql /var/log/mysqld.log /usr/lib64/mysql /etc/my.cnf /usr/my.cnf https://i-blog.csdnimg.cn/direct/7a99ef81d27b475a9d04e64afad507fd.gif
下载mysql
这里下载的mysql版本为mysql8.4.2，假如下载较旧的mysql版本大概与openEuler24.03不兼容。
cd /opt/software
wget https://downloads.mysql.com/archives/get/p/23/file/mysql-8.4.2-1.el9.x86_64.rpm-bundle.tar 注意：假如命令行下载较慢，可以直接使用浏览器访问https链接所在下载，再上传安装文件到Linux的安装包存放目次，比方：/opt/software。
解压mysql
# 创建mysql目录
$ mkdir mysql
#解压
$ tar -xvf mysql-8.4.2-1.el9.x86_64.rpm-bundle.tar -C mysql https://i-blog.csdnimg.cn/direct/eeaed4b264214e89b24b38a539783f9d.gif
删除不必要的rpm包
$ cd mysql
$ ls
mysql-community-client-8.4.2-1.el9.x86_64.rpm
mysql-community-client-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-client-plugins-8.4.2-1.el9.x86_64.rpm
mysql-community-client-plugins-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-common-8.4.2-1.el9.x86_64.rpm
mysql-community-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-debugsource-8.4.2-1.el9.x86_64.rpm
mysql-community-devel-8.4.2-1.el9.x86_64.rpm
mysql-community-icu-data-files-8.4.2-1.el9.x86_64.rpm
mysql-community-libs-8.4.2-1.el9.x86_64.rpm
mysql-community-libs-compat-8.4.2-1.el9.x86_64.rpm
mysql-community-libs-compat-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-libs-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-server-8.4.2-1.el9.x86_64.rpm
mysql-community-server-debug-8.4.2-1.el9.x86_64.rpm
mysql-community-server-debug-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-server-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-test-8.4.2-1.el9.x86_64.rpm
mysql-community-test-debuginfo-8.4.2-1.el9.x86_64.rpm
$ rm -f *debug*
$ ls
mysql-community-client-8.4.2-1.el9.x86_64.rpm mysql-community-libs-8.4.2-1.el9.x86_64.rpm
mysql-community-client-plugins-8.4.2-1.el9.x86_64.rpmmysql-community-libs-compat-8.4.2-1.el9.x86_64.rpm
mysql-community-common-8.4.2-1.el9.x86_64.rpm mysql-community-server-8.4.2-1.el9.x86_64.rpm
mysql-community-devel-8.4.2-1.el9.x86_64.rpm mysql-community-test-8.4.2-1.el9.x86_64.rpm
mysql-community-icu-data-files-8.4.2-1.el9.x86_64.rpm https://i-blog.csdnimg.cn/direct/e64c14e4e37040a7a4c80e53786f3e88.gif
安装mysql
安装命令
$ sudo rpm -ivh *.rpm --force --nodeps https://i-blog.csdnimg.cn/direct/863e136352204031a963a47eb5986e04.gif安装过程
警告：mysql-community-client-8.4.2-1.el9.x86_64.rpm: 头 V4 RSA/SHA256 Signature, 密钥 ID a8d3785c: NOKEY
Verifying... #################################
准备中... #################################
正在升级/安装...
1:mysql-community-common-8.4.2-1.el################################# [ 11%]
2:mysql-community-client-plugins-8.################################# [ 22%]
3:mysql-community-libs-8.4.2-1.el9 ################################# [ 33%]
4:mysql-community-client-8.4.2-1.el################################# [ 44%]
5:mysql-community-icu-data-files-8.################################# [ 56%]
6:mysql-community-server-8.4.2-1.el################################# [ 67%]
7:mysql-community-test-8.4.2-1.el9 ################################# [ 78%]
8:mysql-community-devel-8.4.2-1.el9################################# [ 89%]
9:mysql-community-libs-compat-8.4.2#################################
/usr/lib/tmpfiles.d/dbus.conf:13: Line references path below legacy directory /var/run/, updating /var/run/dbus/containers → /run/dbus/containers; please update the tmpfiles.d/ drop-in file accordingly.
$
启动mysql服务
$ sudo systemctl start mysqld
开机自启动mysql服务
$ sudo systemctl enable mysqld
登录mysql
查看暂时暗码
$ sudo grep 'temporary password' /var/log/mysqld.log 输出如下
2025-03-19T06:43:36.958044Z 6 A temporary password is generated for root@localhost: tFggwGvsx8=j 这里查看到的暂时暗码为：tFggwGvsx8=j
注意：查看到的暂时暗码大概不一样，请使用实际查到的暂时暗码登录mysql。
登录mysql
$ mysql -uroot -p'tFggwGvsx8=j'
mysql: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.Commands end with ; or \g.
Your MySQL connection id is 8
Server version: 8.4.2
Copyright (c) 2000, 2024, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> exit;
Bye
$
修改mysql暗码
修改暗码策略
若在内网环境使用MySQL，想使用简单暗码，则必要设置暗码策略，否则，不必要设置暗码策略。
$ sudo vim /etc/my.cnf
在下面添加如下语句
validate_password.length=4
validate_password.policy=0
重启mysql
$ sudo systemctl restart mysqld
登录mysql并修改暗码
$ mysql -uroot -p'tFggwGvsx8=j'
mysql> set password='000000';
Query OK, 0 rows affected (0.00 sec)
mysql> update mysql.user set host='%' where user='root';
Query OK, 1 row affected (0.01 sec)
Rows matched: 1Changed: 1Warnings: 0
mysql> alter user 'root'@'%' identified by '000000';
Query OK, 0 rows affected (0.01 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
mysql> exit;
Bye
使用新暗码登录
$ mysql -uroot -p000000
mysql: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.Commands end with ; or \g.
Your MySQL connection id is 9
Server version: 8.4.2 MySQL Community Server - GPL
Copyright (c) 2000, 2024, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> exit;
Bye
$
远程连接mysql
使用navicat等工具，通过ip及端口号远程连接
https://i-blog.csdnimg.cn/direct/a106d0b2d8dd45b1b8c23cc204939279.png
安装Hive
在node2安装Hive
下载安装包
浏览器访问以下链接下载Hive安装包
https://archive.apache.org/dist/hive/hive-3.1.3/apache-hive-3.1.3-bin.tar.gz
上传Hive安装包到Linux /opt/software目次
$ ls | grep hive
apache-hive-3.1.3-bin.tar.gz
解压
解压
$ tar -zxvf /opt/software/apache-hive-3.1.3-bin.tar.gz -C /opt/module/
重命名
$ ls /opt/module/
apache-hive-3.1.3-binhadoop-3.3.4jdk1.8.0_271
$ mv /opt/module/apache-hive-3.1.3-bin /opt/module/hive-3.1.3
设置环境变量
$ sudo vim /etc/profile.d/my_env.sh 末尾添加如下内容
#HIVE_HOME
export HIVE_HOME=/opt/module/hive-3.1.3
export PATH=$PATH:$HIVE_HOME/bin
让环境变量收效
$ source /etc/profile
解决日记包辩论
解决日记Jar包依赖与Hadoop日记Jar版本不同辩论，重命名hive的日记包为其他名字，如许就只有hadoop的日记Jar包收效，辩论解决。
$ cd $HIVE_HOME/lib/
$ ls | grep slf4j
log4j-slf4j-impl-2.17.1.jar
$ mv log4j-slf4j-impl-2.17.1.jar log4j-slf4j-impl-2.17.1.jar.bak
将mysql驱动拷贝到lib目次
浏览器访问如下所在，下载mysql驱动包
https://mvnrepository.com/artifact/com.mysql/mysql-connector-j/8.4.0
https://i-blog.csdnimg.cn/direct/1bf02e910ea34908a61f6b65546f2fac.png
点击jar下载，并上传jar包文件到Linux /opt/software目次
$ ls /opt/software | grep connect
mysql-connector-j-8.4.0.jar 将jar文件复制到HIVE_HOME的lib目次
$ cp /opt/software/mysql-connector-j-8.4.0.jar $HIVE_HOME/lib/
$ ls | grep connect
mysql-connector-j-8.4.0.jar
设置Hive
切换到Hive设置目次，查看设置文件
$ cd $HIVE_HOME/conf
$ ls
beeline-log4j2.properties.template ivysettings.xml
hive-default.xml.template llap-cli-log4j2.properties.template
hive-env.sh.template llap-daemon-log4j2.properties.template
hive-exec-log4j2.properties.templateparquet-logging.properties
hive-log4j2.properties.template
创建hive-site.xml
$ vim hive-site.xml 设置内容
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<!--配置Hive保存元数据信息所需的 MySQL URL地址-->
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://node2:3306/metastore?useSSL=false&useUnicode=true&characterEncoding=UTF-8&allowPublicKeyRetrieval=true</value>
</property>
<!--配置Hive连接MySQL的驱动全类名-->
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.cj.jdbc.Driver</value>
</property>
<!--配置Hive连接MySQL的用户名 -->
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
</property>
<!--配置Hive连接MySQL的密码 -->
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>000000</value>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>
</property>
<property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
</property>
<property>
<name>hive.server2.thrift.port</name>
<value>10000</value>
</property>
<property>
<name>hive.server2.thrift.bind.host</name>
<value>node2</value>
</property>
<property>
<name>hive.metastore.event.db.notification.api.auth</name>
<value>false</value>
</property>
<property>
<name>hive.cli.print.header</name>
<value>true</value>
</property>
<property>
<name>hive.cli.print.current.db</name>
<value>true</value>
</property>
</configuration> 设置文件中连接nod2机器mysql的metastore数据库，所以必要先把metastore数据库创建出来。
$ mysql -uroot -p000000
mysql: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.Commands end with ; or \g.
Your MySQL connection id is 13
Server version: 8.4.2 MySQL Community Server - GPL
Copyright (c) 2000, 2024, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> create database metastore;
Query OK, 1 row affected (0.01 sec)
mysql> exit;
Bye
$
调整输出日记级别
默认日记级别为INFO，实行HQL语句过程会输出太多日记，将Hive输出日记级别改为WARN减少日记输出。
cd $HIVE_HOME/conf 实行如下命令新建log4j.properties
cat > log4j.properties <<EOL
log4j.rootLogger=WARN, CA
log4j.appender.CA=org.apache.log4j.ConsoleAppender
log4j.appender.CA.layout=org.apache.log4j.PatternLayout
log4j.appender.CA.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
EOL
初始化Hive元数据库
$ schematool -initSchema -dbType mysql -verbose 末尾部分输出内容如下
0: jdbc:mysql://node2:3306/metastore> !closeall
Closing: 0: jdbc:mysql://node2:3306/metastore?useSSL=false&useUnicode=true&characterEncoding=UTF-8&allowPublicKeyRetrieval=true
beeline>
beeline> Initialization script completed
schemaTool completed 看到schemaTool completed输出，阐明初始化乐成。
初始化本质是在元数据库里创建出相关的表及初始化相关表数据，可以到mysql的metastore数据库查看生成的表数据。
修改元数据库字符集
$ mysql -uroot -p000000
...
mysql> use metastore;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> alter table COLUMNS_V2 modify column COMMENT varchar(256) character set utf8;
Query OK, 0 rows affected, 1 warning (0.03 sec)
Records: 0Duplicates: 0Warnings: 1
mysql> alter table TABLE_PARAMS modify column PARAM_VALUE mediumtext character set utf8;
Query OK, 0 rows affected, 1 warning (0.02 sec)
Records: 0Duplicates: 0Warnings: 1
mysql> quit;
Bye
$
进入Hive命令行
进入Hive命令行之前，必要启动hadoop，假如还没启动，必要先启动hadoop
# 分别启动
$ start-dfs.sh
$ start-yarn.sh
# 或者使用脚本启动
$ hdp.sh start 进入hive命令行
$ hive
which: no hbase in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/module/jdk1.8.0_271/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/home/liang/bin:/opt/module/jdk1.8.0_271/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/home/liang/bin:/opt/module/jdk1.8.0_271/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/opt/module/hive-3.1.3/bin)
Hive Session ID = a2b0c24d-7c45-4e95-9c86-1108b2606f3b
Logging initialized using configuration in jar:file:/opt/module/hive-3.1.3/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Hive Session ID = 5d3c533a-79e3-4690-b54c-1006d18d7aca
hive (default)>
简单使用Hive
查看数据库
hive (default)> show databases;
OK
database_name
default
Time taken: 0.609 seconds, Fetched: 1 row(s) 查看数据表
hive (default)> show tables;
OK
tab_name
Time taken: 0.051 seconds
创建表
hive (default)> CREATE TABLE IF NOT EXISTS test_students (
id INT,
name STRING,
age INT
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','; 插入数据
hive (default)> INSERT INTO TABLE test_students VALUES (103, 'Bob', 21);
查看表数据
hive (default)> select * from test_students;
退出Hive命令行
hive (default)> quit;
$
如有必要，可点击查看：配套视频教程
完成！enjoy it！
免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！更多信息从访问主页：qidao123.com:ToB企服之家，中国第一个企服评测及商务社交产业平台。
页:
[1]