openEuler24.03 LTS下安装Hive3

九天猎人  论坛元老 | 2025-3-31 09:30:50 | 显示全部楼层 | 阅读模式
打印 上一主题 下一主题
楼主

主题 1602|帖子 1602|积分 4806

目次
前提条件
安装MySQL
卸载原有mysql及mariadb
下载mysql
解压mysql
安装mysql
启动mysql服务
开机自启动mysql服务
登录mysql
修改mysql暗码
远程连接mysql
安装Hive
下载安装包
解压
设置环境变量
解决日记包辩论
将mysql驱动拷贝到lib目次
设置Hive
创建hive-site.xml
调整输出日记级别
初始化Hive元数据库
修改元数据库字符集
进入Hive命令行
简单使用Hive
退出Hive命令行

前提条件

Hive依赖于Hadoop，所以必要先安装好Hadoop，可参考：openEuler24.03 LTS下安装Hadoop3完全分布式

安装MySQL

Hive的元数据存储在关系型数据库中，这里使用的关系型数据库为MySQL，所以必要先安装好MySQL。
在node2安装MySQL。

卸载原有mysql及mariadb

  1. sudo systemctl stop mysql mysqld 2>/dev/null
  2. sudo rpm -qa | grep -i 'mysql\|mariadb' | xargs -n1 sudo rpm -e --nodeps 2>/dev/null
  3. sudo rm -rf /var/lib/mysql /var/log/mysqld.log /usr/lib64/mysql /etc/my.cnf /usr/my.cnf
复制代码

下载mysql

这里下载的mysql版本为mysql8.4.2，假如下载较旧的mysql版本大概与openEuler24.03不兼容。
  1. cd /opt/software
  2. wget https://downloads.mysql.com/archives/get/p/23/file/mysql-8.4.2-1.el9.x86_64.rpm-bundle.tar
复制代码
注意：假如命令行下载较慢，可以直接使用浏览器访问https链接所在下载，再上传安装文件到Linux的安装包存放目次，比方：/opt/software。

解压mysql

  1. # 创建mysql目录
  2. [liang@node2 software]$ mkdir mysql
  3. ​
  4. #解压
  5. [liang@node2 software]$ tar -xvf mysql-8.4.2-1.el9.x86_64.rpm-bundle.tar -C mysql
复制代码

删除不必要的rpm包
  1. [liang@node2 software]$ cd mysql
  2. [liang@node2 mysql]$ ls
  3. mysql-community-client-8.4.2-1.el9.x86_64.rpm
  4. mysql-community-client-debuginfo-8.4.2-1.el9.x86_64.rpm
  5. mysql-community-client-plugins-8.4.2-1.el9.x86_64.rpm
  6. mysql-community-client-plugins-debuginfo-8.4.2-1.el9.x86_64.rpm
  7. mysql-community-common-8.4.2-1.el9.x86_64.rpm
  8. mysql-community-debuginfo-8.4.2-1.el9.x86_64.rpm
  9. mysql-community-debugsource-8.4.2-1.el9.x86_64.rpm
  10. mysql-community-devel-8.4.2-1.el9.x86_64.rpm
  11. mysql-community-icu-data-files-8.4.2-1.el9.x86_64.rpm
  12. mysql-community-libs-8.4.2-1.el9.x86_64.rpm
  13. mysql-community-libs-compat-8.4.2-1.el9.x86_64.rpm
  14. mysql-community-libs-compat-debuginfo-8.4.2-1.el9.x86_64.rpm
  15. mysql-community-libs-debuginfo-8.4.2-1.el9.x86_64.rpm
  16. mysql-community-server-8.4.2-1.el9.x86_64.rpm
  17. mysql-community-server-debug-8.4.2-1.el9.x86_64.rpm
  18. mysql-community-server-debug-debuginfo-8.4.2-1.el9.x86_64.rpm
  19. mysql-community-server-debuginfo-8.4.2-1.el9.x86_64.rpm
  20. mysql-community-test-8.4.2-1.el9.x86_64.rpm
  21. mysql-community-test-debuginfo-8.4.2-1.el9.x86_64.rpm
  22. [liang@node2 mysql]$ rm -f *debug*
  23. [liang@node2 mysql]$ ls
  24. mysql-community-client-8.4.2-1.el9.x86_64.rpm          mysql-community-libs-8.4.2-1.el9.x86_64.rpm
  25. mysql-community-client-plugins-8.4.2-1.el9.x86_64.rpm  mysql-community-libs-compat-8.4.2-1.el9.x86_64.rpm
  26. mysql-community-common-8.4.2-1.el9.x86_64.rpm          mysql-community-server-8.4.2-1.el9.x86_64.rpm
  27. mysql-community-devel-8.4.2-1.el9.x86_64.rpm           mysql-community-test-8.4.2-1.el9.x86_64.rpm
  28. mysql-community-icu-data-files-8.4.2-1.el9.x86_64.rpm
复制代码

安装mysql

安装命令
  1. [liang@node2 mysql]$ sudo rpm -ivh *.rpm --force --nodeps
复制代码
安装过程
  1. 警告：mysql-community-client-8.4.2-1.el9.x86_64.rpm: 头 V4 RSA/SHA256 Signature, 密钥 ID a8d3785c: NOKEY
  2. Verifying...                          ################################# [100%]
  3. 准备中...                          ################################# [100%]
  4. 正在升级/安装...
  5.   1:mysql-community-common-8.4.2-1.el################################# [ 11%]
  6.   2:mysql-community-client-plugins-8.################################# [ 22%]
  7.   3:mysql-community-libs-8.4.2-1.el9 ################################# [ 33%]
  8.   4:mysql-community-client-8.4.2-1.el################################# [ 44%]
  9.   5:mysql-community-icu-data-files-8.################################# [ 56%]
  10.   6:mysql-community-server-8.4.2-1.el################################# [ 67%]
  11.   7:mysql-community-test-8.4.2-1.el9 ################################# [ 78%]
  12.   8:mysql-community-devel-8.4.2-1.el9################################# [ 89%]
  13.   9:mysql-community-libs-compat-8.4.2################################# [100%]
  14. /usr/lib/tmpfiles.d/dbus.conf:13: Line references path below legacy directory /var/run/, updating /var/run/dbus/containers → /run/dbus/containers; please update the tmpfiles.d/ drop-in file accordingly.
  15. [liang@node2 mysql]$
复制代码

启动mysql服务

  1. [liang@node2 mysql]$ sudo systemctl start mysqld
复制代码

开机自启动mysql服务

  1. [liang@node2 mysql]$ sudo systemctl enable mysqld
复制代码

登录mysql

查看暂时暗码
  1. [liang@node2 mysql]$ sudo grep 'temporary password' /var/log/mysqld.log
复制代码
输出如下
  1. 2025-03-19T06:43:36.958044Z 6 [Note] [MY-010454] [Server] A temporary password is generated for root@localhost: tFggwGvsx8=j
复制代码
这里查看到的暂时暗码为：tFggwGvsx8=j
注意：查看到的暂时暗码大概不一样，请使用实际查到的暂时暗码登录mysql。

登录mysql
  1. [liang@node2 mysql]$ mysql -uroot -p'tFggwGvsx8=j'
  2. mysql: [Warning] Using a password on the command line interface can be insecure.
  3. Welcome to the MySQL monitor.  Commands end with ; or \g.
  4. Your MySQL connection id is 8
  5. Server version: 8.4.2
  6. ​
  7. Copyright (c) 2000, 2024, Oracle and/or its affiliates.
  8. ​
  9. Oracle is a registered trademark of Oracle Corporation and/or its
  10. affiliates. Other names may be trademarks of their respective
  11. owners.
  12. ​
  13. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
  14. ​
  15. mysql> exit;
  16. Bye
  17. [liang@node2 mysql]$
复制代码

修改mysql暗码

修改暗码策略
若在内网环境使用MySQL，想使用简单暗码，则必要设置暗码策略，否则，不必要设置暗码策略。
  1. [liang@node2 mysql]$ sudo vim /etc/my.cnf
复制代码

在[mysqld]下面添加如下语句
  1. validate_password.length=4
  2. validate_password.policy=0
复制代码

重启mysql
  1. [liang@node2 mysql]$ sudo systemctl restart mysqld
复制代码

登录mysql并修改暗码
  1. [liang@node2 mysql]$ mysql -uroot -p'tFggwGvsx8=j'
  2. ​
  3. mysql> set password='000000';
  4. Query OK, 0 rows affected (0.00 sec)
  5. ​
  6. mysql> update mysql.user set host='%' where user='root';
  7. Query OK, 1 row affected (0.01 sec)
  8. Rows matched: 1  Changed: 1  Warnings: 0
  9. ​
  10. mysql> alter user 'root'@'%' identified by '000000';
  11. Query OK, 0 rows affected (0.01 sec)
  12. ​
  13. mysql> flush privileges;
  14. Query OK, 0 rows affected (0.00 sec)
  15. ​
  16. mysql> exit;
  17. Bye
复制代码

使用新暗码登录
  1. [liang@node2 mysql]$ mysql -uroot -p000000
  2. mysql: [Warning] Using a password on the command line interface can be insecure.
  3. Welcome to the MySQL monitor.  Commands end with ; or \g.
  4. Your MySQL connection id is 9
  5. Server version: 8.4.2 MySQL Community Server - GPL
  6. ​
  7. Copyright (c) 2000, 2024, Oracle and/or its affiliates.
  8. ​
  9. Oracle is a registered trademark of Oracle Corporation and/or its
  10. affiliates. Other names may be trademarks of their respective
  11. owners.
  12. ​
  13. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
  14. ​
  15. mysql> exit;
  16. Bye
  17. [liang@node2 mysql]$
复制代码

远程连接mysql

使用navicat等工具，通过ip及端口号远程连接


安装Hive

在node2安装Hive

下载安装包

浏览器访问以下链接下载Hive安装包
  1. https://archive.apache.org/dist/hive/hive-3.1.3/apache-hive-3.1.3-bin.tar.gz
复制代码

上传Hive安装包到Linux /opt/software目次
  1. [liang@node2 software]$ ls | grep hive
  2. apache-hive-3.1.3-bin.tar.gz
复制代码

解压

解压
  1. [liang@node2 software]$ tar -zxvf /opt/software/apache-hive-3.1.3-bin.tar.gz -C /opt/module/
复制代码

重命名
  1. [liang@node2 software]$ ls /opt/module/
  2. apache-hive-3.1.3-bin  hadoop-3.3.4  jdk1.8.0_271
  3. ​
  4. [liang@node2 software]$ mv /opt/module/apache-hive-3.1.3-bin /opt/module/hive-3.1.3
复制代码

设置环境变量

  1. [liang@node2 software]$ sudo vim /etc/profile.d/my_env.sh
复制代码
末尾添加如下内容
  1. #HIVE_HOME
  2. export HIVE_HOME=/opt/module/hive-3.1.3
  3. export PATH=$PATH:$HIVE_HOME/bin
复制代码

让环境变量收效
  1. [liang@node2 software]$ source /etc/profile
复制代码

解决日记包辩论

解决日记Jar包依赖与Hadoop日记Jar版本不同辩论，重命名hive的日记包为其他名字，如许就只有hadoop的日记Jar包收效，辩论解决。
  1. [liang@node2 software]$ cd $HIVE_HOME/lib/
  2. ​
  3. [liang@node2 lib]$ ls | grep slf4j
  4. log4j-slf4j-impl-2.17.1.jar
  5. ​
  6. [liang@node2 lib]$ mv log4j-slf4j-impl-2.17.1.jar log4j-slf4j-impl-2.17.1.jar.bak
  7. ​
复制代码

将mysql驱动拷贝到lib目次

浏览器访问如下所在，下载mysql驱动包
  1. https://mvnrepository.com/artifact/com.mysql/mysql-connector-j/8.4.0
复制代码

点击jar下载，并上传jar包文件到Linux /opt/software目次
  1. [liang@node2 lib]$ ls /opt/software | grep connect
  2. mysql-connector-j-8.4.0.jar
复制代码
将jar文件复制到HIVE_HOME的lib目次
  1. [liang@node2 lib]$ cp /opt/software/mysql-connector-j-8.4.0.jar $HIVE_HOME/lib/
  2. [liang@node2 lib]$ ls | grep connect
  3. mysql-connector-j-8.4.0.jar
复制代码

设置Hive

切换到Hive设置目次，查看设置文件
  1. [liang@node2 lib]$ cd $HIVE_HOME/conf
  2. [liang@node2 conf]$ ls
  3. beeline-log4j2.properties.template    ivysettings.xml
  4. hive-default.xml.template             llap-cli-log4j2.properties.template
  5. hive-env.sh.template                  llap-daemon-log4j2.properties.template
  6. hive-exec-log4j2.properties.template  parquet-logging.properties
  7. hive-log4j2.properties.template
复制代码

创建hive-site.xml

  1. [liang@node2 conf]$ vim hive-site.xml
复制代码
设置内容
  1. <?xml version="1.0"?>
  2. <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
  3. <configuration>
  4.     <!--配置Hive保存元数据信息所需的 MySQL URL地址-->
  5.     <property>
  6.         <name>javax.jdo.option.ConnectionURL</name>
  7.         <value>jdbc:mysql://node2:3306/metastore?useSSL=false&amp;useUnicode=true&amp;characterEncoding=UTF-8&amp;allowPublicKeyRetrieval=true</value>
  8.     </property>
  10.     <!--配置Hive连接MySQL的驱动全类名-->
  11.     <property>
  12.         <name>javax.jdo.option.ConnectionDriverName</name>
  13.         <value>com.mysql.cj.jdbc.Driver</value>
  14.     </property>
  16.     <!--配置Hive连接MySQL的用户名 -->
  17.     <property>
  18.         <name>javax.jdo.option.ConnectionUserName</name>
  19.         <value>root</value>
  20.     </property>
  22.     <!--配置Hive连接MySQL的密码 -->
  23.     <property>
  24.         <name>javax.jdo.option.ConnectionPassword</name>
  25.         <value>000000</value>
  26.     </property>
  28.     <property>
  29.         <name>hive.metastore.warehouse.dir</name>
  30.         <value>/user/hive/warehouse</value>
  31.     </property>
  33.     <property>
  34.         <name>hive.metastore.schema.verification</name>
  35.         <value>false</value>
  36.     </property>
  38.     <property>
  39.     <name>hive.server2.thrift.port</name>
  40.     <value>10000</value>
  41.     </property>
  43.     <property>
  44.         <name>hive.server2.thrift.bind.host</name>
  45.         <value>node2</value>
  46.     </property>
  48.     <property>
  49.         <name>hive.metastore.event.db.notification.api.auth</name>
  50.         <value>false</value>
  51.     </property>
  52.    
  53.     <property>
  54.         <name>hive.cli.print.header</name>
  55.         <value>true</value>
  56.     </property>
  58.     <property>
  59.         <name>hive.cli.print.current.db</name>
  60.         <value>true</value>
  61.     </property>
  62. </configuration>
复制代码
设置文件中连接nod2机器mysql的metastore数据库，所以必要先把metastore数据库创建出来。
  1. [liang@node2 conf]$ mysql -uroot -p000000
  2. mysql: [Warning] Using a password on the command line interface can be insecure.
  3. Welcome to the MySQL monitor.  Commands end with ; or \g.
  4. Your MySQL connection id is 13
  5. Server version: 8.4.2 MySQL Community Server - GPL
  6. ​
  7. Copyright (c) 2000, 2024, Oracle and/or its affiliates.
  8. ​
  9. Oracle is a registered trademark of Oracle Corporation and/or its
  10. affiliates. Other names may be trademarks of their respective
  11. owners.
  12. ​
  13. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
  14. ​
  15. mysql> create database metastore;
  16. Query OK, 1 row affected (0.01 sec)
  17. ​
  18. mysql> exit;
  19. Bye
  20. [liang@node2 conf]$
复制代码

调整输出日记级别

默认日记级别为INFO，实行HQL语句过程会输出太多日记，将Hive输出日记级别改为WARN减少日记输出。
  1. cd $HIVE_HOME/conf
复制代码
实行如下命令新建log4j.properties
  1. cat > log4j.properties <<EOL
  2. log4j.rootLogger=WARN, CA
  4. log4j.appender.CA=org.apache.log4j.ConsoleAppender
  6. log4j.appender.CA.layout=org.apache.log4j.PatternLayout
  7. log4j.appender.CA.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
  8. EOL
复制代码

初始化Hive元数据库

  1. [liang@node2 conf]$ schematool -initSchema -dbType mysql -verbose
复制代码
  1. 末尾部分输出内容如下
  2. ​
  3. 0: jdbc:mysql://node2:3306/metastore> !closeall
  4. Closing: 0: jdbc:mysql://node2:3306/metastore?useSSL=false&useUnicode=true&characterEncoding=UTF-8&allowPublicKeyRetrieval=true
  5. beeline>
  6. beeline> Initialization script completed
  7. schemaTool completed
复制代码
看到schemaTool completed输出，阐明初始化乐成。
初始化本质是在元数据库里创建出相关的表及初始化相关表数据，可以到mysql的metastore数据库查看生成的表数据。
修改元数据库字符集

  1. [liang@node2 conf]$ mysql -uroot -p000000
  2. ...
  3. mysql> use metastore;
  4. Reading table information for completion of table and column names
  5. You can turn off this feature to get a quicker startup with -A
  6. ​
  7. Database changed
  8. mysql> alter table COLUMNS_V2 modify column COMMENT varchar(256) character set utf8;
  9. Query OK, 0 rows affected, 1 warning (0.03 sec)
  10. Records: 0  Duplicates: 0  Warnings: 1
  11. ​
  12. mysql> alter table TABLE_PARAMS modify column PARAM_VALUE mediumtext character set utf8;
  13. Query OK, 0 rows affected, 1 warning (0.02 sec)
  14. Records: 0  Duplicates: 0  Warnings: 1
  15. mysql> quit;
  16. Bye
  17. [liang@node2 conf]$
  18. ​
复制代码

进入Hive命令行

进入Hive命令行之前，必要启动hadoop，假如还没启动，必要先启动hadoop
  1. # 分别启动
  2. [liang@node2 conf]$ start-dfs.sh
  3. [liang@node3 conf]$ start-yarn.sh
  4. ​
  5. # 或者使用脚本启动
  6. [liang@node2 conf]$ hdp.sh start
复制代码
进入hive命令行
  1. [liang@node2 conf]$ hive
  2. which: no hbase in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/module/jdk1.8.0_271/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/home/liang/bin:/opt/module/jdk1.8.0_271/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/home/liang/bin:/opt/module/jdk1.8.0_271/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/opt/module/hive-3.1.3/bin)
  3. Hive Session ID = a2b0c24d-7c45-4e95-9c86-1108b2606f3b
  4. ​
  5. Logging initialized using configuration in jar:file:/opt/module/hive-3.1.3/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true
  6. Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
  7. Hive Session ID = 5d3c533a-79e3-4690-b54c-1006d18d7aca
  8. hive (default)>
  9. ​
复制代码

简单使用Hive

查看数据库
  1. hive (default)> show databases;
  2. OK
  3. database_name
  4. default
  5. Time taken: 0.609 seconds, Fetched: 1 row(s)
复制代码
查看数据表
  1. hive (default)> show tables;
  2. OK
  3. tab_name
  4. Time taken: 0.051 seconds
复制代码

创建表
  1. hive (default)> CREATE TABLE IF NOT EXISTS test_students (
  2.    id INT,
  3.    name STRING,
  4.    age INT
  5. )
  6. ROW FORMAT DELIMITED FIELDS TERMINATED BY ',';
复制代码
插入数据
  1. hive (default)> INSERT INTO TABLE test_students VALUES (103, 'Bob', 21);
复制代码

查看表数据
  1. hive (default)> select * from test_students;
复制代码

退出Hive命令行

  1. hive (default)> quit;
  2. [liang@node2 conf]$
复制代码

  如有必要，可点击查看：配套视频教程

 完成！enjoy it！

免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！更多信息从访问主页：qidao123.com:ToB企服之家，中国第一个企服评测及商务社交产业平台。
继续阅读请点击广告

本帖子中包含更多资源

您需要 登录 才可以下载或查看，没有账号？立即注册

x
回复

使用道具 举报

0 个回复

倒序浏览
返回列表

快速回复

高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 or 立即注册

本版积分规则

九天猎人

论坛元老
这个人很懒什么都没写!

楼主热帖

标签云

国产数据库 集成商 AI 运维 CIO 存储 服务器
微信订阅号
微信服务号
微信客服
小程序
H5
关于我们 商务合作 网站地图
©Copyright 2022-2024 杭州祥升科技有限公司 版权所有 浙ICP备20004199号
快速回复 返回顶部 返回列表