Install Apache Hive 3 on Ubuntu 18.04.5 | Step By Step | Part 4

Install Apache Hive:

Step 1: Download Apache Hive

https://hive.apache.org/downloads.html

Click on "Download a release now!" hyperlink

Click on https://mirrors.estointernet.in/apache/hive/ hyperlink

Click on hive-3.1.2 hyperlink

apache-hive-3.1.2-bin.tar.gz

https://mirrors.estointernet.in/apache/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz

wget https://mirrors.estointernet.in/apache/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz

apache-hive-3.1.2-bin.tar.gz

Copy apache-hive-3.1.2-bin.tar.gz file into /home/datamaking/softwares/ directory

cp apache-hive-3.1.2-bin.tar.gz /home/datamaking/softwares/

cd /home/datamaking/softwares

tar -xvzf apache-hive-3.1.2-bin.tar.gz

Step 2: Add the HIVE_HOME path in the bash file (.bashrc)

nano ~/.bashrc

export HIVE_HOME=/home/datamaking/softwares/apache-hive-3.1.2-bin
export HIVE_CONF_DIR=/home/datamaking/softwares/apache-hive-3.1.2-bin/conf
export PATH=$HIVE_HOME/bin:$PATH

source ~/.bashrc

Step 3: Create "warehouse" directory in HDFS

Use below HDFS commands to create /tmp and /user/hive/warehouse (aka hive.metastore.warehouse.dir) and set them chmod 777 before you can create a table in Hive.

jps

hdfs dfs -mkdir /tmp

hdfs dfs -chmod 777 /tmp

hdfs dfs -mkdir -p /user/hive/warehouse

hdfs dfs -chmod 777 /user/hive/warehouse

Step 4: Configure Hive with MySQL as Metastore

Install MySQL

Update repository index

sudo apt-get update

Install MySQL Server with apt-get

sudo apt-get install mysql-server

Verify installation (optional)

mysql --version

Set password for root user using below command

sudo mysql_secure_installation

Configuring Root to use MySQL shell

sudo mysql

SELECT user,authentication_string,plugin,host FROM mysql.user;

ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'datamaking';

FLUSH PRIVILEGES;

SELECT user,authentication_string,plugin,host FROM mysql.user;

exit;

sudo systemctl status mysql

sudo systemctl stop mysql

sudo systemctl start mysql

sudo systemctl status mysql

mysql -u root -p

CREATE DATABASE metastore;

USE metastore;

CREATE USER 'hive'@'%' IDENTIFIED BY 'datamaking';

GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%' WITH GRANT OPTION;

FLUSH PRIVILEGES;

exit;

Configure hive-site.xml in ${HIVE_HOME}/conf/

cd ${HIVE_HOME}/conf/

nano hive-site.xml

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at
 
       http://www.apache.org/licenses/LICENSE-2.0
 
   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->
<configuration>
<!-- WARNING!!! This file is auto generated for documentation purposes ONLY! -->
<!-- WARNING!!! Any changes you make to this file will be ignored by Hive.   -->
<!-- WARNING!!! You must make your changes in hive-site.xml instead.         -->
<!-- Hive Execution Parameters -->
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost/metastore?createDatabaseIfNotExist=true&amp;autoReconnect=true&amp;useSSL=false</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>datamaking</value>
</property>
<property>
<name>datanucleus.autoCreateSchema</name>
<value>true</value>
</property>
<property>
<name>datanucleus.fixedDatastore</name>
<value>true</value>
</property>
<property>
<name>datanucleus.autoCreateTables</name>
<value>True</value>
</property>
<property>
<name>hive.server2.transport.mode</name>
<value>http</value>
</property>
<property>
<name> hive.server2.thrift.http.port </name>
<value>10000</value>
</property>
<property>
<name>hive.server2.http.endpoint</name>
<value>cliservice</value>
</property>
</configuration>


Installing the MySQL JDBC Driver

Download the MySQL JDBC driver from http://www.mysql.com/downloads/connector/j/5.1.html (in .tar.gz format).

wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.49.tar.gz

tar -xvzf mysql-connector-java-5.1.49.tar.gz

Copy the JDBC driver, renamed, to /usr/share/java/.

If the target directory does not yet exist, create it. For example:

sudo mkdir -p /usr/share/java/

cd mysql-connector-java-5.1.49

sudo cp mysql-connector-java-5.1.49.jar /usr/share/java/mysql-connector-java.jar

sudo cp mysql-connector-java-5.1.49.jar /home/datamaking/softwares/apache-hive-3.1.2-bin/lib/mysql-connector-java.jar

Create Hive schema (MySQL)

${HIVE_HOME}/bin/schematool -initSchema -dbType mysql

If you get the below error,

Exception in thread "main" java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V


To fix the issue, find guava installed with hadoop and hive

find /home/datamaking/softwares/hadoop-3.2.1/ -type f -name "guava-*.jar"

/home/datamaking/softwares/hadoop-3.2.1/share/hadoop/hdfs/lib/guava-27.0-jre.jar

find /home/datamaking/softwares/apache-hive-3.1.2-bin/ -type f -name "guava-*.jar"

mv /home/datamaking/softwares/apache-hive-3.1.2-bin/lib/guava-19.0.jar /home/datamaking/softwares/apache-hive-3.1.2-bin/lib/guava-19.0.jar.bak

cp /home/datamaking/softwares/hadoop-3.2.1/share/hadoop/hdfs/lib/guava-27.0-jre.jar /home/datamaking/softwares/apache-hive-3.1.2-bin/lib/

${HIVE_HOME}/bin/schematool -initSchema -dbType mysql

datamaking@datamakingvm:~/Downloads/mysql-connector-java-5.1.49$ ${HIVE_HOME}/bin/schematool -initSchema -dbType mysql
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/datamaking/softwares/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/datamaking/softwares/hadoop-3.2.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:mysql://localhost/metastore?createDatabaseIfNotExist=true&autoReconnect=true&useSSL=false
Metastore Connection Driver : com.mysql.jdbc.Driver
Metastore connection User: hive
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.mysql.sql
.
.
.
.
.
Initialization script completed
schemaTool completed
datamaking@datamakingvm:~/Downloads/mysql-connector-java-5.1.49$

Upgrade metastore database in mysql (Optional)

cd ${HIVE_HOME}/scripts/metastore/upgrade/mysql/

mysql -u root -p

USE metastore;

source hive-schema-3.1.0.mysql.sql;

exit;

nohup hive --service metastore &

nohup hive --service hiveserver2 &

Verify port no

netstat -an | grep 9083

sudo apt install net-tools

open Hive CLI

hive

show databases;

show tables;

Hiveserver2 Web UI:

http://localhost:10002/

Happy Learning !!!

Post a Comment

2 Comments

  1. Call Girls contact number in Agra is an excellent place to create your looking for Call Girls contact number in Agra and need to say you really have found Call Girls contact number in Agra you have looking for about best Call Girls contact number in Agra.

    ReplyDelete
  2. You have now land on the true website of model Call Girls contact number in Agra at our expert female Call Girls contact number in Agra. Real mobile number Call Girls contact number in Ahmedabad and discover pleasant Call Girls contact number in Ahmedabad for yourself if you are looking for celebrity escorts in Ahmedabad.

    ReplyDelete
Emoji
(y)
:)
:(
hihi
:-)
:D
=D
:-d
;(
;-(
@-)
:P
:o
:>)
(o)
:p
(p)
:-s
(m)
8-)
:-t
:-b
b-(
:-#
=p~
x-)
(k)