Let's talk about the last step of preparing to upgrade presto, because the pre deployed Presto machine itself deployed a CDH cluster (customer's home), and the machine was Centos6.x. according to the JDK on the Cloudrea official website, it can't move. Later, it was found that his JDK environment variable was written in / etc/profile, that is, he wanted to write a new PATH under the user with the priority of the user's own PATH - this was ok, of course, because he forgot to write $ A handful of bitter tears
The address of the old package first: link: https://pan.baidu.com/s/1U676CujPaD8kfjrkXgj0qQ
Extraction code: zvy0
1, Environmental preparation
1. Service role distribution
IP address | Duty |
---|---|
172.16.180.4 | coordinator |
172.16.180.17 | worker |
172.16.180.3 | worker |
172.16.180.3 | worker |
2. installation package
Need to get it at the beginning of Baidu Online
Two, deployment
1. Install JDK
Do not go into details. It should be noted that the official website explains that presto-server-0.229.tar.gz requires JDK to be at least jdk-6u115. After field testing in the production environment (jdk-8u45-linux-x64.gz), this version is not available. Later, it is OK to change this version (jdk-8u221-linux-x64.tar.gz). The first deployed environment variable is best written to / etc/profile. If there is j DK, the environment variable is written to the corresponding user's.bashrc
2. Install presto
1) Create a new Presto user and upload presto-server-0.229.tar.gz to / home/presto /
# useradd presto # chown -R presto. presto /home/presto # su – presto $ tar -zxvf presto-server-0.229.tar.gz
2) Modify configuration file (operation on 172.16.180.12)
# su – presto $ cd /home/presto/presto-server-0.229 $ mkdir etc $ cd etc/ $ touch node.properties $ touch jvm.config $ touch config.properties $ touch log.properties $ mkdir catalog $ cd catalog $ touch hive.properties $ mkdir hive ##Upload hive's configuration files core-site.xml and hdfs-site.xml to this file $ vi hive.properties connector.name=hive-hadoop2 hive.metastore.uri=thrift://172.16.180.4:9083 hive.config.resources=/home/presto/presto-server-0.229/etc/catalog/hive/core-site.xml,/home/presto/presto-server-0.229/etc/catalog/hive/hdfs-site.xml :wq Save exit
$ cd .. $ vi config.properties coordinator=true node-scheduler.include-coordinator=true http-server.http.port=19999 query.max-memory=8GB query.max-memory-per-node=2GB discovery-server.enabled=true discovery.uri=http://172.16.180.12:19999 :wq Save exit $ vi jvm.config -server -Xmx8G -XX:+UseG1GC -XX:G1HeapRegionSize=32M -XX:+UseGCOverheadLimit :wq Save exit $ vi log.properties com.facebook.presto=INFO #Output log level :wq Save exit $ vi node.properties node.environment=production node.id=cgn_presto_coordinator_node1 #Node unique identification node.data-dir=/home/presto/presto-server-0.229/data :wq Save exit $ exit # scp -r /home/presto/presto-server-0.229 root@172.16.180.17: /home/presto/ # scp -r /home/presto/presto-server-0.229 root@172.16.180.3: /home/presto/
Modify config.properties and node.properties on 172.16.180.17 and 172.16.180.3, where config.properties are the same as the following:
coordinator=false http-server.http.port=19999 query.max-memory=8GB query.max-memory-per-node=2GB discovery.uri=http://172.16.180.12:19999
On 172.16.180.17, the node.properties are as follows:
node.environment=production node.id=cgn_presto_coordinator_node2 node.data-dir=/home/presto/presto-server-0.229/data
On 172.16.180.3, the node.properties are as follows:
node.environment=production node.id=cgn_presto_coordinator_node3 node.data-dir=/home/presto/presto-server-0.229/data
- Start presto on 172.16.180.12, 172.16.180.17, 172.16.180.3
$ cd /home/presto/presto-server-0.229/bin $ ./launcher start
If the above information appears, the startup is successful.
Browser login: http://172.16.180.12:19999
4) Upload presto-cli-0.229-executable.jar to / home/presto/presto-server-0.229/bin directory (connect to hive data warehouse)
$ cd /home/presto/presto-server-0.229/bin $ mv presto-cli-0.229-executable.jar presto $ chmod +x presto $ vi hive.sh ./presto --server 172.16.180.12:19999 --catalog hive --schema test :wq Save exit $ ./hive.sh
Connection succeeded!
3, Error resolution
1) error 1
Solution: add this parameter to config.properties: query.max-total-memory-per-node=2GB(2GB value as reference)
2) If this error occurs: (injection constructor error, java.lang. Invalid memory configuration. The sum of the maximum query total memory (2147483648) and the heap head space (644245094) of each node cannot be greater than the available heap memory (2147483648)
Solution: modify the - Xmx2G parameter of etc/jvm.config file to adjust the size