DSS Deployment-12, DSS Installation

Posted by VanHagar on Sat, 05 Mar 2022 18:38:29 +0100

DataSphere Studio One-Click Installation Use Documentation

Supplement nginx configuration, test whether yum is useful, prepare configuration file

links uses Python by default, python2 is recommended

Install of the web. In sh, you need to install locally, then remove the installation of nginx and some scripts for firewall processing

Modify config for conf in the whole family. nginx port in SH is not 8088 and cannot conflict with yarn

1. Pre-use environmental preparation

a. Installation of basic software

Command tools required by Linkix (scripts automatically detect whether these commands are available before a formal installation, and if no automatic installation is attempted, users will need to manually install the following basic shell command tools if the installation fails):

telnet tar sed dos2unix mysql yum unzip expect
MySQL (5.5+)
JDK (above 1.8.0_141)
Python(2.7)
Nginx
Hadoop(2.7.2, other versions of Hadoop compile Linkis on their own), the installed machine must support execution of the HDFS dfs-ls/command
Hive(2.3.3, other versions of Hive need to compile Linkis on their own), and the machine installed must support the execution of the hive-e "show databases" command
Spark (supports all versions above 2.0), the installed machine must support the execution of the spark-sql-e "show databases" command

Tips:

If you are installing Hadoop for the first time, Hadoop can be deployed on a single machine for reference: Hadoop Single-machine Deployment ; Distributed deployment of Hadoop can be referred to as: Hadoop Distributed Deployment.

If you are installing Hive for the first time, refer to: Hive Quick Install Deployment.

If you are installing Spark for the first time, On Yarn mode is available for reference: Spark on Yarn deployment.

b. Create users

For example, the deployment user is a Hadoop account (it may not be a Hadoop user, but a Hadoop superuser is recommended for deployment, this is just an example)

Create deployment users on all machines that need to be deployed for installation

sudo useradd hadoop

Because Linkis services execute jobs by switching engines in sudo-u ${linux-user}, deploying users requires sudo privileges and is secret-free.

vim /etc/sudoers
hadoop  ALL=(ALL)       NOPASSWD: NOPASSWD: ALL

Ensure that servers deploying DSS and Linkis can properly execute commands such as hdfs, hive-e, and spark-sql-e. In a one-click installation script, components are checked.
If your Pyspark wants to have drawing capabilities, you also need to install the drawing module at all installation nodes. The commands are as follows:

download pip(Applicable to python3)
 curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
 
 download pip(Applicable to python2)
 curl https://bootstrap.pypa.io/pip/2.7/get-pip.py -o get-pip.py
 install pip
 python get-pip.py
 upgrade pip
 pip install --upgrade pip
 install matplotlib
 python -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple matplotlib
 pip install matplotlib

c. Installation preparation

Compile the installation package yourself or download it from the component release page:

Download installation package

Download and unzip the DSS & LINKIS one-click installation deployment package. The following is a hierarchical directory structure for installing deployment packages at one click:

├── dss_linkis # One-Click Deployment Home Directory
 ├── bin # For one-click installation and one-click DSS + Linkis startup
 ├── conf # Parameter Configuration Directory for One-Click Deployment
 ├── wedatasphere-dss-x.x.x-dist.tar.gz # DSS Background Installation Package
 ├── wedatasphere-dss-web-x.x.x-dist.zip # DSS Front End Installation Package
 ├── wedatasphere-linkis-x.x.x-dist.tar.gz # Linkis installation package

d. Modify Configuration

Open conf/config.sh, modify the configuration parameters as needed:

vim conf/config.sh

The parameters are described as follows:

Description, DSS_WEB_PORT ports need to be guaranteed not to conflict with YARN REST URL ports and can be changed to 8099 or other available ports

#################### Basic configuration for one-click installation deployment ####################

# Deployment user, default to current logon user, not necessarily recommended
# deployUser=hadoop

# Not necessarily do not recommend modification
# LINKIS_VERSION=1.0.2

### DSS Web, no modifications required for native installation
#DSS_NGINX_IP=127.0.0.1
#DSS_WEB_PORT=8099

# Not necessarily do not recommend modification
#DSS_VERSION=1.0.0

## Stack size for Java applications. 128M is recommended if the memory of the deployed machine is less than 8G; At least 256M is recommended when 16G is reached; If you want a very good user experience, it is recommended that you deploy a machine with at least 32G of memory.
export SERVER_HEAP_SIZE="128M"

############################################################
##################### Linkis Configuration Start #####################
########### Non-commented parameters must be configured, commented out parameters can be modified as needed ##########
############################################################

### DSS Workspace Directory
WORKSPACE_USER_ROOT_PATH=file:///tmp/linkis/
### User HDFS Root Path
HDFS_USER_ROOT_PATH=hdfs:///tmp/linkis
### Result set path: file or hdfs path
RESULT_SET_ROOT_PATH=hdfs:///tmp/linkis

### Path to store started engines and engine logs, must be local
ENGINECONN_ROOT_PATH=/appcom/tmp

#ENTRANCE_CONFIG_LOG_PATH=hdfs:///tmp/linkis/

### ==HADOOP profile path, must be configured==
HADOOP_CONF_DIR=/appcom/config/hadoop-config
### HIVE CONF DIR
HIVE_CONF_DIR=/appcom/config/hive-config
### SPARK CONF DIR
SPARK_CONF_DIR=/appcom/config/spark-config

# for install
#LINKIS_PUBLIC_MODULE=lib/linkis-commons/public-module


## YARN REST URL
YARN_RESTFUL_URL=http://127.0.0.1:8088

## Engine version configuration, default configuration if not configured
#SPARK_VERSION
#SPARK_VERSION=2.4.3
##HIVE_VERSION
#HIVE_VERSION=1.2.1
#PYTHON_VERSION=python2

## LDAP is for enterprise authorization, if you just want to have a try, ignore it.
#LDAP_URL=ldap://localhost:1389/
#LDAP_BASEDN=dc=webank,dc=com
#LDAP_USER_NAME_FORMAT=cn=%s@xxx.com,OU=xxx,DC=xxx,DC=com

# Microservices Service Registration Discovery Center
#LINKIS_EUREKA_INSTALL_IP=127.0.0.1         
#LINKIS_EUREKA_PORT=20303
#LINKIS_EUREKA_PREFER_IP=true

###  Gateway install information
#LINKIS_GATEWAY_PORT =127.0.0.1
#LINKIS_GATEWAY_PORT=9001

### ApplicationManager
#LINKIS_MANAGER_INSTALL_IP=127.0.0.1
#LINKIS_MANAGER_PORT=9101

### EngineManager
#LINKIS_ENGINECONNMANAGER_INSTALL_IP=127.0.0.1
#LINKIS_ENGINECONNMANAGER_PORT=9102

### EnginePluginServer
#LINKIS_ENGINECONN_PLUGIN_SERVER_INSTALL_IP=127.0.0.1
#LINKIS_ENGINECONN_PLUGIN_SERVER_PORT=9103

### LinkisEntrance
#LINKIS_ENTRANCE_INSTALL_IP=127.0.0.1
#LINKIS_ENTRANCE_PORT=9104

###  publicservice
#LINKIS_PUBLICSERVICE_INSTALL_IP=127.0.0.1
#LINKIS_PUBLICSERVICE_PORT=9105

### cs
#LINKIS_CS_INSTALL_IP=127.0.0.1
#LINKIS_CS_PORT=9108

##################### Linkis Configuration complete #####################

############################################################
####################### DSS Configuration Start #######################
########### Non-commented parameters must be configured, commented out parameters can be modified as needed ##########
############################################################

# Used to store temporary ZIP package files published to Schedulis
WDS_SCHEDULER_PATH=file:///appcom/tmp/wds/scheduler

### This service is used to provide dss-framework-project-server capability.
#DSS_FRAMEWORK_PROJECT_SERVER_INSTALL_IP=127.0.0.1
#DSS_FRAMEWORK_PROJECT_SERVER_PORT=9002

### This service is used to provide dss-framework-orchestrator-server capability.
#DSS_FRAMEWORK_ORCHESTRATOR_SERVER_INSTALL_IP=127.0.0.1
#DSS_FRAMEWORK_ORCHESTRATOR_SERVER_PORT=9003

### This service is used to provide dss-apiservice-server capability.
#DSS_APISERVICE_SERVER_INSTALL_IP=127.0.0.1
#DSS_APISERVICE_SERVER_PORT=9004

### This service is used to provide dss-workflow-server capability.
#DSS_WORKFLOW_SERVER_INSTALL_IP=127.0.0.1
#DSS_WORKFLOW_SERVER_PORT=9005

### dss-flow-Execution-Entrance
### This service is used to provide flow execution capability.
#DSS_FLOW_EXECUTION_SERVER_INSTALL_IP=127.0.0.1
#DSS_FLOW_EXECUTION_SERVER_PORT=9006

### This service is used to provide dss-datapipe-server capability.
#DSS_DATAPIPE_SERVER_INSTALL_IP=127.0.0.1
#DSS_DATAPIPE_SERVER_PORT=9008

##sendemail configuration, affects mail functionality only in DSS workflows
EMAIL_HOST=smtp.163.com
EMAIL_PORT=25
EMAIL_USERNAME=xxx@163.com
EMAIL_PASSWORD=xxxxx
EMAIL_PROTOCOL=smtp 
####################### DSS End of configuration #######################

The following addresses need to be configured

###HADOOP CONF DIR #/appcom/config/hadoop-config
HADOOP_CONF_DIR=/opt/modules/hadoop-2.7.2/etc/hadoop/
###HIVE CONF DIR #/appcom/config/hive-config
HIVE_CONF_DIR=/opt/modules/apache-hive-2.3.3/conf
###SPARK CONF DIR #/appcom/config/spark-config
SPARK_CONF_DIR=/opt/modules/spark-2.3.2-bin-hadoop2.7/conf

e. Modify database configuration

Make sure the configured database is accessible to the installation machine, otherwise DDL and DML import failures will occur.

vi conf/db.sh 
### Configure DSS Database
MYSQL_HOST=127.0.0.1
MYSQL_PORT=3306
MYSQL_DB=dss
MYSQL_USER=root
MYSQL_PASSWORD=asdf1234

## Hive metastore database configuration for Linkis to access Hive's metadata information
HIVE_HOST=127.0.0.1
HIVE_PORT=3306
HIVE_DB=hive
HIVE_USER=root
HIVE_PASSWORD=asdf1234

f. Modify wedatasphere-dss-web-1.0.1-dist configuration

Install. The following sections in SH need to be processed

centos7(){
    # Whether nginx is installed
    #sudo rpm -Uvh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm
    #Adjustment point: 1
    # Part of the nginx installed by yum is missing the'/etc/nginx/conf.d/folder', so nginx is manually installed, see Part 8
    #sudo yum install -y nginx 
    #echo "Nginx installed successfully"

    # Configure nginx
    dssConf

    # Solve 0.0.0.0:8888 problem
    yum -y install policycoreutils-python
    semanage port -a -t http_port_t -p tcp $dss_port

    # Open Front End Access Port
    #Adjustment point 2
    #[If used for testing, the firewall is turned off locally and does not need to be executed]
    #firewall-cmd --zone=public --add-port=$dss_port/tcp --permanent

    #Adjustment point 3
    #service iptables restart
    # [If used for testing, the firewall is turned off locally and does not need to be executed]
    #firewall-cmd --reload

    # start nginx
    systemctl restart nginx

    # Adjusting SELinux parameters
    sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
    # Provisional Effectiveness
    setenforce 0

}

2. Installation and Use

1. Execute the installation script:

sh bin/install.sh
# See what step to take
sh -v bin/install.sh

2. Installation steps

The installation script checks for integrated environment commands, and if not, install them as prompted, the following commands are required [installed when the environment is ready]

yum java mysql zip unzip expect telnet tar sed dos2unix nginx
When you install, the script asks if you need to initialize the database and import metadata, and Linkis and DSS ask.

The first installation must be selected Yes.

3. Successful installation:

View the log information printed by the console to see if the installation was successful.

If you have error information, you can see the specific reason for the error.

4. Start the service

(1) Start the service:

Start all services by executing the following command in the installation directory:

sh bin/start-all.sh

If the boot produces an error message, you can see the specific cause of the error. After startup, each micro-service carries out communication detection, and if there are any anomalies, it can help users locate the anomaly log and cause.

(2) Check to see if launch was successful

You can view the startup of each microservice in the Linkis & DSS background from the Eureka interface.

(3) Google Browser Access:

Please use Google Browser to access the following front-end addresses:

http://DSS_NGINX_IP:DSS_WEB_PORT The startup log prints this access address. The user name and password of the administrator are both deployment user names at logon time. If the deployment user is hadoop, the user name/password of the administrator is hadoop/hadoop.

(4) Stop service:

Execute the following command in the installation directory to stop all services:

sh bin/stop-all.sh

The following script determines if the stop succeeds or if it fails, the process can be terminated by kill ing

ps -ef|grep DSS
ps -ef|grep Linkis
ps -ef|grep eureka

(5) After successful installation, there are six DSS services and eight Linkis services

LINKIS-CG-ENGINECONNMANAGER
LINKIS-CG-ENGINEPLUGIN
LINKIS-CG-ENTRANCE
LINKIS-CG-LINKISMANAGER
LINKIS-MG-EUREKA
LINKIS-MG-GATEWAY
LINKIS-PS-CS
LINKIS-PS-PUBLICSERVICE

5. Install log install.sh

[hadoop@dss dssLinksFamilyMeals]$ sh bin/install.sh


########################################################################################
########################################################################################
Welcome to DSS & Linkis Deployment Service!
Suitable for Linkis and DSS first installation, please be sure the environment is ready.
########################################################################################
########################################################################################

It is recommended to use 5G memory.
Each service is set to 256M, with a minimum of 128M.
The default configuration is 256M.
If you need to modify it, please modify conf/config.sh

Are you sure you have installed the database?
If installed, enter 1, otherwise enter 0

Please input the choice:1

Do you need to check the installation environment?
Enter 1 if necessary, otherwise enter 0

Please input the choice:0



########################################################################
###################### Start to install Linkis #########################
########################################################################
Start to unzip linkis package.
Succeed to + Unzip linkis package to /opt/modules/dssLinksFamilyMeals/linkis-pre-install.
Start to replace linkis field value.
End to replace linkis field value.
<-----start to check used cmd---->
<-----end to check used cmd---->
Succeed to + check env
step1:load config
Succeed to + load config
Do you want to clear Linkis table information in the database?
 1: Do not execute table-building statements
 2: Dangerous! Clear all data and rebuild the tables
 other: exit

Please input the choice:2
You chose Rebuild the table
create hdfs  directory and local directory
Succeed to + create file:///tmp/linkis/ directory
Succeed to + create hdfs:///tmp/linkis directory
Succeed to + create hdfs:///tmp/linkis directory
rm: cannot remove '/opt/modules/dssLinksFamilyMeals/linkis-bak': No such file or directory
mv  /opt/modules/dssLinksFamilyMeals/linkis  /opt/modules/dssLinksFamilyMeals/linkis-bak
create dir LINKIS_HOME: /opt/modules/dssLinksFamilyMeals/linkis
Succeed to + Create the dir of /opt/modules/dssLinksFamilyMeals/linkis
Start to cp /opt/modules/dssLinksFamilyMeals/linkis-pre-install/linkis-package to /opt/modules/dssLinksFamilyMeals/linkis.
Succeed to + cp /opt/modules/dssLinksFamilyMeals/linkis-pre-install/linkis-package to /opt/modules/dssLinksFamilyMeals/linkis
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
Succeed to + source linkis_ddl.sql
mysql: [Warning] Using a password on the command line interface can be insecure.
+-----------------+
| @label_id := id |
+-----------------+
|               1 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|               2 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|               3 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|               4 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              11 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              12 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              13 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              14 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              15 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              16 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              17 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              18 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              19 |
+-----------------+
Succeed to + source linkis_dml.sql
Rebuild the table
Update config...
update conf /opt/modules/dssLinksFamilyMeals/linkis/conf/linkis.properties
update conf /opt/modules/dssLinksFamilyMeals/linkis/conf/linkis-mg-gateway.properties
update conf /opt/modules/dssLinksFamilyMeals/linkis/conf/linkis-ps-publicservice.properties
Congratulations! You have installed Linkis 1.0.3 successfully, please use sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-start-all.sh to start it!
Your default account password ishadoop/a61035488
Succeed to + install Linkis


#########################################################################
###################### Start to install DSS Service #####################
#########################################################################
Succeed to + Create the dir of /opt/modules/dssLinksFamilyMeals/dss-pre-install
Start to unzip dss server package.
Succeed to + Unzip dss server package to /opt/modules/dssLinksFamilyMeals/dss-pre-install
Start to replace dss field value.
End to replace dss field value.
java version "1.8.0_261"
Java(TM) SE Runtime Environment (build 1.8.0_261-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.261-b12, mixed mode)
Succeed to + execute java --version
step1:load config
Do you want to clear Dss table information in the database?
 1: Do not execute table-building statements
 2: Dangerous! Clear all data and rebuild the tables.

Please input the choice:2
You chose Rebuild the table
Simple installation mode
java version "1.8.0_261"
Java(TM) SE Runtime Environment (build 1.8.0_261-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.261-b12, mixed mode)
Succeed to + execute java --version
telnet check for your MYSQL, if you wait for a long time,may be your MYSQL does not prepared
MYSQL is OK.
mysql: [Warning] Using a password on the command line interface can be insecure.
Succeed to + source dss_ddl.sql
mysql: [Warning] Using a password on the command line interface can be insecure.
+---------------------------------+
| @dss_appconn_orchestratorId:=id |
+---------------------------------+
|                               2 |
+---------------------------------+
+-----------------------------+
| @dss_appconn_workflowId:=id |
+-----------------------------+
|                           3 |
+-----------------------------+
+---------------------------------+
| @dss_appconn_eventcheckerId:=id |
+---------------------------------+
|                               5 |
+---------------------------------+
+--------------------------------+
| @dss_appconn_datacheckerId:=id |
+--------------------------------+
|                              6 |
+--------------------------------+
Succeed to + source dss_dml_real.sql
Rebuild the table
step2:update config
rm: cannot remove '/opt/modules/dssLinksFamilyMeals/dss-bak': No such file or directory
mv  /opt/modules/dssLinksFamilyMeals/dss  /opt/modules/dssLinksFamilyMeals/dss-bak
create dir SERVER_HOME: /opt/modules/dssLinksFamilyMeals/dss
Succeed to + Create the dir of /opt/modules/dssLinksFamilyMeals/dss
Succeed to + subsitution /opt/modules/dssLinksFamilyMeals/dss/conf/dss-framework-project-server.properties
Succeed to + subsitution /opt/modules/dssLinksFamilyMeals/dss/conf/dss-framework-orchestrator-server.properties
Succeed to + subsitution /opt/modules/dssLinksFamilyMeals/dss/conf/dss-apiservice-server.properties
Succeed to + subsitution /opt/modules/dssLinksFamilyMeals/dss/conf/dss-datapipe-server.properties
Succeed to + subsitution /opt/modules/dssLinksFamilyMeals/dss/conf/dss-flow-execution-server.properties
Succeed to + subsitution /opt/modules/dssLinksFamilyMeals/dss/conf/dss-workflow-server.properties
Congratulations! You have installed DSS 1.0.1 successfully, please use sbin/dss-start-all.sh to start it!
Succeed to + install DSS Service


###########################################################################
###################### Start to install DSS & Linkis Web ##################
###########################################################################
Succeed to + Create the dir of /opt/modules/dssLinksFamilyMeals/web
Start to unzip dss web package.
Succeed to + Unzip dss web package to /opt/modules/dssLinksFamilyMeals/web
Start to replace dss web field value.
End to replace  dss web field value.
dss front-end deployment script
linux
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.bupt.edu.cn
 * epel: mirror.earthlink.iq
 * extras: mirrors.cn99.com
 * updates: mirrors.cn99.com
Package policycoreutils-python-2.5-34.el7.x86_64 already installed and latest version
Nothing to do
ValueError: Port tcp/8088 already defined
Succeed to + install DSS & Linkis Web

Eureka configuration path of Linkis: linkis/conf/application-linkis.yml
Eureka configuration path of DSS : dss/conf/application-dss.yml

Congratulations! You have installed DSS & Linkis successfully, please use bin/start-all.sh to start it!

6. Start the script start-all.sh


[hadoop@dss dssLinksFamilyMeals]$
[hadoop@dss dssLinksFamilyMeals]$ bin/start-all.sh
########################################################################
###################### Begin to start Linkis ###########################
########################################################################
We will start all linkis applications, it will take some time, please wait
<-------------------------------->
Begin to start mg-eureka
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh restart mg-eureka
server mg-eureka is not running
Start to check whether the mg-eureka is running
Start server, startup script:  /opt/modules/dssLinksFamilyMeals/linkis/sbin/ext/linkis-mg-eureka
nohup: redirecting stderr to stdout
server linkis-mg-eureka start succeeded!
Succeed to + End to start mg-eureka
<-------------------------------->
<-------------------------------->
Begin to start mg-gateway
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh restart mg-gateway
server mg-gateway is not running
Start to check whether the mg-gateway is running
Start server, startup script:  /opt/modules/dssLinksFamilyMeals/linkis/sbin/ext/linkis-mg-gateway
nohup: redirecting stderr to stdout
server linkis-mg-gateway start succeeded!
Succeed to + End to start mg-gateway
<-------------------------------->
<-------------------------------->
Begin to start ps-publicservice
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh restart ps-publicservice
server ps-publicservice is not running
Start to check whether the ps-publicservice is running
Start server, startup script:  /opt/modules/dssLinksFamilyMeals/linkis/sbin/ext/linkis-ps-publicservice
nohup: redirecting stderr to stdout
server linkis-ps-publicservice start succeeded!
Succeed to + End to start ps-publicservice
<-------------------------------->
<-------------------------------->
Begin to start cg-linkismanager
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh restart cg-linkismanager
server cg-linkismanager is not running
Start to check whether the cg-linkismanager is running
Start server, startup script:  /opt/modules/dssLinksFamilyMeals/linkis/sbin/ext/linkis-cg-linkismanager
nohup: redirecting stderr to stdout
server linkis-cg-linkismanager start succeeded!
Succeed to + End to start cg-linkismanager
<-------------------------------->
<-------------------------------->
Begin to start ps-cs
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh restart ps-cs
server ps-cs is not running
Start to check whether the ps-cs is running
Start server, startup script:  /opt/modules/dssLinksFamilyMeals/linkis/sbin/ext/linkis-ps-cs
nohup: redirecting stderr to stdout
server linkis-ps-cs start succeeded!
Succeed to + End to start ps-cs
<-------------------------------->
<-------------------------------->
Begin to start cg-entrance
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh restart cg-entrance
server cg-entrance is not running
Start to check whether the cg-entrance is running
Start server, startup script:  /opt/modules/dssLinksFamilyMeals/linkis/sbin/ext/linkis-cg-entrance
nohup: redirecting stderr to stdout
server linkis-cg-entrance start succeeded!
Succeed to + End to start cg-entrance
<-------------------------------->
<-------------------------------->
Begin to start cg-engineconnmanager
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh restart cg-engineconnmanager
server cg-engineconnmanager is not running
Start to check whether the cg-engineconnmanager is running
Start server, startup script:  /opt/modules/dssLinksFamilyMeals/linkis/sbin/ext/linkis-cg-engineconnmanager
nohup: redirecting stderr to stdout
server linkis-cg-engineconnmanager start succeeded!
Succeed to + End to start cg-engineconnmanager
<-------------------------------->
<-------------------------------->
Begin to start cg-engineplugin
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh restart cg-engineplugin
server cg-engineplugin is not running
Start to check whether the cg-engineplugin is running
Start server, startup script:  /opt/modules/dssLinksFamilyMeals/linkis/sbin/ext/linkis-cg-engineplugin
nohup: redirecting stderr to stdout
server linkis-cg-engineplugin start succeeded!
Succeed to + End to start cg-engineplugin
<-------------------------------->
start-all shell script executed completely
Start to check all linkis microservice
<-------------------------------->
Begin to check mg-eureka
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh status mg-eureka
12210
server mg-eureka is running.
<-------------------------------->
<-------------------------------->
Begin to check mg-gateway
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh status mg-gateway
12248
server mg-gateway is running.
<-------------------------------->
<-------------------------------->
Begin to check ps-publicservice
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh status ps-publicservice
12317
server ps-publicservice is running.
<-------------------------------->
<-------------------------------->
Begin to check ps-cs
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh status ps-cs
12465
server ps-cs is running.
<-------------------------------->
<-------------------------------->
Begin to check cg-linkismanager
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh status cg-linkismanager
12363
server cg-linkismanager is running.
<-------------------------------->
<-------------------------------->
Begin to check cg-entrance
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh status cg-entrance
12519
server cg-entrance is running.
<-------------------------------->
<-------------------------------->
Begin to check cg-engineconnmanager
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh status cg-engineconnmanager
12575
server cg-engineconnmanager is running.
<-------------------------------->
<-------------------------------->
Begin to check cg-engineplugin
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh status cg-engineplugin
12631
server cg-engineplugin is running.
<-------------------------------->
Linkis started successfully
Succeed to + start Linkis


########################################################################
###################### Begin to start DSS Service ######################
########################################################################
We will start all dss applications, it will take some time, please wait
<-------------------------------->
Begin to start dss-framework-project-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh restart dss-framework-project-server
server dss-framework-project-server is not running
Start to check whether the dss-framework-project-server is running
Start to start server, startup script:  /opt/modules/dssLinksFamilyMeals/dss/sbin/ext/dss-framework-project-server
nohup: redirecting stderr to stdout
server dss-framework-project-server start succeeded!
End to start dss-framework-project-server
<-------------------------------->
<-------------------------------->
Begin to start dss-framework-orchestrator-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh restart dss-framework-orchestrator-server
server dss-framework-orchestrator-server is not running
Start to check whether the dss-framework-orchestrator-server is running
Start to start server, startup script:  /opt/modules/dssLinksFamilyMeals/dss/sbin/ext/dss-framework-orchestrator-server
nohup: redirecting stderr to stdout
server dss-framework-orchestrator-server start succeeded!
End to start dss-framework-orchestrator-server
<-------------------------------->
<-------------------------------->
Begin to start dss-apiservice-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh restart dss-apiservice-server
server dss-apiservice-server is not running
Start to check whether the dss-apiservice-server is running
Start to start server, startup script:  /opt/modules/dssLinksFamilyMeals/dss/sbin/ext/dss-apiservice-server
nohup: redirecting stderr to stdout
server dss-apiservice-server start succeeded!
End to start dss-apiservice-server
<-------------------------------->
<-------------------------------->
Begin to start dss-datapipe-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh restart dss-datapipe-server
server dss-datapipe-server is not running
Start to check whether the dss-datapipe-server is running
Start to start server, startup script:  /opt/modules/dssLinksFamilyMeals/dss/sbin/ext/dss-datapipe-server
nohup: redirecting stderr to stdout
server dss-datapipe-server start succeeded!
End to start dss-datapipe-server
<-------------------------------->
<-------------------------------->
Begin to start dss-workflow-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh restart dss-workflow-server
server dss-workflow-server is not running
Start to check whether the dss-workflow-server is running
Start to start server, startup script:  /opt/modules/dssLinksFamilyMeals/dss/sbin/ext/dss-workflow-server
nohup: redirecting stderr to stdout
server dss-workflow-server start succeeded!
End to start dss-workflow-server
<-------------------------------->
<-------------------------------->
Begin to start dss-flow-execution-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh restart dss-flow-execution-server
server dss-flow-execution-server is not running
Start to check whether the dss-flow-execution-server is running
Start to start server, startup script:  /opt/modules/dssLinksFamilyMeals/dss/sbin/ext/dss-flow-execution-server
nohup: redirecting stderr to stdout
server dss-flow-execution-server start succeeded!
End to start dss-flow-execution-server
<-------------------------------->
<-------------------------------->
Begin to check dss-framework-project-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh status dss-framework-project-server
server dss-framework-project-server is running.
<-------------------------------->
<-------------------------------->
Begin to check dss-framework-orchestrator-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh status dss-framework-orchestrator-server
server dss-framework-orchestrator-server is running.
<-------------------------------->
<-------------------------------->
Begin to check dss-apiservice-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh status dss-apiservice-server
server dss-apiservice-server is running.
<-------------------------------->
<-------------------------------->
Begin to check dss-datapipe-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh status dss-datapipe-server
server dss-datapipe-server is running.
<-------------------------------->
<-------------------------------->
Begin to check dss-workflow-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh status dss-workflow-server
server dss-workflow-server is running.
<-------------------------------->
<-------------------------------->
Begin to check dss-flow-execution-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh status dss-flow-execution-server
server dss-flow-execution-server is running.
<-------------------------------->
Succeed to + start DSS Service


########################################################################
###################### Begin to start DSS & Linkis web ##########################
########################################################################
Succeed to + start DSS & Linkis Web

===============================================
There are eight micro services in Linkis: 
linkis-cg-engineconnmanager
linkis-cg-engineplugin
linkis-cg-entrance
linkis-cg-linkismanager
linkis-mg-eureka
linkis-mg-gateway
linkis-ps-cs
linkis-ps-publicservice
-----------------------------------------------
There are six micro services in DSS: 
dss-framework-project-server
dss-framework-orchestrator-server-dev
dss-workflow-server-dev
dss-flow-entrance
dss-datapipe-server
dss-apiservice-server
===============================================

Log path of Linkis: linkis/logs
Log path of DSS : dss/logs

You can check DSS & Linkis by acessing eureka URL: http://192.168.122.67:20303
You can acess DSS & Linkis Web by http://192.168.122.67:8088

[hadoop@dss dssLinksFamilyMeals]$

7. Log description

Under dss path
dss-apiservice-server.out
dss-datapipe-server.out
dss-flow-execution-server.out
dss-framework-orchestrator-server.out
dss-framework-project-server.out
dss-workflow-server.out

Under linkis path
linkis-cg-engineconnmanager.out
linkis-cg-engineplugin.out
linkis-cg-entrance.out
linkis-cg-linkismanager.out
linkis-mg-eureka.out
linkis-mg-gateway.out
linkis-ps-cs.out
linkis-ps-publicservice.out

3. Relevant Access Addresses

nginx http://192.168.122.67/

hadoop http://192.168.122.67:50070/dfshealth.html

spark http://192.168.122.67:8080/

spark http://192.168.122.67:8081/

eureka URL: http://192.168.122.67:20303

DSS & Linkis Web by http://192.168.122.67:8088

The login password can be found in the log, refer to the following information

Your default account password is hadoop/5f8a94fae
68.122.67:20303
You can acess DSS & Linkis Web by http://192.168.122.67:8088

[hadoop@dss dssLinksFamilyMeals]$

### 7. Log description
**dss Under Path**
dss-apiservice-server.out
dss-datapipe-server.out
dss-flow-execution-server.out
dss-framework-orchestrator-server.out
dss-framework-project-server.out
dss-workflow-server.out



**linkis Under Path**
linkis-cg-engineconnmanager.out
linkis-cg-engineplugin.out
linkis-cg-entrance.out
linkis-cg-linkismanager.out
linkis-mg-eureka.out
linkis-mg-gateway.out
linkis-ps-cs.out
linkis-ps-publicservice.out


## 3. Relevant Access Addresses

>  nginx	http://192.168.122.67/
>
>  hadoop	http://192.168.122.67:50070/dfshealth.html
>
>  spark	http://192.168.122.67:8080/
>
>  spark	http://192.168.122.67:8081/
>
>  
>
>  eureka URL: http://192.168.122.67:20303
>
>  DSS & Linkis Web by http://192.168.122.67:8088
>
>  The login password can be found in the log, refer to the following information
>
>  Your default account password is hadoop/5f8a94fae

Topics: AI IoT

Programmer Think