Ambari Custom Service

Posted by justindublin on Sun, 09 Jun 2019 20:57:13 +0200

1. Ambari Basic Architecture


img016.jpg

Ambari Server reads the configuration files for Stack and Service.When creating services with Ambari, Ambari Server passes configuration files for Stack and Service and control scripts for the service life cycle to Ambari Agent.When the Agent gets the configuration file, it downloads and installs the package from the public source (Redhat, which uses the yum service).When the installation is complete, Ambari Server notifies the Agent to start the service.Ambari Server then periodically sends commands to the Agent to check the status of the service, which is reported to the Server and presented on Ambari's GUI.

2. Create Ambari Custom Service

#AmbariServer resource files in the / var/lib/ambari-server/resources directory
#cd to Ambari Stack directory (current version 2.6)
cd /var/lib/ambari-server/resources/stacks/HDP/2.6/services
#Create a custom Service directory (using the uppercase ServiceName command, for example, My Service)
mkdir MYSERVICE
cd MYSERVICE

1. Edit metainfo.xml

<?xml version="1.0"?>
<metainfo>
 <schemaVersion>2.0</schemaVersion>
 <services>
 <service>
 <!--  -->
 <!-- To write Service Name and Service information -->
 <name>MYSERVICE</name>
 <displayName>My Service</displayName>
 <comment>this is comment</comment>
 <version>1.0</version>
 <components>
 <component>
 <!-- To write Master assembly -->
 <name>MYMASTER</name>
 <displayName>My Master</displayName>
 <category>MASTER</category>
 <cardinality>1</cardinality>
 <commandScript>
 <script>scripts/master.py</script>
 <scriptType>PYTHON</scriptType>
 <timeout>5000</timeout>
 </commandScript>
 </component>
 <component>
 <!-- To write Slave assembly -->
 <name>MYSALVE</name>
 <displayName>My Slave</displayName>
 <category>SLAVE</category>
 <cardinality>1+</cardinality>
 <commandScript>
 <script>scripts/slave.py</script>
 <scriptType>PYTHON</scriptType>
 <timeout>5000</timeout>
 </commandScript>
 </component>
 </components>
 <osSpecifics>
 <osSpecific>
 <osFamily>any</osFamily>
 </osSpecific>
 </osSpecifics>
 </service>
 </services>
</metainfo>
  • Write multiple components under components.
  • category is a component role that supports Master, Slave, and Client
  • cardinality is the number of nodes, which can be 1, 1+, or a range of 1-2
  • CommScript is a script for component lifecycle callbacks

2. Write life cycle callback scripts for Master components

mkdir -p package/scripts
vim package/scripts/master.py
import sys, os
from resource_management import *
from resource_management.core.exceptions import ComponentIsNotRunning
from resource_management.core.environment import Environment
from resource_management.core.logger import Logger

class Master(Script):
    def install(self, env):
        print "Install My Master"

    def configure(self, env):
        print "Configure My Master"

    def start(self, env):
        print "Start My Master"

    def stop(self, env):
        print "Stop My Master"

    def status(self, env):
        print "Status..."

if __name__ == "__main__":
    Master().execute()

3. Write life cycle callback scripts for Slave components

package/scripts/slave.py
import sys, os
from resource_management import *
from resource_management.core.exceptions import ComponentIsNotRunning
from resource_management.core.environment import Environment
from resource_management.core.logger import Logger

class Slave(Script):
    def install(self, env):
        print "Install My Slave"

    def configure(self, env):
        print "Configure My Slave"

    def start(self, env):
        print "Start My Slave"

    def stop(self, env):
        print "Stop My Slave"

    def status(self, env):
        print "Status..."

if __name__ == "__main__":
    Slave().execute()

4. Restart AmbariServer

ambari-server restart

5. Join the My Service service you just added

  • On Ambari Web.Click Actions-> Add Service to add My Service



    WX20170728-150942@2x.png

3. Enrich Custom Service Functions

1. Add Service Check logic

In Service's metainfo.xml, the commandScript field is used to configure the service check script entry.If a service's metainfo.xml has this field, the command "Run Service Check" appears in the Service's Action list.

When a user clicks "Run Service Check" in the WEB, Ambari Server randomly notifies an Agent process on the machine where the service is located, and the Agent executes the Service check script.

<commandScript>
<script>scripts/master/my_check.py</script>
<scriptType>PYTHON</scriptType> <timeout>300</timeout>
</commandScript>

WX20170728-152404@2x.png

2. Increase Service Configuration Items

This requires adding the <configuration-dependencies>field to the Service's metainfo.xml.This field is used to associate a Service with a profile entry

Each <config-type>field specifies a configuration file.A Service can specify multiple profiles at the same time.However, all configuration files must be placed in the configuration directory of the Service.

<!-- with HDFS take as an example -->
<metainfo>
  <services>
    <service>
    <!-- ellipsis... -->
      <configuration-dependencies>
      <!-- Specify the profile below -->
        <config-type>core-site</config-type>
        <config-type>hdfs-site</config-type>
        <config-type>hadoop-env</config-type>
        <config-type>hadoop-policy</config-type>
        <config-type>hdfs-log4j</config-type>
        <config-type>ranger-hdfs-plugin-properties</config-type>
        <config-type>ssl-client</config-type>
        <config-type>ssl-server</config-type>
        <config-type>ranger-hdfs-audit</config-type>
        <config-type>ranger-hdfs-policymgr-ssl</config-type>
        <config-type>ranger-hdfs-security</config-type>
      </configuration-dependencies>
      <restartRequiredAfterRackChange>true</restartRequiredAfterRackChange>
    </service>
  </services>
</metainfo>
#Files in the configuration directory:
[root@node1 2.1.0.2.0]# ll configuration/
total 84
-rwxr-xr-x 1 admin root  7948 May 27 10:11 core-site.xml
-rwxr-xr-x 1 admin root 16723 May 27 10:11 hadoop-env.xml
-rwxr-xr-x 1 admin root  6201 May 27 10:11 hadoop-policy.xml
-rwxr-xr-x 1 admin root  8879 May 27 10:11 hdfs-log4j.xml
-rwxr-xr-x 1 admin root  8192 May 27 10:11 hdfs-logsearch-conf.xml
-rwxr-xr-x 1 admin root 19139 May 27 10:11 hdfs-site.xml
-rwxr-xr-x 1 admin root  2627 May 27 10:11 ssl-client.xml
-rwxr-xr-x 1 admin root  2959 May 27 10:11 ssl-server.xml

In the configuration file, you specify the properties of some key-value pairs and a description.When this service is added to Ambari's WEB, Ambari Server reads this information and displays it on the service's configuration page (Customize Service and config pages).By default, if a configuration item does not have a default value configured, the user must enter it.If an item is allowed to be empty, the attribute require-input="false" needs to be added to <property>.

3. Add custom Command

Using Rebalance HDFS as an example, add the following to Service's metainfo.xml
The scripts/namenode.py script is triggered when Rebalance HDFS is clicked

<customCommands>
  <customCommand>
    <name>REBALANCEHDFS</name>
    <background>true</background>
    <commandScript>
      <script>scripts/namenode.py</script>
      <scriptType>PYTHON</scriptType>
    </commandScript>
  </customCommand>
</customCommands>

WX20170728-153212@2x.png

Topics: xml SSL Python Hadoop