Introduction: RocketMQ Operator: a powerful tool for RocketMQ operation and maintenance control in the cloud native era:
Introduction: RocketMQ Operator has joined OperatorHub and officially entered the Operator community. Starting from practice and combining with cases, this paper will illustrate how to quickly build a RocketMQ cluster on Kubernetes through RocketMQ Operator, and provide some RocketMQ cluster management functions, including Broker expansion.
Author Liu Rui, Du Heng
Introduction: RocketMQ Operator has joined OperatorHub and officially entered the Operator community. Starting from practice and combining with cases, this paper will illustrate how to quickly build a RocketMQ cluster on Kubernetes through RocketMQ Operator, and provide some RocketMQ cluster management functions, including Broker expansion.
This paper is mainly divided into three parts:
- First, briefly introduce the relevant knowledge of RocketMQ Operator;
- Then, combined with a case, the user-defined resources and usage methods provided by RocketMQ Operator are introduced in detail;
- Finally, it introduces the current situation of the Operator community and looks forward to the next development direction of RocketMQ Operator.
Relevant background knowledge
1. RocketMQ
From 2012 to 2013, Alibaba middleware team independently developed and opened source the third generation distributed messaging engine RocketMQ. Its high-performance, low latency and anti accumulation characteristics stably supported Alibaba's double 11 trillion data peak business. Its cloud product Aliware MQ is widely used in micro services, stream computing, IoT, asynchronous decoupling Numerous working conditions such as data synchronization shine brightly.
In 2016, Alibaba donated RocketMQ to the Apache Software Foundation. The following year, RocketMQ successfully graduated from the foundation and became the top open source project of Apache. Together with Apache Hadoop and Apache Spark, RocketMQ brought good news to developers in the field of global distribution and big data. However, in today's era, how to achieve the most value of RocketMQ as a distributed operation and maintenance system is a great challenge.
RocketMQ supports multiple deployment modes. Take the basic dual master and dual slave architecture as an example, as shown in the figure below.
RocketMQ dual master dual slave architecture
This includes a total of 7 RocketMQ service instances: 3 name server instances, 2 master broker instances, and 2 slave broker instances.
The traditional deployment method requires manual or scripting to configure the environment and files on each node. In addition, with the increase of user services, there is a need for seamless expansion of clusters. The traditional way is that the operation and maintenance personnel visit different nodes and rely on the operation manual and script to complete the operation step by step, which consumes manpower and may be misoperated. Some companies may use some platforms and tools such as Ansible to help automate operation and maintenance. In addition, more and more companies begin to integrate and use the cloud native ecosystem based on Kubernetes.
Using native resources such as Deployment and stateful set provided by Kubernetes can well solve the management problem of stateless applications, but there are many limitations for stateful applications such as database and RocketMQ. For example, for RocketMQ, the capacity expansion is not only completed by pulling up the new instance Pod, but also needs to synchronously copy the Broker's status information, including Topic information and subscription relationship metadata. At the same time, the config parameters of the new Broker, including brokerName and NameServer IP List, should be configured correctly to make the newly expanded Broker available, These can't be done only by users writing stateful sets, modifying size or replicas, and then apply ing.
In fact, Kubernetes developers also found these problems, so they introduced the concept of user-defined resources and controllers, so that developers can directly call Kubernetes API with Go language and write user-defined resources and corresponding controller logic to solve the management problems of complex stateful applications, Such code components that provide custom resources related to specific applications are called operators. The Operator is written by experts with RocketMQ domain knowledge, which shields the professional knowledge in the application field and allows users to only care about and define the desired cluster end state. This is also the design philosophy of Kubernetes declarative API.
2. Kubernetes Operator
Based on Kubernetes, Operator extends Kubernetes API to create, configure and manage complex stateful applications, such as distributed databases. Based on the concept of user-defined Controller introduced since Kubernetes version 1.7, Operator is built on user-defined resources and controllers, and contains application specific domain knowledge. The key to implementing an Operator is the design of CRD (custom resource) and Controller.
From the internal perspective of Kubernetes, the Operator opens the door to a new world for cloud protobiochemistry. Custom resources allow developers to extend and add new functions, update existing functions, and automatically perform some management tasks. These custom controllers are like the native components of Kubernetes. Operators can directly use Kubernetes API for development, That is, they can create and change Pods / Services and expand the capacity of running applications according to the custom rules written by these controllers.
Quick start
This article uses RocketMQ Operator version 0.2.1 to show how to use RocketMQ Operator to quickly create and deploy a RocketMQ service cluster on Kubernetes.
- Prepare the K8s environment. You can use the K8s provided with docker desktop or minicube;
- Clone the rocketmq operator warehouse to your K8s node;
$ git clone https://github.com/apache/rocketmq-operator.git $ cd rocketmq-operator
- Run the script to install RocketMQ Operator;
$ ./install-operator.sh
- Check whether the RocketMQ Operator is installed successfully
$ kubectl get pods NAME READY STATUS RESTARTS AGE rocketmq-operator-564b5d75d-jllzk 1/1 Running 0 108s
When the installation is successful, the rocketmq operator pod is in the running state similar to the above example.
- Apply Broker and NameService custom resources to create a RocketMQ cluster;
Apply RocketMQ in RocketMQ operator / example_ v1alpha1_ RocketMQ_ cluster. Yaml file to quickly deploy a RocketMQ cluster. rocketmq_v1alpha1_rocketmq_cluster.yaml file contents are as follows:
apiVersion: rocketmq.apache.org/v1alpha1
kind: Broker
metadata:
# name of broker cluster
name: broker
spec:
# size is the number of the broker cluster, each broker cluster contains a master broker and [replicaPerGroup] replica brokers.
size: 1
# nameServers is the [ip:port] list of name service
nameServers: ""
# replicationMode is the broker replica sync mode, can be ASYNC or SYNC
replicationMode: ASYNC
# replicaPerGroup is the number of each broker cluster
replicaPerGroup: 1
# brokerImage is the customized docker image repo of the RocketMQ broker
brokerImage: apacherocketmq/rocketmq-broker:4.5.0-alpine
# imagePullPolicy is the image pull policy
imagePullPolicy: Always
# resources describes the compute resource requirements and limits
resources:
requests:
memory: "2048Mi"
cpu: "250m"
limits:
memory: "12288Mi"
cpu: "500m"
# allowRestart defines whether allow pod restart
allowRestart: true
# storageMode can be EmptyDir, HostPath, StorageClass
storageMode: EmptyDir
# hostPath is the local path to store data
hostPath: /data/rocketmq/broker
# scalePodName is broker-[broker group number]-master-0
scalePodName: broker-0-master-0
# volumeClaimTemplates defines the storageClass
volumeClaimTemplates:
- metadata:
name: broker-storage
spec:
accessModes:
- ReadWriteOnce
storageClassName: rocketmq-storage
resources:
requests:
storage: 8Gi
---
apiVersion: rocketmq.apache.org/v1alpha1
kind: NameService
metadata:
name: name-service
spec:
# size is the the name service instance number of the name service cluster
size: 1
# nameServiceImage is the customized docker image repo of the RocketMQ name service
nameServiceImage: apacherocketmq/rocketmq-nameserver:4.5.0-alpine
# imagePullPolicy is the image pull policy
imagePullPolicy: Always
# hostNetwork can be true or false
hostNetwork: true
# Set DNS policy for the pod.
# Defaults to "ClusterFirst".
# Valid values are 'ClusterFirstWithHostNet', 'ClusterFirst', 'Default' or 'None'.
# DNS parameters given in DNSConfig will be merged with the policy selected with DNSPolicy.
# To have DNS options set along with hostNetwork, you have to specify DNS policy
# explicitly to 'ClusterFirstWithHostNet'.
dnsPolicy: ClusterFirstWithHostNet
# resources describes the compute resource requirements and limits
resources:
requests:
memory: "512Mi"
cpu: "250m"
limits:
memory: "1024Mi"
cpu: "500m"
# storageMode can be EmptyDir, HostPath, StorageClass
storageMode: EmptyDir
# hostPath is the local path to store data
hostPath: /data/rocketmq/nameserver
# volumeClaimTemplates defines the storageClass
volumeClaimTemplates:
- metadata:
name: namesrv-storage
spec:
accessModes:
- ReadWriteOnce
storageClassName: rocketmq-storage
resources:
requests:
storage: 1Gi
Note that in this example, storageMode: EmptyDir means that EmptyDir is used for storage, and the data will be erased with the deletion of Pod. Therefore, this method is only used for development and testing. Generally, hostPath or storageclass is used for persistent storage of data. When using hostPath, you need to configure hostPath to declare the directory mounted on the host. When using storageclass, you need to configure volumeClaimTemplates to declare PVC templates. For details, please refer to the RocketMQ Operator documentation.
Apply the yaml file above and enter the command:
$ kubectl apply -f example/rocketmq_v1alpha1_rocketmq_cluster.yaml broker.rocketmq.apache.org/broker created nameservice.rocketmq.apache.org/name-service created
To view the status of the cluster Pod:
$ kubectl get pods -owide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES broker-0-master-0 1/1 Running 0 27s 10.1.2.27 docker-desktop <none> <none> broker-0-replica-1-0 1/1 Running 0 27s 10.1.2.28 docker-desktop <none> <none> name-service-0 1/1 Running 0 27s 192.168.65.3 docker-desktop <none> <none> rocketmq-operator-76b4b9f4db-x52mz 1/1 Running 0 3h25m 10.1.2.17 docker-desktop <none> <none>
Use the default rocketmq_ v1alpha1_ rocketmq_ cluster. According to the yaml file configuration, we can see that one name server service (name-service-0) and two broker services (one master and one slave) are pulled up in the cluster.
All right! Here, you have successfully deployed a RocketMQ service cluster through the user-defined resources provided by the Operator.
- Access the Pod in the RocketMQ cluster to verify whether the cluster can work normally;
Use RocketMQ's tools SH script run Producer example:
$ kubectl exec -it broker-0-master-0 bash bash-4.4# sh ./tools.sh org.apache.rocketmq.example.quickstart.Producer OpenJDK 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0 OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0 06:56:29.145 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J as the default logging framework SendResult [sendStatus=SEND_OK, msgId=0A0102CF007778308DB1206383920000, offsetMsgId=0A0102CF00002A9F0000000000000000, messageQueue=MessageQueue [topic=TopicTest, brokerName=broker-0, queueId=0], queueOffset=0] ... 06:56:51.120 [NettyClientSelector_1] INFO RocketmqRemoting - closeChannel: close the connection to remote address[10.1.2.207:10909] result: true bash-4.4#
Run Consumer example on another node:
$ kubectl exec -it name-service-0 bash
bash-4.4# sh ./tools.sh org.apache.rocketmq.example.quickstart.Consumer
OpenJDK 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
07:01:32.077 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J as the default logging framework
Consumer Started.
ConsumeMessageThread_1 Receive New Messages: [MessageExt [queueId=0, storeSize=273, queueOffset=19845, sysFlag=0, bornTimestamp=1596768410268, bornHost=/30.4.165.204:53450, storeTimestamp=1596768410282, storeHost=/100.81.180.84:10911, msgId=6451B45400002A9F000014F96A0D6C65, commitLogOffset=23061458676837, bodyCRC=532471758, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message{topic='TopicTest', flag=0, properties={MIN_OFFSET=19844, TRACE_ON=true, eagleTraceId=1e04a5cc15967684102641001d0db0, MAX_OFFSET=19848, MSG_REGION=DefaultRegion, CONSUME_START_TIME=1596783715858, UNIQ_KEY=1E04A5CC0DB0135FBAA421365A5F0000, WAIT=true, TAGS=TagA, eagleRpcId=9.1}, body=[72, 101, 108, 108, 111, 32, 77, 101, 116, 97, 81, 32, 48], transactionId='null'}]]
ConsumeMessageThread_4 Receive New Messages: [MessageExt [queueId=1, storeSize=273, queueOffset=19637, sysFlag=0, bornTimestamp=1596768410296, bornHost=/30.4.165.204:53450, storeTimestamp=1596768410298, storeHost=/100.81.180.84:10911, msgId=6451B45400002A9F000014F96A0D7141, commitLogOffset=23061458678081, bodyCRC=1757146968, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message{topic='TopicTest', flag=0, properties={MIN_OFFSET=19636, TRACE_ON=true, eagleTraceId=1e04a5cc15967684102961002d0db0, MAX_OFFSET=19638, MSG_REGION=DefaultRegion, CONSUME_START_TIME=1596783715858, UNIQ_KEY=1E04A5CC0DB0135FBAA421365AB80001, WAIT=true, TAGS=TagA, eagleRpcId=9.1}, body=[72, 101, 108, 108, 111, 32, 77, 101, 116, 97, 81, 32, 49], transactionId='null'}]]
...
- Delete the cluster and clean up the environment;
To clear the RocketMQ service cluster instance:
$ kubectl delete -f example/rocketmq_v1alpha1_rocketmq_cluster.yaml
Clear RocketMQ Operator:
$ ./purge-operator.sh
Install RocketMQ Operator according to the instructions on the official website of OperatorHub
- At operatorhub IO web search RocketMQ Operator;
Select the streaming & Messaging category and click RocketMQ Operator:
- Enter the RocketMQ Operator page and click the Install button;
- Follow the installation instructions of rocketmq and OLM;
Install OLM locally to use RocketMQ Operator
- Install and operate the OLM(Operator Lifecycle Manager) console locally;
Reference: OLM installation documentation.
- Start the UI interface console locally;
$ make run-console-local
- visit http://localhost:9000 View the console;
OperatorHub
- Search RocketMQ or click streaming & messaging in the All Items category to find and install RocketMQ Operator;
- After installing the RocketMQ Operator, you can find the RocketMQ Operator in the Installed Operators;
Installed Operators interface
RocketMQ Operator introduction interface
Create NameService custom resource through UI interface
You can create NameService and Broker instances under the specified Namespace in the UI, and browse and manage the created instances. We can also view the Pod status in the current K8s cluster through the command, for example:
$ kubectl get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE docker compose-78f95d4f8c-8fr5z 1/1 Running 0 32h docker compose-api-6ffb89dc58-nv9rh 1/1 Running 0 32h kube-system coredns-5644d7b6d9-hv6r5 1/1 Running 0 32h kube-system coredns-5644d7b6d9-mkqb6 1/1 Running 0 32h kube-system etcd-docker-desktop 1/1 Running 0 32h kube-system kube-apiserver-docker-desktop 1/1 Running 0 32h kube-system kube-controller-manager-docker-desktop 1/1 Running 1 32h kube-system kube-proxy-snmxh 1/1 Running 0 32h kube-system kube-scheduler-docker-desktop 1/1 Running 1 32h kube-system storage-provisioner 1/1 Running 1 32h kube-system vpnkit-controller 1/1 Running 0 32h marketplace broker-0-master-0 1/1 Running 0 5h3m marketplace broker-0-replica-1-0 1/1 Running 0 5h3m marketplace name-service-0 1/1 Running 0 5h3m marketplace marketplace-operator-69756457d8-42chk 1/1 Running 0 32h marketplace rocketmq-operator-0.2.1-c9fffb5f-cztcl 1/1 Running 0 32h marketplace rocketmq-operator-84c7bb4ddc-7rvqr 1/1 Running 0 32h marketplace upstream-community-operators-5b79db455f-7t47w 1/1 Running 1 32h olm catalog-operator-7b788c597d-gjz55 1/1 Running 0 32h olm olm-operator-946bd977f-dhszg 1/1 Running 0 32h olm operatorhubio-catalog-fvxp9 1/1 Running 0 32h olm packageserver-789c7b448b-7ss7m 1/1 Running 0 32h olm packageserver-789c7b448b-lfxrw 1/1 Running 0 32h
You can see that the corresponding name server and broker instances have also been successfully created in the namespace of the marketplace.
The above is based on the case of installing and using RocketMQ Operator by OperatorHub and OLM. We will continue to push and maintain the new version of RocketMQ Operator to the platform to facilitate users to obtain the latest updates or select the appropriate version of Operator.
community
RocketMQ Operator is an open source project of Apache community. It serves Alibaba SaaS class delivery of proprietary cloud, product private cloud environment deployment and other scenarios. At the same time, it also receives code submissions from open source contributors of iqiyi and other Internet companies. Welcome users to give feedback in the community project. Click the link below to leave your information, so that we can better improve RocketMQ Operator.
Link: https://github.com/apache/rocketmq-operator/issues/20
Currently, RocketMQ Operator v0 The PR of 2.1 has been merged into the community operators warehouse, and the RocketMQ Operator has entered the operator hub After IO, users can install and subscribe to RocketMQ Operator by using OLM(Operator Lifecycle Manager) to obtain continuous service support.
Future outlook
RocketMQ Operator v0.2.1 the supported functions mainly include: automatic creation of Name Server and Broker cluster, seamless expansion of Name Server cluster (automatically notify Broker cluster to update Name Server IP list), seamless expansion of Broker cluster under non sequential message (New Broker instance will synchronize metadata from source Broker Pod specified by Broker CRD, including Topic information and subscription information), And Topic migration.
Next, we hope to work with the community to further improve the RocketMQ Operator project, including gray publishing, full life cycle management of data, disaster recovery, backup and recovery, traffic and other indicator monitoring, automatic elastic capacity expansion and contraction, and finally realize the management of the full life cycle of RocketMQ services through the Operator.
Welcome to use RocketMQ Operator and put forward valuable suggestions.
https://developer.aliyun.com/article/783489?utm_content=g_1000262525
This article is the original content of Alibaba cloud and cannot be reproduced without permission.