Pulsar, a popular messaging middleware recently, wanted to learn. I found that many articles on the Internet introduce performance and compare Kafka, and there are few articles on practice! So I practiced with the official documents and wrote this article, which is estimated to be the first pulsar actual combat article in China. I hope it will be helpful to you!

SpringBoot e-commerce project mall (40k+star) address: https://github.com/macrozheng/mall

Introduction to Pulsar

Pulsar is a server-to-server messaging middleware, which has the advantages of multi tenancy and high performance. Pulsar was originally developed by Yahoo and is currently managed by the Apache Software Foundation. Pulsar adopts the publish subscribe design mode. Producer publishes messages to Topic, and Consumer subscribes to Topic and processes messages in Topic.

Pulsar has the following characteristics:

• A single instance of Pulsar supports native clusters.
• Extremely low release latency and end-to-end latency.
• It can be seamlessly extended to more than one million topics.
• Easy to use client API, supporting Java, Go, Python and C + +.
• Support multiple Topic subscription modes (exclusive subscription, shared subscription, failover subscription).
• Message delivery is guaranteed through the persistent message storage mechanism provided by Apache BookKeeper.

Pulsar installation

Installing Pulsar with Docker is the simplest. This time, we use Docker to install Pulsar.

• First, download the Docker image of Pulsar;

docker pull apachepulsar/pulsar:2.7.1

• After downloading, run the pulsar container. Port 8080 is used for http protocol access and port 6650 is used for pulsar protocol (Java, Python and other clients).

docker run --name pulsar \
-p 6650:6650 \
-p 8080:8080 \
--mount source=pulsardata,target=/pulsar/data \
--mount source=pulsarconf,target=/pulsar/conf \
-d apachepulsar/pulsar:2.7.1 \
bin/pulsar standalone

Pulsar visualization

Pulsar Manager is an official visualization tool that can visually manage multiple pulsars. Although it has few functions, it is basically sufficient and supports Docker deployment.

• Download the Docker image of pulsar manager;

docker pull apachepulsar/pulsar-manager:v0.2.0

• After downloading, run the pulsar manager container to access the Web page from port 9527;

docker run -it --name pulsar-manager\
    -p 9527:9527 -p 7750:7750 \
    -e SPRING_CONFIGURATION_FILE=/pulsar-manager/pulsar-manager/application.properties \
    -d apachepulsar/pulsar-manager:v0.2.0

• After running successfully, we can't access it at first. We need to create an administrator account. The account created here is admin:apachepulsar:

CSRF_TOKEN=$(curl http://localhost:7750/pulsar-manager/csrf-token)
curl \
    -H "X-XSRF-TOKEN: $CSRF_TOKEN" \
    -H "Cookie: XSRF-TOKEN=$CSRF_TOKEN;" \
    -H 'Content-Type: application/json' \
    -X PUT http://localhost:7750/pulsar-manager/users/superuser \
    -d '{"name": "admin", "password": "apachepulsar", "description": "test", "email": "username@test.org"}'

• After the creation is successful, log in through the login page and visit the address: http://192.168.5.78:9527

• After successful login, we need to configure an environment first, that is, configure the Pulsar service to be managed. The configured Service URL is: http://192.168.5.78:8080

• You can view the Tenant list;

• You can view the Topic list and manage topics;

• You can also view the details of Topic.

Pulsar is used in conjunction with SpringBoot

It is also very simple to use Pulsar with SpringBoot. We can use Pulsar's official Java SDK or a third-party SpringBoot Starter. Using Starter here is very simple!

• First, in POM Add Pulsar related dependencies to XML;

<!--SpringBoot integration Pulsar-->
<dependency>
    <groupId>io.github.majusko</groupId>
    <artifactId>pulsar-java-spring-boot-starter</artifactId>
    <version>1.0.4</version>
</dependency>

• Then in application Add the Service URL configuration of Pulsar in YML;

pulsar:
  service-url: pulsar://192.168.5.78:6650

• Then add the Java configuration of Pulsar, declare two topics, and determine the message type to be sent;

/**
 * Pulsar Configuration class
 * Created by macro on 2021/5/21.
 */
@Configuration
public class PulsarConfig {
    @Bean
    public ProducerFactory producerFactory() {
        return new ProducerFactory()
                .addProducer("bootTopic", MessageDto.class)
                .addProducer("stringTopic", String.class);
    }
}

• Create a Pulsar producer and send messages to the Topic. It can be found that Pulsar supports sending message objects directly;

/**
 * Pulsar Message producer
 * Created by macro on 2021/5/19.
 */
@Component
public class PulsarProducer {
    @Autowired
    private PulsarTemplate<MessageDto> template;

    public void send(MessageDto message){
        try {
            template.send("bootTopic",message);
        } catch (PulsarClientException e) {
            e.printStackTrace();
        }
    }
}

• Create Pulsar consumers, get and consume messages from topics, and you can also get message objects directly;

/**
 * Pulsar Message consumer
 * Created by macro on 2021/5/19.
 */
@Slf4j
@Component
public class PulsarRealConsumer {

    @PulsarConsumer(topic="bootTopic", clazz= MessageDto.class)
    public void consume(MessageDto message) {
        log.info("PulsarRealConsumer consume id:{},content:{}",message.getId(),message.getContent());
    }

}

• Add a test interface and call the producer to send a message;

/**
 * Pulsar functional testing
 * Created by macro on 2021/5/19.
 */
@Api(tags = "PulsarController", description = "Pulsar functional testing")
@Controller
@RequestMapping("/pulsar")
public class PulsarController {

    @Autowired
    private PulsarProducer pulsarProducer;

    @ApiOperation("send message")
    @RequestMapping(value = "/sendMessage", method = RequestMethod.POST)
    @ResponseBody
    public CommonResult sendMessage(@RequestBody MessageDto message) {
        pulsarProducer.send(message);
        return CommonResult.success(null);
    }
}

• The interface is invoked in Swagger for testing.

• After the call is successful, the console will enter the following information, indicating that the message has been successfully received and consumed.

2021-05-21 16:25:07.756  INFO 11472 --- [al-listener-3-1] c.m.m.tiny.component.PulsarRealConsumer  : PulsarRealConsumer consume id:1,content:SpringBoot Message!

summary

I wrote an article last time "Eagle, the Kafka graphical tool for the sky, must be recommended to you!" This paper introduces the basic use of Kafka, which is compared with pulsar. Pulsar's support for Docker is undoubtedly better, and the official documents are more complete. Compared with the graphical tools Pulsar Manager and Kafka Eagle, pulsar's graphical tools feel a little crude. At present, large Internet manufacturers such as Yahoo, Tencent and 360 are using pulsar. The performance and stability of pulsar should be very good!

reference material

Pulsar's official documents are complete and the style is good. You can get started by basically following the documents.

• Pulsar official documents: https://pulsar.apache.org/docs/en/standalone-docker/
• SpringBoot Starter official documentation: https://github.com/majusko/pulsar-java-spring-boot-starter

Project source code address

https://github.com/macrozheng/mall-learning/tree/master/mall-tiny-pulsar

This article GitHub https://github.com/macrozheng/mall-learning Already included, welcome to Star!