EMQ server problem handling and analysis

Posted by just_nobody on Sun, 29 Mar 2020 20:09:36 +0200

Problem description

Restart EMQ by operating back and forth on a server. During cluster and exit cluster operations, the following errors are reported in the log.
EMQ uses rpm command to install, but it can't locate relevant problems.

2018-04-27 10:13:02.032 [error] <0.1274.0> Supervisor emqttd_sup had child emqttd_broker started with emqttd_broker:start_link() at <0.1325.0> exit with reason bad argument in call to ets:lookup(mqtt_route, <<"$SYS/brokers/emq@127.0.0.1/uptime">>) in emqttd_router:'-match/1-lc$^0/1-0-'/1 line 94 in context child_terminated
2018-04-27 10:13:02.032 [error] <0.1274.0> Supervisor emqttd_sup had child emqttd_broker started with emqttd_broker:start_link() at <0.1325.0> exit with reason reached_max_restart_intensity in context shutdown
2018-04-27 10:14:10.203 [error] <0.1325.0> gen_server emqttd_broker terminated with reason: bad argument in call to ets:lookup(mqtt_route, <<"$SYS/brokers/emq@127.0.0.1/uptime">>) in emqttd_router:'-match/1-lc$^0/1-0-'/1 line 94
2018-04-27 10:14:10.203 [error] <0.1325.0> CRASH REPORT Process emqttd_broker with 0 neighbours crashed with reason: bad argument in call to ets:lookup(mqtt_route, <<"$SYS/brokers/emq@127.0.0.1/uptime">>) in emqttd_router:'-match/1-lc$^0/1-0-'/1 line 94
2018-04-27 10:14:10.204 [error] <0.1274.0> Supervisor emqttd_sup had child emqttd_broker started with emqttd_broker:start_link() at <0.1325.0> exit with reason bad argument in call to ets:lookup(mqtt_route, <<"$SYS/brokers/emq@127.0.0.1/uptime">>) in emqttd_router:'-match/1-lc$^0/1-0-'/1 line 94 in context child_terminated
2018-04-27 10:14:10.204 [error] <0.1274.0> Supervisor emqttd_sup had child emqttd_broker started with emqttd_broker:start_link() at <0.1325.0> exit with reason reached_max_restart_intensity in context shutdown

Finally, uninstall the installed EMQ package, for example:

Uninstall EMQ

To view the installation package:

[root@home configs]# rpm -qa | grep emqttd
emqttd-2.3-1.el6.x86_64

Uninstall the installation package:
rpm -e emqttd-2.3-1.el6.x86_64

Install the specified package:
rpm -ivh emqttd-centos6.8-v2.3-1.el7.centos.x86_64.rpm
Several different versions have been replaced and the same error has been reported all the time. Only the rpm installation mode can be abandoned.

General package installation

Finally, use the general package installation method
Start in console debugging mode, check whether EMQ can start normally:
/usr/local/emqttd/bin/emqttd console

CTRL+c to close the console. Daemons mode start:
./bin/emqttd start
The startup error log will be output in the log / directory.

EMQ message server process status query:
./bin/emqttd_ctl status

In normal operation status, the query command returns:

$ ./bin/emqttd_ctl status
Node 'emqttd@127.0.0.1' is started
emqttd 2.0 is running

Stop server:
./bin/emqttd stop
Sometimes it's not easy to use. It's more effective to kill the process directly

The prompt for repeatedly and quickly shutting down and restarting the service mainly appears after other EMQ clusters

Protocol 'inet_tcp': register/listen error: eaddrinuse

The reason is that the port is always occupied

sudo netstat -apn | grep  1883

Just kill the relevant process

Topics: RPM supervisor CentOS sudo