How to clear ALL retained mqtt messages from Mosquitto?

Question

I've seen the mosquitto_pub -h [server] -r -n -t [XYZ] syntax for clearing out one off messages. My problem is the device developers have posted a lot of garbage messages.

I have a Java/Paho code base that I'd like to modify to do this automatically as needed, but I can't seem to publish a zero byte message. I tried

client.publish(topic,null);

...but that didn't seem to work.

Any suggestions on how to delete everything, en mass?

Solution 1

There are 2 options for this using the paho client code depending on which of the 2 publish methods you use.

MqttMessage msg = new MqttMessage(new byte[0]);
msg.setRetained(true);
client.publish(topic, msg);

or

client.publish(topic, new byte[0],0,true);

The other option would be to stop mosquitto and delete the persistence file and restart

Solution 2

Here is how to do it properly with a shell script.

#!/bin/sh
echo "cleaning " $1 " :: usage: cleanmqtt <host>"
mosquitto_sub -h $1 -t "#" -v --retained-only | while read line; do mosquitto_pub -h $1 -t "${line% *}" -r -n; done

Just put it in a file called somthing like

finally_a_working_way_to_remove_all_those_annoying_messages.sh

Then run

sh finally_a_working_way_to_remove_all_those_annoying_messages.sh localhost

This solution is quite crude. You cant specify what to delete or anything. You may have to abort with ctrl-c after you can assume that it has received all the messages.

Solution 3

Mosquitto client provides --remove-retained option:

mosquitto_sub -h $host --remove-retained -t '#' -E

Tunning -t can handle specific topics to be cleared.

Solution 4

This should work:

client.publish(topic, new byte[]{}, 0, true);

Also, you may be interested in this script from Eclipse Paho Python, to clear a given topic hierarchy. You may want to implement a similar behavior in Java, but it looks like you may be looking for a one-off solution, so maybe just use the Python script :)

Solution 5

Since I don't have enough points to comment, running

#!/bin/sh
echo "cleaning " $1 " :: usage: cleanmqtt <host>"
mosquitto_sub -h $1 -t "#" -v | while read line; do mosquitto_pub -h $1 -t "${line% *}" -r -n; done

could cause an infinite loop due to pub/sub delays. Adding --retained-only to mosquitto_sub seems to remove the infinite loop.

View more solutions

Comments

[kartben]

Actually, first version will not work, as the message needs to be published with the retained flag set.

[JohnL]

Deleting the persistence file turned out to be the option I really needed.

[Gussoh]

The python script doesnt seem to work with non-ascii topics or payloads. The shell script I just posted seems to work better.

[Tobias Holm]

This is how I deleted the persistence file in Ubuntu 16.04: sudo service mosquitto stop sudo rm /var/lib/mosquitto/mosquitto.db sudo service mosquitto start You can check in your mosquitto.conf where your persistence_file is located.

[hardillb]

It is important to understand that deleting the persistence file also wipes out all queued messages and any persistent subscriptions. It is better to clear individual topics with publishing a null message in a production environment

[Trevor]

"this script" link is now 404

[moster67]

Could you show how to modify the script to include username and password? When I run it, I get "Connection Refused: not authorised". Thanks.

[Gussoh]

Sure, add -u username -P password in the line to both the mosquitto_sub and mosquitto_pub

[JackWu]

@TobiasHolm Sovle my problem! Thanks!!

[kellzerIrl]

If your topic has spaces in it, you can change the command to: mosquitto_sub -h $1 -t "#" -v | while read line; do mosquitto_pub -h $1 -t "${line% *}" -r -n; done The change is to remove the _ after read line and change $line to ${line% *}

[Gussoh]

Cool Robert, I changed to your suggestion

[hardillb]

That just stops retained messages being stored, across restarts, it doesn't actually clear them (in a running broker)

[Manav Akela]

Agreed, for that remove/re-create db file (manually or by such code:file_operation) which is stored in persistence_location (or in my case /var/lib/mosquitto/mosquitto.db)

[Gussoh]

Nice! I added it. This also solves the ctrl-c issue. There is also a --remove-retained but I can't seem to understand how that works.

[Nik]

This is the most efficient way, however, when using -E the client exits immediately after Client X received SUBACK i.e. after subscribing, but before receiving any messages, so actual removal doesn't happen. The only reliable way to make the client disconnect and exit that I found is using -W 1 flag, i.e. process messages for 1 sec then exit.