MQTT reconnection issues

Hello everyone!

I’m having some troubles when trying to set up a reconnect on the mqtt client. So far, the relevant code after some tests is:

def breconnect_cb(client):
    global mqttReconnects
    print("[MQTT] > Reconnection failed."), 
    print("Trying to reconnect (Attempt %d)." % mqttReconnects)
    mqttReconnects += 1
    sleep(1000)

def aconnect_cb(client):
    global mqttReconnects
    print("[MQTT] > Connection successful.")
    mqttReconnects = 1


    # Start connection to the broker 
    try:
        client.connect("192.168.0.142", 300, port = 1883, ssl_ctx = None, 
        breconnect_cb = breconnect_cb, aconnect_cb = aconnect_cb,
        sock_keepalive = None)
        break
    except Exception as e:
        print("[MQTT] > Unable to connect. Retrying.")
        sleep(2000)

After this, some subscriptions and a while loop to start publishing data to the broker. The problem comes when the broker is initially connected, but later it is stopped or the internet connection is down. My next steps were:

  • I set a breconnect callback with a sleep(1000) and a reconnection tries counter, to see what was happening (prior to this the VM was reseting instantly anter mqtt service shup down). I found that after 26 retries, the MQTTConnectionError was raised and the thread exited:

Thread 6 exited wi > Data error:th exc eptioMQTTConnectionError @[00D2:0038:0000:0000:0000:0000:0000:0000]n
MQTTConnectionError @[00C0:0053:00BF:01CC:00C0:00CA:00C7:010D]
ets Jun 8 2016 00:22:57

  • Checking the mqtt module I found the reconnect method: It has a reconnection_max_retry set up to 25 in a decrement loop, to raise a MQTTConnectionError exception if <0.

  • I tried to handle the MQTTConnectionError, trying to avoid the VM crash, but I have not been successful at all. The try-except enclosing the client.connect does not get it, and the client.loop also doesn’t.

  • I disabled the VM reset on unhandled exceptions. All the threads were running after the MQTTConnectionError, even the one that launches the connect and the loop (It keeps cicling in the publish loop).

I’m now stuck here. I think the solution is to catch the exception raised on the thread created by the loop, but with my current knowledge I don’t know how to do that (I’m very new in Python). After that maybe I can terminate and relaunch the MQTT thread (maybe raising an exception and relaunching it in the main?). Other solution may be just to modify the mqtt.py in any way, but I’m reluctant to do that.

Sorry in advance for my lack of skill trying to explain my problems! I will gladly answer any questions raised asap.

Thank you very much in advance!

Hi @NicolasDiaz, we are slowly deprecating MQTT library in favour of the new LW MQTT library that allows users to have full control over reconnection and failure handling.

You can check an example here: https://docs.zerynth.com/latest/official/lib.zerynth.lwmqtt/examples/examples.html#lwmqtt-at-mosquitto-org.

(Full docs at https://docs.zerynth.com/latest/official/lib.zerynth.lwmqtt/docs/official_lib.zerynth.lwmqtt_mqtt.html#client-class)

The syntax is very similar, you probably will just need to modify your import statement and register a loop_failure handler.

Let me know if you find any problem,
Cheers,
Antonio

1 Like

Hi @a.pitasi, thanks for the fast reply!

I had a look at the lwmqtt library, but I started using the mqtt one so I discarded it. But if I have more control with the lwmqtt, there is nothing more to say: I will jump into it asap.

Thanks again, I will report how it goes :smiley:

Hi! I’m working in the transition to the lwmqtt library, but I’m stuck with a compilation error that cannot be solved no matter what I try:

The mqqt handling is done in a thread invoked by the main program, and it’s definition is in a separate .py file. Code is the following one:

##############################################################################
# LW MQTT client definition
##############################################################################

def threadLWMQTT(id):
    # Import lwmqtt
    from lwmqtt import mqtt
    # Import globals.py to use threading locks and events variables
    import globals
    
    # Subscription callback
    def print_payload_cb(mqttClient, payload, topic):
        # Print payload published on channel
        print("> Received from", topic, ":", payload)
    
    # After connection callback
    def aconnect(mqttClient):
        # Subscribe to channel
        mqttClient.subscribe("temp/samples", print_payload_cb)
    
    # Called when packet handle loop fails    
    def loop_failure(mqttClient):
        # Reconnection logic
        while True:
            try:
                print("[MQTT] > Reconnecting...")
                mqttClient.reconnect()
                break
            except Exception as e:
                print("[MQTT] > Reconnection failed.")
            sleep(1000)
        return mqtt.RECOVERED
    
    # Set the mqtt id to "deBecito"
    mqttClient = mqtt.Client("deBecito")
    
    # Wait for wifi connection to happen
    globals.wifiEvent.wait()
    
    print("[MQTT] > Attempting connection to MQTT broker.")        
    
    # Start connection to the broker
    while True:
        try:
            mqttClient.connect("192.168.0.142", 
            60, port = 1883, ssl_ctx = None, 
            breconnect_cb = None, 
            aconnect_cb = aconnect, 
            loop_failure = loop_failure,
            start_loop = True)
            break
        except Exception as e:
            print("[MQTT] > Unable to connect. Retrying.")
        sleep(5000)
        
    # Endless loop for sending samples to topic
    x = 0
    while True:
        if mqttClient.connected():
            x += 1
            print("> Publishing.")
            mqttClient.publish("temp/samples", str(x))
        sleep(1000)

The thing is I’m getting the following error:

[fatal] Unexpected exception
Traceback (most recent call last):
File “C:\Users\nicolas.diaz\zerynth2\dist\r2.3.2\ztc\compiler\astwalker.py”, line 120, in visit_Import mod = self.hooks.getModuleCode(alias.name)
File “C:\Users\nicolas.diaz\zerynth2\dist\r2.3.2\ztc\compiler\compiler.py”, line 154, in getModuleCode return self.modules[name]
KeyError: ‘lwmqttDef’
During handling of the above exception, another exception occurred:
Traceback (most recent call last): > File “C:\Users\nicolas.diaz\zerynth2\dist\r2.3.2\ztc\compiler\compilercmd.py”, line 80, in _zcompile binary, reprs = compiler.compile()
File “C:\Users\nicolas.diaz\zerynth2\dist\r2.3.2\ztc\compiler\compiler.py”, line 515, in compile self.compileModule(self.mainfile)
File “C:\Users\nicolas.diaz\zerynth2\dist\r2.3.2\ztc\compiler\compiler.py”, line 470, in compileModule mc.visit(tree)
File “C:\Users\nicolas.diaz\zerynth2\sys\python\lib\ast.py”, line 245, in visit return visitor(node)
File “C:\Users\nicolas.diaz\zerynth2\dist\r2.3.2\ztc\compiler\astwalker.py”, line 103, in visit_Module self.code.addCode(self.visit(stmt))
File “C:\Users\nicolas.diaz\zerynth2\sys\python\lib\ast.py”, line 245, in visit return visitor(node)
File “C:\Users\nicolas.diaz\zerynth2\dist\r2.3.2\ztc\compiler\astwalker.py”, line 122, in visit_Import self.hooks.importHook(alias.name,node.lineno,self.filename)
File “C:\Users\nicolas.diaz\zerynth2\dist\r2.3.2\ztc\compiler\compiler.py”, line 170, in importHook self.compileModule(name,line,filename)
File “C:\Users\nicolas.diaz\zerynth2\dist\r2.3.2\ztc\compiler\compiler.py”, line 470, in compileModule mc.visit(tree)
File “C:\Users\nicolas.diaz\zerynth2\sys\python\lib\ast.py”, line 245, in visit return visitor(node)
File “C:\Users\nicolas.diaz\zerynth2\dist\r2.3.2\ztc\compiler\astwalker.py”, line 107, in visit_Module self.generateCodeObjs()
File “C:\Users\nicolas.diaz\zerynth2\dist\r2.3.2\ztc\compiler\astwalker.py”, line 853, in generateCodeObjs fcode.addCode(self.genCodeList(node.body))
File “C:\Users\nicolas.diaz\zerynth2\dist\r2.3.2\ztc\compiler\astwalker.py”, line 92, in genCodeList listcode.append(self.visit(cc)) > File “C:\Users\nicolas.diaz\zerynth2\sys\python\lib\ast.py”, line 245, in visit return visitor(node)
File “C:\Users\nicolas.diaz\zerynth2\dist\r2.3.2\ztc\compiler\astwalker.py”, line 810, in visit_FunctionDef code.addCode(OpCode.BUILD_TUPLE(len(cellvars)))
UnboundLocalError: local variable ‘cellvars’ referenced before assignment

I suspect the problem is in the function definitions, but no matter what I try I cannot solve it :sob:

Any help would be very appreciated! Thank you very much in advance :smiley:

In your original code, why do you have a comma at the end of the 2nd line of the reconnect method?

You have 2 errors, a key error, and a local error.

Deal with the local error first and it might help you figure out the key error.

Hi! I’m afraid that was an error because of learning with Python 2 and trying to code with Python 3. In Python 2 you can print in a single output line by adding a comma to the end of the code line, but that doesn’t work in Python 3 (there is another way to do this).
, Python 2:

print (“Print this”),
print (“in a single line”)

and in Python 3:

print (“Print this”, end =’ ')
print (“in a single line”)

are equivalents.

I’m having a hard time trying to make this work. I’ve wrapped all the variables and objects to control the threads as class attributes, and it “works”. But some troubles seems to happen when I touch the code: threads crashses and unexpected errors from the cores.

I will recap everything that bugs me and post it here as soon as I can, and hopefully get some kind help!