Rebooting issue

Hi,

I’m struggling to solve an issue in my Zerynth project.

My custom board(based on esp32 wroom module) sends the readings from the sensors to the server, but the board sometimes occurs rebooting while sending data to the server.

image

The firmware includes FOTA function. If I disabled FOTA function, then reboot doesn’t occur.
I tried to figure out why rebooting happened, but I couldn’t find any error in the firmware.

I hope you to help me solve this problem as soon as possible.

Thanks in advance.

Anatoli.

Hi,

Did you have a chance to look at my previous topic?

I’m still having a hard time because of the reboot.

I’m sure it’s because of Zerynth’s limitation regarding the core use.

Is it possible to get ESP32 dual core working with Zerynth? Zerynth VM still runs only on the first core?

Would it be a good alternative if I use another processor with higher performance supported by Zerynth? (for example, ATSAM21D, Coterx-M4, etc)

I also wonder if I can use the current firmware as it is when using another processor. How portable the code was?

Actually, there is no part of the code related to the processor except FOTA feature.

At this moment, your advice and support are very urgent for me.

Looking forward to hearing something from you soon.

Anatoli.

Hi @Anatoli_Juny
In the console message you posted, before the uC resets, there is: esp_wifi_connect 952 wifi not started, Are you sure this is not related to the wifi network and the device disconnecting and reconnecting again?

If you’re sure this error is due to the FOTA, Let’s have a look at the FOTA code.
Also, Have a look at the FOTA in Zerynth advanced device manager, Much easier and more efficient to work with.

In Zerynth, the code is independent of the hardware, the same code should work on other supported boards. (Expect hardware drivers of boards, like ESP32wifi needs an ESP32).

Hi Karim,
Thanks for your reply.

I can’t see this error (esp_wifi_connect 952 wifi not started) in the serial monitor when the board first starts.

As for the reboot, I’m sure it’s not related to the wifi network and device disconnecting and reconnecting again. The wifi network here is good and It occurs when data is sent to the server.
There are a few APIs to send the data to the server. This reboot occurs when two APIs are called among them. The very two APIs are intended to send 2 array data with 100 elements are sent to the server at a time. Other APIs send only a few values to the server at a time. They work properly with FOTA function without any reboot.

I’m also sure it’s not related to FOTA engine. I’ve used the same FOTA code that is provided by Zerynth and it works well.

As you know, Zerynth VM runs only on one core. The flash size for zerynth VM is 1Mbyte. So I think this is because of the Zerynth’s limitation regarding the core use.
image

This is also why I’m looking for another processor with higher performance than ESP32.
Would you recommend another processor that are allocated more flash for Zerynth VM and supported by Zerynth. It would be better If it had the wifi feature. If not, it will be fine.

Thanks.
Anatoli.

Hi Karim,

I hope you’re doing well. I sincerely waited for your reply.
I think I should explain to you the problem in detail I’m facing up.
Below is a brief description of the project.
ESP32 WROOM module on my custom board communicates with MCP3913(adc chip) via SPI.

ESP32 receives 3600 sampling data from the MCP3913 and produces 1,200 data to be sent to the server based on them.

1200 data is stored in a list variable.

image.png

As you can see, the list variable consists of 12 sub arrays with a length of 100.

The 1200 data is not sent to the server at once. They are transmitted six times, 200 at a time.

image.png

Rebooting occurs if more than 200 data are transmitted at a time. Also, if the length of sub arrays is greater than 100, sometimes reboot occurs while adding values to the sub array.

That’s why I limited the length of the sub array to 100.

However, I really don’t understand why.

One data is calculated from three sampling data. So we get 1200 data from 3600 sampling data. The following code is for calculating 1200 data from 3600 sampling data from mcp.


The functions so far have already been realized and they work well.

The problem is from now on. I wanted to increase the number of data to be sent to the server from 1200 to 2048.

So I added a few sub arrays to the list variable as following and tried to send them 10 times, 200 at a time.


However, when I increase the number of sub arrays, rebooting occurs when adding values to the sub array(in the code shown above) and sending data to the server. With the firmware running reliably, the maximum number of data that could be transmitted was 1,200. Of course it’s a conclusion from many tests.

When the rebooting occurs, the serial monitor displays the following information.

image.png

For reference, the firmware has a FOTA feature. If I disabled the FOTA function, the rebooting incidence is significantly reduced.

So I first thought it’s because of flash memory limitation and Zerynth’s limitation regarding the core use. But the size of bytecode for the current firmware is only about 110Kbyte. So I don’t think it’s because of memory limitation.

And the firmware doesn’t have BLE feature.

image.png

At this point, this issue is the biggest obstacle to progress.

What do you think about debugging ESP32 using JTAG probe?

I wonder if it’s possible in Zerynth IDE environment and it would actually help.

I sincerely hope that you will provide professional help on what is the main cause of this issue and how to solve it!

Looking forward to hearing something from you as soon as possible.

Thanks.
Anatoli

Hi @Anatoli_Juny
This is interesting, Could you post the last two pictures in a new reply? I cannot see them.
Could you also tell me the output of the garbage collector gc.info() before sending the data to the server? (before rebooting)
I just want to check the available RAM for all of these stored data points.
Please post the two pictures again and the gc.info result and I will follow up with you asap.

Hi @karimhamdy1

Thanks for your reply. I sent the last two pictures again.
I’ll try to execute garbage collector gc info() in my firmware.
In fact, that’s new to me.
image
image

I’ll get back to you with the output of the garbage collector gc.info() soon.
Thanks.
Anatoli

Hi @karimhamdy1

I’m trying to send 1600 data to the server. Reboots occur in two places in the firmware.
One is while reading the sampling data from MCP3913 and another is while calculating the data and adding them to sub arrays.

image

I think you can understand the pictures above.
Don’t hesitate to ask me if there’s anything unclear in my code.

Looking forward to hearing a good news from you soon.

Thanks.
Anatoli.

Hi @Anatoli_Juny
Could you try having some delays every 1000 samples for instance?
Like using sleep(1000), Sleep is not blocking so the other threads will continue execution.

Could you also tell me which IoT platform are you working on, which protocol are you using to send the data? Are you also using any IoT library for that like Ubidots or AWS?

Hi @karimhamdy1

I have already tried the delays, but they didn’t work.
The data is sent to AWS EC2 Server.

I think the problem is in appending the lists.
Appending process is very resource intensive, this error maybe due to overlapping memory segments
This problem happens at python because python doesn’t force programmer to state the length of the list, In contrary to C.

A better approach would be to make a list with a certain length, assign the value at the right index, instead of making smaller sub-arrays and appending them.

Please try this and let me know how it goes.

Hi… I would like to know, do the esp-idf examples run?
You can also find hints to which line is crashing your code by using “make monitor” or manually resolving the address using addr2line.