DevLog: Orange Sentry Sprint 3 - Unix Domain Sockets & Async MQTT

Introuction

This one was a doozy. The main objective was to write a functional async subscribing MQTT client, but to get there I ended up rewriting the entire IPC stack. I moved from FIFO named pipes to Unix Domain Sockets. A lot of time was spent laying the groundwork and learning the quirks of C development. The result is a fully finished and debugged MQTT client, an upgraded memory Arena system, and a much cleaner architecture that communicates using raw binary structs.

The engineering

I have seen low level programming being described as building a skyscraper. The first half of the project is spent just digging on the ground. That could not describe this sprint better. So much time has been dedicated to laying foundations to this project. From structural to technological to knowledge, I had to work a lot for seemingly not that much result. But let me tell you, when you start to actually code on main.c it feels amazing.

I realized that I did not make my homework well enough on the last sprint. I went with fifo pipes because they were easy and friendly to implement, and fell in the trap of not considering my options nicely before.

But Unix Domain Sockets (UDSs) are so much better for this application! For one, they are bidirectional. This means that instead of having two pipes per connecting process, I only need to have one.

Another thing is that they can be message-based, unlike FIFO that are stream-based. This means that if I send 50 bytes I can guarantee that the other side receives that exact amount in one go, while with FIFO the other side could read 10 then 30 then 10. This means that, even if the implementation is a bit bigger and requires a bit more work to debug, I have to write way less logic to accomplish the same thing, which is very good.

It also allows me to handle writing the controller in a more concise way. Since the UDS API is the same as the normal network socket API, it is very optimized for handling many connections. That being said, it took me a bit of work and a lot of time procrastinating because I thought it would be worse than it actually was, but I did it, and I can say that this is actually a lot better. Since I can use discrete messages to communicate, I can do everything with structs and unions, avoiding having to parse json or something like that, which is good.

The rest of the sprint was focused on the MQTT client itself. I successfully implemented the async subscribing functionality using the Eclipse Paho library. I also furthered the memory arena functionality and made many other small changes for both quality of developer life and the betterment of the project's structure.

The Main blockers

My main blockers were life stuff outside the project, the sheer amount of stuff you need to write before writing the actual software, and my general ineptitude with C. My solutions were putting in the time, reading manuals, studying the Paho docs, finding random tutorials, and consulting AI.

The Next Steps

The next sprint will be writing some kernel modules to manage some button presses. There are some other, better ways to do this, but I really want to learn about writing kernel modules, and this one just seemed like the best way to sneak a bit or kernel into the project without being too disruptive.

I am a bit sick of MQTT lol

#MQTT #programming #devlog #C #Linux #Embedded #buildinpublic