Message queuing is one of those things that distributed systems have to wrestle with. There are many solutions in the marketplace from commercial offerings to open source ones. A quick search on the web will reveal the many choices. Some time ago, I had a need for such a solution. However, the project did not need the operational overhead that comes along with having such a message queue solution in place.

I needed a decent amount of performance from the message queue and have it work with NodeJS. The design had to be simple enough that it can be deployed with the application or hosted separately. The solution also required a guarantee that at least one and at most one client will process a message. The decision was made to try out Redis Pub/Sub.I already had Redis in production and the thinking leaned towards leveraging this technology.

One problem immediately surfaced, I cannot meet the guarantee that was required for the project. Before completely giving up on Redis and moving on to something else, I decided to perform a quick proof of concept to see if this can work with Redis as a data store. The result was that it could be done and the elegance of Redis shone through.

With Lua scripting inside of Redis, it makes addressing such a problem possible. I have made the solution available on Github for those of you interested in the technology and dealing with a similar requirement. The code is also available as a NodeJS module on NPM.

If you find this useful, I would love to hear all about your use case.

Happy experimenting!