RabbitMQ serializes messages from a queue with multiple consumers
I am having an issue where I have a shared mode queue set up and multiple consumers bound to it. The problem is that it looks like rabbitmq is serializing messages i.e. Only one consumer can operate at a time. I need this to be in parallel, however I cannot figure out how to do this.
Each consumer works in their own process. There are many messages in the queue. I am using py-amqplib to interact with RabbitMQ.
Any thoughts?
a source to share
how about prefetch (QOS)? on a small queue, I cast the appearance of parallelism, declaring the queue, getting the number of messages currently available, adding a consumer, consuming messages, and then closing it once the number of messages has been destroyed. Closing a channel without acknowledging messages makes the messages available to other consumers, polling the queue is fast enough and you can have a parallel solution.
a source to share
Re-establish your preferred AMQP model, which appears to be the consumer of the connection queue. You must create a "direct" exchange and agree on a routing key that all of your users will listen to. Then each consumer that connects must create an exclusive, private, short-lived queue and use it queue_bind()
to subscribe to their queue for messages that match the exchange's public routing key. By using this scheme, my workers start working in parallel, instead of having their operations serialized!
a source to share