Message Acknowledgement types and different delivery methods in a Distributed Message Queuing system

In today’s world of communication, message queues are ubiquitous.

In the world of distributed systems, message queue plays a vital role in enabling reliable and asynchronous communication between components.

One critical aspect of this communication is message acknowledgement.

Acknowledgement ensures that messages are delivered successfully and processed by the intended recipients.

In this blog post, we will explore the various types of message acknowledgements and the different delivery methods employed in a distributed message queuing system.

Let’s start with a high level architecture of any Message Queuing system:

Main components:

Producer:

A producer is responsible for creating and sending messages to the message queue. It generates the messages and specifies their destination or topic. Producers play a vital role in initiating the communication flow by publishing messages to the queue.

Message Queue:

The message queue is the central component that holds and manages the messages. It acts as a buffer between the producer and the consumer, ensuring that messages are stored until they are consumed. The queue provides a way for producers to send messages and for consumers to receive them asynchronously.

Consumer:

A consumer receives and processes messages from the message queue. It subscribes to specific topics or queues and waits for messages to become available. Once a message is retrieved, the consumer performs the necessary actions based on the content of the message. Consumers can be designed to process messages in parallel, allowing for scalable and efficient processing of messages.

Broker:

The broker acts as an intermediary between producers and consumers. It manages the message queue, facilitates the transfer of messages between producers and consumers, and ensures reliable delivery. The broker may implement various mechanisms such as message routing, load balancing, and persistence to handle the communication between producers and consumers effectively.

Depending on the desired throughput or latency, the choice of acknowledgement type can vary when a publisher publishes a message in the queue.

ACK=All:

In this particular acknowledgement method, achieving strong consistency is prioritized. This means that the producer will receive an acknowledgement only when the message is successfully persisted across all the necessary replicas. However, this process can take a significant amount of time, making it less suitable for high throughput systems.

ACK=0:

In this type of acknowledgement method, high throughput is achieve because producer will not wait for any acknowledgement when a message is published. Hence, in this case consistency is not guaranteed. This type of method will be useful for a system where sometimes a few message loss is acceptable, for example: Log, Alert or Monitoring based system

ACK=1:

In this particular acknowledgement method, at least one acknowledgment will be needed. This means that the producer will receive an acknowledgement only when the message is successfully persisted in at least one replica. This is a balanced approach between achieving high throughput and low latency.

Now let’s talk about the different type of message delivery methods:

At-Least-Once Delivery(No of times msg delivery>=1):

At-least-once delivery guarantees that each message will be delivered to the recipient at least once. To achieve this, the sender retries message delivery until it receives a positive acknowledgement from the recipient. This method ensures that no messages are lost in transit but may result in duplicate messages being processed if the acknowledgement is not received in a timely manner. At-least-once delivery is commonly used in scenarios where message integrity and reliability are of utmost importance.

Consumer in this case won’t commit the message till it’s fully processed.

Use cases: Financial systems, E-commerce order processing, Distributed computing and job processing, Messaging and Notification

At-Most-Once Delivery(No of times msg delivery<=1):

At-most-once delivery ensures that each message is delivered at most once, eliminating the possibility of duplicates. In this method, producer will not wait for any acknowledgement hence no retry will happened. While this approach prevents duplicate processing, it may result in message loss if the recipient fails to process the message.

Use cases: Broad casting, Event Logging and Audit Trail, Non-idempotent operations, Reliable File Transfer, Billing and Payment Processing

Exactly-Once Delivery(No of times msg delivery=1):

Exactly-once delivery aims to provide the strongest level of message delivery guarantee. It ensures that messages are delivered exactly once, without duplicates or losses. Achieving exactly-once delivery is challenging and often requires additional mechanisms, such as unique message IDs and transactional processing. This method is commonly used in scenarios where duplicate processing or message loss can have severe consequences, such as in financial systems or critical infrastructure.

Use cases: Financial Transactions, Deduplication and Idempotent Operations, Critical Command and Control Systems, Reliable Data Replication, Workflow and Process Orchestration, Online Transactions and E-commerce

Please comment and share your use case and where you use a combination of Acknowledgement and delivery method, it will be interesting to know.

Follow me here as well: https://hubpages.com/@ajhawrites