Event-Driven architecture is itself a distributed system, a field renowned for being difficult, particularly when things go wrong. The fallacies of distributed computing always lurk around the corner. One of the fallacies is that the network is unreliable. This necessitates planning for retries in case of failures.
One issue with this is that retries can result in duplicate processing, and this can cause very real problems. Taking a payment twice from someone’s account will lead to an incorrect balance and an irate user.
To avoid processing the event multiple times, we would need Exactly-once semantics to be applied. Exactly-once processing is impossible to guarantee without some sort of cooperation between the broker and the consumer. If the broker uses acknowledgements to producers for publishing message, each message must have unique IDs for the broker to deduplicate retires.On the consumer side, deduplication can be used to ensure that duplicate messages are not processed.the consumption logic needs to be made idempotent.
Essentially, being idempotent means that when the same event/message is received multiple times, there is no change to the actor’s state. Here are some approaches:
- De-duplicate incoming messages by recognizing duplicate messages by identity.
- Design state transitioning messages to cause the same impact each time they are received.
- Allow the state transition to render harmless the receipt of duplicate messages.
The Idempotent Consumer pattern acts like a stateful filter that allows logic wrapped by it to be executed only once. Two elements are needed to implement this:
- A way to uniquely identify each message by a business key.
- An idempotent repository. Idempotent repositories are containers for a durable set of keys that will survive restarts of the consumer and can be implemented in database tables, journals, or similar.
When an event arrives, the consumer needs to uniquely identify it using an event key assigned to the event. The idempotent repository is checked to see whether it contains the key, and if it does not, the logic wrapped by it is executed, otherwise it is skipped.
Many messaging systems, such as Apache ActiveMQ, Apache Kafka, Apache Camel have capabilities to eliminate duplicate messages. The Idempotent consumer pattern is used to provide this functionlaity in these systems.