> For the complete documentation index, see [llms.txt](https://engineering.umh.app/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://engineering.umh.app/engineering/development-process/how-to-build/coding-standards/architecture/data-integrity/retries-and-dead-letter-queues.md).

# Retries and dead-letter queues

What happens to a message that fails.

**A plugin retries a message only as long as a retry can fix the problem.** [Error management](/engineering/development-process/how-to-build/coding-standards/architecture/error-management.md) sorts every error into transient, persistent transient or persistent. This page says what happens to the message in each case. A dead-letter queue is where a message goes that fails for good.

Read: DDIA chapter 12, "Acknowledgments and redelivery"; [Redpanda Connect, route to a dead-letter queue](https://docs.redpanda.com/connect/configuration/error_handling/#route-to-a-dead-letter-queue).

## Retries and error classes

**A failed message is never dropped.** What happens to it depends on its error class from [Error management](/engineering/development-process/how-to-build/coding-standards/architecture/error-management.md).

* Transient error: the plugin retries the message with exponential backoff, a limited number of times. Because the retries are limited, the messages behind it wait only for a short time. A transient error never blocks. If the retries keep failing, it becomes a persistent transient error.
* Persistent transient error, such as an MES that stays unreachable: after the last retry, the plugin either blocks or sends the message to the dead-letter queue. Each write flow has a setting for this.
* Persistent error, such as a poison message: the plugin does not retry it. The message goes to the dead-letter queue.

Endless retries would hold up every message behind the failed one without anyone noticing. The user can change the number of retries or turn them off. With retries turned off, every error that a retry could fix becomes a persistent transient error at once.

**A time-critical write goes to the dead-letter queue.** In process locking, the machine waits for the MES. After the last retry, the write flow sends the message to the dead-letter queue. The machine then gets an error instead of waiting forever.

**A write that is not time-critical blocks until the system it writes to is reachable again.** An example is a write flow that stores time-series values in the historian database. If the database is unreachable, the write flow stops and goes to [degraded](/engineering/development-process/how-to-build/coding-standards/architecture/fsmv2-workers.md#states). Once the database is back, it writes all values in their sequence.

**The dead-letter queue is part of the UNS.** A failed message goes back under the [data contract](https://docs.umh.app/usage/data-modeling/data-contracts) of its input topic, with `_error` added to the contract's name. Every error then has the same shape. It sits next to the data it belongs to. The sender of the message reads its outcome there.

The user can configure a write flow to also send a success message into the UNS. Other flows can then start when the write succeeds. In process locking, the bridge waits for this message to answer the machine.

Read: DDIA chapter 9, "Timeouts and Unbounded Delays".

## Poison message

**A message that fails for good goes to a dead-letter queue. The sender still gets an answer.** A Kafka topic keeps the sequence of its messages. One message that the MES rejects every time blocks every request behind it.

In the example, the write flow writes the rejection into the UNS. The waiting bridge can then answer the machine.

## Detecting faults

**Only an answer that the receiving system sent on purpose counts as success.** A gateway in front of the MES can answer with the same HTTP status for its own errors as the MES uses for "already approved". Read the body of the answer before you count it as an approval. Otherwise an MES outage approves every part.

## Timeouts

**When the bridge times out, it does not know whether the MES approved the part.** It answers the machine with an error. A second later the MES can still approve the part. When the machine sends the part again, the write flow asks the MES again with the same part number and station. The MES then answers that it already approved this part.

Read: DDIA chapter 13, "Timeliness and Integrity".

## Use control loops

**A data flow that waits for an answer is a** [**control loop**](/engineering/development-process/how-to-build/coding-standards/architecture/control-loops.md)**. Every cycle, it compares the desired state with the observed state.**

Such a data flow can have two inputs. The first reads the observed state from the UNS and stores it in the cache of a `nodered_js` processor. The second is a `generate` input that creates an empty message every second. On each of these messages, the processor compares the observed state in its cache with the desired state from its config. It acts only if they differ and the cache is filled.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://engineering.umh.app/engineering/development-process/how-to-build/coding-standards/architecture/data-integrity/retries-and-dead-letter-queues.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.