Enterprise Integration Patterns
Overview
GoCamel implements the Enterprise Integration Patterns (EIP) from the classic book by Gregor Hohpe and Bobby Woolf.
Message Routing
Choice
Content-based routing with conditional branches.
Syntax:
When(expression)- Adds a conditional branchOtherwise()- Adds a default branchEndChoice()- Ends the Choice block and adds it to the current container (the route, or a surroundingSplit/Loop/Multicastso each part is evaluated)End()- Alternative method
Expression Operators:
| Operator | Description | Example |
|---|---|---|
== | Equal | ${header.type} == 'order' |
!= | Not equal | ${header.status} != 'error' |
> | Greater than | ${header.count} > 10 |
>= | Greater or equal | ${header.count} >= 10 |
< | Less than | ${header.price} < 100 |
<= | Less or equal | ${header.price} <= 100 |
&& | AND | a > 5 && b == 'x' |
|| | OR | a > 5 || b == 'y' |
Filter
Filter messages based on a condition. Exchanges that do not match are silently dropped (ErrStopRouting).
The drop only ends the current route: when that route was called through direct:, the
caller carries on (as with an incomplete Aggregator or a Resequencer). Use Stop() to end
the exchange everywhere.
Throttle
Rate-limit exchange processing to at most N messages per time window. Exchanges that exceed the rate are delayed, not dropped, until the window resets.
Notes:
- Uses a fixed-window counter; for smoother behaviour use smaller windows
- Blocking: the goroutine sleeps until the window allows the exchange through
maxRequestsandtimePeriodmust be greater than zero; invalid values return an error instead of blocking indefinitely- If the exchange context is cancelled while waiting for the next window,
Processreturns an error
Sample
Let only a sample of the exchanges through. Unlike Throttle, which delays exchanges, Sample drops the others silently, regardless of their content.
Notes:
- Dropped exchanges stop with
ErrStopRouting: they are not treated as failures - No background goroutine: only a mutex guards the timestamp or counter
Samplekeeps the first exchange of each period, not the latest one- A non-positive period or count is reported by
Route.Validate();Processreturns an error instead of passing everything
Delay
Pause exchange processing for a fixed or dynamically computed duration. The sleep respects context cancellation.
Notes:
- Blocking: the goroutine sleeps for the computed duration
- If the exchange context is cancelled during the sleep,
Processreturns an error - Return
0fromDelayFuncto skip the delay for a given exchange - Passing
niltoDelayFuncfails route construction;NewDelayerFunc(nil)remains constructible but returns a processing error instead of panicking
Content Enricher
Fetch additional data from an external resource and merge it into the current exchange.
How it works:
- A copy of the current exchange is sent to the enrichment URI
- The response is received
AggregationStrategy.Aggregate(original, response)merges the two- If no strategy is provided, the response body replaces the original body
PollEnrich
Poll on-demand from a consumer endpoint (with timeout and optional aggregation strategy).
Supported endpoints are seda, chan, and custom implementations of PollingEndpoint. One call receives at most one exchange and leaves the queue open with its remaining messages intact. Endpoints without this contract, or queues with an existing consumer or concurrent poll, return an error before consuming a message. A zero timeout is non-blocking; a negative timeout waits until a message arrives or the exchange context is cancelled. A positive timeout leaves the original message unchanged when no message arrives.
Message Validator
Validate in-flight messages against a predicate, failing the exchange with ErrValidationFailed if the condition is not met.
Idempotent Consumer
Prevent duplicate messages from being processed.
How it works:
- Evaluates the expression to get a unique key.
- Checks if the key exists in the repository.
- If not present, adds the key eagerly (blocking concurrent duplicates) and proceeds.
- If present, stops routing (
ErrStopRouting). - If the exchange later fails downstream, the key is removed from the repository again
(via an exchange
Synchronization), so a redelivery of the same message is processed instead of being dropped β at-least-once semantics.
Repositories:
NewMemoryIdempotentRepository(): In-memory store (default).NewRedisIdempotentRepository(): Distributed store backed by Redis.- SQL-based repositories can be implemented via the
IdempotentRepositoryinterface.
Circuit Breaker
Protect routes from cascading failures when calling unstable external services.
States:
- Closed: Requests pass through. Failures are counted.
- Open: Requests fail immediately or go to fallback.
- Half-Open: After
OpenTimeout, a single trial request at a time probes the downstream service; concurrent requests are diverted to the fallback as if the circuit were still open. OnceSuccessThresholdsuccessful probes complete, the circuit closes. If a probe fails, it opens again.
ErrStopRouting (Filter, incomplete aggregation, Stop EIP) is not a failure:
it does not increment the failure count and does not open the circuit.
Load Balancer
Distribute messages across multiple endpoints.
Strategies:
RoundRobin(): Circular distribution (default).Random(): Random distribution.Custom(strategy): Provide a custom implementation of theLoadBalancerStrategyinterface.
SEDA
Staged Event-Driven Architecture. Decouples producers and consumers using an internal queue (Go channel).
How it works:
- The producer sends a copy of the exchange to an internal queue.
- The producer returns immediately (non-blocking if queue is not full).
- One or more consumer workers read from the queue and process the exchange.
Parameters:
size: The capacity of the internal queue (default 1000).concurrentConsumers: The number of worker goroutines (default 1).shutdownTimeout: How longStopkeeps processing the queued exchanges, in ms (default 30000;0discards them immediately).
Invalid values (negative size, non-positive concurrentConsumers) fall back to the
defaults with a warning. Parameters are applied when the endpoint is first created;
later references to the same queue name reuse the existing endpoint, and a reference
with different options logs a warning.
Context. Workers process each exchange under the consumer’s context, not the
sender’s. A route such as From("http://...").To("seda:work") returns as soon as the
message is queued, which cancels the HTTP request context; the asynchronous leg is not
affected and its HTTP calls, delays and SQL queries run normally.
Shutdown. Stop first rejects new sends (and unblocks producers waiting on a full
queue), then lets the workers finish the exchanges already queued, for up to
shutdownTimeout. It then cancels the consumer context; exchanges still queued are
completed as failures. CamelContext.GracefulStop cancels the context up front, so it
does not wait for the queue.
Resequencer
Reorder messages that arrived out of order based on a sequence number or timestamp.
How it works:
- Evaluates the expression to get the sequence number.
- If the sequence is the next expected one, it’s processed immediately.
- If it’s a future sequence, it’s buffered and sorted.
- If a gap persists longer than the
Timeout, the resequencer skips the missing sequence and continues with the next available one in the buffer.
Notes:
- Messages are delivered downstream through the resequencer’s internal pipeline (the steps
between
Resequence(...)andEnd()). The original exchange stops there (ErrStopRouting), so steps placed afterEnd()never see out-of-order messages. - Delivery is serialized to guarantee ordering; downstream steps must not route back into the same resequencer.
- Duplicate or outdated sequence numbers are dropped (logged as warnings).
- A buffered exchange is delivered later, by another message or by the timeout: it runs downstream under the route’s lifecycle context, not the context of the request that brought it (which is usually over by then).
- A message only fails for its own downstream error. The failure of another buffered exchange delivered in the same batch is logged; it is not reported to the message that triggered the delivery, and does not go through the route’s error handler.
Claim Check
Reduce the size of messages by storing the payload in an external repository and passing only a reference (key).
Operations:
ClaimCheckPush: Stores the body and replaces it with a key. If no key is provided, one is generated.ClaimCheckPop: Retrieves the data by key, restores the body, and removes it from the repository.ClaimCheckGet: Retrieves the data by key and restores the body (keeps it in the repository).ClaimCheckDiscard: Removes the data from the repository without restoring it.
Multicast
Send a copy of the message to multiple destinations.
With Parallel Processing:
With Strategy:
Message Transformation
Splitter
Divide a message into parts.
With Aggregation:
Parallel processing:
Exchange Properties during Split:
| Property | Type | Description |
|---|---|---|
CamelSplitIndex | int | Current index (0-based) |
CamelSplitSize | int | Total number of parts |
CamelSplitComplete | bool | Last part indicator |
These properties are branch-scoped
They describe one part, not the whole message, and are removed when the
aggregated result is published back onto the parent exchange. A processor
placed after the Split does not see CamelSplitComplete=true left over
from the last part, and a nested Split does not overwrite the bookkeeping
of the enclosing one. The same rule applies to CamelMulticast*,
CamelRecipientList* and CamelRoutingSlip*.
A []byte body is not a collection
Splitting a []byte body yields a single part, matching Apache Camel,
which does not split byte[]. Treating it as a list produced one exchange
per byte β a 10 KB payload became 10 000 exchanges. To divide binary
content deliberately, split on a decoded value instead:
Aggregator
Combine multiple messages into one.
At least one completion condition is required: SetCompletionSize,
SetCompletionTimeout, or SetCompletionPredicate. Without one, Process
returns an error instead of buffering forever.
Aggregation Strategy:
Completion Conditions:
| Method | Description |
|---|---|
SetCompletionSize(n) | Complete after n messages |
SetCompletionTimeout(ms) | Complete after timeout |
SetCompletionPredicate(fn) | Complete when predicate returns true |
Storage Options:
When a route with a completion timeout starts, the groups already held by the
repository get a full CompletionTimeout again: a group left pending by a
stopped route, or persisted in SQL across a process restart, still expires
instead of waiting forever for its size or predicate. Custom repositories get
this by implementing ListableAggregationRepository (Keys).
Transformer
Transform message content.
Transform
A dedicated message transformation EIP that modifies the message body dynamically using a custom Go function or a Simple language expression.
Messaging Systems
Pipeline
Sequential processing within multicast.
ToD (Dynamic To)
Send to dynamically computed endpoint.
Escaping. Exchange values are data, not URI syntax. Each value is escaped
according to where its placeholder sits in the template (InterpolateURI):
- in the query string, it is query-escaped, so a header such as
a&allowHeaderOverride=truestays inside its option and cannot enable another one; - in the path,
?and#are escaped, so a value cannot start a query string.
A placeholder that stands for the whole URI or its scheme (ToD("${header.dest}"))
is inserted as-is: the route explicitly lets exchange data choose the endpoint,
so that data must come from a trusted source.
Each distinct URI keeps a started producer in the context’s producer cache,
bounded to DefaultProducerCacheSize (1000) producers. Beyond that, the least
recently used idle producers are stopped. Change the bound with
CamelContext.SetProducerCacheSize(n) (n <= 0 = unbounded).
Recipient List
Send to multiple recipients computed at runtime.
Dynamic Router
Route an exchange through a sequence of endpoints resolved dynamically by calling a control function after each step.
Routing ends when the function returns an empty string or nil.
How it works:
- Calls the control function/expression.
- If result is non-empty, sends exchange to that URI.
- Propagates output to input and repeats from step 1.
Routing Slip
Route an exchange through a dynamic sequence of endpoints. Unlike Recipient List, each step receives the output from the previous step.
Exchange Properties during Routing Slip:
| Property | Type | Description |
|---|---|---|
CamelRoutingSlipIndex | int | Current step index (0-based) |
CamelRoutingSlipSize | int | Total number of routing steps |
CamelRoutingSlipComplete | bool | Last step indicator |
Control Flow
Stop
Stop routing without error.
Note: Subsequent processors won’t be executed. Unlike Filter, Stop() ends the whole
exchange (it sets the CamelRouteStop property): a route that called this one through
direct: stops too, and its Send returns ErrStopRouting.
Loop
Iterate a route segment a static or dynamic number of times.
Exchange Properties during Loop:
| Property | Type | Description |
|---|---|---|
CamelLoopIndex | int | Current iteration index (0-based) |
CamelLoopSize | int | Total number of loop iterations |
CamelLoopComplete | bool | Boolean indicating if this is the last iteration |
Note: Stop() inside a loop ends routing of the exchange: the steps after End() do not
run. Any other stop signal raised in the loop body (Filter, Sample…) only ends the loop,
and routing carries on after End().
Message Headers
SetHeader
RemoveHeader
Exchange Properties
SetProperty
Exchange-scoped variables (not in message).
RemoveProperty
Error Handling
Do-Try-Catch-Finally
Structured error handling within a route.
How it works:
- Executes the processors in the
DoTryblock. - If an error occurs, it searches for a matching
DoCatchblock (empty string matches any error). - If a match is found, the error is considered “handled” and routing continues after the
DoTryblock (unless the catch block itself fails). - The
DoFinallyblock is always executed, regardless of whether an error occurred or was caught.
Route error handler and Dead Letter Channel
A route-level ErrorHandler receives every error a step returns. Its return
value drives the route:
nilasks the route to retry the failed step (redelivery). It does not mean “ignore the error and continue”: the route gives up afterMaxErrorHandlerRetries(10000) consecutive retries of the same step and returns an error wrappingErrTooManyRetries.- an error wrapping
ErrStopRoutingends routing without failing the exchange. - any other error fails the exchange.
DeadLetterChannel retries up to MaxRedeliveries, then sends the exchange to
the dead letter endpoint and, by default, still returns the original error to
the consumer (the HTTP consumer answers 500). With DeadLetterHandled(true) a
dead-lettered exchange counts as handled, as in Apache Camel:
OnException
OnException declares route-scoped handlers selected by error type, like
Apache Camel’s onException. Each clause lists one or more targets:
- an error value (a sentinel such as
io.EOF) matches witherrors.Is; - a non-nil pointer to an error or interface type matches with
errors.As:gocamel.ErrorType[*MyError]()(same asnew(*MyError)),&MyValueError{}for a value-receiver error,gocamel.ErrorType[net.Error](). A&MyError{}whoseErrormethod has a pointer receiver is an error value and only matches itself.
A missing, nil or unsupported target fails Build().
How it works:
- When a top-level route step fails, the clauses are tried in declaration order and the first matching one wins: declare specific clauses first.
- A matched clause replaces the route’s
ErrorHandlerfor that error. The failed step is redelivered up toMaximumRedeliveriestimes (default 0), waitingRedeliveryDelay(default 1 s) multiplied byBackOffMultiplier(default 2, use 1 for a fixed delay) at each attempt, capped byMaximumRedeliveryDelay(default 60 s). The wait aborts when the exchange’s context is cancelled. - If the step still fails, the clause’s processors (
Process,ProcessFunc,To,SetBody,SetHeader,Log) run on the message the step received, with theCamelExceptionCaught,CamelRedeliveryCounterandCamelFailureRouteIdproperties set. - Then:
Handled(true): routing stops (no further route step runs) and no error reaches the consumer or sender, which receives the exchange as left by the handler. The route returns an error wrappingErrStopRoutingand the original error.Continued(true): routing continues with the next step as if the error had not occurred; that step receives the handler’s output.Continuedtakes precedence overHandled.- neither (default): the original error propagates.
- If a handler processor fails, its error (wrapping the original one) is returned instead.
Unmatched errors keep the ErrorHandler/DeadLetterChannel behaviour. An
exchange already dead-lettered (IsDeadLettered), e.g. by a route called via
direct: that propagates the original error, never triggers a clause: its
failure has already been taken care of.
Limitations: clauses are route-scoped (no context-wide OnException yet),
match errors returned by top-level steps only β a redelivery re-runs the whole
top-level step, e.g. a whole Split block β and have no OnWhen predicate.
Transactions & Sagas
Saga
The Saga EIP provides a mechanism to coordinate a series of local transactions across distributed services without locking databases. It defines a set of actions and their corresponding compensating actions (rollbacks).
If any action in the Saga block fails or returns an error, the Saga orchestrator automatically triggers the compensating steps for all completed actions in reverse order (LIFO - Last In, First Out). Action panics are recovered and treated as failures (compensations still run). ErrStopRouting from an action stops the saga without compensation β it is a control signal, not a failure.
You can also use custom Go functions for actions and compensations directly using ActionFunc:
A nil compensation in ActionFunc means the step has nothing to undo.
Persistent saga state and recovery
Each saga runs under an ID taken from the CamelSagaId header (a UUID is generated and set on the header when it is absent). After every action whose step has a compensation, the saga checkpoints its state to a SagaRepository: the list of pending compensations plus a snapshot of the In message (body and headers). It saves again after each compensation, and removes the state once the saga has completed or been fully compensated. A compensation that fails stays in the repository so a later recovery can retry it.
| Option | Default | Effect |
|---|---|---|
Repository(repo) | MemorySagaRepository | Where the state is persisted. SQLSagaRepository (root module, database/sql only β bring your own driver) survives a process crash. |
Timeout(d) | 0 (none) | Bounds each action and each compensation. An action exceeding it fails with ErrSagaStepTimeout (wrapping context.DeadlineExceeded) and the completed steps are compensated; the timed-out step itself is not, as for any failed step. The step runs on a copy of the exchange whose Context carries the deadline: a step that ignores its context keeps running in the background after the timeout, and its result is discarded. |
RecoverIncomplete(true) | off | At CamelContext.Start, once every route is started, incomplete sagas found in the repository are compensated last step first. When off, each incomplete saga is only logged as a warning telling how to enable recovery. |
Recovery sends each compensation to its URI through the context’s producer cache, with an exchange rebuilt from the last checkpoint’s message and the CamelSagaId header. SagaProcessor.Recover(ctx, camelCtx) runs it on demand. Sagas still running in the current process are never recovered.
Warning
Only compensations declared by endpoint URI survive a process restart. A compensation passed to ActionFunc is a Go function: it cannot be serialised, so after a restart there is nothing to rebuild it from. Such steps are still persisted (as InProcessCompensation), and when recovery meets one it reports ErrSagaCompensationNotRecoverable in an ERROR log line naming the saga and step β it never skips it silently β and keeps it in the repository, status COMPENSATING, for manual handling. The remaining URI compensations of that saga still run. Declare steps that must be recoverable with Action(actionURI, compensationURI).
Delivery guarantees. The checkpoint is written after the action returns and before the next action starts. A crash between an action and its checkpoint loses that step (its compensation does not run on recovery); a crash after a compensation and before the state update runs that compensation again. Compensation on recovery is therefore at-least-once for checkpointed steps: compensations must be idempotent. If a checkpoint cannot be saved, the saga fails and compensates its completed steps, including the one just completed.
Persistence format. SQLSagaRepository stores the state as JSON (saga_id, status, state, updated_at as Unix nanoseconds): message body and header values must be JSON-serialisable and come back as JSON types (numbers as float64, []byte as a base64 string). ListIncomplete returns at most DefaultSagaListLimit (1000) sagas, oldest first; the rest are picked up by a later start once earlier ones are resolved.
Limits. A saga nested in another saga’s action gets its own ID (the enclosing saga’s header is restored afterwards); sagas are not propagated or joined across routes. Recovery does not coordinate several processes sharing one database: enable RecoverIncomplete on a single instance, otherwise one process may compensate a saga another is still running.
Message Tracking & Diagnostics
Message History
The Message History EIP enables tracing the chronological journey of a message as it flows through the various route nodes and processors.
To enable tracking for a route, invoke .EnableMessageHistory() on your route builder. You can then retrieve and inspect the history using gocamel.GetMessageHistory(exchange).
EIP Pattern Summary
| Pattern | Category | Description |
|---|---|---|
| Choice | Routing | Content-based routing |
| Filter | Routing | Conditional filtering |
| Idempotent Consumer | Routing | Prevent duplicate messages |
| Circuit Breaker | Routing | Resilience pattern for failures |
| Load Balancer | Routing | Load distribution across endpoints |
| Dynamic Router | Routing | Dynamic decision-based routing |
| Resequencer | Routing | Reorder messages by sequence |
| Claim Check | Routing | Store payload and pass reference |
| Throttle | Routing | Rate limiting |
| Delay | Routing | Pause before next step |
| Sample | Routing | Keep a sample, drop the rest |
| Multicast | Routing | Multiple destinations |
| Recipient List | Routing | Dynamic recipient routing |
| Routing Slip | Routing | Dynamic sequential routing |
| Wire Tap | Routing | Async fire-and-forget copy |
| Split | Transformation | Message splitting |
| Aggregate | Transformation | Message aggregation |
| Content Enricher | Transformation | Enrich with external data |
| Transform | Transformation | Content transformation |
| ToD | Endpoint | Dynamic endpoint |
| Stop | Control | Stop routing |
| DoTry | Control | Try-Catch-Finally error handling |
| SetHeader | Headers | Header manipulation |
| SetProperty | Properties | Exchange properties |
| Loop | Control | Iterate with static/dynamic counter |
| Saga | Control | Saga transactional pattern |
| Message History | Diagnostics | Track message processing trace |
Branch completion and aggregation
Split, Multicast and RecipientList complete each processed branch on success or failure, including idempotency rollback callbacks. Parallel branches copy supported mutable bodies, including each split part, before workers start. Custom mutable types still require application-defined copying or immutable use. Workers finish before aggregation begins; the strategy is called serially in input order, and strategy panics become ErrProcessorPanic errors. Sequential processing stops at the first failing branch. A parallel Split or RecipientList runs at most MaxConcurrency(n) branches at a time (default DefaultMaxConcurrency, 10): both expand from message content, so an unbounded fan-out would start one goroutine per element. A parallel Multicast runs all its branches at once, their number being fixed by the route.
Invalid Resequence expressions accumulate a build error while leaving the DSL chain usable; Timeout, Capacity and End do not panic because an earlier expression was invalid.