sdlc.dev

AWS fundamentals path · Messaging · Topic 15 of 18

Fan-out and events: SNS and EventBridge

Not started

SQS gives each message to one consumer. When several services each need their own copy of the same fact ("order placed", "file uploaded", "instance stopped"), you publish it once and let the infrastructure fan it out. **SNS** does this with **topics** and subscriptions. **EventBridge** does it with **event buses** and **rules** that match on event content. Both deliver at least once, both retry, and both drop what they cannot deliver unless you give them a dead-letter queue.

Why it matters: Queues connect one producer to one consumer group. Topics and event buses let many services react to the same fact without the producer knowing them, and fail in different ways.

  1. 1SNS pushes to subscribers

    An SNS topic pushes each message to every subscription (SQS queues, Lambda functions, HTTP endpoints, email, SMS), optionally filtered.

  2. 2SNS to SQS is the classic fan-out

    Each consuming service subscribes its own queue, so it can fall behind, retry and scale without affecting the others.

  3. 3EventBridge routes by content

    An event bus matches each event against rules with patterns on its fields and sends matches to targets, including AWS service events and SaaS sources.

  4. 4Both retry, then drop

    Delivery is at least once with retries. Without a dead-letter queue, events that keep failing are eventually lost.

1

How it works

The explanation, with examples you can run.

SNS: topics and subscriptions

A publisher sends to a topic; SNS pushes a copy to every subscription: SQS queues, Lambda functions, HTTP(S) endpoints, Amazon Data Firehose, email, SMS or mobile push. Filter policies on a subscription let a consumer receive only some messages, matched on message attributes or the body. FIFO topics, which only deliver to SQS queues, preserve ordering per message group and deduplicate.

The standard pattern is SNS to SQS fan-out:

OrderPlaced -> SNS topic orders                 |-> SQS fulfilment-queue  -> fulfilment workers                 |-> SQS email-queue       -> email sender                 '-> SQS analytics-queue   -> analytics loader

Each service owns its queue, scales its consumers and has its own DLQ. If the email sender is down for an hour, its queue fills and drains later; fulfilment is unaffected. Delivering straight from SNS to Lambda or HTTP removes that buffer, so failures fall back on SNS's retry policy.

EventBridge: buses and rules

An event is JSON with a standard envelope (source, detail-type, time, detail). Rules on a bus match events with patterns:

{  "source": ["com.example.orders"],  "detail-type": ["OrderPlaced"],  "detail": { "total": [{ "numeric": [">", 100] }], "country": ["DE", "FR"] }}

Matching events go to up to five targets per rule: Lambda, SQS, Step Functions, API destinations (any HTTP API), another bus, and many more. The default bus receives events from AWS services themselves (an EC2 state change, a CodePipeline failure), and partner buses receive SaaS events. Extras include the schema registry, archive and replay of past events, Pipes for point-to-point source-to-target integrations, and Scheduler for cron-like and one-off invocations.

How they fail

  • Retries and drops: EventBridge retries a failing target for up to 24 hours and 185 attempts by default, then drops the event. Errors such as a missing permission or a deleted target are not retried. SNS retry policies depend on the protocol. Configure a DLQ (an SQS queue) on every target or subscription, and alarm on it.
  • Duplicates: at-least-once delivery means consumers must be idempotent.
  • No ordering: standard SNS topics and EventBridge do not guarantee order. Include versions or timestamps and let consumers ignore stale updates.
  • Permissions: the queue's or function's resource policy must allow the topic or the rule to deliver; a missing policy silently routes to the DLQ, or nowhere.
  • Payload drift: consumers depend on fields producers may change. Version events and document their schemas.
2

Check yourself

3 questions. Get them all right to finish this topic.

  1. 1.Orders, emails and analytics all need every OrderPlaced message, and each can be slow or down independently. Best design?
  2. 2.By default, how long does EventBridge keep retrying delivery to a failing target?
  3. 3.Which is a strength of EventBridge over SNS for routing?

0 of 3 right so far. Get all three to finish this topic.

Go deeper