Delivering webhooks to a queue instead of a URL

Why a URL is sometimes the wrong ask

The request usually arrives from your customer, not from you: “can you put the events on our SQS queue?” Sometimes it is architecture — they already consume everything through a broker and a lone HTTPS handler would be the odd one out. Just as often it is policy: their security team will not open an inbound port for a third party, and no amount of IP allowlisting changes the answer. Either way, if all you can send is HTTP, their request becomes your engineering project.

Here it is a field on the endpoint. Fan-out, retries, the delivery timeline, replay and metering are identical for every destination kind — the transport is the only thing that changes.

The kinds

SQS

curl -X POST https://api.hookget.com/v1/endpoints \
  -H "authorization: Bearer $HOOKGET_KEY" \
  -H "content-type: application/json" \
  -d '{
    "kind": "sqs",
    "event_types": ["order.created"],
    "bus": {
      "target": "https://sqs.eu-central-1.amazonaws.com/123456789012/orders",
      "access_key_id": "AKIA…",
      "secret_access_key": "…"
    }
  }'

The region is derived from the queue URL, so there is one less field to get wrong. The right credential is a user or role whose entire policy is sqs:SendMessage on that one queue — we are a publisher to it, and a publisher needs nothing else.

SNS

curl -X POST https://api.hookget.com/v1/endpoints \
  -H "authorization: Bearer $HOOKGET_KEY" \
  -H "content-type: application/json" \
  -d '{
    "kind": "sns",
    "bus": {
      "target": "arn:aws:sns:eu-central-1:123456789012:orders",
      "access_key_id": "AKIA…",
      "secret_access_key": "…"
    }
  }'

The topic ARN carries the region in its fourth field, and that is where we read it from.

Google Pub/Sub

curl -X POST https://api.hookget.com/v1/endpoints \
  -H "authorization: Bearer $HOOKGET_KEY" \
  -H "content-type: application/json" \
  -d '{
    "kind": "pubsub",
    "bus": {
      "project": "acme-prod",
      "topic": "orders",
      "client_email": "[email protected]",
      "private_key": "-----BEGIN PRIVATE KEY-----\n…"
    }
  }'

We mint the OAuth2 token from the service account key the same way Google’s own libraries do, per delivery batch, and the key itself is encrypted at rest and never returned by any read.

Azure Service Bus

curl -X POST https://api.hookget.com/v1/endpoints \
  -H "authorization: Bearer $HOOKGET_KEY" \
  -H "content-type: application/json" \
  -d '{
    "kind": "servicebus",
    "bus": {
      "namespace": "acme.servicebus.windows.net",
      "entity": "orders",
      "key_name": "hookget-send",
      "sas_key": "…"
    }
  }'

Create a SAS policy on the queue or topic with only the Send claim and name it for us, so revoking it later is one click that cannot break anything else.

AWS EventBridge

curl -X POST https://api.hookget.com/v1/endpoints \
  -H "authorization: Bearer $HOOKGET_KEY" \
  -H "content-type: application/json" \
  -d '{
    "kind": "eventbridge",
    "bus": {
      "target": "arn:aws:events:eu-central-1:123456789012:event-bus/orders",
      "access_key_id": "AKIA…",
      "secret_access_key": "…"
    }
  }'

The region comes out of the bus ARN. One honest wrinkle: EventBridge parses and re-serialises the event detail on its way to every target, so the exact signed bytes cannot travel as the detail — they travel in it, base64-encoded in a data field with the signature attributes beside it, exactly as Pub/Sub forces. Decode, then verify.

Kafka — through the REST proxy

curl -X POST https://api.hookget.com/v1/endpoints \
  -H "authorization: Bearer $HOOKGET_KEY" \
  -H "content-type: application/json" \
  -d '{
    "kind": "kafka",
    "bus": {
      "target": "https://pkc-xxxxx.eu-central-1.aws.confluent.cloud:443",
      "topic": "orders",
      "username": "CLUSTER_API_KEY",
      "password": "CLUSTER_API_SECRET"
    }
  }'

Stated plainly: we do not speak the Kafka wire protocol — that belongs to a client library this product deliberately does not take. What we speak is the Confluent REST proxy's produce API, which Confluent Cloud hosts for every cluster and self-hosters run as a sidecar. The record key is the delivery id, so a replay lands on the same partition; the value is an envelope carrying the exact signed bytes base64-encoded plus the three signature attributes. If your cluster has no HTTP door and cannot grow one, a vendor that speaks native Kafka is the honest fit.

RabbitMQ — through the management API

curl -X POST https://api.hookget.com/v1/endpoints \
  -H "authorization: Bearer $HOOKGET_KEY" \
  -H "content-type: application/json" \
  -d '{
    "kind": "rabbitmq",
    "bus": {
      "target": "https://mq.example.com",
      "entity": "webhook-events",
      "vhost": "prod",
      "routing_key": "orders",
      "username": "hookget-publisher",
      "password": "…"
    }
  }'

Publishes through the management plugin's HTTP API — per-delivery traffic, which is exactly what that API is rated for, and every hosted RabbitMQ (CloudAMQP included) exposes it over TLS. The payload travels base64 so the signed bytes survive; the signature attributes ride in the AMQP message headers where any consumer on any client library finds them. The routing key defaults to the event type, so consumers bind queues the way they already think.

An S3-compatible bucket

curl -X POST https://api.hookget.com/v1/endpoints \
  -H "authorization: Bearer $HOOKGET_KEY" \
  -H "content-type: application/json" \
  -d '{
    "kind": "bucket",
    "bucket": {
      "bucket": "acme-events",
      "prefix": "hookget/",
      "region": "eu-central-1",
      "access_key_id": "AKIA…",
      "secret_access_key": "…"
    }
  }'

Each delivery becomes one object under the prefix. “S3-compatible” is literal: anything that speaks the S3 API with SigV4 works, not only AWS.

Polling — no destination at all

The last option removes the destination entirely. An endpoint of kind poll is fanned out to like any other, but nothing is sent: the consumer reads GET /v1/poll/{id} from a cursor whenever it likes, authenticated with a portal grant. It sees precisely the set of events a pushed endpoint would have received — same event types, same filters, same transformations.

curl -X POST https://api.hookget.com/v1/endpoints \
  -H "authorization: Bearer $HOOKGET_KEY" \
  -H "content-type: application/json" \
  -d '{"kind": "poll", "event_types": ["order.created"]}'

The signature still travels

There are no HTTP headers on the far side of a queue, so the three Standard Webhooks values — webhook-id, webhook-timestamp, webhook-signature — travel as message attributes, alongside hookget-event-type. The message body is the exact bytes that were signed. A consumer reading the queue verifies with the same function an HTTP consumer uses; the signature guide has it in full.