0% found this document useful (0 votes)
1 views22 pages

Webhook System Design

The document outlines the design of a webhook system for an authentication microservice, detailing event creation, database structure, webhook endpoint registration, event emission, payload signing, and delivery processing. It emphasizes reliability through asynchronous delivery with retries, signature verification, and a replay system for failed deliveries. Best practices for production are also provided, including security measures and idempotency considerations.

Uploaded by

Sandip Stha
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
1 views22 pages

Webhook System Design

The document outlines the design of a webhook system for an authentication microservice, detailing event creation, database structure, webhook endpoint registration, event emission, payload signing, and delivery processing. It emphasizes reliability through asynchronous delivery with retries, signature verification, and a replay system for failed deliveries. Best practices for production are also provided, including security measures and idempotency considerations.

Uploaded by

Sandip Stha
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Webhook System Design

how to create wehook system like sending some events


with webhoook . i am creating auth microservice and
creating webhook so that other service can use that . like
clerk

For your auth microservice, the safest pattern is: save the auth action, save an event, then
deliver it asynchronously with signing + retries. That makes it reliable like Clerk instead of
just “fire and forget.”

Here is the clean architecture for a Clerk-like webhook system in your auth microservice.

1. Basic idea
Your auth service creates events like:

TypeScript
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]

Other services register a webhook endpoint like:

TypeScript
POST [Link]

Then your auth service sends them payloads:

JSON
{
"id": "evt_123",
"type": "[Link]",
"createdAt": "2026-06-16T10:00:00.000Z",
"data": {
"userId": "usr_123",
"email": "test@[Link]",
"name": "Prakash"

1/22
}
}

The receiving service verifies the signature and handles the event.

2. Database design
Use this kind of structure:

prisma
model WebhookEndpoint {
id String @id @default(cuid())
name String
url String
secret String
events Json // ["[Link]", "[Link]"]
active Boolean @default(true)

createdAt DateTime @default(now())


updatedAt DateTime @updatedAt

deliveries WebhookDelivery[]
}

model WebhookEvent {
id String @id @default(cuid())
type String
payload Json
createdAt DateTime @default(now())

deliveries WebhookDelivery[]
}

model WebhookDelivery {
id String @id @default(cuid())
eventId String
endpointId String

status String @default("PENDING") // PENDING, SUCCESS, FAILED


attempts Int @default(0)
nextAttemptAt DateTime @default(now())

responseStatus Int?
error String?

createdAt DateTime @default(now())


updatedAt DateTime @updatedAt

2/22
event WebhookEvent @relation(fields: [eventId], references: [id])
endpoint WebhookEndpoint @relation(fields: [endpointId], references: [id])

@@index([status, nextAttemptAt])
}

3. Register webhook endpoint


Example API:

TypeScript
[Link]("/webhooks/endpoints", async (req, res) => {
const { name, url, events } = [Link];

const secret = `whsec_${[Link](32).toString("hex")}`;

const endpoint = await [Link]({


data: {
name,
url,
events,
secret,
},
});

[Link]({
id: [Link],
name: [Link],
url: [Link],
events: [Link],
secret, // show only once
});
});

Example request:

JSON
{
"name": "Order Service",
"url": "[Link]
"events": ["[Link]", "[Link]"]
}

4. Emit event from auth service

3/22
When user signs up:

TypeScript
await [Link]({
data: {
email,
passwordHash,
name,
},
});

await publishWebhookEvent("[Link]", {
userId: [Link],
email: [Link],
name: [Link],
});

Webhook publisher:

TypeScript
async function publishWebhookEvent(type: string, payload: any) {
const endpoints = await [Link]({
where: {
active: true,
},
});

const subscribedEndpoints = [Link]((endpoint) => {


const events = [Link] as string[];
return [Link](type) || [Link]("*");
});

const event = await [Link]({


data: {
type,
payload,
},
});

if ([Link] > 0) {
await [Link]({
data: [Link]((endpoint) => ({
eventId: [Link],
endpointId: [Link],
})),
});
}

4/22
return event;
}

5. Sign webhook payload


Never send webhooks without signature.

TypeScript
import crypto from "crypto";

function createSignature(payload: string, secret: string, timestamp: number) {


const signedPayload = `${timestamp}.${payload}`;

return crypto
.createHmac("sha256", secret)
.update(signedPayload)
.digest("hex");
}

Headers:

TypeScript
{
"x-webhook-id": [Link],
"x-webhook-event": [Link],
"x-webhook-timestamp": [Link](),
"x-webhook-signature": signature
}

6. Delivery worker
This should run separately from your API server.

TypeScript
async function processWebhookDeliveries() {
const deliveries = await [Link]({
where: {
status: "PENDING",
nextAttemptAt: {
lte: new Date(),
},
},
include: {
event: true,

5/22
endpoint: true,
},
take: 20,
});

for (const delivery of deliveries) {


const body = [Link]({
id: [Link],
type: [Link],
createdAt: [Link],
data: [Link],
});

const timestamp = [Link]([Link]() / 1000);

const signature = createSignature(


body,
[Link],
timestamp
);

try {
const response = await fetch([Link], {
method: "POST",
headers: {
"content-type": "application/json",
"x-webhook-id": [Link],
"x-webhook-event": [Link],
"x-webhook-timestamp": [Link](),
"x-webhook-signature": signature,
},
body,
});

if ([Link]) {
await [Link]({
where: { id: [Link] },
data: {
status: "SUCCESS",
attempts: [Link] + 1,
responseStatus: [Link],
},
});
} else {
await markDeliveryFailed([Link], [Link], [Link]);
}
} catch (error: any) {
await markDeliveryFailed([Link], [Link], null, [Link]);
}

6/22
}
}

Retry helper:

TypeScript
async function markDeliveryFailed(
deliveryId: string,
attempts: number,
responseStatus?: number | null,
error?: string
) {
const nextAttempts = attempts + 1;

const maxAttempts = 5;

if (nextAttempts >= maxAttempts) {


await [Link]({
where: { id: deliveryId },
data: {
status: "FAILED",
attempts: nextAttempts,
responseStatus,
error,
},
});

return;
}

const delayMinutes = [Link](2, nextAttempts); // 2, 4, 8, 16...

await [Link]({
where: { id: deliveryId },
data: {
status: "PENDING",
attempts: nextAttempts,
responseStatus,
error,
nextAttemptAt: new Date([Link]() + delayMinutes * 60 * 1000),
},
});
}

Run worker every few seconds:

TypeScript
setInterval(() => {
processWebhookDeliveries().catch([Link]);

7/22
}, 5000);

Later you can replace this with RabbitMQ, BullMQ, or Temporal.

7. Receiver service verification


In another service:

TypeScript
import crypto from "crypto";

function verifyWebhookSignature({
rawBody,
signature,
timestamp,
secret,
}: {
rawBody: string;
signature: string;
timestamp: string;
secret: string;
}) {
const now = [Link]([Link]() / 1000);
const requestTime = Number(timestamp);

if ([Link](now - requestTime) > 300) {


return false; // older than 5 minutes
}

const expectedSignature = crypto


.createHmac("sha256", secret)
.update(`${timestamp}.${rawBody}`)
.digest("hex");

return [Link](
[Link](signature),
[Link](expectedSignature)
);
}

Receiver route:

TypeScript
[Link]("/webhooks/auth", [Link]({ type: "application/json" }), async (req, res) => {
const rawBody = [Link]();

const signature = [Link]("x-webhook-signature");

8/22
const timestamp = [Link]("x-webhook-timestamp");

const isValid = verifyWebhookSignature({


rawBody,
signature: signature!,
timestamp: timestamp!,
secret: [Link].AUTH_WEBHOOK_SECRET!,
});

if (!isValid) {
return [Link](401).json({ message: "Invalid webhook signature" });
}

const event = [Link](rawBody);

switch ([Link]) {
case "[Link]":
[Link]("Create local user profile:", [Link]);
break;

case "[Link]":
[Link]("Delete/disable local user:", [Link]);
break;

default:
[Link]("Unhandled event:", [Link]);
}

return [Link]({ received: true });


});

8. Important production rules


Use these rules from day one:
1. Always sign payloads using HMAC.
2. Never send passwords, OTPs, refresh tokens, or private session tokens inside
webhook data.
3. Use retry system because other services can be down.
4. Store delivery logs for debugging.
5. Use idempotency on receiver side using [Link] , so the same webhook does not
create duplicate data.
6. Return 2xx only when processed successfully.
7. Timeout webhook requests, for example after 5–10 seconds.
8. Allow services to subscribe only to selected events.

9/22
Recommended flow

User signs up

Auth service creates user

Auth service creates WebhookEvent: [Link]

Auth service creates WebhookDelivery for subscribed services

Worker sends POST request to each service

Service verifies signature

Service handles event

Auth service stores SUCCESS / FAILED delivery log

For your case, this is enough structure:

auth-service
├── users
├── sessions
├── webhook_endpoints
├── webhook_events
├── webhook_deliveries
└── webhook_worker

Later, when traffic grows, move delivery processing to RabbitMQ:

Auth Service → RabbitMQ → Webhook Worker → External Services

how can i create event replay system?

Replay is basically “take an already stored event/delivery and create a new delivery
attempt again.” The key is not to recreate the original event, only create a new delivery/log
for it.

For your auth microservice, create replay like this:

Original event stays same


Replay creates a new delivery

10/22
Worker sends it again
Receiver verifies same signature logic
Receiver uses [Link] for idempotency

1. Better schema for replay


Use separate event, delivery, and attempt tables.

prisma
model WebhookEvent {
id String @id @default(cuid())
type String
payload Json
createdAt DateTime @default(now())

deliveries WebhookDelivery[]
}

model WebhookEndpoint {
id String @id @default(cuid())
name String
url String
secret String
events Json
active Boolean @default(true)

createdAt DateTime @default(now())


updatedAt DateTime @updatedAt

deliveries WebhookDelivery[]
}

model WebhookDelivery {
id String @id @default(cuid())

eventId String
endpointId String

status String @default("PENDING")


// PENDING, SUCCESS, FAILED, CANCELLED

attemptsCount Int @default(0)


nextAttemptAt DateTime @default(now())

replayOfDeliveryId String?
replayReason String?

createdAt DateTime @default(now())


updatedAt DateTime @updatedAt

11/22
event WebhookEvent @relation(fields: [eventId], references: [id])
endpoint WebhookEndpoint @relation(fields: [endpointId], references: [id])

attempts WebhookDeliveryAttempt[]

@@index([status, nextAttemptAt])
@@index([eventId])
@@index([endpointId])
}

model WebhookDeliveryAttempt {
id String @id @default(cuid())
deliveryId String

attemptNumber Int
status String // SUCCESS, FAILED

responseStatus Int?
responseBody String?
error String?

createdAt DateTime @default(now())

delivery WebhookDelivery @relation(fields: [deliveryId], references: [id])


}

Important: do not make eventId + endpointId unique, because replay needs multiple
deliveries for the same event and endpoint.

2. Replay a single failed delivery


Example API:

POST /webhook-deliveries/:deliveryId/replay

Code:

TypeScript
[Link]("/webhook-deliveries/:deliveryId/replay", async (req, res) => {
const { deliveryId } = [Link];

const oldDelivery = await [Link]({


where: { id: deliveryId },
include: {
event: true,
endpoint: true,
},

12/22
});

if (!oldDelivery) {
return [Link](404).json({
message: "Webhook delivery not found",
});
}

if (![Link]) {
return [Link](400).json({
message: "Cannot replay to inactive endpoint",
});
}

const replayDelivery = await [Link]({


data: {
eventId: [Link],
endpointId: [Link],
status: "PENDING",
attemptsCount: 0,
nextAttemptAt: new Date(),
replayOfDeliveryId: [Link],
replayReason: [Link] ?? "Manual replay",
},
});

return [Link]({
message: "Webhook replay queued",
deliveryId: [Link],
eventId: [Link],
endpointId: [Link],
});
});

This does not create a new event. It only creates a new delivery for the same event.

3. Replay one event to all subscribed endpoints


Example:

POST /webhook-events/:eventId/replay

Code:

TypeScript
[Link]("/webhook-events/:eventId/replay", async (req, res) => {
const { eventId } = [Link];

13/22
const event = await [Link]({
where: { id: eventId },
});

if (!event) {
return [Link](404).json({
message: "Webhook event not found",
});
}

const endpoints = await [Link]({


where: {
active: true,
},
});

const subscribedEndpoints = [Link]((endpoint) => {


const events = [Link] as string[];

return [Link]([Link]) || [Link]("*");


});

const deliveries = await [Link]({


data: [Link]((endpoint) => ({
eventId: [Link],
endpointId: [Link],
status: "PENDING",
nextAttemptAt: new Date(),
replayReason: [Link] ?? "Manual event replay",
})),
});

return [Link]({
message: "Webhook event replay queued",
eventId: [Link],
deliveriesCreated: [Link],
});
});

4. Replay event to one specific endpoint


This is useful when only one service failed.

POST /webhook-events/:eventId/replay/:endpointId

TypeScript

14/22
[Link]("/webhook-events/:eventId/replay/:endpointId", async (req, res) => {
const { eventId, endpointId } = [Link];

const event = await [Link]({


where: { id: eventId },
});

if (!event) {
return [Link](404).json({
message: "Webhook event not found",
});
}

const endpoint = await [Link]({


where: { id: endpointId },
});

if (!endpoint || ![Link]) {
return [Link](400).json({
message: "Endpoint not found or inactive",
});
}

const subscribedEvents = [Link] as string[];

const isSubscribed =
[Link]([Link]) || [Link]("*");

if (!isSubscribed) {
return [Link](400).json({
message: "Endpoint is not subscribed to this event type",
});
}

const delivery = await [Link]({


data: {
eventId,
endpointId,
status: "PENDING",
nextAttemptAt: new Date(),
replayReason: [Link] ?? "Manual endpoint replay",
},
});

return [Link]({
message: "Webhook replay queued",
deliveryId: [Link],
});
});

15/22
5. Worker should process replay normally
Your worker does not need special replay logic. It just picks PENDING deliveries.

TypeScript
async function processWebhookDeliveries() {
const deliveries = await [Link]({
where: {
status: "PENDING",
nextAttemptAt: {
lte: new Date(),
},
},
include: {
event: true,
endpoint: true,
},
take: 20,
});

for (const delivery of deliveries) {


await sendWebhookDelivery(delivery);
}
}

6. Send delivery with same event ID, new delivery ID


Very important:

TypeScript
const body = [Link]({
id: [Link],
type: [Link],
createdAt: [Link],
data: [Link],
});

Headers:

TypeScript
headers: {
"content-type": "application/json",
"x-webhook-event-id": [Link],
"x-webhook-delivery-id": [Link],
"x-webhook-event": [Link],
"x-webhook-timestamp": [Link](),

16/22
"x-webhook-signature": signature,
}

[Link] stays the same.

[Link] changes on every replay.

That is how the receiver knows:

Same business event


Different delivery attempt

7. Store every attempt

TypeScript
async function sendWebhookDelivery(delivery: any) {
const body = [Link]({
id: [Link],
type: [Link],
createdAt: [Link],
data: [Link],
});

const timestamp = [Link]([Link]() / 1000);

const signature = createSignature(


body,
[Link],
timestamp
);

const attemptNumber = [Link] + 1;

try {
const response = await fetch([Link], {
method: "POST",
headers: {
"content-type": "application/json",
"x-webhook-event-id": [Link],
"x-webhook-delivery-id": [Link],
"x-webhook-event": [Link],
"x-webhook-timestamp": [Link](),
"x-webhook-signature": signature,
},
body,
});

17/22
const responseBody = await [Link]();

await [Link]({
data: {
deliveryId: [Link],
attemptNumber,
status: [Link] ? "SUCCESS" : "FAILED",
responseStatus: [Link],
responseBody: [Link](0, 5000),
},
});

if ([Link]) {
await [Link]({
where: { id: [Link] },
data: {
status: "SUCCESS",
attemptsCount: attemptNumber,
},
});

return;
}

await scheduleRetry(delivery, attemptNumber, [Link]);


} catch (error: any) {
await [Link]({
data: {
deliveryId: [Link],
attemptNumber,
status: "FAILED",
error: [Link],
},
});

await scheduleRetry(delivery, attemptNumber, null, [Link]);


}
}

8. Retry helper

TypeScript
async function scheduleRetry(
delivery: any,
attemptNumber: number,
responseStatus?: number | null,
error?: string

18/22
) {
const maxAttempts = 5;

if (attemptNumber >= maxAttempts) {


await [Link]({
where: { id: [Link] },
data: {
status: "FAILED",
attemptsCount: attemptNumber,
},
});

return;
}

const delayMinutes = [Link](2, attemptNumber);

await [Link]({
where: { id: [Link] },
data: {
status: "PENDING",
attemptsCount: attemptNumber,
nextAttemptAt: new Date([Link]() + delayMinutes * 60 * 1000),
},
});
}

9. Receiver-side idempotency
The receiving service should save processed event IDs.

prisma
model ProcessedWebhookEvent {
id String @id @default(cuid())
eventId String @unique
type String
processedAt DateTime @default(now())
}

Receiver code:

TypeScript
[Link]("/webhooks/auth", [Link]({ type: "application/json" }), async (req, res) => {
const rawBody = [Link]();

const event = [Link](rawBody);

19/22
const existing = await [Link]({
where: {
eventId: [Link],
},
});

if (existing) {
return [Link]({
received: true,
duplicate: true,
});
}

switch ([Link]) {
case "[Link]":
await [Link]({
data: {
authUserId: [Link],
email: [Link],
name: [Link],
},
});
break;

case "[Link]":
await [Link]({
where: {
authUserId: [Link],
},
data: {
deletedAt: new Date(),
},
});
break;
}

await [Link]({
data: {
eventId: [Link],
type: [Link],
},
});

return [Link]({
received: true,
});
});

20/22
10. Admin APIs you should add
For a good webhook system, add these APIs:

GET /webhook-events
GET /webhook-events/:eventId
POST /webhook-events/:eventId/replay

GET /webhook-deliveries
GET /webhook-deliveries/:deliveryId
POST /webhook-deliveries/:deliveryId/replay

GET /webhook-endpoints
POST /webhook-endpoints
PATCH /webhook-endpoints/:endpointId
DELETE /webhook-endpoints/:endpointId

Useful filters:

/webhook-events?type=[Link]
/webhook-deliveries?status=FAILED
/webhook-deliveries?endpointId=xxx
/webhook-deliveries?eventId=xxx

11. Replay flow

Admin clicks replay



Find old delivery or event

Create new WebhookDelivery row

Status = PENDING

Worker picks it

Send same event again

Create WebhookDeliveryAttempt log

Mark SUCCESS or FAILED

Best structure:

21/22
WebhookEvent

WebhookDelivery

WebhookDeliveryAttempt

So in your auth microservice:

auth-service
├── users
├── sessions
├── webhook_endpoints
├── webhook_events
├── webhook_deliveries
├── webhook_delivery_attempts
└── webhook_worker

This gives you replay, retry, logs, debugging, and future dashboard support.

22/22

You might also like