pgmq

Start Here

Kafka is great when you need Kafka. Most early products do not need Kafka yet.

They need a place to put work after a database write. They need a worker to claim that work. They need retries when the worker fails. That is a queue problem, not always a streaming-platform problem.

pgmq gives you a queue inside Postgres. That means fewer moving parts while the product is still changing.

What To Build First

Create one queue for one clear job flow.

create extension if not exists pgmq;

select pgmq.create('email_delivery');

select pgmq.send(
  'email_delivery',
  '{"task":"send_welcome_email","user_id":42}'
);

A worker reads from the queue, does the work, then deletes or archives the message when it is done.

select *
from pgmq.read('email_delivery', 30, 5);

That 30 is the visibility timeout in seconds. While the worker has the message, other workers should not see it. If the worker dies, the message comes back after the timeout.

Good First Uses

• Sending email after signup.
• Running small background jobs.
• Retrying webhook delivery.
• Moving events from an outbox to another service.
• Kicking off product tasks that are tied to Postgres data.

These are jobs where the hard part is usually correctness, not streaming scale.

The Part People Skip

Make workers idempotent. The queue can redeliver a message. Your code must be safe when that happens.

For email, store that an email was sent before you archive the message. For payments, use the payment provider’s idempotency key. For webhooks, record attempts and payloads.

Also add a failed-message path early. You do not want the first production failure to be a mystery message looping forever.

What This Does Not Replace

pgmq is not Kafka. It does not give you Kafka’s full log model, consumer groups, long retention, stream processing, or large fan-out story.

That is fine. You are not trying to fake Kafka. You are trying to avoid running Kafka before the product needs it.

Move To Kafka When

• Many independent teams consume the same event stream.
• Replay and long retention are core requirements.
• You need high-throughput fan-out across many consumers.
• Stream processing is part of the product, not just plumbing.
• Kafka is already a platform your team operates well.

Until then, a Postgres-backed queue is usually easier to build, deploy, and debug.

References

• pgmq