Your Next Message Queue is Postgres

Dedicated message brokers like SQS, Kafka, and RabbitMQ introduce significant, often hidden, operational overhead. You manage separate infrastructure, incurring costs for provisioning, deploying, monitoring, and scaling distinct services. This adds network latency and critically increases cognitive load for developers integrating and debugging multiple distributed systems, forcing unnecessary complexity into your stack.

Your existing Postgres database offers a robust, reliable alternative, eliminating broker bloat. Built on ACID compliance and battle-tested transactional power, Postgres provides a proven foundation for reliable queueing, ensuring data integrity even in failures. It comfortably handles millions of rows and thousands of requests per second; for example, a test with 2 CPUs and 2GB RAM processed 100,000 messages in 9 seconds, averaging 11,100 messages/second.

PGMQ formalizes this powerful pattern as a lightweight, simple extension. It delivers an SQS-like API directly within Postgres, enabling queue creation, message sending with optional delays, and consumption with visibility timeouts (VT) for guaranteed exactly once delivery. This means you gain a fully functional message queue, with client libraries for Rust, Python, and TypeScript, without adding a single new service to your stack.

Create queues with pure SQL. Call pgmq.create('my_queue'); every queue becomes its own Postgres table. Send messages using pgmq.send('my_queue', '{"job_id": 123}'). This embeds messages as JSON directly within Postgres, simplifying your data model.

Implement scheduled delivery with pgmq.send(). Add a delay parameter (e.g., pgmq.send('my_queue', '{"task": "future"}', 30)). The message enters the queue but remains unconsumable for 30 seconds, enabling advanced job scheduling without external cron services.

Consume messages via pgmq.read('my_queue', 30, 1). The critical vt (visibility timeout) parameter makes read messages invisible for the specified duration (e.g., 30 seconds). This ensures exactly-once delivery: no other worker can process the same message during its active window. If not processed or deleted within the VT, it reappears in the queue.

Once processed, remove messages. Use pgmq.delete('my_queue', message_id) to permanently remove a message. Alternatively, pgmq.archive('my_queue', message_id) deletes the message from the active queue and moves it to a dedicated archive table, providing a historical log for auditing or re-processing.

"But it doesn't scale," you say. This is the common refrain against databases as queues. Postgres, however, comfortably handles millions of rows and thousands of requests per second. PGMQ leverages this inherent capability, turning a perceived weakness into a strength for distributed applications.

A recent stress test validated PGMQ's performance on constrained resources. Researchers provisioned a Docker container with just 2 CPUs and 2GB RAM. They then injected 100,000 messages into a PGMQ queue. This setup mirrors typical live service environments, making the results highly relevant.

One hundred concurrent workers processed all 100,000 messages in a mere 9 seconds. Each worker read, logged, and deleted messages in batches. This translates to a combined throughput exceeding 11,100 messages per second. For comprehensive documentation and further examples, consult the official Postgres Message Queue (PGMQ) repository.

Such performance unequivocally invalidates the 'it doesn't scale' myth for virtually every real-world application. PGMQ proves Postgres is not just viable but highly performant as a message queue, eliminating the operational overhead of dedicated brokers for most use cases. Keep your infrastructure simple.

Integrate PGMQ directly into your application stack. Move beyond raw SQL with robust client libraries for popular languages. Official support powers Python and Rust integrations, offering idiomatic interfaces. Community-driven libraries further extend PGMQ to Ruby and multiple TypeScript flavors, including a seamless experience with Prisma.

Build producers with minimal code. A Python producer sends a message with a simple send() call, specifying the queue name and JSON payload. This mirrors the pgmq.send() SQL command, abstracting database interaction for you.

Workers consume messages efficiently. Use a client library's read() function to fetch a batch of messages, respecting the visibility timeout (VT). After processing, call delete() or archive() to remove messages, ensuring exactly-once delivery and preventing reprocessing. This pattern handles high-volume tasks.

Unifying your data and messaging layers within Postgres simplifies your infrastructure. Eliminate the operational overhead of separate message brokers, reducing network latency and cognitive load. This consolidation streamlines development, testing, and deployment, yielding a more robust and maintainable system.

What is PGMQ?

PGMQ (Postgres Message Queue) is a lightweight extension for PostgreSQL that enables message queuing functionality directly within the database, offering an alternative to services like AWS SQS, RabbitMQ, or Kafka.

How does PGMQ ensure exactly-once delivery?

PGMQ uses a 'visibility timeout'. When a message is read, it becomes invisible to other consumers for a set period. The consumer must delete or archive the message within that time. If it fails, the message reappears for another consumer to process, preventing data loss.

Can PGMQ handle production-level traffic?

Yes. Benchmarks show PGMQ can process over 11,000 messages per second on modest hardware (e.g., a 2-CPU container). This is more than sufficient for many high-volume distributed applications.

What is the main benefit of using Postgres as a message queue?

The primary benefit is infrastructure simplification. By leveraging your existing database, you reduce dependencies, lower operational overhead, and simplify your entire tech stack without sacrificing performance for most common use cases.