The Postgres Extension Killing Cron

PG Durable is a new, open-source Postgres Extension developed by Microsoft that empowers everyone to run durable, crash-proof functions directly inside Postgres. It transforms your database into a workflow orchestrator, eliminating the need for external services.

For too long, managing complex, database-centric workflows meant wrestling with outside systems like cron jobs, Temporal, or message queues. PG Durable solves this by bringing scheduled tasks, automatic retries, and long-running processes natively into your database, simplifying your architecture significantly.

What makes these functions truly "durable"? Unlike a plain `BEGIN...COMMIT` transaction, PG Durable persists your function's state to disk at every single step of its execution. This ensures that if your database crashes or restarts mid-workflow, your process picks up exactly where it left off, without losing progress.

This crash-proof guarantee is a game-changer. It provides robust fault tolerance for critical operations, from human-in-the-loop approvals to multi-step data transformations that previously demanded elaborate external orchestration. Think of it as a resilient workflow engine, built right into the heart of your data.

Imagine building complex workflows right inside your database, not just querying it. PG Durable gives you new SQL superpowers, transforming your database into a full-fledged orchestrator. It introduces a SQL-native DSL with powerful operators to define the flow of execution.

You can sequence operations with `~>`, run them in parallel using `&`, or create eternal loops with `@>`. This allows you to construct sophisticated graphs of steps directly within your SQL statements, making complex logic surprisingly concise.

PG Durable also brings essential built-in functions. Use `df.start()` to kick off a workflow asynchronously, immediately receiving an ID for tracking. Need cron-like scheduling? `df.wait_for_schedule` lets you pause until a specified time. For human approvals or external events, `df.wait_for_signal` parks your workflow until it receives a wake-up call.

Crucially, when you start a process with `df.start()`, it runs in a Postgres background worker. This means your client connection immediately returns an ID and doesn't block, even for long-running or perpetually looping workflows. Your database handles the heavy lifting, freeing your application.

When we talk about "90% less code," that's not just marketing fluff. Consider the video's example: over 300 lines of boilerplate code—for queue setup, error handling, worker management, and state tracking—can be replaced by a mere seven lines of PG Durable SQL. This dramatic reduction comes from moving complex orchestration logic into the database itself.

Imagine building a robust system that manages message queues, polls for new tasks, handles message processing, tracks state across multiple steps, and implements error handling with automatic retries. Typically, this requires significant custom code or external services. PG Durable abstracts all that away, letting you focus on the business logic.

This conciseness makes PG Durable shine for specific workflows. It's ideal for: - ETL pipelines that need reliable, multi-step data transformations. - Scheduled maintenance, like refreshing materialized views or running daily cleanup tasks. - Multi-step business logic, such as order fulfillment or invoice processing. - API integration workflows that need to manage external calls and their responses durably.

The power lies in the extension handling the hard parts. Developers no longer need to build custom solutions for durability, automatic retries, or complex state management. PG Durable provides these foundational capabilities out of the box, ensuring your workflows are crash-proof and self-healing with minimal code.

PG Durable's true power emerges when your workflow's logic and state are deeply intertwined with data already residing in Postgres. It perfectly handles use cases like automating materialized view refreshes, orchestrating multi-step data transformations, or managing human-in-the-loop approval processes directly where your core data lives. This tight, in-database integration drastically simplifies development, eliminates external service dependencies, and often reduces operational overhead for data-centric tasks.

However, external orchestrators still hold their ground for broader, more complex scenarios. Consider workflows spanning many heterogeneous systems—coordinating microservices, external APIs, and cloud functions across different platforms. For these, a dedicated orchestrator offers the flexibility and robust integration capabilities that extend far beyond a single database, especially when dealing with intricate application logic ill-suited for SQL. These external tools provide a specialized layer for managing distributed systems.

Installation permissions are a crucial consideration for PG Durable. As a Postgres extension, it requires specific privileges to install and configure within your database instance. It also leverages a dedicated background worker to execute durable functions, making it an integral part of your database server. While this architecture provides crash-proof reliability, it also means PG Durable is a powerful, specialized tool for in-database orchestration, not a universal substitute for every orchestration challenge across your entire infrastructure.

What is PG Durable?

PG Durable is an open-source PostgreSQL extension from Microsoft that allows developers to create and manage durable, fault-tolerant workflows and scheduled jobs directly within the database using a SQL-native DSL.

How does PG Durable differ from a traditional cron job?

Unlike a cron job, which is a simple scheduler, PG Durable provides stateful, durable execution. Its workflows are persisted to disk at every step, meaning they can survive database crashes and restarts, automatically retrying and resuming where they left off.

Is PG Durable a replacement for tools like Temporal or Airflow?

PG Durable is an excellent replacement for database-centric workflows, eliminating the need for external services. However, tools like Temporal or Airflow are better suited for complex orchestration across many heterogeneous systems or when the primary logic lives outside the database.

What version of PostgreSQL is required for PG Durable?

PG Durable is designed for modern PostgreSQL versions. The official Docker image from Microsoft runs on Postgres 17, and it's best to consult the official documentation for the latest compatibility information.

Can PG Durable workflows make external HTTP calls?

Yes, PG Durable includes functionality for making HTTP requests directly from a SQL workflow, allowing you to integrate with external APIs as part of your durable function.