Understanding OpenQueue and event-driven architecture
Event-driven architecture (EDA) is a software design pattern where components communicate by producing and consuming events. Instead of direct, synchronous calls between services, systems emit events when something happens, and other services react to those events asynchronously.
Client waits for processing to complete
Client gets immediate confirmation, work happens in background
Producers and workers don't need to know about each other. Just enqueue work and move on.
Add more workers to handle increased load. The queue absorbs spikes gracefully.
Jobs persist in the queue until processed. No work is lost if a worker crashes.
Clients get immediate responses. Heavy processing happens asynchronously in the background.
Jobs can be prioritized. Critical tasks jump to the front of the line.
Delay execution with run_at. Perfect for cron jobs, retries, and deferred work.
OpenQueue implements a job queue pattern - one of the most common ways to achieve event-driven processing. Here's how each concept maps to OpenQueue:
Any service that creates work. It enqueues a job with a payload and gets back a job ID immediately. The producer doesn't care who processes it or when.
# Producer enqueues work
job_id = client.enqueue(
queue_name="send_email",
payload={"to": "user@example.com", "subject": "Welcome!"}
)
# Returns immediately - job is queuedServices that consume and process work. Workers lease jobs from the queue, process them, and report success or failure. You can run multiple workers for parallel processing.
# Worker leases and processes
leased = client.lease(queue_name="send_email", worker_id="worker-1")
if leased:
send_email(leased.job.payload)
client.ack(leased.job.id, leased.lease_token)The buffer between producers and workers. OpenQueue uses PostgreSQL as the backing store - jobs are rows in a table, giving you durability, querying, and ACID guarantees.
FOR UPDATE SKIP LOCKED). This prevents two workers from processing the same job - no race conditions, no duplicates.run_at parameter delays execution. Jobs stay pending until their scheduled time, then become available for leasing. No cron needed.