Job System

Every AI generation in CutX is tracked as a job. Jobs follow a strict state machine that guarantees reliable processing, automatic retries, and credit safety.

Job States

State	Description
pending	Job created, credits deducted, awaiting queue
queued	Sent to Cloudflare Queue for processing
processing	Pipeline worker is actively running the job
completed	Output generated and stored successfully
failed	Error occurred — credits refunded automatically
canceled	Manually canceled by user or admin
expired	Timed out waiting for a webhook callback

State Machine

Jobs can only transition through valid paths:

pending ──→ queued ──→ processing ──→ completed
  │            │           │
  │            │           ├──→ failed
  │            │           │
  │            │           ├──→ canceled
  │            │           │
  │            │           └──→ expired
  │            │
  │            ├──→ failed
  │            │
  │            └──→ canceled
  │
  ├──→ processing (skip queue for sync jobs)
  │
  ├──→ failed
  │
  └──→ canceled

failed ──→ pending (retry)
expired ──→ pending (retry)

Invalid transitions are rejected with an error. Every transition is logged.

Job Lifecycle

1. Creation

POST /api/generate/copy  (or /static-ad, /video)
  → Validate input
  → Check credit balance
  → Deduct credits atomically
  → Insert job row (status: pending)
  → Enqueue to Cloudflare Queue (status: queued)
  → Return job ID

2. Processing

The pipeline worker dequeues jobs and routes them by type:

Job Type	Processing	Completion
copy	Workers AI (synchronous)	Immediate
scrape	HTTP fetch (synchronous)	Immediate
static_ad	Replicate API (async)	Via webhook
ugc_video	Replicate API (async)	Via webhook
tts	Replicate API (async)	Via webhook

Synchronous jobs complete in the pipeline worker. Async jobs submit to Replicate and wait for a webhook callback.

3. Completion

• Sync jobs: Pipeline worker stores the output, marks completed
• Async jobs: Webhook worker receives the Replicate callback, downloads output to R2, marks completed

4. Failure

When a job fails:

1. Error message is recorded on the job row
2. Status transitions to failed
3. Credits are automatically refunded
4. Structured log entry is emitted

Idempotency

Generation endpoints accept an optional idempotency_key to prevent duplicate processing:

POST /api/generate/static-ad
Content-Type: application/json

{
  "product_id": "uuid",
  "style": "lifestyle",
  "idempotency_key": "user-123-product-456-static-v1"
}

The key is scoped per-user with a unique database constraint (user_id + idempotency_key). If a duplicate key is submitted:

• If the existing job is terminal (completed, failed, canceled, expired): the existing job is returned (no new credits deducted)
• If the existing job is active (pending, queued, processing): the existing job is returned

This ensures that retried requests (network timeouts, client retries) never double-charge credits.

Retries

Failed and expired jobs can be retried:

POST /api/admin/jobs/<id>/retry
Authorization: Bearer <admin-token>

This resets the job to pending status, re-deducts credits, and re-enqueues it. Only failed and expired states allow retry.

The pipeline worker also handles transient errors with automatic retry (up to 3 attempts per queue message) before acknowledging the message and marking the job as failed.

Structured Logging

Every job event emits a structured JSON log:

{
  "ts": "2026-03-09T15:30:00.000Z",
  "event": "job.transition",
  "job_id": "uuid",
  "from_status": "queued",
  "to_status": "processing"
}

Key events:

• job.created — new job with type and credits cost
• job.transition — status change with from/to
• job.transition_denied — invalid transition attempt
• job.completed — success with output metadata
• job.failed — error details and refund info