PPactDocs
Data & ETL

Pipeline detail

The per-pipeline view: run history from etl_runs, per-run record counts and timing, and the inline trigger and pause controls.

Pipeline detail

Open a pipeline at Data → Pipelines → to see its definition and its full execution history. Unlike a data source — which keeps only a single last-sync snapshot — a pipeline records every run as its own etl_runs row, so the detail view is a genuine run-by-run timeline.

Run history

The list at GET /v1/etl/pipelines/{id}/runs returns paginated etl_runs rows, newest first. Each run carries:

  • status — running, succeeded, failed, or cancelled
  • started_at / finished_at — execution window
  • records_in / records_out / records_failed — throughput counts
  • error — populated on failure
  • log_json — the per-run log payload

The pipeline row itself keeps a rolled-up snapshot (last_run_at, last_run_status) so lists and cards don't need to join the runs table.

Counts are simulated

The run-history plumbing is real, but the numbers come from _run_pipeline_mock() — records_in / records_out / records_failed and the timing are generated per-destination, not measured from a live transform. A run always completes inline; there is no queue latency or partial-progress state to observe yet.

Controls

  1. 1

    Trigger a run

    POST /v1/etl/pipelines/{id}/runs inserts a running row, executes the inline mock, and returns the terminal run. The pipeline card shows a last-run badge and an inline trigger button.

  2. 2

    Pause / resume

    PATCH /v1/etl/pipelines/{id} with {"status":"paused"} or {"status":"active"} toggles whether the pipeline is eligible to run.

  3. 3

    Edit the definition

    PATCH /v1/etl/pipelines/{id} updates name, description, destination, transform, or schedule.

bash
# Trigger a run, then read the history
curl -X POST https://api.pact.place/v1/etl/pipelines/7/runs \
  -H "Authorization: Bearer $PACT_API_KEY"

curl "https://api.pact.place/v1/etl/pipelines/7/runs?limit=20" \
  -H "Authorization: Bearer $PACT_API_KEY"

The {id} path param is the internal integer id — the routes in api/routes/etl.py type it as int, so a UUID public_id is not yet accepted here. Every query is tenant-scoped, so a pipeline id belonging to another tenant resolves to 404.

PipelinesThe pipeline list, destinations, and full endpoint reference.Data source detailThe upstream source a pipeline reads from.