Data sources

A data source is one tenant-owned connection to an external system you want to pull records from. Sources are managed through a real CRUD surface at /v1/data/sources (backed by the data_sources table, alembic 0097_data_sources) and surfaced in the app under Data → Sources.

Every row carries its operational state inline — status, last_synced_at, last_error, and last_records_count — while per-provider connection parameters live in a free-form config_json blob so the row shape never changes when a new connector kind is added.

Sync execution is a demo path today

CRUD (create, read, update, delete) is fully real and tenant-isolated. The sync itself runs inline as a simulation — _run_sync_mock() in api/routes/data_sources.py flips the row to syncing, generates a per-kind record count, and writes it back as last_records_count. No records are actually ingested from the remote system yet. The route layer is designed so a real worker can replace _run_sync_mock() with provider-specific logic without changing the API contract.

Supported connector kinds

The kind field is validated against a fixed allowlist (ALLOWED_KINDS):

Kind	Description
`s3`	Amazon S3 bucket / prefix
`postgres`	External Postgres database
`http_csv`	CSV fetched over HTTP
`salesforce`	Salesforce org
`hubspot`	HubSpot portal
`webhook`	Inbound webhook feed
`manual_csv`	Operator-uploaded CSV
`snowflake`	Snowflake warehouse
`bigquery`	Google BigQuery

A source's status is one of idle, syncing, error, or disabled. Only idle and disabled are user-settable via PATCH; syncing and error are written by the sync flow itself.

Endpoints

code

POST   /v1/data/sources            create
GET    /v1/data/sources            list (filter by kind, status; paginated)
GET    /v1/data/sources/{id}       fetch one
PATCH  /v1/data/sources/{id}       update name / config / status
DELETE /v1/data/sources/{id}       delete
POST   /v1/data/sources/{id}/sync  run a sync (inline mock)

The list endpoint accepts kind, status, limit (1–200, default 50), and offset query parameters. Every read and write is scoped to your tenant, so a {id} from another tenant resolves to 404.

Path param is the int id today

The source {id} path param is the internal integer id — the routes in api/routes/data_sources.py type it as int, so a UUID public_id is not yet accepted here. The public_id column exists on the table (alembic 0099) for the platform-wide IDOR migration, but this route has not been switched to the dual-accept resolver yet.

1
Create a source
POST /v1/data/sources with name, kind, and a config object. The tenant is derived from your auth context — never sent in the body.
2
Configure connection params
Provider credentials and settings go into config_json. Patch them any time with PATCH /v1/data/sources/{id}.
3
Trigger a sync
POST /v1/data/sources/{id}/sync flips the row to syncing and records a result. The source detail page polls every 3s while a sync is in flight.

bash

curl -X POST https://api.pact.place/v1/data/sources \
  -H "Authorization: Bearer $PACT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"Prod Postgres","kind":"postgres","config":{"host":"…"}}'

Data source detailConnection status, sync history, and per-source controls.PipelinesTransform and load source data into Pact objects.Data vaultExport Pact data outbound to your own warehouse or object store.