Linking Promptwatch as a source

The Promptwatch connector syncs your LLM monitoring data – prompts, responses, monitors, tags, topics, and personas – into PostHog, so you can analyze AI model performance alongside your product data.

Prerequisites

• A Promptwatch account on the Explore plan or higher (required for API access).
• A read-only API key from Settings > API Keys in Promptwatch. Read-only keys are enough – PostHog only reads data.
• If you're using an organization API key, you also need the project ID for the project you want to sync.

Adding a data source

1. In PostHog, go to the Sources tab of the data pipeline section.
2. Click + New source and click Link next to this source.
3. Enter your credentials (see Configuration below) and click Next.
4. Select the tables you want to sync, choose a sync method and frequency, then click Import.

Once the syncs are complete, you can start querying this data in PostHog.

You'll be asked for:

• API key (required) – a read-only API key from Promptwatch Settings > API Keys.
• Project ID (optional) – required only when using an organization API key. You can find it in your Promptwatch project settings.
• Response start date (required) – the earliest date to sync responses from, in YYYY-MM-DD format. Only the responses table uses this value.

Sync modes

Each table can be synced in one of several modes, depending on what the source supports:

• Webhook (when available) – the source pushes changes to PostHog in real time. Fastest freshness, lowest ongoing cost, and the only mode that reliably captures updates and deletes.
• Incremental – only new or updated rows are synced on each run, using a cursor field (such as an updated_at timestamp). Cheaper than a full refresh, but deletes aren't captured.
• Append only – new rows are appended using a cursor field; existing rows are never updated. Ideal for immutable, append-only tables like event logs.
• Full refresh – the whole table is reloaded on every sync. Use it when a table has no reliable cursor or when you need deletions reflected.

See sync methods for a full explanation of how each mode works and how to choose between them.

The responses table supports incremental sync using the createdAt field – each sync picks up new responses since the last run. All other tables (prompts, monitors, tags, topics, personas) use full refresh.

Sync limits

Syncs count against your Promptwatch account's hourly API request quota. Each paginated table (prompts and responses) syncs up to 1,000 rows per run (50 rows per page, 20 pages max).

• For a full refresh sync that hits the 1,000-row cap, the sync fails with an error. Use a smaller project or narrow the response start date.
• For an incremental responses sync, the sync stops at the cap and picks up where it left off on the next run.

Configuration

Supported tables

The tables available from this source are discovered from your account when you connect it, so the exact list depends on your data. Once connected, you can pick which tables to sync from the sources tab.

Troubleshooting

• 401 error – Promptwatch rejected your API key. Check the key in Settings > API Keys and make sure it hasn't been revoked.
• 403 error – Promptwatch denied access. If you're using an organization API key, make sure you entered the correct project ID.
• Hourly quota exceeded – Promptwatch reached its hourly API request limit. Wait until the next UTC hour or upgrade your Promptwatch plan.
• Row limit exceeded – The table has more than 1,000 rows. Use a later response start date or a smaller project to reduce the row count.

If your sync is failing or data looks wrong, see the Data warehouse troubleshooting guide. If that doesn't help, contact support – we're happy to help.