---
description: Calling the XDive web API (scrape, crawl, map, search, research, extract) from this codebase
globs:
alwaysApply: false
---

# XDive web API

- Reference: https://xdive.dev/llms.txt (Markdown docs) and https://xdive.dev/openapi.json (OpenAPI 3.1). Look parameters up there; do not invent them.
- Base URL `https://xdive.dev/api`; header `Authorization: Bearer ${XDIVE_API_KEY}` from the environment. Never hard-code, log or commit a `ck_live_` key.
- Every work endpoint is `/orgs/{orgId}/...`; fetch `orgId` once with `GET /whoami`.
- One URL: `POST /orgs/{id}/scrape` with `{"url", "formats": ["markdown"]}`, read `data.artifacts.markdown.content`. Find pages: `/search`. Cited answer: `/research`. Structured data from up to 10 URLs: `/extract` with `prompt` and/or `schema`. List a site's URLs: `/map`. Many pages: `/crawl` or `/batch/scrape`, which return `202` and an id to poll at `GET /orgs/{id}/crawls/{crawlId}` (honour `Retry-After`).
- Errors: JSON with `error` and often `code`; branch on status and `code`. Retry `429`/`503` after `Retry-After` with a cap; never retry `402` or other `4xx` unchanged.
- POSTs that may be retried send `Idempotency-Key: <uuid>` and reuse it on retry (`Idempotent-Replayed: true` on a replay; `422 IDEMPOTENCY_KEY_REUSED` if the body changed).
- HTTP timeouts of at least 120 s; 180 s for deep research.
- In a terminal, `npx xdive scrape <url>` / `search` / `research` print Markdown on stdout (`--json` for JSON); exit codes 3 = not signed in (`npx xdive login`), 4 = quota, 5 = rate limited.
- Long pages: `"max_chars": 8000`, then `"offset": next_offset`. A page with `data.warnings` may be incomplete: retry once with `"wait_for": 3000`.