Quickstart

Go from a new account to your first scrape, search and research call. You need an account, an API key and either curl, Python 3.9+ or Node.js 18+.

1. Create an account

Sign up at xdive.dev/register. Signing up also creates your first organization, which owns your API keys, usage and plan. New organizations start on the free plan (see Usage and billing).

Servers that review sign-ups

A server can be configured to review new accounts (REGISTRATION_APPROVAL=required). On such a server, signing up records an access request instead of signing you in, and you receive an email when an administrator approves it. Until then, signing in returns 403 with the code ACCOUNT_PENDING_APPROVAL.

2. Create an API key

In the dashboard, open API Keys and click Issue new key.
Give the key a name, for example Production server.
Keep the default Expiry (days) of 90, or enter 0 for a key that never expires. Optionally restrict Allowed IPs / CIDRs.
Choose Scopes. New keys start with crawl and scrape; also select search, which search and research need.
Click Create and copy the key. It starts with ck_live_ and is shown only once.

Scopes and key restrictions are covered in Authentication.

3. Set up your environment

Store the key in an environment variable. Both SDKs read XDIVE_API_KEY when you don't pass a key.

bash

export XDIVE_API_KEY="ck_live_…"

Install an SDK if you are not using cURL:

bash

# Nothing to install. The examples below also use jq to read JSON.

bash

pip install xdive

bash

npm install xdive

4. Find your organization id

Organization endpoints live under /orgs/{id}/. GET /whoami returns the organization your key belongs to. The SDKs make this call for you on first use and cache the result, so only cURL users need to save the id.

bash

curl https://xdive.dev/api/whoami \
  -H "X-API-Key: $XDIVE_API_KEY"
 
# Save the id for the next steps:
export ORG_ID=$(curl -s https://xdive.dev/api/whoami \
  -H "X-API-Key: $XDIVE_API_KEY" | jq -r .orgId)

python

from xdive import XDive
 
xdive = XDive()  # reads XDIVE_API_KEY
print(xdive.whoami())

import { XDive } from "xdive";
 
const xdive = new XDive(); // reads XDIVE_API_KEY
console.log(await xdive.whoami());

json

{
  "auth": "api_key",
  "keyId": "key_5b1e9c7a20d34f8e86a1c0d7e3f92b64",
  "orgId": "org_8f2c41d09a7b4e6f93d0c5a1b2e47f18",
  "plan": "free",
  "scopes": ["crawl", "scrape", "search"],
  "suspended": false
}

5. Scrape a page

Fetch one page as Markdown. Scrape is synchronous: the content comes back in the response.

bash

curl -X POST "https://xdive.dev/api/orgs/$ORG_ID/scrape" \
  -H "X-API-Key: $XDIVE_API_KEY" -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

python

from xdive import XDive
 
xdive = XDive()
page = xdive.scrape("https://example.com")
print(page["title"])
print(page["artifacts"]["markdown"]["content"])

import { XDive, pageMarkdown } from "xdive";
 
const xdive = new XDive();
const page = await xdive.scrape("https://example.com");
console.log(page.title);
console.log(pageMarkdown(page));

The API wraps the page in data; the SDKs return data directly.

json

{
  "data": {
    "url": "https://example.com",
    "title": "Example Domain",
    "depth": 0,
    "status_code": 200,
    "word_count": 19,
    "fetched_at": "2026-10-15T08:42:10Z",
    "fetch_route": "http",
    "fetch_attempts": 1,
    "fetch_cost_units": 1,
    "artifacts": {
      "markdown": {
        "media_type": "text/markdown; charset=utf-8",
        "content": "# Example Domain\n\nThis domain is for use in documentation examples …",
        "bytes": 167
      }
    }
  },
  "security": {
    "cache_state": "eligible",
    "network_policy": "live",
    "outbound_permitted": true,
    "zero_data_retention": false
  },
  "success": true
}

Ask for other formats with formats, for example ["markdown", "links"]. See Scrape for every option.

6. Search the web

Search returns ranked results from the configured search engines. This call needs the search scope.

bash

curl -X POST "https://xdive.dev/api/orgs/$ORG_ID/search" \
  -H "X-API-Key: $XDIVE_API_KEY" -H "Content-Type: application/json" \
  -d '{"query": "robots.txt specification", "limit": 3}'

python

from xdive import XDive
 
xdive = XDive()
results = xdive.search("robots.txt specification", limit=3)
for result in results["results"]:
    print(result["rank"], result["title"], result["url"])

import { XDive } from "xdive";
 
const xdive = new XDive();
const { results } = await xdive.search("robots.txt specification", { limit: 3 });
for (const result of results) console.log(result.rank, result.title, result.url);

json

{
  "provider": "brave",
  "query": "robots.txt specification",
  "results": [
    {
      "id": "3fa94c1e0b7d2a85",
      "source": "web",
      "rank": 1,
      "title": "RFC 9309: Robots Exclusion Protocol",
      "url": "https://www.rfc-editor.org/rfc/rfc9309.html",
      "description": "The standard that defines how crawlers read robots.txt rules."
    }
  ],
  "search_depth": ""
}

Add "include_answer": true for a short cited answer, or "search_depth": "advanced" to also read the top results and get their most relevant passages. See Search.

7. Ask a research question

Research searches the web, reads the best sources and writes an answer whose citations are checked against the pages it read. Quick research (the default) takes seconds; "depth": "deep" plans several searches and can take up to about two minutes. It needs the search scope.

bash

curl -X POST "https://xdive.dev/api/orgs/$ORG_ID/research" \
  -H "X-API-Key: $XDIVE_API_KEY" -H "Content-Type: application/json" \
  -d '{"query": "What does RFC 9309 standardize?"}'

python

from xdive import XDive
 
xdive = XDive()
result = xdive.research("What does RFC 9309 standardize?")
print(result["answer"])
for citation in result["citations"]:
    print(f'[{citation["number"]}] {citation["url"]}')

import { XDive } from "xdive";
 
const xdive = new XDive();
const result = await xdive.research("What does RFC 9309 standardize?");
console.log(result.answer);
for (const citation of result.citations) console.log(`[${citation.number}] ${citation.url}`);

json

{
  "query": "What does RFC 9309 standardize?",
  "answer": "RFC 9309 standardizes the Robots Exclusion Protocol, the robots.txt rules that tell crawlers which paths of a site they may fetch [1]. …",
  "citations": [
    {
      "number": 1,
      "url": "https://www.rfc-editor.org/rfc/rfc9309.html",
      "title": "RFC 9309: Robots Exclusion Protocol",
      "quote": "…"
    }
  ],
  "sources": [
    {
      "number": 1,
      "url": "https://www.rfc-editor.org/rfc/rfc9309.html",
      "title": "RFC 9309: Robots Exclusion Protocol",
      "passages": [{ "text": "…", "score": 7.42, "start": 0 }],
      "score": 7.42
    }
  ],
  "queries": ["What does RFC 9309 standardize?"],
  "confidence": "high",
  "usage": {
    "searches": 1,
    "pages_read": 4,
    "rounds": 1,
    "plan": "single",
    "duration_ms": 6120,
    "search_ms": 830,
    "read_ms": 2410,
    "model_ms": 2760
  }
}

Research writes its answer with a language model: the one your organization configured under AI Extraction in the dashboard, or one the server provides. Without either, the call returns 501. Asking the same question with the same options again within 10 minutes can return the cached answer, marked "cached": true, and a cached answer is not metered.

Next steps

Crawl a whole site with Crawl, or list its URLs first with Map.
Learn the client libraries: Python SDK and JavaScript SDK.
Handle limits and retries before going live: Errors and limits.
Connect an AI assistant: MCP.