Research

Ask a question and get an answer written from the web, in which every claim cites a source that was actually read. A research run searches, reads the best pages, keeps the passages relevant to the question, and has a language model write the answer; the citations are then checked in code. Use it for questions you would otherwise answer by searching and reading several pages yourself. For raw results to choose from, use Search.

Requires an API key with the search scope, an LLM configured for your organization and a search engine on the server. Research is synchronous: a quick run takes seconds and is stopped after 45 seconds, a deep run after 150 seconds.

curl -X POST "https://xdive.dev/api/orgs/$ORG_ID/research" \
  -H "X-API-Key: $XDIVE_API_KEY" -H "Content-Type: application/json" \
  -d '{
    "query": "How long does Example Cloud keep audit logs by default?",
    "depth": "quick",
    "freshness": "py"
  }'

from xdive import XDive
 
xdive = XDive()  # reads XDIVE_API_KEY
result = xdive.research(
    "How long does Example Cloud keep audit logs by default?",
    depth="quick",
    freshness="py",
)
print(result["answer"])
for citation in result["citations"]:
    print(f"[{citation['number']}] {citation['url']}")

import { XDive } from "xdive";
 
const xdive = new XDive(); // reads XDIVE_API_KEY
const result = await xdive.research("How long does Example Cloud keep audit logs by default?", {
  depth: "quick",
  freshness: "py",
});
console.log(result.answer);
for (const citation of result.citations) console.log(`[${citation.number}] ${citation.url}`);

Parameters

How a run works

Quick

1. The question is searched (10 results). Pages found by several queries rank higher, and no site gets more than two places.
2. Up to 10 candidate pages are read in parallel, through the page cache, within 6 seconds. Once 3 pages have arrived, the run waits at most 0.7 seconds for more instead of holding the answer for the slowest site. Pages that were not read in time contribute their search snippet instead; such sources are marked snippet: true and rank below pages that were read.
3. Each page is split into passages and the passages most relevant to the question are kept, up to 4 per source, across up to 6 sources.
4. The model writes the answer from those numbered passages, citing them as [1], [2].

Two-step questions

Some questions describe a thing indirectly and then ask about it, such as "Name a co-founder of the company that develops the V8 JavaScript engine." A single search for that question finds pages about V8, not about the company's founders.

In quick mode, while the first search runs, the model triages the question. When it finds a two-step question, the intermediate thing (here, the company) is identified from the pages already read, never from the model's own knowledge; if those pages do not name it, one search for it is made. The remaining question is then searched with the name filled in, adding up to 4 sources. usage.plan reports the triage (single, multi_hop, or unavailable when it failed or was not ready 2 seconds after the first round, and was ignored) and usage.hops is 1 when a bridge was made. The triage is skipped when you pass a schema.

Follow-up search

When the writer reports that the sources lack a fact the question needs, it names a search for exactly that fact. Quick mode makes that one follow-up search, reads up to 4 more sources within 4 seconds, and rewrites the answer. If the writer asked for more without naming a search, a short gap check decides what to search. The follow-up is skipped when a two-step bridge was already made or a schema was passed, and if the rewrite fails the first answer is returned. A quick run makes at most three search rounds in total.

If the writer still returns no answer, it is asked once more to answer from the sources it has, or to say in one sentence that they do not contain the answer and what they do say, with citations. A run that found sources does not come back with an empty answer.

Deep

The model first plans 2 to 4 sub-queries that cover the question's aspects; they are searched together with the question. After each round a gap check lists up to 3 queries for what is still missing, and the next round searches them, for up to 3 rounds. Each round reads up to 24 candidates within 20 seconds, and up to 15 sources are kept. The default output is a report.

Citations

Sources are numbered from 1 and passed to the model as untrusted data, with instructions to ignore any instructions inside them; the model has no tools, so a page can change the words of the answer but not what is fetched or cited. After the model writes, every [n] marker is checked in code against the numbered sources:

• a number that matches no source is removed from the text;
• groups such as [1, 3] and ranges such as [1-3] are expanded, and repeats collapse;
• citations lists exactly the sources the answer cites, in number order, each with a quote: up to 300 characters of that source's most relevant passage.

A fabricated citation cannot survive this check. sources lists every source that was given to the model, cited or not.

Confidence

confidence summarizes how well the answer is grounded:

Structured output

Pass schema to get JSON instead of prose. The model fills the schema from the sources and uses null for anything they do not support. The response has data instead of answer, and because structured output carries no citation markers, every source is listed in citations. The schema must be a JSON object.

{
  "query": "Audit log retention and export options in Example Cloud",
  "schema": {
    "type": "object",
    "properties": {
      "default_retention_days": { "type": "integer" },
      "max_retention_days": { "type": "integer" },
      "export_formats": { "type": "array", "items": { "type": "string" } }
    }
  }
}

Caching

An identical request from your organization within 10 minutes is answered from a cache, with cached: true, and is not metered. Requests count as identical after trimming the query and filling in the default depth and output. Identical requests that arrive while a run is in progress share that run. Failed runs are not cached. Set no_cache: true to force a fresh run.

Response

{
  "query": "How long does Example Cloud keep audit logs by default?",
  "answer": "Example Cloud keeps audit log entries for 90 days by default [1]. Enterprise plans can extend retention to 365 days [1][2].",
  "citations": [
    {
      "number": 1,
      "url": "https://docs.example.com/security/audit-logs",
      "title": "Audit logs",
      "quote": "Audit log entries are retained for 90 days. On the Enterprise plan, retention can be extended to 365 days from the security settings."
    },
    {
      "number": 2,
      "url": "https://example.com/pricing",
      "title": "Pricing",
      "quote": "Enterprise: SSO, SCIM, audit log retention up to 365 days, and a 99.95% uptime SLA."
    }
  ],
  "sources": [
    {
      "number": 1,
      "url": "https://docs.example.com/security/audit-logs",
      "title": "Audit logs",
      "passages": [
        {
          "text": "Audit log entries are retained for 90 days. On the Enterprise plan, retention can be extended to 365 days from the security settings.",
          "score": 7.412,
          "start": 318
        }
      ],
      "score": 7.412
    },
    {
      "number": 2,
      "url": "https://example.com/pricing",
      "title": "Pricing",
      "passages": [
        {
          "text": "Enterprise: SSO, SCIM, audit log retention up to 365 days, and a 99.95% uptime SLA.",
          "score": 4.106,
          "start": 2210
        }
      ],
      "score": 4.106
    },
    {
      "number": 3,
      "url": "https://blog.example.org/audit-logging-compared",
      "title": "Audit logging compared",
      "passages": [
        {
          "text": "Example Cloud stores audit events for three months unless you upgrade.",
          "score": 3.877,
          "start": 0
        }
      ],
      "score": 2.326,
      "snippet": true
    }
  ],
  "queries": ["How long does Example Cloud keep audit logs by default?"],
  "confidence": "high",
  "usage": {
    "searches": 1,
    "pages_read": 5,
    "rounds": 1,
    "plan": "single",
    "duration_ms": 4210,
    "search_ms": 812,
    "read_ms": 1630,
    "model_ms": 1544
  }
}

Sources

Usage

A fresh run counts searches against your search quota and pages_read against your page quota. Cached results cost nothing. A request is refused with 402 when either quota is already used up.

Errors

Errors with a code also have "success": false.

See Errors and limits for authentication errors and retries.

Related

• Search for ranked results with passages, and include_answer for a short answer from the top results.
• Extract to fill a schema from pages you already know.
• Usage and billing for what is metered.