Providers and keys

Ask AI calls an LLM with your own key. You choose the provider and model in orbitdocs.config.ts. The key stays in an environment variable on the server that serves the docs.

Choose a providerCopy Anchor Link

ai: {
  provider: 'anthropic',
  model: 'claude-sonnet-5-5',
  apiKeyEnv: 'ANTHROPIC_API_KEY',
},

model is passed to the provider as is. Use any model id your provider accepts. A fast, inexpensive model is usually enough: answers come from a few passages, not from the model's own knowledge.

Keep the key safeCopy Anchor Link

• The config holds only the variable name. .orbitdocs/ai.json and the build hold no key.
• The key is read on the server for each answer and never sent to the browser.
• To rotate it, change the environment variable and restart the server. No rebuild is needed.

Where to set it depends on what serves the docs:

Rate limitsCopy Anchor Link

Each reader can ask 30 questions per 10 minutes. The 31st gets 429 Too many questions. Try again in a few minutes. Readers are counted by IP address, taken from X-Forwarded-For, then X-Real-IP.

With Nest you can change the limit:

mountOrbitDocs(app, {
  root: join(__dirname, '../docs/out'),
  path: '/docs',
  ai: { rateLimit: 60, env: process.env },
});

mountOrbitDocs ai options:

• rateLimit: questions per reader per 10 minutes (default 30).
• env: where to read apiKeyEnv from (default process.env).
• ai: false turns Ask AI and MCP off even when the build has an AI index.

In Next server mode the limit is fixed at 30.

Limits are kept in memory per server process. Several instances each keep their own count. Your provider's own limits and spending caps apply on top; set a monthly budget with your provider.

Track questions with onAskCopy Anchor Link

createAi from @orbitdocs/ai takes an onAsk callback, called for every question with the number of passages found. Zero passages means your docs don't cover the question, which tells you what to write next. The self-hosted platform uses it for its built-in analytics (see Analytics).

mountOrbitDocs and orbitProxy don't expose onAsk. To use it, serve the AI routes with your own handler:

import { readFileSync } from 'node:fs';

import { type AiManifest, createAi } from '@orbitdocs/ai';

const manifest = JSON.parse(readFileSync('docs/out/orbitdocs-ai.json', 'utf8')) as AiManifest;

export const ai = createAi({
  manifest,
  rateLimit: 30,
  onAsk: ({ question, sources }) => {
    console.log(JSON.stringify({ event: 'docs_question', question, sources }));
  },
});

// ai.handle(request: Request) answers /_ai/* and /mcp[/<api>] and returns undefined for anything else.

createAi options:

Without access and auth, every page is treated as public. Pass both when your docs are private.

createAuth from @orbitdocs/auth takes the same fetch option, for its OpenID Connect, app-session, personalization and audit-webhook requests. The self-hosted platform passes both a fetch that refuses private addresses (see Outbound requests).

Next stepsCopy Anchor Link