From 9e4530bcaa33e598a8b8b416410fef112dc0d780 Mon Sep 17 00:00:00 2001 From: Luis Castillo Date: Sat, 5 Sep 2026 19:02:50 +0200 Subject: [PATCH] docs: add The Grid custom LLM guide The Grid is an OpenAI-compatible inference marketplace. Document it as a custom-llm provider alongside the existing OpenRouter/Together guidance, and note that Vapi's default maxTokens of 250 is shared with reasoning tokens on several instruments. --- fern/customization/custom-llm/thegrid.mdx | 66 +++++++++++++++++++++++ fern/docs.yml | 2 + 2 files changed, 68 insertions(+) create mode 100644 fern/customization/custom-llm/thegrid.mdx diff --git a/fern/customization/custom-llm/thegrid.mdx b/fern/customization/custom-llm/thegrid.mdx new file mode 100644 index 000000000..a8292301b --- /dev/null +++ b/fern/customization/custom-llm/thegrid.mdx @@ -0,0 +1,66 @@ +--- +title: The Grid +subtitle: Use The Grid instruments in Vapi through the custom-llm provider +description: Wire The Grid, an OpenAI-compatible inference marketplace, into Vapi with the custom-llm provider and Custom Credentials. +slug: customization/custom-llm/thegrid +--- + +The Grid is an inference marketplace that serves models from several labs behind one OpenAI-compatible API. You address capabilities rather than a specific lab's model name: `text-standard`, `code-prime` and `agent-max` each route to a current model for that tier, so an assistant keeps working when the underlying model is replaced. + +Because The Grid implements the OpenAI chat completions contract, it plugs into Vapi as a [custom LLM](/customization/custom-llm/fine-tuned-openai-models) — the same path used for OpenRouter or Together AI. Streaming and tool calling both work over that path. + +A The Grid account and API key are required. Keys are created in the dashboard at [thegrid.ai](https://thegrid.ai). + +## Add your credential + +POST your key to the `/credential` endpoint with the `custom-llm` provider: + +```json +{ + "provider": "custom-llm", + "apiKey": "" +} +``` + +## Create the assistant + +Point `url` at The Grid's base URL and set `model` to the instrument you want: + +```json +{ + "name": "My Assistant", + "model": { + "provider": "custom-llm", + "url": "https://api.thegrid.ai/v1", + "model": "text-standard", + "maxTokens": 600, + "messages": [ + { + "role": "system", + "content": "You are a helpful voice assistant. Keep replies short." + } + ] + } +} +``` + +## Choosing an instrument + +`GET https://api.thegrid.ai/v1/models` returns the current list with context window, pricing and capability flags for each entry. The tiers most relevant to voice are: + +| Instrument | Use it for | +| --- | --- | +| `text-standard` | General conversation. The usual starting point. | +| `text-prime`, `text-max` | Harder reasoning, at higher latency and cost. | +| `agent-standard`, `agent-max` | Multi-step tool use. | +| `code-standard`, `code-prime` | Assistants that read or write code. | + +## Set maxTokens above the default + +Vapi's `maxTokens` defaults to **250**, and on The Grid that budget is shared with reasoning tokens, which are billed and counted but never spoken. Several instruments reason before answering, so a 250-token cap can truncate the spoken reply mid-sentence. + +Measured on a single short prompt: `text-standard` spent 114 of 250 tokens on reasoning, and `agent-max` reached the cap and stopped with `finish_reason: "length"`. Setting `maxTokens` to 600 or higher leaves room for the spoken turn. Keep the system prompt explicit about brevity — that shortens replies more reliably than a low token cap, and without the risk of cutting one off. + +## Tool calling + +Tools work over the standard OpenAI function-calling fields, so the [tool calling integration guide](/customization/custom-llm/tool-calling-integration) applies unchanged. `agent-standard` and `agent-max` are tuned for multi-step tool use. diff --git a/fern/docs.yml b/fern/docs.yml index ef815129a..c8fb6d672 100644 --- a/fern/docs.yml +++ b/fern/docs.yml @@ -324,6 +324,8 @@ navigation: path: customization/custom-llm/using-your-server.mdx - page: Tool calling integration path: customization/custom-llm/tool-calling-integration.mdx + - page: The Grid + path: customization/custom-llm/thegrid.mdx - section: Examples icon: fa-light fa-code contents: