Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions evals/migrate-sonar-to-agent-api/check.sh
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,38 @@ ABSENT=(
"\\[[\"']citations[\"']\\]|\\.citations\\b|get\\([\"']citations[\"']"
"[\"']sonar(-[a-z]+)*[\"']"
)
# Only max_tokens belonging to the web_search object is a valid tool budget.
request_max_tokens() {
hit "(^|[^_a-zA-Z])max_tokens\\b" | "$PY" -c '
import importlib.util, sys
from collections import defaultdict
spec = importlib.util.spec_from_file_location("scanner", sys.argv[1])
scanner = importlib.util.module_from_spec(spec)
spec.loader.exec_module(scanner)
hits = defaultdict(list)
for row in sys.stdin:
path, num, _ = row.split(":", 2)
hits[path].append(int(num))
for path, nums in hits.items():
lines = open(path, encoding="utf-8", errors="ignore").read().splitlines()
budget_lines = scanner.web_search_budget_lines(lines)
for n in nums:
if n not in budget_lines:
print(f"{path}:{n}:{lines[n - 1]}")
' "$(dirname "$0")/../../skills/migrate-sonar-to-agent-api/scripts/scan_sonar_usage.py"
}

for pat in "${ABSENT[@]}"; do
if [ "$pat" = "(^|[^_a-zA-Z])max_tokens\\b" ]; then
if [ -n "$(request_max_tokens)" ]; then
echo "FAIL leftover: $pat"
request_max_tokens | head -3
fail=1
else
echo " ok: no $pat"
fi
continue
fi
if hit "$pat" >/dev/null; then
echo "FAIL leftover: $pat"
hit "$pat" | head -3
Expand Down
53 changes: 53 additions & 0 deletions evals/migrate-sonar-to-agent-api/test_search_budgets.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
import importlib.util
from pathlib import Path
import subprocess
import tempfile
import unittest


ROOT = Path(__file__).resolve().parents[2]
spec = importlib.util.spec_from_file_location(
"scanner", ROOT / "skills/migrate-sonar-to-agent-api/scripts/scan_sonar_usage.py"
)
scanner = importlib.util.module_from_spec(spec)
spec.loader.exec_module(scanner)


class SearchBudgetTests(unittest.TestCase):
def test_only_nested_tool_budgets_are_exempt(self):
cases = [
('tools=[{"type": "web_search", "max_tokens": 2000}]', False),
('const tools = [{type: "web_search", max_tokens: 2000}];', False),
('tools=[{\n"max_tokens": 2000,\n"type": "web_search"\n}]', False),
('responses.create(\n tools=[{"type": "web_search"}],\n max_tokens=100,\n)', True),
('payload={"tools": [{"type": "web_search"}], "max_tokens": 100}', True),
('payload={"max_tokens_per_page": 2000, "max_tokens": 100}', True),
('tools=[{"type": "web_search", "max_tokens": 2000}]; max_tokens=100', True),
('tools=[{"type": "web_search"}, {"type": "other", "max_tokens": 100}]', True),
]
for source, expected in cases:
with self.subTest(source=source):
findings, _ = scanner.scan_lines(source.splitlines())
self.assertEqual(any(row[3] == "Sonar request param" for row in findings), expected)

def test_grader_rejects_request_budget_next_to_tool(self):
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "app.py"
source = '''response = client.responses.create(
tools=[{"type": "web_search", "max_tokens": 2000}],
max_output_tokens=100,
)
search_results = response.status
'''
path.write_text(source)
grader = Path(__file__).with_name("check.sh")
accepted = subprocess.run([str(grader), directory, "--style", "sdk"], capture_output=True, text=True)
self.assertEqual(accepted.returncode, 0, accepted.stdout + accepted.stderr)
path.write_text(source.replace("max_output_tokens=100,", "max_output_tokens=100, max_tokens=100,"))
rejected = subprocess.run([str(grader), directory, "--style", "sdk"], capture_output=True, text=True)
self.assertNotEqual(rejected.returncode, 0, rejected.stdout + rejected.stderr)
self.assertIn("FAIL leftover:", rejected.stdout)


if __name__ == "__main__":
unittest.main()
6 changes: 3 additions & 3 deletions skills/migrate-sonar-to-agent-api/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ Leave framework-internal LLMs pointed at other providers alone.
A consumer that only exits on `response.completed` hangs on failed runs.

6. **No `perplexity/sonar-pro` slug exists.**
Map `sonar-pro` per the two-option rule (preset `"low"` vs `perplexity/sonar` + web_search) in [models-and-presets.md](references/models-and-presets.md).
Map `sonar-pro` per the two-option rule (preset `"fast"` vs `perplexity/sonar` + web_search) in [models-and-presets.md](references/models-and-presets.md).

7. **`max_output_tokens` is REQUIRED for `anthropic/*` models** (400 otherwise); values under 16 floor to 16.
On presets and reasoning models, tight caps can be consumed by reasoning tokens before any text or search appears - size generously or omit.
Expand Down Expand Up @@ -127,7 +127,7 @@ Leave framework-internal LLMs pointed at other providers alone.
The live run is MANDATORY when an API key is available - compile-clean SDK code can still crash on call (see the typed-kwarg trap in integration-styles.md); if you truly cannot run it, say so explicitly in your summary.

**Minimal-diff rule:** keep the author's structure, naming, and voice; do not refactor unrelated code.
When a Sonar feature has no Agent equivalent (images, related questions, academic search mode), apply the documented workaround from [request-mapping.md](references/request-mapping.md) if it is small.
When a Sonar feature has no direct Agent equivalent (related questions, academic search mode), apply the documented workaround from [request-mapping.md](references/request-mapping.md) if it is small.
For `return_related_questions` specifically, use the lightweight prompt-delimiter recipe in request-mapping.md - it fits free-text flows too.
If no workaround fits, remove the feature AND call the removal out in your migration notes; never drop user-facing output silently.

Expand All @@ -136,7 +136,7 @@ If no workaround fits, remove the feature AND call the removal out in your migra
- Compile or typecheck every changed file.
- Run one real request per distinct call shape against production, with a small `max_output_tokens` and a cheap model (except on presets and reasoning models - see pitfall 7).
- Capture real response JSON for any printed samples; never fabricate response JSON.
- Grep for leftovers: `chat.completions`, `choices[0]`, `max_tokens` (as a request field), top-level `search_recency_filter`, `citations`, `delta.content`, `sonar-` model slugs.
- Grep for leftovers: `chat.completions`, `choices[0]`, `max_tokens` (as a request field, not a `web_search` tool budget), top-level `search_recency_filter`, `citations`, `delta.content`, `sonar-` model slugs.
Every hit must be intentional legacy prose.
- Full checklist in [verification.md](references/verification.md).

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,7 @@ Whichever you pick, state the choice explicitly in your migration notes.
- `max_tool_calls: 0` disables tool calls entirely - the closest analog of `disable_search` when a preset forces tools you cannot remove (note: it disables ALL tools).
- Built-in tools, one line each:
- `web_search` - grounded search; filters per [request-mapping.md](request-mapping.md); `max_results` <= 50.
- `image_search` - image retrieval; `max_results` 1..30; domain/format filters per [request-mapping.md](request-mapping.md); returns `image_search_results` output items.
- `fetch_url` - fetch pages; `max_urls` 1..10; `total_budget_tokens` <= 120000.
- `people_search` - professional profiles; bills as `search_people`.
- `finance_search` - quotes, fundamentals, filings; replaces `search_mode: "sec"`.
Expand Down
29 changes: 16 additions & 13 deletions skills/migrate-sonar-to-agent-api/references/models-and-presets.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,29 +4,31 @@ Read this when choosing a target model or preset for a migrated call site.

## Valid model slugs

`GET https://api.perplexity.ai/v1/models` (no auth required) is authoritative - always verify slugs there before writing them into code.
Agent API slugs are provider-prefixed, e.g. `perplexity/sonar`, `openai/gpt-5.1`, `anthropic/claude-sonnet-4-5`, `google/gemini-3-flash-preview`.
Verify slugs against the current [model reference](https://docs.perplexity.ai/docs/agent-api/models) or `GET https://api.perplexity.ai/v1/models` with your API key before writing them into code.
Agent API slugs are provider-prefixed, e.g. `perplexity/sonar` and `openai/gpt-6-luna`.
Check the [changelog](https://docs.perplexity.ai/docs/resources/changelog) for retirement dates when choosing a model.
Do not invent slugs: there is NO `perplexity/sonar-pro` slug.

## Model mapping table

| Sonar model | Migrate to | Rationale and caveats |
|---|---|---|
| `sonar` | `model: "perplexity/sonar"` + `tools: [{"type": "web_search"}]`, or `preset: "fast"` | Closest by name. CAUTION: output is not byte-identical to chat `sonar` and pricing differs (see the public pricing page) - A/B compare quality and cost before cutover. |
| `sonar-pro` | Two options - pick per the heuristic below | Option 1, behavioral match: `preset: "low"` (pro-search, multi-step search). Option 2, lighter direct substitution: `model: "perplexity/sonar"` + web_search. |
| `sonar-reasoning` | `preset: "low"`, or a reasoning model (e.g. `openai/gpt-5.1` + `reasoning: {"effort": "medium"}` + web_search) | Retired; do not carry the name forward. Note the substitution in your migration notes. |
| `sonar-reasoning-pro` | `preset: "low"` (closest), or a reasoning model + `reasoning: {"effort": "medium"}` + web_search + a generously sized `max_output_tokens` | `medium` is a strictly heavier, costlier tier - offer it only as an explicit upgrade. |
| `sonar-deep-research` | `preset: "medium"` (deep-research) | The medium (deep-research) preset is the closest published equivalent of sonar-deep-research. `high` is a costlier upgrade, not an equivalent; offer it only as an explicit upsell note. |
| `sonar` | `preset: "fast"`, or `model: "perplexity/sonar"` + `tools: [{"type": "web_search"}]` | `fast` is the documented starting point. Output is not byte-identical and pricing differs - A/B compare quality and cost before cutover. |
| `sonar-pro` | Two options - pick per the heuristic below | Option 1, documented starting point: `preset: "fast"`. Option 2, model-slug substitution: `model: "perplexity/sonar"` + web_search. |
| `sonar-reasoning` | `preset: "low"`, or a current reasoning model + web_search | Retired; do not carry the name forward. Verify the model's supported reasoning efforts and note the substitution in your migration notes. |
| `sonar-reasoning-pro` | `preset: "low"`, or a current reasoning model + web_search + a generously sized `max_output_tokens` | `low` is the documented starting point; evaluate heavier presets separately. |
| `sonar-deep-research` | `preset: "high"` | The current migration guide recommends `high` for deep research. |

These are starting points from the [migration guide](https://docs.perplexity.ai/docs/agent-api/migrate-from-sonar/how-to), not guarantees of equivalent behavior or cost.

### The sonar-pro two-option rule

- **Option 1 - behavioral match: `preset: "low"`.**
Caveats to state inline in migration notes: `preset` is a separate request field, not a model slug; it may resolve to a third-party model (`response.model` can echo e.g. a `google/*` model); higher latency and cost per call; via the OpenAI SDK pass `extra_body={"preset": "low"}` and omit `model`.
- **Option 2 - lighter substitution: `model: "perplexity/sonar"` + `tools: [{"type": "web_search"}]`.**
- **Option 1 - documented starting point: `preset: "fast"`.**
Caveats to state inline in migration notes: `preset` is a separate request field, not a model slug; it may resolve to a third-party model; compare quality, latency, and cost; via the OpenAI SDK pass `extra_body={"preset": "fast"}` and omit `model`.
- **Option 2 - model-slug substitution: `model: "perplexity/sonar"` + `tools: [{"type": "web_search"}]`.**
Required when the code's contract is a model slug: a `--model` CLI flag, a model allowlist, or a framework model-class bridge that takes a model string (presets do not fit there).
Also preferred for lightweight or high-frequency call sites where pro-search depth is not the point.

Selection heuristic: model-slug contract -> option 2; hardcoded single model where search depth matters -> option 1; otherwise either, but document the substitution.
Selection heuristic: model-slug contract -> option 2; otherwise start with option 1, but document and evaluate the substitution.

**Preset plus search filters:** per the presets documentation, a tool you list in the request overrides only the preset's tool of the same type; the preset's other tools stay enabled.
When the flow needs guaranteed `search_domain_filter` / recency / date filters, the simplest setup to reason about is still option 2 - an explicit model plus a `web_search` tool carrying `filters` - because nothing else is bundled alongside it.
Expand All @@ -35,7 +37,8 @@ When the flow needs guaranteed `search_domain_filter` / recency / date filters,

Know these before using presets:

- Short names `fast | low | medium | high | xhigh` are the canonical preset names; `fast-search | pro-search | deep-research | advanced-deep-research` are accepted as previous names for the first four.
- Short names `fast | low | medium | high | xhigh` are the canonical preset names; `fast-search | pro-search | deep-research | advanced-deep-research | ultra` are their respective previous names.
- Preset models and tool configuration can change; consult the current [preset reference](https://docs.perplexity.ai/docs/agent-api/presets) rather than freezing a model from an old example.
- Any request field set alongside `preset` overrides the preset's value for that field (request-over-preset precedence).
In particular, set `max_output_tokens` in the request when you need a cap.
- `instructions` alongside a preset REPLACES the preset's system prompt entirely; `""` clears it; omit the field to keep it.
Expand Down
Loading
Loading