Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 94 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
name: Release

# Publishes @diffbot/pi to npm via trusted publishing (OIDC; no NPM_TOKEN).
# Triggered by pushing a version tag (e.g. v0.4.0). workflow_dispatch runs the
# gates and `npm publish --dry-run` only; the publish job runs for tag pushes.
on:
push:
tags: ["v*"]
workflow_dispatch:

# Never cancel an in-flight run: cancelling a publish midway could leave npm
# and the GitHub Release out of sync. Runs for the same ref queue instead.
concurrency:
group: release-${{ github.ref }}
cancel-in-progress: false

jobs:
verify:
name: Verify and dry-run
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- uses: actions/checkout@v5

- uses: actions/setup-node@v5
with:
node-version: 22
cache: npm

# The tag must match package.json so the GitHub Release can never
# disagree with the version published to npm. (No VERSION constant in
# this repo's sources.)
- name: Check tag matches package version
if: startsWith(github.ref, 'refs/tags/')
run: |
pkg=$(node -p "require('./package.json').version")
tag="${GITHUB_REF_NAME#v}"
if [ "$pkg" != "$tag" ]; then
echo "::error::Tag ${GITHUB_REF_NAME} does not match package.json version ${pkg}. Bump package.json and re-tag."
exit 1
fi
echo "Tag and package version agree: ${pkg}"

# Same gates as ci.yml (typecheck is the only one; the repo has no
# lint or test scripts and no live tests).
- run: npm ci
- run: npm run typecheck

# The package ships TypeScript sources (extensions/, skills/), so there
# is no build step.
- name: Publish dry run
run: npm publish --dry-run --access public

publish:
name: Publish to npm and GitHub
needs: verify
if: startsWith(github.ref, 'refs/tags/')
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: write
# Required for npm trusted publishing and provenance attestation.
id-token: write
steps:
- uses: actions/checkout@v5

- uses: actions/setup-node@v5
with:
node-version: 22
registry-url: "https://registry.npmjs.org"

# Trusted publishing needs npm >= 11.5.1; Node 22 ships older.
- name: Update npm
run: |
npm install -g npm@latest
npm --version

# No token: npm authenticates via the trusted publisher configured on
# npmjs.com (org diffbot, repo diffbot-pi, workflow release.yml).
- name: Publish to npm
run: npm publish --access public --provenance

- name: Create GitHub Release
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
if gh release view "$GITHUB_REF_NAME" >/dev/null 2>&1; then
echo "Release $GITHUB_REF_NAME already exists; skipping."
else
gh release create "$GITHUB_REF_NAME" --title "$GITHUB_REF_NAME" --generate-notes --verify-tag
fi
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Changelog

## 0.4.0 - 2026-11-17

### Changed

- Depends on `@diffbot/typescript` `^0.6.0` (cumulative 0.4.0–0.6.0). DQL goes through the SDK `dql()` helper, which POSTs a JSON body; the token stays the query parameter.
- `dql()` format is the SDK `ExportFormat`. Chat messages are checked against `system`, `user`, and `assistant`.
- A missing crawl job (`crawlGetJob()` returning `undefined`) is reported as an error.

`package-lock.json` still records the published 0.3.0 tarball. Run `npm update @diffbot/typescript` once 0.6.0 is on npm, then tag `v0.4.0`. Publishing is npm trusted publishing from `.github/workflows/release.yml` (no token, no GitHub environment).
14 changes: 13 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,19 @@ pi -e .
pi install .
```

Depends on [`@diffbot/typescript`](https://www.npmjs.com/package/@diffbot/typescript) from npm.
0.4.0 depends on [`@diffbot/typescript`](https://www.npmjs.com/package/@diffbot/typescript) ^0.6.0. DQL goes through the SDK `dql()` helper, which posts a JSON body; the token stays a query parameter. `package-lock.json` still records the published 0.3.0 tarball — run `npm update @diffbot/typescript` once 0.6.0 is on npm. A missing crawl job from `crawlGetJob()` is `undefined`.

## Release

Publishing uses npm trusted publishing from `.github/workflows/release.yml` (no npm token).

1. Bump `version` in `package.json` (and update `package-lock.json`) on a branch.
2. Merge to `main`.
3. Tag the merge commit and push the tag: `git tag vX.Y.Z && git push origin vX.Y.Z`.

The workflow checks the tag matches `package.json`, runs `npm ci` and `npm run typecheck`, then runs `npm publish --access public --provenance` and creates the GitHub Release with generated notes. Running the workflow manually (`workflow_dispatch`) only does the checks and `npm publish --dry-run`.

One-time npm setup (package settings → Trusted publisher): org `diffbot`, repo `diffbot-pi`, workflow `release.yml`, no environment.

## Package layout

Expand Down
32 changes: 29 additions & 3 deletions extensions/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ import {
entities,
extract,
webSearch,
type ChatMessage,
type ExportFormat,
type JsonObject,
} from "@diffbot/typescript";
import { FileOntologyStore, resolveToken } from "@diffbot/typescript/node";
Expand Down Expand Up @@ -65,6 +67,23 @@ function fail(
};
}

const EXPORT_FORMATS = ["json", "csv", "xls", "xlsx", "jsonl"] as const;

function asExportFormat(value: string | undefined): ExportFormat {
if (value === undefined || value === "json") return "json";
if ((EXPORT_FORMATS as readonly string[]).includes(value)) return value as ExportFormat;
throw new Error(`Unsupported format "${value}". Use json, csv, xls, xlsx, or jsonl.`);
}

function toChatMessages(messages: Array<{ role: string; content: string }>): ChatMessage[] {
return messages.map((message) => {
if (message.role !== "system" && message.role !== "user" && message.role !== "assistant") {
throw new Error(`Message role must be system, user, or assistant (got "${message.role}")`);
}
return { role: message.role, content: message.content };
});
}

function asObject(value: unknown): JsonObject {
return value && typeof value === "object" && !Array.isArray(value)
? (value as JsonObject)
Expand Down Expand Up @@ -159,7 +178,7 @@ export default function (pi: ExtensionAPI) {
async execute(_toolCallId, params, _signal, _onUpdate, _ctx) {
try {
const db = getClient();
const format = params.format ?? "json";
const format = asExportFormat(params.format);
// The API rejects `;`-joined bare field paths (422). If the filter is
// not JsonPath (no `$`), convert `;`-separated field names into the
// space-separated dot-notation form the API expects.
Expand Down Expand Up @@ -448,15 +467,16 @@ export default function (pi: ExtensionAPI) {
}),
async execute(_toolCallId, params, signal, _onUpdate, _ctx) {
try {
const messages =
const drafted =
params.messages && params.messages.length > 0
? params.messages
: params.prompt
? [{ role: "user", content: params.prompt }]
: null;
if (!messages) {
if (!drafted) {
throw new Error("Provide prompt or messages");
}
const messages = toChatMessages(drafted);
const db = getClient();
if (params.schema) {
const answer = await askJson(db, messages, params.schema as JsonObject);
Expand Down Expand Up @@ -574,6 +594,12 @@ export default function (pi: ExtensionAPI) {
? await crawlGetJob(db, params.jobName)
: await crawlListJobs(db);
await db.close();
if (params.jobName && data === undefined) {
return fail(
"diffbot_crawl_jobs",
new Error(`No crawl job named "${params.jobName}"`),
);
}
return ok(formatJson(data));
} catch (err) {
return fail("diffbot_crawl_jobs", err);
Expand Down
12 changes: 6 additions & 6 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@diffbot/pi",
"version": "0.3.0",
"version": "0.4.0",
"description": "Pi package for Diffbot — structured web knowledge tools for the Pi coding agent",
"type": "module",
"license": "MIT",
Expand Down Expand Up @@ -39,7 +39,7 @@
"typecheck": "tsc --noEmit"
},
"dependencies": {
"@diffbot/typescript": "^0.3.0"
"@diffbot/typescript": "^0.6.0"
},
"peerDependencies": {
"@earendil-works/pi-coding-agent": "*",
Expand Down
2 changes: 1 addition & 1 deletion skills/diffbot-web-search/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ current state matters.
| `maxTokens` | none | Caps total response tokens — trims from the bottom of the ranking |
| `snippetLength` | `long` | `short`, `medium`, `long` or `full`: how much of each page's matching content comes back in `content`. **Always pass `"short"` for searches** — ~1–1.5 KB per result instead of 2.5–6 KB, and the matching sentence is still in the chunk. **Pass `"full"` for a `url:` read** of one whole page; it disables reranking, so query terms are ignored |

Requires `@diffbot/typescript` >= 0.3.0 (`snippetLength` was added in 0.3.0).
Requires `@diffbot/typescript` ^0.6.0 (`snippetLength` was added in 0.3.0).

## Workflow

Expand Down
Loading