Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Runs on Bun >= 1.2 or Node >= 24. The built `dist/` entry is the default import.
bun add @corbits/reranking
```

`rerankDocuments(query, docs, config, options)` validates config with `RerankConfigSchema`, short-circuits empty input with no request, and returns ranked results. Options carry `deps`, with optional `retryPolicy`, `registry`, and `signal`. Config takes `baseURL` and `apiStyle`, with optional `model` (used by Cohere and Voyage), `apiKey`, and `timeoutMs`.
`rerankDocuments(query, docs, config, options?)` validates config with `RerankConfigSchema`, short-circuits empty input with no request, and returns ranked results. Options are optional: `deps` (defaults to global `fetch` and `createDefaultScheduler()`), `retryPolicy`, `registry`, and `signal`. Config takes `baseURL` and `apiStyle`, with optional `model` (used by Cohere and Voyage), `apiKey`, and `timeoutMs`.

A retrieval pipeline reads its reranker endpoint from host config at boot — a half-configured reranker should fail boot, not degrade silently later — then reranks each query's fused candidates, falling back to the caller's own ordering on failure so a reranker outage never fails search:

Expand Down Expand Up @@ -77,7 +77,7 @@ Reranking has no single OpenAI-style standard, so this package draws the adapter

Jina's `/rerank` follows the Cohere shape and is served by the `cohere` adapter.

`RerankAdapter` mirrors inference's `ProviderAdapter` with `buildRequest` / `parseResponse` plus optional `extractRetryAfterMs`. Styles resolve through `createRerankAdapterRegistry`, which keeps a private `Map` copy so config-supplied style names resolve safely. Pass a custom `registry` to add a house format alongside the built-ins (`rerankAdapters`, `rerankAdapterRegistry`).
`RerankAdapter` mirrors inference's `ProviderAdapter` with `buildRequest` / `parseResponse` plus optional `extractRetryAfterMs`. Styles resolve through `createRerankAdapterRegistry`, which keeps a private `Map` copy so config-supplied style names resolve safely. Pass a custom `registry` to add a house format alongside the built-ins in `rerankAdapterRegistry`.

Every protocol addresses documents by position in the request array, and TEI replies unordered. Results map back through `docs[index]`, with an out-of-range index raising so a score always lands on the right document. `RerankDoc.id` carries your stable identifier across that index-based wire, and the final list sorts by score descending.

Expand Down
2 changes: 1 addition & 1 deletion src/adapters.ts
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ export function createRerankAdapterRegistry(
};
}

export const rerankAdapters: Readonly<Record<RerankAPIStyle, RerankAdapter>> = {
const rerankAdapters: Readonly<Record<RerankAPIStyle, RerankAdapter>> = {
tei,
cohere,
voyage,
Expand Down
1 change: 0 additions & 1 deletion src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ export {
type RerankOptions,
} from "./rerank.js";
export {
rerankAdapters,
rerankAdapterRegistry,
createRerankAdapterRegistry,
type RerankAdapter,
Expand Down
26 changes: 12 additions & 14 deletions src/request.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ import {
classifyHTTPError,
classifyNetworkError,
classifyProtocolMismatch,
createDefaultRetryPolicy,
type BuiltRequest,
type Dependencies,
} from "@intx/inference";
Expand Down Expand Up @@ -59,29 +58,25 @@ export type RetryAfterExtractor = (headers: Headers) => number | undefined;

export type RunRequestOptions = {
deps: RequestDependencies;
/** Defaults to Interchange's policy: back off retryables, abort the rest. */
retryPolicy?: RetryPolicy;
retryPolicy: RetryPolicy;
/**
* Per-attempt ceiling, enforced alongside any caller `signal` rather than
* instead of it. A cold local model can take a while to page in.
*/
timeoutMs?: number;
timeoutMs: number;
/**
* Reads `Retry-After` off a failed response. Mirrors `ProviderAdapter`'s
* member of the same name, so a provider that signals pacing its own way can
* override without touching the transport. Defaults to
* {@link extractRetryAfterMs}.
* override without touching the transport.
*/
extractRetryAfterMs?: RetryAfterExtractor;
signal?: AbortSignal;
extractRetryAfterMs: RetryAfterExtractor;
signal: AbortSignal | undefined;
};

type Attempt =
| { ok: true; body: unknown }
| { ok: false; error: InferenceError };

const DEFAULT_TIMEOUT_MS = 30_000;

/**
* `Retry-After` in seconds or as an HTTP date; undefined when absent.
*
Expand Down Expand Up @@ -199,10 +194,13 @@ export async function runJSONRequest(
request: BuiltRequest,
options: RunRequestOptions,
): Promise<unknown> {
const { deps, signal } = options;
const policy = options.retryPolicy ?? createDefaultRetryPolicy();
const timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS;
const extractRetryAfter = options.extractRetryAfterMs ?? extractRetryAfterMs;
const {
deps,
signal,
retryPolicy: policy,
timeoutMs,
extractRetryAfterMs: extractRetryAfter,
} = options;

// Time comes from the harness scheduler, not globals, so a virtual-clock
// test scheduler drives the retry loop deterministically.
Expand Down
10 changes: 5 additions & 5 deletions src/rerank.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import { rerankDocuments, type RerankConfig } from "./rerank";
import {
createRerankAdapterRegistry,
rerankAdapterRegistry,
rerankAdapters,
type RerankAdapter,
type RerankAPIStyle,
} from "./adapters";
import { extractRetryAfterMs, RerankRequestError } from "./request";
Expand Down Expand Up @@ -232,7 +232,7 @@ describe("retry behaviour inherited from the inference policy", () => {
const seen: Array<string | null> = [];
const registry = createRerankAdapterRegistry({
tei: {
...rerankAdapters.tei,
...rerankAdapterRegistry.resolve("tei"),
extractRetryAfterMs: (headers) => {
seen.push(headers.get("x-ratelimit-reset"));
return 0;
Expand Down Expand Up @@ -301,11 +301,11 @@ describe("adapter registry", () => {
});

it("is not mutable through the map it was built from", () => {
const source: Record<string, (typeof rerankAdapters)["tei"]> = {
tei: rerankAdapters.tei,
const source: Record<string, RerankAdapter> = {
tei: rerankAdapterRegistry.resolve("tei"),
};
const registry = createRerankAdapterRegistry(source);
source.voyage = rerankAdapters.voyage;
source.voyage = rerankAdapterRegistry.resolve("voyage");
expect(registry.has("voyage")).toBe(false);
});
});
34 changes: 18 additions & 16 deletions src/rerank.ts
Original file line number Diff line number Diff line change
@@ -1,29 +1,33 @@
import { type } from "arktype";
import {
classifyProtocolMismatch,
createDefaultRetryPolicy,
createDefaultScheduler,
ProtocolMismatchError,
} from "@intx/inference";
import type { RetryPolicy } from "@intx/types/runtime";

import {
rerankAdapterRegistry,
type RerankAdapterRegistry,
type RerankAPIStyle,
type RerankDoc,
type RerankResult,
} from "./adapters.js";
import {
extractRetryAfterMs,
RerankRequestError,
runJSONRequest,
type RequestDependencies,
} from "./request.js";

const DEFAULT_TIMEOUT_MS = 30_000;

export const RerankConfigSchema = type({
/** Provider root, e.g. `http://localhost:8085` for a TEI server. */
/** Provider root, e.g. `http://localhost:8080` for a TEI server. */
baseURL: "string",
/**
* A key into the registry. Left open rather than pinned to
* {@link RerankAPIStyle} so a custom registry can name a house format; the
* `RerankAPIStyle` so a custom registry can name a house format; the
* registry rejects an unknown style by name. The built-ins are `tei`,
* `cohere` (also Jina) and `voyage`.
*/
Expand All @@ -36,7 +40,8 @@ export const RerankConfigSchema = type({
export type RerankConfig = typeof RerankConfigSchema.infer;

export type RerankOptions = {
deps: RequestDependencies;
/** Defaults to global `fetch` and `createDefaultScheduler()`. */
deps?: RequestDependencies;
retryPolicy?: RetryPolicy;
/** Defaults to the built-in registry: TEI, Cohere/Jina and Voyage. */
registry?: RerankAdapterRegistry;
Expand All @@ -55,7 +60,7 @@ export async function rerankDocuments(
query: string,
docs: readonly RerankDoc[],
config: RerankConfig,
options: RerankOptions,
options: RerankOptions = {},
): Promise<RerankResult[]> {
const parsed = RerankConfigSchema.assert(config);

Expand All @@ -70,15 +75,14 @@ export async function rerankDocuments(
});

const body = await runJSONRequest(request, {
deps: options.deps,
...(options.retryPolicy !== undefined
? { retryPolicy: options.retryPolicy }
: {}),
...(parsed.timeoutMs !== undefined ? { timeoutMs: parsed.timeoutMs } : {}),
...(adapter.extractRetryAfterMs !== undefined
? { extractRetryAfterMs: adapter.extractRetryAfterMs }
: {}),
...(options.signal !== undefined ? { signal: options.signal } : {}),
deps: options.deps ?? {
fetch: globalThis.fetch,
scheduler: createDefaultScheduler(),
},
retryPolicy: options.retryPolicy ?? createDefaultRetryPolicy(),
timeoutMs: parsed.timeoutMs ?? DEFAULT_TIMEOUT_MS,
extractRetryAfterMs: adapter.extractRetryAfterMs ?? extractRetryAfterMs,
signal: options.signal,
});

let scored: Array<{ index: number; score: number }>;
Expand Down Expand Up @@ -108,5 +112,3 @@ export async function rerankDocuments(
})
.sort((a, b) => b.score - a.score);
}

export type { RerankAPIStyle, RerankDoc, RerankResult };
Loading