diff --git a/docs/architecture-decisions/013-integration-version-migrations.md b/docs/architecture-decisions/013-integration-version-migrations.md index ddce36a8f..b397a4ad1 100644 --- a/docs/architecture-decisions/013-integration-version-migrations.md +++ b/docs/architecture-decisions/013-integration-version-migrations.md @@ -112,5 +112,6 @@ separate ADR, to be authored independently; this ADR keys off whatever that cont - [ADR-012: Database Schema Migrations](./012-database-schema-migrations.md) - [ADR-010: Reporting as an Admin Operation](./010-reporting-as-admin-operation.md) (shared runner + isolation) - [ADR-005: Admin Script Runner Service](./005-admin-script-runner.md) -- Integration Versioning ADR — *to be authored separately* (defines the version contract this keys off). +- [ADR-030: Integration Versioning](./030-integration-versioning.md) (defines the version contract this keys off). +- [ADR-063: Integration Definitions as Data](./063-integration-definitions-as-data.md) — for data-defined integrations, the migrations here reduce to `planUpgrade` rewriting `Integration.config` plus link rebuilds after an identity change (ADR-063 §17). - Implementation: `packages/devtools/migrations/` (`MigrationManager`, `Migrator`, `Options`). diff --git a/docs/architecture-decisions/030-integration-versioning.md b/docs/architecture-decisions/030-integration-versioning.md index 7f001dedb..c10219897 100644 --- a/docs/architecture-decisions/030-integration-versioning.md +++ b/docs/architecture-decisions/030-integration-versioning.md @@ -231,6 +231,19 @@ Enumerated to make the failure modes concrete. Any chosen path must handle each - **Punt to per-adopter.** Rejected. Each adopter would build a slightly different version story, and ADR-013's migration runner needs a shared version contract to key off. - **Fold this into ADR-013.** Rejected in ADR-013 itself. Migration and versioning are distinct concerns with distinct lifecycles. +## Note: data-defined integrations (ADR-063) + +[ADR-063](./063-integration-definitions-as-data.md) adds a second authoring form, in which an integration is a JSON +definition run by one shared interpreter class. **For that form only**, Path D is taken: an installation pins an +exact, immutable definition version (`Integration.config.definition`, mirrored in `Integration.version`), queued +messages carry the version they were routed under and finish on it, and rollback moves the pin back. The same note +answers open question 3 for that form: the change level is computed from the two documents, and a publish whose +declared semver bump is smaller is refused (`checkBump`). ADR-063 §17 goes through this ADR's serverless concerns +one by one. + +This does not decide anything for code-defined classes. A code-defined integration still has one `Definition.version` +per deploy, and every path and open question in this ADR still applies to it. + ## Open questions 1. **Which path?** The path selection is the primary open question. Suggested criteria: which path minimizes the fleet-operation cost of a version bump? Which path handles signed webhook URLs without re-registration? Which path is compatible with the shipped `Integration Router v2` (ADR-006)? @@ -256,5 +269,6 @@ Explicit follow-ups Sean called out for further investigation before landing a d - [ADR-006: Integration Router v2](./006-integration-router-v2.md): current routing surface any versioning scheme must integrate with. - [ADR-012: Database Schema Migrations](./012-database-schema-migrations.md): schema evolution, distinct from record versioning. - [ADR-013: Integration Version Migrations](./013-integration-version-migrations.md): record transformation between versions. This ADR provides the version contract ADR-013 keys off. +- [ADR-063: Integration Definitions as Data](./063-integration-definitions-as-data.md): takes Path D for data-defined integrations only (see the note above). - [ADR-004: Project Structure Migration Tool](./004-migration-tool-design.md): project-scaffold migration; distinct from all of the above. - Implementation surface: `packages/core/integrations/integration-base.js` (`Definition.version`, `supportedVersions`, `getCurrentVersion`), `packages/core/prisma-postgresql/schema.prisma` (`Integration.version`), `packages/core/integrations/repositories/integration-repository-*.js`. diff --git a/docs/architecture-decisions/063-integration-definitions-as-data.md b/docs/architecture-decisions/063-integration-definitions-as-data.md index e93c63083..d87e5e707 100644 --- a/docs/architecture-decisions/063-integration-definitions-as-data.md +++ b/docs/architecture-decisions/063-integration-definitions-as-data.md @@ -1,6 +1,6 @@ # ADR-063: Integration Definitions as Data -**Status**: Proposed (exploratory: direction set, mechanics deliberately open) +**Status**: Proposed. Direction set; v0.1 implemented in `packages/core/integrations/data-defined/` (see Implementation) **Date**: 2026-10-15 **Deciders**: Sean Matthews @@ -141,6 +141,540 @@ Because a definition can only compose allow-listed, versioned code, a workspace- - Whether code integrations can be lowered to data. Probably not in general; possibly for the subset that only uses object routes plus mapping. - How a visual editor round-trips with hand-edited files. +## Implementation (v0.1) + +The decision above was written before any code existed. v0.1 is a working implementation in +`packages/core/integrations/data-defined/`, with 128 tests (the persistence suite runs against real PostgreSQL and +MongoDB). It runs 16 example definitions (a phone-to-CRM definition is the +reference: seven workflows, one two-way sync, two UI surfaces) against fake API modules, and checks 21 capability +manifests. This section records how it works, where it changed the sketch above, and what it does not do yet. File +references are relative to that directory. + +v0.1 is the version to iterate on for about a month before anything becomes v0.2. The gaps the examples exposed were +therefore closed inside v0.1 rather than deferred (§20). + +### 7. Three layers, stored separately + +An integration splits into three things that change at different rates. Mixing them is what makes "edit an +integration" dangerous today, because a class holds all three. + +| Layer | What it holds | Where | Changes when | +|---|---|---|---| +| **Definition** | What the integration *can* do: modules, settings, syncs, workflows, queries, UI, templates, lifecycle | `IntegrationDefinitionVersion` (new table), one immutable row per published version | A version is published, then never again | +| **Installation** | What one customer turned on and chose, plus the version they pin | `Integration.config` (existing column, new shape) | The customer configures, or an upgrade moves the pin | +| **State** | What has happened | `IntegrationMapping` (existing) and `WorkflowRun` (new) | Every event | + +The two new tables are real Prisma models in both schemas (`prisma-postgresql/schema.prisma`, +`prisma-mongodb/schema.prisma`), with the PostgreSQL migration `20261018000000_add_data_defined_integrations`. +`WorkflowRun` rows are deleted with their `Integration`. `schema/records.md` explains the fields. Worked rows, +generated by running the code rather than written by hand, are in `examples/records/`. + +The repositories (`repositories/`) store rows only: one class each for MongoDB, PostgreSQL and DocumentDB, behind a +factory that reads `DB_TYPE`. The rules live in `use-cases/definition-use-cases.js`. `PublishDefinitionVersion` +validates both layers (§11). Republishing an identical document is a no-op, and a different document at a published +version is refused (`IMMUTABLE`). It also refuses a version that is not newer than the latest published one +(`NOT_NEWER`) and a bump smaller than the changes (`BUMP_TOO_SMALL`). A draft is rewritten only while it is still a +draft: `updateIfStatus` is a conditional write the database checks. Two concurrent publishes of one version therefore +cannot both win. A loser with the same checksum gets the winner's row, and one with a different checksum gets +`CONFLICT`. `createDefinitionCatalog()` is the store the runtime reads. It serves only published, deprecated or +retired rows, never a draft, and caches them by `id@version`. It verifies a pinned checksum on every read +(`CHECKSUM_MISMATCH`). + +- **`IntegrationDefinitionVersion`**: `definitionId`, `version` (semver), `schemaVersion`, `status` + (`DRAFT` → `PUBLISHED` → `DEPRECATED` → `RETIRED`), `document` (the definition verbatim), `checksum` + (`sha256:` of canonical JSON with sorted keys, `canonical.js`), `changeLevel` and `previousVersion`. Unique on + `(definitionId, version)`. Only a `DRAFT` may be edited. Publishing runs `validate` and `checkBump` and freezes the + row. A `DEPRECATED` version keeps running for its pins but cannot be newly installed. A version is `RETIRED` only + when no installation pins it and no queued message names it. +- **`Integration.config`**: + ```jsonc + { + "definition": { "id": "phone-crm", "version": "1.4.0", "checksum": "sha256:…" }, // the pin + "settings": { "fromNumber": "+15125550100" }, + "syncs": { "contacts": { "enabled": true, "mapping": [ /* optional full override */ ] } }, + "workflows": { "missed-call-task": { "enabled": true, "settings": { "owner": "81273645" } } }, + "ui": { "phone-card": { "enabled": true } }, + "runtime": { "installed": { "hook": { "id": "WHk2c9" } } } // written by the runtime only + } + ``` + An absent `enabled` means "use the definition's `default`". The existing `Integration.version` column mirrors the + pin so it can be indexed. `runtime.installed` holds each `onInstall` step's result by step id (§15). +- **`IntegrationMapping`** holds three kinds of row, distinguished by key prefix: + - `link:::`: "this record on this side is linked to these ids on the other side". `` is a + sync id or a link set id (§8). + - `idx:::`: the identity index, for example `idx:contacts:phone:+15125550142`. + - `evt:::`: the dedupe claim, written insert-only. + + The mapping repository already exists, so no new table is needed for state. +- **`WorkflowRun`** (new) holds one row per run. Its fields are `definition {id, version}`, `workflowId`, `trigger`, + `status` (`completed | stopped | filtered | duplicate | failed`), `reason`, `error`, `retryable` and a step trace. + This is the "step-level traces from day one" that the Negative consequences below demand. + Every run is written after it finishes. A failed write is logged and never fails the run, because the run's effects + have already happened. + +### 8. The definition document + +JSON Schema: `schema/integration-definition.v0.1.schema.json`. Top-level keys: + +| Key | Holds | +|---|---| +| `id`, `version`, `schemaVersion`, `display` | Identity | +| `modules` | `{ : { module: "hubspot", version: "^3" } }`. The key is how steps refer to the module | +| `settings` | Integration-level settings: `type`, `label`, `required`, `default`, `options`. An option setting names a module's option list (`hubspot.owner`), or `{ source, with }` when the list takes inputs read from earlier choices: `{ "source": "asana.project", "with": { "workspaceGid": "$.settings.workspace" } }` | +| `syncs[]` | `id`, `source`/`target` (`module.object`), `direction` (`one_way | two_way`), `identity.match[]` (`{key, source, target, transform}`), `mapping[]` (rows read a `source` or write a fixed `value`), `backfill`, `ongoing` (`webhook | poll | none`), `default`. A definition may be syncs only, with no workflows | +| `links[]` | Link sets: `id`, `between` (two `module.object` on different modules), `identity.match[]` (`{key, transform}`). Records known to be the same thing with no sync behind them, such as a Calendly invitee and the Salesforce Lead it booked as | +| `workflows[]` | `id`, `label`, `description`, `default` (`on | off`), `trigger`, per-workflow `settings`, `steps[]` | +| `queries[]` | Named read-only step lists that feed UI surfaces | +| `ui[]` | `id`, `host` (module key), `placement`, `query`, `view` (a component tree), `default` | +| `templates` | Named text templates | +| `lifecycle` | `onInstall` and `onUninstall` step lists, for example creating and deleting a provider webhook subscription | + +The reference document (phone to CRM) and more are in `examples/`. Example ids in this ADR are generic; the files keep their own names. + +**Mappings: where v0.1 departs from ADR-057.** The `syncs[].mapping` rows above (a `source` or a fixed `value`, plus a `transform`) belong to the definition. They are not [ADR-057](./057-canonical-schema.md) module mappings (`extensions.mapping`) or `canonical.rules`, and the transforms they name (`trim`, `to_e164_phone`, `split_full_name`, `to_iso_date`, ... in §10) have their own names, where ADR-057 names its transforms differently (`joinName`, for example). ADR-057 is still an open PR (#676). §1 states the goal: one transform vocabulary, and module mappings used wherever a module ships one. Closing the gap means reading ADR-057's module mappings when they exist and reconciling the transform names so one operation has one name. Until then the schema's transform list is pinned to the runtime's (§11), which keeps this form consistent with itself, not with ADR-057. + +### 9. Capability manifests: how a definition reaches code without containing any + +The allow-list in §1 needs something to list. In v0.1 each API module ships a **capability manifest** next to its +code (`schema/capability-manifest.v0.1.schema.json`; mocks for HubSpot and a phone module in `capabilities/`): + +- **`objects`**: fields, with type, read-only flag and an ontology tag (`person.email`, `person.phone`). The validator + checks mapping rows and identity keys against them. +- **`events`**: each topic the module can deliver, with `record` (where the record sits in the payload; a phone module uses + `data.object`), `idPath` (the provider's event id, used for dedupe) and an example payload. +- **`capabilities`**: `.` → `{ kind: read | write, method, input }`. `method` names a method on the + module's existing `Api` class. A step `hubspot.task.create` therefore resolves to `HubSpotApi.createTask(args)`, and + nothing else is reachable. +- **`options`**: the lists a settings dropdown can draw from, such as owners or phone numbers: a read capability, + `label` and `value` expressions over `$.item`, and `inputs` when the capability needs arguments (Asana projects need + a workspace). +- **`account`**: values about the connected account that the module persists after auth (`apiPropertiesToPersist`), + such as QuickBooks' `realmId` or a Zendesk subdomain. Definitions read them as `$.install.accounts..`; + nothing undeclared is exposed. +- **`ui`**: the placements this host offers, such as HubSpot's `crm.record.sidebar` and a phone module's `conversation.sidebar`. +- **`webhook`**: how a POST from this provider becomes module events (`webhooks.js`), written with the same template + expressions definitions use, over `body`, `headers` (lower-case), `query`, `item` and `parent`: + ```jsonc + "webhook": { "scope": "app", // or "installation": a URL per installation + "batch": "body", // the array of events; [outer, inner] for two levels (QuickBooks) + "topic": "{{item.subscriptionType}}", // a list tries each until one is non-empty (Slack) + "eventId": "{{item.eventId}}", "account": "{{item.portalId}}", + "verify": "verifyWebhookSignature", // the module's Api method; required to return true + "handshake": { "when": "…", "body": { "challenge": "$.body.challenge" } }, + "unwrap": "payload" } // a form field holding the JSON body (Slack interactivity) + ``` + Each event names the provider's topic (`"match": "orders/create"`, default: the event key) and may add a condition + (`"when"`): a phone module reports a missed call as `call.completed` with status `missed`, so `call.missed` is that subset. An + event's example `payload` (with `headers` when the topic is in one) is an exact delivery, and validation requires it + to route to its own event through the same code a live webhook uses. + +The manifest is the module author's promise of what is safe to call. Adding a capability is a module release, which +is code that gets reviewed. Using it from a definition is a publish. This is the line between the two authoring forms. + +Because definitions are checked against manifests, a broken manifest would let a broken definition through, so +manifests are validated too (`manifests.js` `validateManifest`): capability kinds, option lists that call a read +capability and declare its required arguments as inputs, option labels that are one reference or a template, enum +fields with values, and event `record` paths that exist in the example payload. Writing it found three real faults +in the mocks: HubSpot's owner label rendered `$.item.firstName` literally, a Shopify capability had a kind the schema +does not allow, and a HubSpot enum listed no values. + +### 10. Steps and expressions (as built) + +A step is `{ id?, use, with?, when?, onError? }` (`steps.js`). `use` is either a built-in or +`..`. There is no `http`, `eval` or `script` step. + +| Built-in | Does | +|---|---| +| `links.find` / `links.upsert` | Read and write the link and identity rows above, by `sync` or by `link` (a link set), normalising keys with that set's identity transform | +| `map` | Apply a sync's mapping, or the installation's override of it, or inline rows (`transforms.js`: `trim`, `lowercase`, `to_e164_phone`, `split_full_name`, `to_iso_date`, `map_values`, `units_to_cents`, `from_unix_seconds`, …). A row reads `source` or writes a fixed `value` | +| `merge` | Deep-merge objects into one: later wins, lists are replaced, inputs are untouched | +| `template` | Render a named or inline template | +| `set` | Write into `$.vars` | +| `stop` | End the run as `stopped`, a normal outcome and not an error | +| `foreach` | Nested steps per item, `max` defaults to 100, hard cap `FOREACH_HARD_CAP` = 1000. `where` skips items (checked before the cap, so `max` bounds the work done); `collect` makes the step's result a list of one value per item, which may read that iteration's steps | +| `branch` | First case whose `when` holds, else `else` | +| `code` | Call a registered, versioned code step: `"pkg#fn@1.2.0"`. This is the only way custom code runs | + +`onError` is `fail` (the default: the run fails and the queue retries), `continue` or `stop`. + +**Expressions** (`expressions.js`) take three forms: +- a reference, `$.event.data.object.from`, which keeps its type; +- a template, `"Call · {{record.duration | minutes}}m"`. A filter argument may itself be a path. A string that is + exactly one tag keeps the value's type, so `"{{item.Amount | to_cents}}"` is a number and `"{{steps.ids}}"` a list. + Filters are named, total functions (bad input gives blank, never a throw): + - text and dates: `upper`, `lower`, `truncate`, `default`, `date`, `json`, `minutes`, `seconds`; + - lists: `join`, `size`, `first`; + - arithmetic: `plus`, `minus`, `times`, `divided_by` (by zero is blank), `round`, `abs`, `number`; + - money: `to_cents` / `from_cents`, and `to_minor_units` / `from_minor_units` with a currency argument, because JPY + has no minor unit and KWD has three. Rounding is half-up in decimal, so 19.995 is 2000 cents, not the 1999 that + `19.995 * 100` gives in binary floating point; + - time: `from_unix` / `to_unix`. +- a condition, `"$.steps.contact == null && $.record.direction in ['incoming']"`. + +Conditions are tokenised and parsed by a small recursive-descent parser. Nothing is passed to `eval` or `Function`. +The roots a path may start from are fixed: `event`, `record`, `steps`, `settings`, `install`, `vars`, `item`, +`index`, `data`, `row` and `now`. + +**Where this departs from §1:** +- §1 said "the day a definition needs a loop, it should become a code step". The stale-contacts workflow needed one, + so v0.1 has `foreach`, but bounded. It cannot recurse or exceed 1000 items, so the "total, side-effect-free" + property holds in practice. +- The condition language is slightly richer than `eq/in/exists/and/or/not`: it also has comparisons, `contains` and + `!`. It has no arithmetic and no calls. +- Arithmetic exists, but only as named filters. Converting cents or unix time is the most common reason the examples + reached for a `code` step, and a code step for `amount / 100` costs a reviewed release. A filter is total and + side-effect free, which keeps §1's property; an expression language would not. + +Treat all three as the ceiling, not a first step. + +### 11. Validation is the compile step + +`validate(def, manifests)` (`validate.js`) returns `{ valid, errors[], warnings[] }`, and every error carries a +`path`. Beyond JSON Schema, it resolves every reference a schema cannot see: + +- Ids are unique across all kinds, link sets included. +- Every module key has a manifest. +- A setting's `options` source exists, its required inputs are supplied, it reads no undeclared input, and inputs + read only `$.settings` at the same level (a workflow setting may also read `$.install`), never the setting itself + and never in a loop. +- Sync fields exist on both objects, identity keys are declared, and a write never targets a read-only field. Every + mapping row has a source or a fixed value, never both. +- A link set joins two existing objects on different modules. +- Every capability step exists, required arguments are present and undeclared ones are refused. +- `$.steps.x` only refers to an *earlier* step, `$.settings.y` to a declared setting, and `$.item` appears only + inside `foreach`. +- `links.*` names exactly one sync or link set; its keys are identity keys of that set, and `from` / `ids` name + modules the set joins. Templates exist, and `code` refs are well formed. `foreach` has `do`, `collect` or both, and + `max` is 1 to 1000. +- Every template tag, anywhere (step arguments, named templates, views), uses a known filter. A named template that + no step renders is a warning. +- `$.install` paths exist: an integration setting, `accounts..` the manifest declares, or + `runtime.installed.`, which install steps themselves cannot read. +- Uninstall undoes install: an `onInstall` step with no `id`, or whose kept result `onUninstall` never reads, is a + warning (`--strict` makes it fail), because it is usually a webhook nobody deletes. +- A webhook trigger's topic is in the manifest's `events`, and a cron has five fields. +- A `userAction` with `from` names a real UI surface, and a `syncRecord` trigger names a real sync. +- A UI placement is offered by the host, its query exists, components are in the fixed set, and every `Button` + targets a `userAction` workflow. A view reads only `$.data` (and only keys its query returns), `$.record` and + `$.install`; `$.item` only inside a `List` row and `$.row` only inside a `Table` column. A typo there would render + blank, so it is an error here instead. + +An invalid definition cannot be published. This is the "checks still compile" property from Context, made literal. + +Validation is two layers, run together by `schema-check.js`: the JSON Schemas (the published contract) and the +reference checks above (what the runtime enforces). A document is valid only when both pass. They run in three +places: +- `scripts/validate.js` (`npm run validate:definitions` in core): any definitions and manifests, exit `0` valid, `1` + invalid, `2` could not run; `--strict` fails on warnings and `--json` is for pipelines. It is what a publish step + calls before writing an `IntegrationDefinitionVersion` row. +- `tests/schemas.test.js`, in CI on every change: every bundled example and manifest, plus the CLI driven as a child + process, and the schema's transform list pinned to the runtime's so the two cannot drift. +- Anything that copies definitions elsewhere refuses to write invalid output; a product's definitions page + sync is one. + +`ajv` is already a dependency of `@friggframework/schemas`; core's runtime stays dependency-free and loads it only +for validation. + +### 12. Compiling an installation into ordinary Frigg events + +`compile(def, installation)` (`compile.js`) is pure. It turns a definition plus *one installation's choices* into +exactly what a hand-written integration builds in its constructor: a `this.events` map. It also returns the routing +tables the runtime needs: + +| Definition item | Compiled event | Frigg type | +|---|---|---| +| sync `contacts`, backfill on user action | `SYNC_CONTACTS_BACKFILL` | `USER_ACTION` | +| sync page worker | `SYNC_CONTACTS_PAGE` | `QUEUE` | +| sync ongoing (webhook or poll) | `SYNC_CONTACTS_ONGOING` | `WEBHOOK` or `CRON` | +| workflow `log-call` | `WF_LOG_CALL` | `WEBHOOK`, `CRON`, `USER_ACTION` or `QUEUE`, from its trigger | +| query `recent-activity` | `Q_RECENT_ACTIVITY` | `ROUTE` | +| UI surface `phone-card` | `UI_PHONE_CARD`, rendered by `POST /api/integrations/:id/actions/UI_PHONE_CARD` | `ROUTE` | + +The other outputs are `webhooks` (module → topic → event names; a pattern may end in `.*`), `schedules` and +`userActions`. + +**Both authoring forms compile to the same two things.** A code-defined integration builds `static Definition` and `this.events` in its class. A definition compiles to the same pair: the `Definition` the host reads for routes, queues and schedules, and the `this.events` map the dispatcher reads. Everything downstream (dispatcher, queue worker, scheduler, mapping repository) therefore cannot tell the forms apart. + +**Where this should meet the one-file and trigger ADRs.** Three facts the compiled output depends on are described again here and should come from the module or the framework instead: +- *Recurring work* is [ADR-055](./055-recurring-triggers.md)'s `Definition.triggers`, which ADR-055 governs for both forms. A later section (§15a, recurring schedules) will follow ADR-055. +- *Module webhook facts* (how a delivery is verified, answered, split into events and attributed to an account) come from [ADR-054](./054-one-file-integrations.md) extension contract 2: `verify`, `handshake`, `receive`, `resolve`, and `emits` / `normalize`. The manifest's `webhook` block (§9) is v0.1's own description of the same facts. When contract 2 exists, the manifest should be derived from it, not kept as a second description of one module. +- *Sync object access* (list, get, upsert, ids and modified-at paths) comes from ADR-054 §6's `objects` descriptor, and the `sync` blocks in §15b are the same kind of fact. They should be derived from `objects` when it exists. + +ADR-054 and ADR-055 are Proposed. This section records the intent, and does not depend on either being built. + +**Only enabled items are compiled.** A workflow the customer has off has no event, no route and no schedule. It +does not exist at runtime, so "disabled but still firing" cannot happen. + +### 13. Two deployment shapes + +`data-defined-integration.js` produces an `IntegrationBase` subclass in one of two ways: + +| | Shape A: `defineIntegrationClass({ id, … })` | Shape B: `defineSharedIntegrationClass({ name: 'data-defined', … })` | +|---|---|---| +| Classes | One per definition id; name and modules come from the definition | One for every definition | +| `static Definition.modules` | That definition's modules | Every API module the app has installed | +| Which definition an installation runs | The id is fixed; the version comes from the pin | `config.definition.{id, version}` | +| New version | Publish only, no deploy | Publish only, no deploy | +| New integration | A new class, so a deploy and three more functions plus a queue | A row, no deploy | +| Fits | A product team with a handful of integrations | A product that lists many integrations, where definitions accumulate without bound | + +Shape B is what actually makes §4's "constant infrastructure" true. Shape A still grows the stack per definition +id, just not per version. + +### 14. Runtime: how an event loads and runs + +**Cold start (unchanged).** `loadAppDefinition()` registers the generated classes like any others. No definition is +read at cold start. + +**Hydrating an installation.** Every Frigg use case that touches an integration (create, update, delete, the queue +worker, user actions, config options) constructs the class from its record and calls `initialize()` first. The +data-defined class loads its definition there, so every real entry point hydrates through the same path. A dry +instance with no record (an app-level webhook before its installation is known) has no pin and loads nothing. +`loadDefinition()` does five things: +1. Reads the pin from `config.definition`. With no pin, it fails loudly: "This installation does not name a + definition". +2. Fetches `definitions.get(id, version)`. Versions are immutable, so the result can be cached by + `id@version` (or by checksum) for the life of the process. +3. Runs `compile(def, config)`. +4. Builds the **ports** (`runtime.js` `makePorts`): + - `call(moduleKey, capability, args)` looks the capability up in the manifest and calls the connected module's `api[method](args)`, so credentials, token refresh and encryption stay exactly where they are today. The shared class registers modules by module name, so a definition key that differs (`crm: { module: 'hubspot' }`) is mapped to it. + - `links.find` and `links.upsert` sit over `getMapping` and `upsertMapping`. + - `dedupe.claim` sits over `createMapping`, which is insert-only and throws `MappingAlreadyExistsError` when the row exists; `dedupe.release` sits over `deleteMapping`. + - `runCode` calls registered code steps. +5. Replaces the previously compiled events in `this.events` with the compiled map, every handler being + `dispatch(ref, payload)`, then calls `registerEventHandlers()`. Replace, not merge: a workflow switched off since + the last compile loses its event immediately. + +From there, the existing dispatcher, queue worker, scheduler and mapping repository do what they already do. + +**A webhook arrives.** A provider's POST carries no Frigg vocabulary: nothing says which module sent it or which +event it is. Install steps register a per-module URL, `$.install.webhookUrls.`, which names the module, and the +module's manifest says how to read the rest (§9, `webhook` block). For example a phone module's `call.completed`: + +``` +POST /api/data-defined-integration/webhooks/?module=phone + → webhook function (no database): a handshake (Slack url_verification, Asana X-Hook-Secret) is answered here; + anything else is queued as { integrationId, body, headers, query } ── the unchanged Frigg path + → queue worker: hydrate the record → initialize() → loadDefinition() + → onWebhook({ data }) + module = ?module, else the one manifest that recognises the body + verify = manifest.webhook.verify → the module's Api method; anything but true drops the delivery (fail closed) + events = eventsIn(manifest, delivery): split a batch, read each event's topic, id and account, keep the + manifest events whose `match` (and `when`) fit → [{ topic: 'call.completed', eventId: 'EVc0a1', … }] + for each event, each enabled workflow listening for it: + queueWebhook({ event: 'WF_LOG_CALL', body: event, eventId, definitionVersion: '1.4.0' }) ← one message per workflow + → queue worker, per message: dispatch(ref, msg) + runWorkflow(): + record = manifest.events['call.completed'].record → body.data.object + filter = trigger.filter evaluated → 'filtered' (nothing claimed, nothing written) + claim = createMapping('evt:phone-crm:log-call:') → false ⇒ 'duplicate' + steps = links.find(phone) → template('call-note') → hubspot.engagement.createCall + result = WorkflowRun { status, trace[{id, use, at, status, ms, error?}] } + failed → release the claim, throw → the queue retries (the DLQ after max receives) +``` + +Fanning out one message per workflow is deliberate. One workflow's failure or retry cannot re-run, or block, the +others that listened for the same event. + +**An app-level webhook** (one URL for every account: HubSpot, Stripe Connect, GitHub Apps, QuickBooks, Slack, Linear, +Notion) is registered once on the provider's app as `…/webhooks?module=hubspot`, with no integration id. The worker +runs it on a dry instance, reads each event's account (`webhook.account`, e.g. HubSpot's `portalId`), finds the +installations of this class connected to that account (`listIntegrationsByEntityExternalId`, then each record's +type, because one portal can also be used by integrations of other types and the worker would hydrate those as +their own class), and queues the delivery once per installation. Each installation then keeps only the events +whose account is its own. + +**Delivery guarantees, stated plainly:** +- An event runs **at most once per workflow when it succeeds**, because of the claim. +- If a run fails, it releases the claim so the retry actually runs. This was a bug in the first cut, which left the + claim behind so every retry read as "duplicate". It is now pinned by a test. +- Within a retried run, steps are **at-least-once**. If step 3 wrote to HubSpot and step 4 failed, the retry writes + step 3 again. Idempotent capabilities (upserts, `links.find` before create) are the v0.1 answer. +- Per-step checkpointing comes after v0.1. + +**A schedule fires.** `CRON` → `WF_STALE_CONTACTS`, with no event id, so there is no claim. Steps such as +`hubspot.contact.search` followed by `foreach` run in the same interpreter. + +**A user clicks a button** in a UI surface. `USER_ACTION` → `WF_TEXT_FROM_CARD`, with the surface's record as +`$.record`. + +**Running an installation uses Frigg's existing, authenticated routes.** None is unauthenticated: + +| Route (all authenticated, all hydrate via `GetIntegrationInstance`) | Event | Data-defined handling | +|---|---|---| +| `GET /api/integrations/:id/config/options` | `GET_CONFIG_OPTIONS` | The configure form, generated from the definition (`config-form.js`); its data is the `Integration.config` shape, so what the UI PATCHes back is the config | +| `POST …/config/options/refresh` | `REFRESH_CONFIG_OPTIONS` | The same form re-filled from the values chosen so far: dependent dropdowns (`resolveOptions`) | +| `PATCH /api/integrations/:id` | `onUpdate` → `validateConfig` | NEEDS_CONFIG from the definition's required settings, checked without calling any API | +| `GET …/actions`, `POST …/actions/:id/options` | `GET_USER_ACTIONS`, `GET_USER_ACTION_OPTIONS` | Button workflows and backfills; a button's inputs are its options form | +| `POST …/actions/:id` | any registered event | Runs a button workflow (`WF_*`), a backfill (`SYNC_*_BACKFILL`) or renders a UI surface (`UI_*`, body `{ record }`) | + +Data-defined apps add a few routes of their own (`router.js`, mounted by `createIntegrationRouter` only when a +registered class is data-defined). These are the only new routes, and every one is authenticated: + +| Route | Who | What | +|---|---|---| +| `GET /api/data-defined/definitions` | signed-in user | The install catalogue: the latest published version of each definition | +| `GET /api/integrations/:id/upgrade?to=` | signed-in user, own installation | The upgrade plan (`planUpgrade`); changes nothing | +| `POST /api/integrations/:id/upgrade` `{ to? }` | signed-in user, own installation | Applies the plan: uninstall steps of the old version, then install steps of the new one if it asks for a reinstall, then the new config and a reload. Answers 409 with the plan when a person must supply something first | +| `GET /api/integrations/:id/runs` | signed-in user, own installation | `WorkflowRun` history, newest first (`workflowId`, `status`, `before`, `limit`) | +| `POST /admin/integrations/definitions` `{ document, notes?, publish?, strict? }` | `x-frigg-admin-api-key` | Validate and publish (or save a draft). 201 created, 200 for an identical republish, 400 invalid, 409 refused | +| `GET /admin/integrations/definitions/:id` | admin key | Every version, without documents | +| `POST /admin/integrations/definitions/:id/versions/:version/deprecate` | admin key | `PUBLISHED` → `DEPRECATED` | + +The admin routes live in core's existing `/admin/*` namespace, beside `/admin/scripts` and `/admin/db-migrate`, behind the same +`validateAdminApiKey` middleware (`handlers/middleware/admin-auth`). They are deliberately not under `/api/admin`, which would be a +third namespace. The user routes stay where they are until the `/api/v2` route registry (PR #671) lands; they move then. + +Installing needs no new route. `POST /api/integrations` with `config.definition: { id }` pins the latest published +version, and its checksum, in `onCreate`. An app that bundles its definitions (`memoryDefinitions`) has no store to +publish to, so the admin routes answer 501. + +A host renderer (the HubSpot card) therefore calls Frigg through an authenticated path. Proving the host request is +genuine (HubSpot signs its UI-extension requests) is the renderer's job, not Frigg core's. + +### 15. Lifecycle + +| Moment | What runs | Code | +|---|---|---| +| **Install** | The customer connects both modules (the existing auth flow), picks settings, and the record is created with the pin to the latest `PUBLISHED` version. `onCreate` loads the definition and runs `lifecycle.onInstall`, for example `phone.webhook.create` pointed at this installation's webhook URL. Every install step with an `id` has its result saved to `config.runtime.installed.` with `patchConfig` | `onCreate` | +| **Configure** | The customer toggles workflows, changes settings or overrides a mapping. `onUpdate` merges config, then recompiles: enabling a workflow adds its event, routes and schedule, and disabling removes them. No deploy, and no other installation is affected | `onUpdate` | +| **Update** (new version) | Nothing happens to running installations. Each stays on its pin until it is moved, as in §17 | `planUpgrade` | +| **Uninstall** | `onDelete` runs `lifecycle.onUninstall`, which reads what install created (`phone.webhook.delete` with `$.install.runtime.installed.hook.id`; a `foreach` over several webhook ids for Shopify), then the framework removes the record | `onDelete` | + +`$.install` is built by one function, `installContext`, everywhere a definition runs: workflows, queries, UI +surfaces, lifecycle steps and option lists. It holds the integration settings with their defaults, `webhookUrl`, +`integrationId`, `accounts` and `runtime.installed`. The first cut built it four ways, and the query and UI versions +had no `runtime`, so a query reading an install result silently got nothing. + +### 16. UI surfaces + +A UI extension is a `query` plus a `view`. The view is a tree of a **fixed component set**: `Stack`, `Heading`, +`Text`, `KeyValue`, `List`, `Table`, `Badge`, `Button`, `Empty` and `Divider`. It holds no HTML and no script. + +- `renderSurface` (`runtime.js`) runs the query and then `resolveView`, which substitutes expressions and expands + lists. It returns a **resolved tree**: plain JSON with data already in it. +- Each host gets **one generic renderer, deployed once**. For example, there is one HubSpot UI-extension card that + calls `POST /api/integrations/:id/actions/UI_` (authenticated) for whatever installation it sits in and draws the tree. New surfaces and new + integrations need no new host deployment. +- A `Button` can only point at a `userAction` workflow (checked by `validate`). A surface can therefore read data + and trigger declared workflows, and nothing else. + +The renderers themselves are not in v0.1. + +### 17. Versioning (answers ADR-030 for data-defined integrations) + +`versioning.js` computes the change level from the two documents, so nobody has to remember the rules. + +| Level | Changes | +|---|---| +| **patch** | Steps, templates, view and query bodies, default mapping rows, labels and help text, a setting's default | +| **minor** | Additive: a new workflow (with its `default`), a new optional setting, a new sync or link set, a new or moved UI surface, a changed trigger topic, filter or schedule, a removed setting, changed lifecycle steps, changed option-list inputs | +| **major** | Anything an installation may rely on goes away or changes meaning: a removed workflow, sync or link set, a new module, a replaced module, a new required setting with no default, a setting whose type or option list changed or that became required, changed identity keys (sync or link set), a link set joining different objects, a changed sync direction, a changed trigger source | + +- **`checkBump(from, to)`** refuses a publish whose declared semver bump is smaller than the computed level. Shipping + an identity change as 1.5.0 is an error, not a review comment. +- **`planUpgrade(installation, from, to)`** returns three things for one installation: + - `needs`: what a human must supply, such as a value for the new required setting or a connection to the new module; + - `actions`: what the runtime will do, such as re-running `onInstall` because lifecycle changed, or rebuilding + links because identity changed. A link set has no backfill to rebuild from, so a changed link-set identity is a + `reindex` action that says plainly that links by id still resolve and lookups by the new keys miss until each + pair is linked again; + - the new `config`. + + An upgrade with unmet `needs` cannot be applied. Patches and minors with no needs can be applied in bulk. +- **Draining.** Every queued message carries `definitionVersion`. `dispatch` runs a message on the version it was + queued under, not on the installation's new pin. In-flight work therefore finishes on the definition that started + it, and is never re-interpreted halfway. +- **Rollback** means moving the pin back. Both versions are still published rows, so this is a config write, not a + redeploy. A rollback across an identity change still needs the links rebuilt, and `planUpgrade` says so in the + reverse direction too. + +This is ADR-030's Path D, "version as a routing dimension outside the class", for the cost of one column. + +**ADR-030's serverless concerns, answered for data-defined integrations.** ADR-030 itself stays undecided for +code-defined classes; this is the answer only where the class is the shared interpreter. + +| ADR-030 concern | Data-defined answer (v0.1) | +|---|---| +| Lambda module-scope caching | Nothing version-specific is read at module scope. The definition is fetched when an installation is hydrated for an event and cached by `id@version`, so one warm Lambda serves several versions at once and a publish needs no deploy | +| In-flight SQS jobs | Every message carries `definitionVersion`, and `dispatch` runs it on that version: ADR-030's "dispatch by the payload's declared version" | +| EventBridge Scheduler entries | The rule: a scheduled run carries no version and runs on the installation's current pin, because a schedule belongs to the installation, not to the version that created it. `compile` produces each installation's `schedules`, so an upgrade that changes a cron yields a different list. **Registering that list is not built in v0.1** (§19), so this row is the contract for when it is | +| Signed webhook URLs | URLs are per installation, not per version, so an upgrade re-registers nothing. Only a change to `lifecycle.onInstall` re-runs install, and `planUpgrade` lists that as a `reinstall` action | +| Rollback | Moving the pin back is a config write (ADR-030's Path C behaviour). An identity change still needs links rebuilt in both directions, and `planUpgrade` says so | +| Global routing / edge | Unchanged: routing is by installation id, and the version is resolved at the origin from the record | +| Encryption schema | Unchanged: credentials stay in the module's entity and credential rows. A definition never holds a secret, and `$.install.accounts` exposes only fields a manifest declares | + +**Relation to [ADR-013](./013-integration-version-migrations.md).** ADR-013 transforms persisted records between +versions; it explicitly deferred the version model to ADR-030. For data-defined integrations the version model is +the above, and the record migrations ADR-013 describes reduce to two things: `planUpgrade` rewrites +`Integration.config` (settings dropped or added, workflows enabled or offered, the pin moved), and an identity change +rebuilds `link:` / `idx:` rows. v0.1 performs the first and reports the second as `relink` / `reindex` actions; it +does not rebuild the rows itself. + +### 18. Hosting + +A hosted multi-tenant deployment runs on AWS, one sub-account per tenant, using **Shape B**. Each +tenant's stack is constant-size: one HTTP function, one webhook function, one queue worker and one queue for every +data-defined integration. Netlify was considered for the HTTP side and rejected as the primary host for this case. [ADR-028](./028-multi-provider-support.md) (multi-provider support) is the route to a Netlify provider where that is wanted: +- function time and size limits; +- no native queue; +- no per-tenant isolation boundary to match "one sub-account per tenant". + +The noisy-neighbour work in §4 still applies inside a tenant: per-installation concurrency keys, and interactive +work kept apart from bulk work. + +### 19. Not in v0.1 + +| Missing | Why it matters | Plan | +|---|---|---| +| **Sync engine**: backfill pages, ongoing deltas, two-way master and conflict rules | `SYNC_*` events compile, but `runSync` throws "not implemented in v0.1" | Reuse the existing `SYNC_PAGE` and `Process` patterns from `BaseCRMIntegration` | +| `delay`, approvals, per-module rate limits | Common in workflows; each needs durable state between steps | With step checkpointing | +| Rebuilding links after an identity change | `planUpgrade` reports `relink` / `reindex`; nothing performs it | With the sync engine, which owns backfill | +| Registering `compiled.schedules` | Schedule-triggered workflows and polling syncs compile to `CRON` events, but nothing creates the recurring schedule. Frigg's scheduler commands schedule one-time jobs, so recurring per-installation schedules need either an EventBridge schedule group per installation or one app-level tick that fans out | A design decision of its own | +| Host renderers (HubSpot card, Phone panel) | `renderSurface` returns trees that nothing draws yet | Per host | +| Real manifests | `capabilities/*.json` are mocks written to match the Api classes | Generated from, or checked against, each module's `Api` | + +Their order is decided by a month of iteration on v0.1, not fixed here. + +**Rate limits belong to the runtime's ports, under ADR-049.** The "per-module rate limits" item is not a definition feature. [ADR-049](./049-rate-limit-awareness.md) puts the budget in the ports: the Requester waits in process for a short wait, and pacing is its §6. A module's `static rateLimit` is descriptive only (provider facts such as windows and scope, ADR-054 §6), so a definition never states a budget. A wait too long for the Requester throws `RateLimitError`, and the queue worker defers the message to `retryAt` (frigg#657) instead of failing it and letting the queue retry. A definition run must therefore let that error through unchanged, so that §14's "a failed run releases its claim and the queue retries" is not what happens to it. The "token bucket per module and installation" in the table above is ADR-049's pacing, built there and not here. + +### 20. Gaps the examples exposed, closed in v0.1 + +Writing 14 more examples against the first cut found seven places the format was thin. Each was closed inside v0.1: + +| Gap | Was | Now | +|---|---|---| +| Links needed a sync | Two Calendly syncs that never run existed only to store links | `links[]` link sets (§8); `links.*` take `{ link }` | +| No fixed values, no merge | A `default` transform on a field that does not exist | Mapping rows with `value`; a `merge` step | +| `foreach` could not collect | Digests grew a string with `set` on every item | `collect` and `where`; `join` and `size` filters | +| Templates and views unchecked | A wrong filter or `$.data` key rendered blank at runtime | Validated statically (§11), plus unused-template warnings | +| Option lists took no inputs | Asana projects and Linear states made a required argument optional | `options: { source, with }`; `resolveOptions`; manifest `inputs` | +| No arithmetic | Code steps for cents and unix time | Named, total filters (§10); currency-aware minor units | +| No sync-only example | Every example had workflows | `pipedrive-hubspot`: one two-way sync, no workflows | + +Closing them found more, each fixed and pinned by a test: +- **Install results.** `onCreate` saved only `steps.hook.id`, under a hard-coded `subscriptions.phone`. Every other + install step's result was lost, so the Pipedrive-to-phone example, both Shopify examples and zendesk-jira would have uninstalled + against ids nobody stored. Now `runtime.installed.` (§15). +- **Account values.** Four examples read `$.install.runtime.realmId`-style values nothing wrote. They come from the + connected account, so manifests declare `account` fields (§9). +- **`$.install` differed by entry point** (§15). +- **Rounding.** `round: 2` turned 1.005 into 1, and `to_cents` turned 19.995 into 1999, both from binary floating + point. Rounding is now in decimal. +- **Manifests**: three faults found by `validateManifest` (§9), and drifted JSON Schemas (transform list, required + `source`, string-only `options`) caught by running them (§11). + +Each new check was mutation-tested: 24 mutations, one per rule (a link set read as a sync, `collect` or `where` +ignored, a check switched off, currency ignored, binary rounding, install results not saved, accounts exposing every +Api property, a failed run keeping its claim, …), and every one makes the suite fail. + +**Code map.** `expressions.js`, `transforms.js` and `steps.js` define the vocabulary. `interpreter.js` runs a step +list and produces the trace. `validate.js`, `manifests.js` and `compile.js` are the compile side. `runtime.js` holds +the ports, `runWorkflow`, `runQuery`, `renderSurface`, `runLifecycle`, `resolveOptions` and `installContext`. +`versioning.js` and `canonical.js` handle publishing. `data-defined-integration.js` is the bridge to +`IntegrationBase`. The schemas are in `schema/` and the tests in `tests/`. Run the tests from that directory with +`jest --config '{}' --rootDir . tests`. + ## Where this leads: Frigg in other runtimes The bigger consequence is not "fewer CloudFormation resources". **Once an integration is a portable document, the runtime is the only thing that must exist per environment.** Today Frigg's portability is limited by being Node-on-Lambda-with-SQS. A data definition is indifferent to all three. This section maps that space; choosing runtime targets is a follow-up ADR. @@ -225,7 +759,8 @@ So "Frigg in Rust" splits into options with very different costs: - [ADR-023](./023-integration-templates.md): a template may ship as a definition document (see its Amendment (ADR-063)) - [ADR-021](./021-ontology.md): agent context for authoring definitions (not a field vocabulary) - [ADR-027](./027-ssm-parameter-offload-and-env-scoping.md): per-function environment, relevant once many integrations share functions -- [ADR-030](./030-integration-versioning.md): versioning, which the definition store makes concrete +- [ADR-013](./013-integration-version-migrations.md): record migrations between versions; §17 says what they reduce to here +- [ADR-030](./030-integration-versioning.md): versioning, which the definition store makes concrete (§17 answers its serverless concerns for data-defined integrations) - [ADR-057](./057-canonical-schema.md) (PR #676): data-only mappings this format reuses - [ADR-059](./059-app-level-entity-registry.md), open question 1: named registry entries are the closest existing proposal for more than one instance of a module in an app. The `modules.*.instance` key in the sketch depends on how that is resolved and is not decided here - [ADR-056](./056-app-profiles-tenancy-entity-scopes.md): owner, `organizationId`, `visible(context)` and binding scope for workspace-specific definitions (follow-up, §5) diff --git a/packages/core/integrations/data-defined/README.md b/packages/core/integrations/data-defined/README.md new file mode 100644 index 000000000..a4af338a9 --- /dev/null +++ b/packages/core/integrations/data-defined/README.md @@ -0,0 +1,169 @@ +# Data-defined integrations (v0.1) + +ADR-063, implemented to the point where a definition can be validated, compiled into Frigg events, executed, rendered +and versioned. An integration becomes a **document** (what it connects, what it syncs, which workflows and UI it has) +run by **one** runtime class, instead of a hand-written class per integration. + +``` +definition (JSON, immutable per version) ─┐ +capability manifests (one per API module) ─┼─ validate ─ compile ─▶ this.events (same shape as hand-written) +installation (Integration.config) ─┘ │ + event arrives ─▶ runWorkflow ─▶ steps ─▶ Api methods +``` + +## Layout + +| Path | What | +|---|---| +| `schema/integration-definition.v0.1.schema.json` | The definition format (JSON Schema 2020-12) | +| `schema/capability-manifest.v0.1.schema.json` | What an API module declares it can do | +| `schema/installation.v0.1.schema.json` | `Integration.config` for a data-defined installation | +| `schema/records.md` | Database records: definition versions, installation, mappings, workflow runs | +| `capabilities/*.capabilities.json` | Mock manifests (HubSpot, Phone, and the apps the examples use) | +| `examples/*.definition.json` | Example definitions; `examples/records/` has Phone ↔ HubSpot database records | +| `expressions.js` | `$.ref`, `{{template \| filter}}` and conditions. No eval. | +| `steps.js`, `interpreter.js`, `transforms.js` | The step catalogue and the interpreter | +| `validate.js` | Structure + cross-references against the manifests (steps, templates, views, settings, `$.install`) | +| `manifests.js` | `validateManifest`: the manifests themselves (kinds, option lists, account fields) | +| `compile.js` | Definition + installation → events, webhook routes, schedules, user actions | +| `runtime.js` | Ports (Api calls, link store), `runWorkflow`, `runQuery`, `renderSurface`, `runLifecycle`, `resolveOptions`, `installContext` | +| `versioning.js` | `diff`, `checkBump`, `planUpgrade` | +| `data-defined-integration.js` | `defineIntegrationClass`: the IntegrationBase subclass that runs it all | +| `use-cases/definition-use-cases.js` | Publish, deprecate, catalogue, history, upgrade plan; `createDefinitionCatalog()` (the store the runtime reads) | +| `repositories/` | `IntegrationDefinitionVersion` and `WorkflowRun` rows (MongoDB, PostgreSQL, DocumentDB) | +| `router.js` | The publish, catalogue, upgrade and run-history routes | + +## The pieces of a definition + +- **modules** — module keys used everywhere else (`phone`, `hubspot`), each naming a capability manifest. +- **settings** — integration-level choices (`fromNumber`); workflows can declare their own. An option setting draws + its choices from a module's option list, and may pass it inputs read from earlier choices + (`{ "source": "asana.project", "with": { "workspaceGid": "$.settings.workspace" } }`); `resolveOptions` fills the + dropdown and returns nothing until those inputs are chosen. +- **syncs** — records kept as copies on both sides: identity keys, field mapping (with transforms and master rules), + backfill and ongoing mode. A sync maintains the link + identity index every workflow uses (`links.find`). + A definition can have no workflows at all (`pipedrive-hubspot` is sync only). +- **links** — link sets: records on two modules known to be the same thing, with no sync behind them (a Calendly + invitee and the Salesforce Lead it booked as). Same storage and identity index as a sync, addressed as + `links.find { link: "people", ... }`. +- **workflows** — trigger (`webhook`, `schedule`, `userAction`, `syncRecord`) + steps. Default on or off. +- **queries** — read-only steps that return data, used by UI surfaces. +- **ui** — a surface in a host app (`hubspot` `crm.record.sidebar`): a query for data and a view tree from a fixed + component set. A generic renderer deployed once per host draws it; changing the card is a publish, not a deploy. +- **templates** — message bodies. **lifecycle** — steps run on install / uninstall (webhook subscriptions). Each + `onInstall` step with an `id` keeps its result as `$.install.runtime.installed.`, which is how `onUninstall` + deletes exactly what install created. + +## How it plugs into Frigg + +- **Hydration**: Frigg's use cases call `initialize()` on every hydrated integration; the class loads its pinned + definition there. +- **Webhooks**: install steps register `$.install.webhookUrls.` (the URL names the module). The manifest's + `webhook` block (`webhooks.js`) splits batches and reads each event's topic, id and account; `verify` must return + true; handshakes are answered at receipt. App-level webhooks (`webhook.scope: "app"`, one URL per provider app) + are fanned out to the installations connected to each account. +- **Routes**: Frigg's existing authenticated ones. `GET/POST /api/integrations/:id/config/options[/refresh]` is the + generated configure form (dependent dropdowns on refresh); `POST /api/integrations/:id/actions/:event` runs a + button workflow (`WF_*`), a backfill, or renders a UI surface (`UI_*`, body `{ record }`). + +`$.install` is the same everywhere a definition runs: integration settings, `webhookUrl`, `integrationId`, +`runtime.installed`, and `accounts..` — values about the connected account that the module's +manifest declares under `account` (`$.install.accounts.quickbooks.realmId`). Nothing undeclared is exposed. + +## Publishing, installing, upgrading + +Definitions live in the `IntegrationDefinitionVersion` table (Prisma models in both schemas, PostgreSQL migration +`20261018000000_add_data_defined_integrations`), and runs in `WorkflowRun`. Point the class at the store: + +```js +const { defineSharedIntegrationClass, createDefinitionCatalog } = require('@friggframework/core/integrations/data-defined'); +// A database store is read asynchronously, so it goes with the shared class (one class, every definition). +const DataDefined = defineSharedIntegrationClass({ definitions: createDefinitionCatalog(), manifests, modules }); +``` + +| Route | Auth | | +|---|---|---| +| `POST /admin/integrations/definitions` `{ document, notes?, publish?, strict? }` | admin key | Validate (both layers) and publish, or save a draft with `publish: false` | +| `GET /admin/integrations/definitions/:id` | admin key | Version history | +| `POST /admin/integrations/definitions/:id/versions/:v/deprecate` | admin key | Stop new installs of a version | +| `GET /api/data-defined/definitions` | user | What can be installed | +| `POST /api/integrations` with `config: { type, definition: { id } }` | user | Install; pins the latest published version and its checksum | +| `GET` / `POST /api/integrations/:id/upgrade` `{ to? }` | user | Plan / apply an upgrade (409 with the plan while something is needed) | +| `GET /api/integrations/:id/runs` | user | Run history, newest first | + +Published versions cannot change. Publishing a document that is identical to a published one returns that row. A +different document at the same version is refused. So is a version that is not newer than the latest, and so is a +semver bump smaller than the changes. Publishing goes through a conditional write, so two concurrent publishes of one +version cannot both win. With `memoryDefinitions()` (definitions bundled in the app) the admin routes answer 501. + +## Steps (v0.1) + +Built-ins: `links.find`, `links.upsert`, `map`, `merge`, `template`, `set`, `stop`, `foreach` (≤1000), `branch`, `code`. + +- `map` rows read a `source` field or write a fixed `value` (`{ "target": "Line[0].DetailType", "value": "SalesItemLineDetail" }`). +- `merge` deep-merges objects (later wins, lists replaced). +- `foreach` takes `where` (skip items, checked before the cap) and `collect` (the step's result becomes one value per + item, which may read that iteration's steps); `do` is optional when `collect` is set. A digest is + `collect` + `{{steps.lines | join: '\n'}}`, not a string grown with `set`. +- Template filters include arithmetic as named, total functions: `plus`, `minus`, `times`, `divided_by`, `round`, + `abs`, `number`, `to_cents` / `from_cents`, `to_minor_units` / `from_minor_units` (currency-aware: JPY has no minor + unit, KWD three), `from_unix` / `to_unix`, and `join`, `size`, `first`. Bad input gives blank, never a throw. A + string that is exactly one tag keeps the value's type, so `"{{item.Amount | to_cents}}"` is the number 2000. +Everything else is `..`, a capability the module's manifest declares and maps to one Api +method. There is no http, script or eval step. `code` calls a named, versioned function shipped in reviewed code +(`@scope/pkg#fn@1.2.0`) for logic that does not fit steps. Per step: `when` (skip), `onError` +(`fail` → the queue retries the run, `continue`, `stop`). + +## Versioning + +Published versions are immutable and each installation pins one. `checkBump` refuses a release whose declared semver +bump is smaller than its changes (patch: steps/templates/view/mapping; minor: additive, including a new link set or +changed option-list inputs; major: removed workflow or link set, required setting, identity change, new module). `planUpgrade` lists what a person must supply first and what the +runtime does (enable/disable workflows, rebuild links, re-run install steps). Queued messages carry the version +they were routed under and finish on it. + +## Running the tests + +```bash +npx jest --config '{}' --rootDir packages/core/integrations/data-defined tests +``` + +`tests/persistence.test.js` runs the repositories and the publish rules against real databases. Each leg runs only +when its URL is set. Point them at a migrated PostgreSQL database and a MongoDB replica set with the schema pushed: + +```bash +DATA_DEFINED_POSTGRES_URL=postgresql://… DATA_DEFINED_MONGO_URL=mongodb://… npx jest … tests +``` + +## Validating + +Validation is two layers, and a document is valid only when both pass (`schema-check.js`): + +1. **JSON Schema** (`schema/*.schema.json`): the published shape. +2. **References** (`validate.js`, `manifests.js`): what a schema cannot see — steps name declared capabilities, + `$.steps.x` is an earlier step, view paths exist, option inputs are declared, `$.install` paths resolve. + +```bash +npm run validate:definitions --workspace @friggframework/core # bundled manifests + examples +node packages/core/integrations/data-defined/scripts/validate.js my.definition.json --strict +node packages/core/integrations/data-defined/scripts/validate.js --manifests ./capabilities --json defs/ +``` + +Exit codes: `0` valid, `1` invalid (or a warning with `--strict`), `2` could not run (unreadable JSON, missing file, +unknown option, `ajv` not installed). Manifests are checked first and an invalid manifest fails the run, because a +definition is only as trustworthy as the manifests it is checked against. + +`ajv` comes from the monorepo root (`@friggframework/schemas` depends on it); core's runtime stays dependency-free and +loads it only for validation. `tests/schemas.test.js` runs both layers over every example and manifest and drives the +CLI as a child process; it fails, rather than skips, if `ajv` is missing. Core's `npm test`, and so CI's +`npm run test:all`, runs it on every change. + +## Not in v0.1 + +- The sync engine (backfill pages, ongoing deltas, two-way master rules) — specified by the schema, to be executed by + the existing `SYNC_PAGE` pattern. `runSync` throws until then. +- `delay` / wait steps, approvals that pause a run, and per-tenant rate limits. +- The host renderers (the HubSpot app extension and Phone panel that draw `renderSurface` output). +- Registering `compiled.schedules` as recurring schedules (Frigg's scheduler commands are one-time jobs). They compile + today; nothing creates them yet. +- Rebuilding `link:` / `idx:` rows after an identity change; `planUpgrade` reports it as `relink` / `reindex`. diff --git a/packages/core/integrations/data-defined/canonical.js b/packages/core/integrations/data-defined/canonical.js new file mode 100644 index 000000000..29c078e68 --- /dev/null +++ b/packages/core/integrations/data-defined/canonical.js @@ -0,0 +1,11 @@ +/** Canonical JSON (sorted keys, no whitespace) and its sha256: the checksum a published version is stored under. */ +const crypto = require('crypto'); +// Canonical order is by UTF-16 code unit (what a bare sort() does), never locale: the checksum must not depend on the host's locale. +const byCodeUnit = (a, b) => (a < b ? -1 : a > b ? 1 : 0); +function canonical(v) { + if (Array.isArray(v)) return '[' + v.map(canonical).join(',') + ']'; + if (v && typeof v === 'object') return '{' + Object.keys(v).sort(byCodeUnit).filter((k) => v[k] !== undefined).map((k) => JSON.stringify(k) + ':' + canonical(v[k])).join(',') + '}'; + return JSON.stringify(v); +} +const checksum = (doc) => 'sha256:' + crypto.createHash('sha256').update(canonical(doc)).digest('hex'); +module.exports = { canonical, checksum }; diff --git a/packages/core/integrations/data-defined/capabilities/acmeerp.capabilities.json b/packages/core/integrations/data-defined/capabilities/acmeerp.capabilities.json new file mode 100644 index 000000000..d2e06a6e3 --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/acmeerp.capabilities.json @@ -0,0 +1,608 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "acmeerp", + "label": "Acme ERP", + "package": "@acme/frigg-api-module", + "version": "0.9.0", + "auth": "api_key", + "objects": { + "customer": { + "label": "Customer", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "display_name", + "type": "string", + "tag": "record.displayName" + }, + { + "name": "first_name", + "type": "string", + "tag": "person.givenName" + }, + { + "name": "last_name", + "type": "string", + "tag": "person.familyName" + }, + { + "name": "company_name", + "type": "string", + "tag": "organization.name" + }, + { + "name": "email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "mobile_number", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "address.street", + "type": "string", + "tag": "address.line1" + }, + { + "name": "address.city", + "type": "string", + "tag": "address.locality" + }, + { + "name": "address.state", + "type": "string", + "tag": "address.region" + }, + { + "name": "address.zip", + "type": "string", + "tag": "address.postalCode" + }, + { + "name": "tags", + "type": "string[]", + "tag": "record.tags" + }, + { + "name": "updated_at", + "type": "datetime", + "tag": "record.updatedAt", + "readOnly": true + } + ] + }, + "job": { + "label": "Job", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "job_number", + "type": "number", + "tag": "record.displayId", + "readOnly": true + }, + { + "name": "customer_id", + "type": "string" + }, + { + "name": "status", + "type": "enum", + "tag": "job.status", + "values": [ + "scheduled", + "dispatched", + "in_progress", + "finished", + "invoiced", + "paid", + "canceled" + ] + }, + { + "name": "description", + "type": "text" + }, + { + "name": "scheduled_start", + "type": "datetime", + "tag": "job.scheduledStart" + }, + { + "name": "scheduled_end", + "type": "datetime", + "tag": "job.scheduledEnd" + }, + { + "name": "finished_at", + "type": "datetime", + "tag": "job.completedAt" + }, + { + "name": "technician_ids", + "type": "string[]", + "tag": "record.owner" + }, + { + "name": "subtotal", + "type": "integer", + "tag": "order.subtotal", + "readOnly": true + }, + { + "name": "tax_amount", + "type": "integer", + "tag": "order.tax", + "readOnly": true + }, + { + "name": "total_amount", + "type": "integer", + "tag": "order.total", + "readOnly": true + }, + { + "name": "payment_status", + "type": "enum", + "tag": "order.paymentStatus", + "values": [ + "unpaid", + "partially_paid", + "paid" + ] + }, + { + "name": "external_invoice_id", + "type": "string", + "tag": "record.externalId" + } + ] + }, + "lineItem": { + "label": "Line item", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "job_id", + "type": "string" + }, + { + "name": "kind", + "type": "enum", + "values": [ + "service", + "labor", + "material", + "fee", + "discount" + ] + }, + { + "name": "name", + "type": "string" + }, + { + "name": "description", + "type": "text" + }, + { + "name": "sku", + "type": "string" + }, + { + "name": "quantity", + "type": "number" + }, + { + "name": "unit_price", + "type": "integer" + }, + { + "name": "unit_cost", + "type": "integer" + }, + { + "name": "taxable", + "type": "boolean" + } + ] + }, + "technician": { + "label": "Technician", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "name", + "type": "string", + "tag": "person.fullName" + }, + { + "name": "email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "phone", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "role", + "type": "string" + } + ] + } + }, + "events": { + "job.finished": { + "delivery": "webhook", + "object": "job", + "subscribe": "webhook.create", + "payload": { + "event_id": "evt_01JAF4T9ZQ", + "event": "job.finished", + "occurred_at": "2026-10-20T16:42:03Z", + "company_id": "co_acme_plumbing", + "data": { + "job": { + "id": "job_8Kq2Lm4N", + "job_number": 1042, + "customer_id": "cus_3Hx9Pq1R", + "status": "finished", + "description": "Replace water heater (50 gal) and haul away old unit", + "scheduled_start": "2026-10-20T13:00:00Z", + "scheduled_end": "2026-10-20T16:00:00Z", + "finished_at": "2026-10-20T16:42:00Z", + "technician_ids": [ + "tech_A1", + "tech_B7" + ], + "subtotal": 184500, + "tax_amount": 15221, + "total_amount": 199721, + "payment_status": "unpaid", + "external_invoice_id": null, + "address": { + "street": "4410 Shoal Creek Blvd", + "city": "Austin", + "state": "TX", + "zip": "78756" + } + } + } + }, + "record": "data.job", + "idPath": "event_id" + }, + "job.scheduled": { + "delivery": "webhook", + "object": "job", + "subscribe": "webhook.create", + "payload": { + "event_id": "evt_01JAF4T9ZQ", + "event": "job.scheduled", + "occurred_at": "2026-10-20T16:42:03Z", + "company_id": "co_acme_plumbing", + "data": { + "job": { + "id": "job_8Kq2Lm4N", + "job_number": 1042, + "customer_id": "cus_3Hx9Pq1R", + "status": "scheduled", + "description": "Replace water heater (50 gal) and haul away old unit", + "scheduled_start": "2026-10-20T13:00:00Z", + "scheduled_end": "2026-10-20T16:00:00Z", + "finished_at": null, + "technician_ids": [ + "tech_A1", + "tech_B7" + ], + "subtotal": 184500, + "tax_amount": 15221, + "total_amount": 199721, + "payment_status": "unpaid", + "external_invoice_id": null, + "address": { + "street": "4410 Shoal Creek Blvd", + "city": "Austin", + "state": "TX", + "zip": "78756" + } + } + } + }, + "record": "data.job", + "idPath": "event_id" + }, + "customer.created": { + "delivery": "webhook", + "object": "customer", + "subscribe": "webhook.create", + "payload": { + "event_id": "evt_01JAF4T9ZQ", + "event": "customer.created", + "occurred_at": "2026-10-20T16:42:03Z", + "company_id": "co_acme_plumbing", + "data": { + "customer": { + "id": "cus_3Hx9Pq1R", + "display_name": "Marisol Ortega", + "first_name": "Marisol", + "last_name": "Ortega", + "company_name": null, + "email": "marisol.ortega@example.com", + "mobile_number": "(512) 555-0188", + "address": { + "street": "4410 Shoal Creek Blvd", + "city": "Austin", + "state": "TX", + "zip": "78756" + }, + "tags": [ + "residential", + "maintenance-plan" + ], + "updated_at": "2026-10-20T12:00:00Z" + } + } + }, + "record": "data.customer", + "idPath": "event_id" + }, + "customer.updated": { + "delivery": "webhook", + "object": "customer", + "subscribe": "webhook.create", + "payload": { + "event_id": "evt_01JAF4T9ZQ", + "event": "customer.updated", + "occurred_at": "2026-10-20T16:42:03Z", + "company_id": "co_acme_plumbing", + "data": { + "customer": { + "id": "cus_3Hx9Pq1R", + "display_name": "Marisol Ortega", + "first_name": "Marisol", + "last_name": "Ortega", + "company_name": null, + "email": "marisol@ortega.family", + "mobile_number": "(512) 555-0188", + "address": { + "street": "4410 Shoal Creek Blvd", + "city": "Austin", + "state": "TX", + "zip": "78756" + }, + "tags": [ + "residential", + "maintenance-plan" + ], + "updated_at": "2026-10-20T12:00:00Z" + } + } + }, + "record": "data.customer", + "idPath": "event_id" + } + }, + "webhook": { + "scope": "installation", + "topic": "{{body.event}}", + "eventId": "{{body.event_id}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "customer.get": { + "kind": "read", + "method": "getCustomer", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "customer" + }, + "customer.list": { + "kind": "read", + "method": "listCustomers", + "input": { + "updated_since": { + "type": "string" + }, + "page": { + "type": "number" + }, + "page_size": { + "type": "number" + } + }, + "output": "page" + }, + "customer.create": { + "kind": "write", + "method": "createCustomer", + "input": { + "customer": { + "type": "object", + "required": true + } + }, + "output": "customer" + }, + "customer.update": { + "kind": "write", + "method": "updateCustomer", + "input": { + "id": { + "type": "string", + "required": true + }, + "customer": { + "type": "object", + "required": true + } + }, + "output": "customer" + }, + "job.get": { + "kind": "read", + "method": "getJob", + "input": { + "id": { + "type": "string", + "required": true + }, + "expand": { + "type": "array" + } + }, + "output": "job" + }, + "job.search": { + "kind": "search", + "method": "searchJobs", + "input": { + "external_invoice_id": { + "type": "string" + }, + "job_number": { + "type": "number" + }, + "customer_id": { + "type": "string" + } + }, + "output": "job[]" + }, + "job.lineItems": { + "kind": "read", + "method": "listJobLineItems", + "input": { + "jobId": { + "type": "string", + "required": true + } + }, + "output": "lineItem[]" + }, + "job.update": { + "kind": "write", + "method": "updateJob", + "input": { + "id": { + "type": "string", + "required": true + }, + "job": { + "type": "object", + "required": true + } + }, + "output": "job" + }, + "job.markPaid": { + "kind": "write", + "method": "recordJobPayment", + "input": { + "id": { + "type": "string", + "required": true + }, + "amount": { + "type": "integer", + "required": true + }, + "paid_at": { + "type": "string" + }, + "method": { + "type": "string" + }, + "reference": { + "type": "string" + } + }, + "output": "job" + }, + "job.addNote": { + "kind": "write", + "method": "createJobNote", + "input": { + "jobId": { + "type": "string", + "required": true + }, + "body": { + "type": "string", + "required": true + } + }, + "output": "note" + }, + "technician.list": { + "kind": "read", + "method": "listTechnicians", + "input": {}, + "output": "technician[]" + }, + "webhook.create": { + "kind": "subscribe", + "method": "createWebhookSubscription", + "input": { + "url": { + "type": "string", + "required": true + }, + "events": { + "type": "array", + "required": true + } + }, + "output": "webhook" + }, + "webhook.delete": { + "kind": "delete", + "method": "deleteWebhookSubscription", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "void" + } + }, + "options": { + "technician": { + "capability": "technician.list", + "label": "{{item.name}}", + "value": "$.item.id" + } + }, + "ui": { + "placements": [ + "job.sidebar", + "customer.sidebar" + ], + "renderer": "acme-erp-extension-panel" + } +} diff --git a/packages/core/integrations/data-defined/capabilities/asana.capabilities.json b/packages/core/integrations/data-defined/capabilities/asana.capabilities.json new file mode 100644 index 000000000..0af0692d4 --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/asana.capabilities.json @@ -0,0 +1,265 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "asana", + "label": "Asana", + "package": "@friggframework/api-module-asana", + "version": "1.5.1", + "auth": "oauth2", + "objects": { + "task": { + "label": "Task", + "fields": [ + { + "name": "gid", + "type": "id", + "readOnly": true + }, + { + "name": "name", + "type": "string", + "tag": "task.title" + }, + { + "name": "notes", + "type": "text", + "tag": "task.description" + }, + { + "name": "completed", + "type": "boolean", + "tag": "task.completed" + }, + { + "name": "completed_at", + "type": "datetime", + "tag": "task.completedAt", + "readOnly": true + }, + { + "name": "due_on", + "type": "date", + "tag": "task.dueDate" + }, + { + "name": "due_at", + "type": "datetime" + }, + { + "name": "assignee.gid", + "type": "string", + "tag": "record.owner" + }, + { + "name": "assignee.name", + "type": "string", + "readOnly": true + }, + { + "name": "assignee.email", + "type": "email", + "readOnly": true + }, + { + "name": "completed_by.name", + "type": "string", + "readOnly": true + }, + { + "name": "projects", + "type": "json" + }, + { + "name": "permalink_url", + "type": "string", + "tag": "record.url", + "readOnly": true + }, + { + "name": "modified_at", + "type": "datetime", + "tag": "record.updatedAt", + "readOnly": true + } + ] + }, + "project": { + "label": "Project", + "fields": [ + { + "name": "gid", + "type": "id", + "readOnly": true + }, + { + "name": "name", + "type": "string", + "tag": "project.name" + }, + { + "name": "archived", + "type": "boolean" + }, + { + "name": "permalink_url", + "type": "string", + "readOnly": true + } + ] + } + }, + "events": { + "task.changed": { + "delivery": "webhook", + "object": "task", + "subscribe": "webhook.create", + "description": "A webhook on a project (X-Hook-Secret handshake). Events are compact: fetch the task to read it. Filtered to the completed field at registration.", + "payload": { + "events": [ + { + "user": { + "gid": "1203511920193211", + "resource_type": "user" + }, + "created_at": "2026-10-20T18:30:12.400Z", + "action": "changed", + "resource": { + "gid": "1208123456789012", + "resource_type": "task", + "resource_subtype": "default_task" + }, + "parent": null, + "change": { + "field": "completed", + "action": "changed", + "new_value": true + } + } + ] + }, + "record": "resource" + } + }, + "webhook": { + "scope": "installation", + "batch": "body.events", + "topic": "{{item.resource.resource_type}}.{{item.action}}", + "eventId": "{{item.resource.gid}}:{{item.action}}:{{item.created_at}}", + "verify": "verifyWebhookSignature", + "handshake": { + "when": "$.headers.x-hook-secret exists", + "status": 200, + "headers": { + "X-Hook-Secret": "$.headers.x-hook-secret" + } + } + }, + "capabilities": { + "task.get": { + "kind": "read", + "method": "getTask", + "input": { + "taskGid": { + "type": "string", + "required": true + }, + "opt_fields": { + "type": "string" + } + }, + "output": "task" + }, + "task.search": { + "kind": "search", + "method": "searchTasksInWorkspace", + "input": { + "workspaceGid": { + "type": "string", + "required": true + }, + "due_on": { + "type": "string" + }, + "completed": { + "type": "boolean" + }, + "projects_any": { + "type": "string" + }, + "assignee_any": { + "type": "string" + }, + "limit": { + "type": "number" + } + }, + "output": "task[]" + }, + "project.list": { + "kind": "read", + "method": "getProjects", + "input": { + "workspaceGid": { + "type": "string", + "required": true + }, + "archived": { + "type": "boolean" + } + }, + "output": "project[]" + }, + "workspace.list": { + "kind": "read", + "method": "getWorkspaces", + "input": {}, + "output": "workspace[]" + }, + "webhook.create": { + "kind": "subscribe", + "method": "createWebhook", + "input": { + "resource": { + "type": "string", + "required": true + }, + "target": { + "type": "string", + "required": true + }, + "filters": { + "type": "array" + } + }, + "output": "webhook" + }, + "webhook.delete": { + "kind": "delete", + "method": "deleteWebhook", + "input": { + "webhookGid": { + "type": "string", + "required": true + } + }, + "output": "void" + } + }, + "options": { + "workspace": { + "capability": "workspace.list", + "label": "{{item.name}}", + "value": "$.item.gid" + }, + "project": { + "capability": "project.list", + "label": "{{item.name}}", + "value": "$.item.gid", + "inputs": { + "workspaceGid": { + "type": "string", + "required": true + } + } + } + } +} diff --git a/packages/core/integrations/data-defined/capabilities/calendly.capabilities.json b/packages/core/integrations/data-defined/capabilities/calendly.capabilities.json new file mode 100644 index 000000000..770d47698 --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/calendly.capabilities.json @@ -0,0 +1,385 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "calendly", + "label": "Calendly", + "package": "@friggframework/api-module-calendly", + "version": "1.0.4", + "auth": "oauth2", + "account": { + "description": "Values about the connected account, persisted by the module (apiPropertiesToPersist). Definitions read them as $.install.accounts...", + "fields": [ + { + "name": "organization", + "type": "string", + "description": "Organization URI of the connected Calendly user" + }, + { + "name": "user", + "type": "string", + "description": "User URI of the connected Calendly user" + } + ] + }, + "objects": { + "invitee": { + "label": "Invitee", + "fields": [ + { + "name": "uri", + "type": "id", + "readOnly": true + }, + { + "name": "email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "name", + "type": "string", + "tag": "person.fullName" + }, + { + "name": "first_name", + "type": "string", + "tag": "person.givenName" + }, + { + "name": "last_name", + "type": "string", + "tag": "person.familyName" + }, + { + "name": "text_reminder_number", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "timezone", + "type": "string", + "tag": "person.timezone" + }, + { + "name": "status", + "type": "enum", + "values": [ + "active", + "canceled" + ], + "readOnly": true + }, + { + "name": "questions_and_answers", + "type": "json" + }, + { + "name": "tracking.utm_source", + "type": "string", + "tag": "marketing.source" + }, + { + "name": "tracking.utm_campaign", + "type": "string", + "tag": "marketing.campaign" + }, + { + "name": "cancellation.reason", + "type": "text" + }, + { + "name": "created_at", + "type": "datetime", + "tag": "record.createdAt", + "readOnly": true + } + ] + }, + "scheduledEvent": { + "label": "Scheduled event", + "fields": [ + { + "name": "uri", + "type": "id", + "readOnly": true + }, + { + "name": "name", + "type": "string", + "tag": "meeting.title" + }, + { + "name": "status", + "type": "enum", + "values": [ + "active", + "canceled" + ] + }, + { + "name": "start_time", + "type": "datetime", + "tag": "meeting.start" + }, + { + "name": "end_time", + "type": "datetime", + "tag": "meeting.end" + }, + { + "name": "location.join_url", + "type": "string", + "tag": "meeting.joinUrl" + }, + { + "name": "event_type", + "type": "string" + }, + { + "name": "event_memberships", + "type": "json", + "tag": "record.owner" + } + ] + } + }, + "events": { + "invitee.created": { + "delivery": "webhook", + "object": "invitee", + "subscribe": "webhook.create", + "payload": { + "event": "invitee.created", + "created_at": "2026-10-18T14:30:01.000000Z", + "created_by": "https://api.calendly.com/users/AAAA", + "payload": { + "uri": "https://api.calendly.com/scheduled_events/GBGBDCAADAEDCRZ2/invitees/AAAAAAAAAAAAAAAA", + "email": "priya@acme.example", + "name": "Priya Natarajan", + "first_name": "Priya", + "last_name": "Natarajan", + "status": "active", + "timezone": "America/Chicago", + "text_reminder_number": "+15125550177", + "rescheduled": false, + "created_at": "2026-10-18T14:30:00.000000Z", + "cancel_url": "https://calendly.com/cancellations/AAAAAAAAAAAAAAAA", + "reschedule_url": "https://calendly.com/reschedulings/AAAAAAAAAAAAAAAA", + "questions_and_answers": [ + { + "question": "Company", + "answer": "Acme Corp", + "position": 0 + }, + { + "question": "What would you like to discuss?", + "answer": "Migrating our Salesforce integration", + "position": 1 + } + ], + "tracking": { + "utm_source": "google", + "utm_medium": "cpc", + "utm_campaign": "salesforce-integration" + }, + "scheduled_event": { + "uri": "https://api.calendly.com/scheduled_events/GBGBDCAADAEDCRZ2", + "name": "Discovery call (30 min)", + "status": "active", + "start_time": "2026-10-22T19:00:00.000000Z", + "end_time": "2026-10-22T19:30:00.000000Z", + "event_type": "https://api.calendly.com/event_types/GBGBDCAADAEDCRZ2", + "location": { + "type": "google_conference", + "join_url": "https://meet.google.com/abc-defg-hij" + }, + "event_memberships": [ + { + "user": "https://api.calendly.com/users/AAAA", + "user_email": "alex@acme.example", + "user_name": "Alex Morgan" + } + ] + } + } + }, + "record": "payload", + "idPath": "payload.uri" + }, + "invitee.canceled": { + "delivery": "webhook", + "object": "invitee", + "subscribe": "webhook.create", + "payload": { + "event": "invitee.canceled", + "created_at": "2026-10-21T09:00:00.000000Z", + "created_by": "https://api.calendly.com/users/AAAA", + "payload": { + "uri": "https://api.calendly.com/scheduled_events/GBGBDCAADAEDCRZ2/invitees/AAAAAAAAAAAAAAAA", + "email": "priya@acme.example", + "name": "Priya Natarajan", + "first_name": "Priya", + "last_name": "Natarajan", + "status": "canceled", + "timezone": "America/Chicago", + "text_reminder_number": "+15125550177", + "rescheduled": false, + "created_at": "2026-10-18T14:30:00.000000Z", + "cancel_url": "https://calendly.com/cancellations/AAAAAAAAAAAAAAAA", + "reschedule_url": "https://calendly.com/reschedulings/AAAAAAAAAAAAAAAA", + "questions_and_answers": [ + { + "question": "Company", + "answer": "Acme Corp", + "position": 0 + }, + { + "question": "What would you like to discuss?", + "answer": "Migrating our Salesforce integration", + "position": 1 + } + ], + "tracking": { + "utm_source": "google", + "utm_medium": "cpc", + "utm_campaign": "salesforce-integration" + }, + "scheduled_event": { + "uri": "https://api.calendly.com/scheduled_events/GBGBDCAADAEDCRZ2", + "name": "Discovery call (30 min)", + "status": "active", + "start_time": "2026-10-22T19:00:00.000000Z", + "end_time": "2026-10-22T19:30:00.000000Z", + "event_type": "https://api.calendly.com/event_types/GBGBDCAADAEDCRZ2", + "location": { + "type": "google_conference", + "join_url": "https://meet.google.com/abc-defg-hij" + }, + "event_memberships": [ + { + "user": "https://api.calendly.com/users/AAAA", + "user_email": "alex@acme.example", + "user_name": "Alex Morgan" + } + ] + }, + "cancellation": { + "canceled_by": "Priya Natarajan", + "reason": "Need to push to next week", + "canceler_type": "invitee" + } + } + }, + "record": "payload", + "idPath": "payload.uri" + } + }, + "webhook": { + "scope": "installation", + "topic": "{{body.event}}", + "eventId": "{{body.payload.uri}}:{{body.event}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "invitee.get": { + "kind": "read", + "method": "getInvitee", + "input": { + "uri": { + "type": "string", + "required": true + } + }, + "output": "invitee" + }, + "event.get": { + "kind": "read", + "method": "getScheduledEvent", + "input": { + "uri": { + "type": "string", + "required": true + } + }, + "output": "scheduledEvent" + }, + "event.list": { + "kind": "read", + "method": "listScheduledEvents", + "input": { + "organization": { + "type": "string" + }, + "user": { + "type": "string" + }, + "invitee_email": { + "type": "string" + }, + "min_start_time": { + "type": "string" + }, + "status": { + "type": "string" + }, + "count": { + "type": "number" + } + }, + "output": "scheduledEvent[]" + }, + "eventType.list": { + "kind": "read", + "method": "listEventTypes", + "input": { + "organization": { + "type": "string" + } + }, + "output": "eventType[]" + }, + "webhook.create": { + "kind": "subscribe", + "method": "createWebhookSubscription", + "input": { + "url": { + "type": "string", + "required": true + }, + "events": { + "type": "array", + "required": true + }, + "organization": { + "type": "string", + "required": true + }, + "scope": { + "type": "string", + "required": true + }, + "signing_key": { + "type": "string" + } + }, + "output": "webhookSubscription" + }, + "webhook.delete": { + "kind": "delete", + "method": "deleteWebhookSubscription", + "input": { + "uri": { + "type": "string", + "required": true + } + }, + "output": "void" + } + }, + "options": { + "eventType": { + "capability": "eventType.list", + "label": "{{item.name}}", + "value": "$.item.uri" + } + } +} diff --git a/packages/core/integrations/data-defined/capabilities/docusign.capabilities.json b/packages/core/integrations/data-defined/capabilities/docusign.capabilities.json new file mode 100644 index 000000000..35c8c33ab --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/docusign.capabilities.json @@ -0,0 +1,314 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "docusign", + "label": "DocuSign eSignature", + "package": "@friggframework/api-module-docusign", + "version": "1.2.0", + "auth": "oauth2", + "objects": { + "envelope": { + "label": "Envelope", + "fields": [ + { + "name": "envelopeId", + "type": "id", + "readOnly": true + }, + { + "name": "status", + "type": "enum", + "tag": "agreement.status", + "values": [ + "created", + "sent", + "delivered", + "signed", + "completed", + "declined", + "voided" + ], + "readOnly": true + }, + { + "name": "emailSubject", + "type": "string", + "tag": "agreement.title" + }, + { + "name": "sentDateTime", + "type": "datetime", + "readOnly": true + }, + { + "name": "completedDateTime", + "type": "datetime", + "tag": "agreement.completedAt", + "readOnly": true + }, + { + "name": "declinedDateTime", + "type": "datetime", + "readOnly": true + }, + { + "name": "sender.email", + "type": "email", + "tag": "record.owner", + "readOnly": true + }, + { + "name": "recipients.signers", + "type": "json", + "tag": "agreement.signers" + }, + { + "name": "customFields.textCustomFields", + "type": "json" + } + ] + } + }, + "events": { + "envelope.completed": { + "delivery": "webhook", + "object": "envelope", + "subscribe": "connect.create", + "description": "DocuSign Connect (JSON SIM) event envelope-completed.", + "payload": { + "event": "envelope-completed", + "apiVersion": "v2.1", + "uri": "/restapi/v2.1/accounts/9c1b2f3e/envelopes/7d2e8f14-3a6b-4c91-9e20-5b1f7c8a2d63", + "retryCount": 0, + "configurationId": 10457, + "generatedDateTime": "2026-10-20T16:00:02.112Z", + "data": { + "accountId": "9c1b2f3e-1a2b-4c3d-8e9f-0a1b2c3d4e5f", + "userId": "4b2a1e8c-0a1b-4c55-9d1b-6f4b2a1e8c33", + "envelopeId": "7d2e8f14-3a6b-4c91-9e20-5b1f7c8a2d63", + "envelopeSummary": { + "envelopeId": "7d2e8f14-3a6b-4c91-9e20-5b1f7c8a2d63", + "status": "completed", + "emailSubject": "Please sign: Acme Corp MSA and Work Order 3", + "sentDateTime": "2026-10-19T14:00:00.000Z", + "sender": { + "userName": "Alex Morgan", + "email": "alex@acme.example" + }, + "recipients": { + "signers": [ + { + "recipientId": "1", + "name": "Priya Natarajan", + "email": "priya@acme.example", + "status": "completed", + "routingOrder": "1" + } + ] + }, + "customFields": { + "textCustomFields": [ + { + "fieldId": "10498", + "name": "SalesforceOpportunityId", + "value": "0068c00000ABCDeAAP", + "show": "false" + } + ] + }, + "completedDateTime": "2026-10-20T15:59:40.000Z" + } + } + }, + "record": "data.envelopeSummary", + "match": "envelope-completed" + }, + "envelope.declined": { + "delivery": "webhook", + "object": "envelope", + "subscribe": "connect.create", + "payload": { + "event": "envelope-declined", + "apiVersion": "v2.1", + "uri": "/restapi/v2.1/accounts/9c1b2f3e/envelopes/7d2e8f14-3a6b-4c91-9e20-5b1f7c8a2d63", + "retryCount": 0, + "configurationId": 10457, + "generatedDateTime": "2026-10-20T16:00:02.112Z", + "data": { + "accountId": "9c1b2f3e-1a2b-4c3d-8e9f-0a1b2c3d4e5f", + "userId": "4b2a1e8c-0a1b-4c55-9d1b-6f4b2a1e8c33", + "envelopeId": "7d2e8f14-3a6b-4c91-9e20-5b1f7c8a2d63", + "envelopeSummary": { + "envelopeId": "7d2e8f14-3a6b-4c91-9e20-5b1f7c8a2d63", + "status": "declined", + "emailSubject": "Please sign: Acme Corp MSA and Work Order 3", + "sentDateTime": "2026-10-19T14:00:00.000Z", + "sender": { + "userName": "Alex Morgan", + "email": "alex@acme.example" + }, + "recipients": { + "signers": [ + { + "recipientId": "1", + "name": "Priya Natarajan", + "email": "priya@acme.example", + "status": "declined", + "declinedReason": "Legal needs to redline section 7 first." + } + ] + }, + "customFields": { + "textCustomFields": [ + { + "fieldId": "10498", + "name": "SalesforceOpportunityId", + "value": "0068c00000ABCDeAAP", + "show": "false" + } + ] + }, + "declinedDateTime": "2026-10-20T13:12:00.000Z" + } + } + }, + "record": "data.envelopeSummary", + "match": "envelope-declined" + }, + "envelope.sent": { + "delivery": "webhook", + "object": "envelope", + "subscribe": "connect.create", + "payload": { + "event": "envelope-sent", + "apiVersion": "v2.1", + "uri": "/restapi/v2.1/accounts/9c1b2f3e/envelopes/7d2e8f14-3a6b-4c91-9e20-5b1f7c8a2d63", + "retryCount": 0, + "configurationId": 10457, + "generatedDateTime": "2026-10-20T16:00:02.112Z", + "data": { + "accountId": "9c1b2f3e-1a2b-4c3d-8e9f-0a1b2c3d4e5f", + "userId": "4b2a1e8c-0a1b-4c55-9d1b-6f4b2a1e8c33", + "envelopeId": "7d2e8f14-3a6b-4c91-9e20-5b1f7c8a2d63", + "envelopeSummary": { + "envelopeId": "7d2e8f14-3a6b-4c91-9e20-5b1f7c8a2d63", + "status": "sent", + "emailSubject": "Please sign: Acme Corp MSA and Work Order 3", + "sentDateTime": "2026-10-19T14:00:00.000Z", + "sender": { + "userName": "Alex Morgan", + "email": "alex@acme.example" + }, + "recipients": { + "signers": [ + { + "recipientId": "1", + "name": "Priya Natarajan", + "email": "priya@acme.example", + "status": "sent", + "routingOrder": "1" + } + ] + }, + "customFields": { + "textCustomFields": [ + { + "fieldId": "10498", + "name": "SalesforceOpportunityId", + "value": "0068c00000ABCDeAAP", + "show": "false" + } + ] + } + } + } + }, + "record": "data.envelopeSummary", + "match": "envelope-sent" + } + }, + "webhook": { + "scope": "installation", + "topic": "{{body.event}}", + "eventId": "{{body.data.envelopeId}}:{{body.event}}:{{body.generatedDateTime}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "envelope.get": { + "kind": "read", + "method": "getEnvelope", + "input": { + "envelopeId": { + "type": "string", + "required": true + }, + "include": { + "type": "string" + } + }, + "output": "envelope" + }, + "envelope.list": { + "kind": "read", + "method": "listStatusChanges", + "input": { + "from_date": { + "type": "string" + }, + "status": { + "type": "string" + }, + "custom_field": { + "type": "string" + }, + "count": { + "type": "number" + } + }, + "output": "envelope[]" + }, + "document.combined": { + "kind": "read", + "method": "getCombinedDocument", + "input": { + "envelopeId": { + "type": "string", + "required": true + }, + "certificate": { + "type": "boolean" + } + }, + "output": "{ base64: string, contentType: string }" + }, + "connect.create": { + "kind": "subscribe", + "method": "createConnectConfiguration", + "input": { + "name": { + "type": "string", + "required": true + }, + "urlToPublishTo": { + "type": "string", + "required": true + }, + "events": { + "type": "array", + "required": true + } + }, + "output": "connectConfiguration" + }, + "connect.delete": { + "kind": "delete", + "method": "deleteConnectConfiguration", + "input": { + "connectId": { + "type": "string", + "required": true + } + }, + "output": "void" + } + } +} diff --git a/packages/core/integrations/data-defined/capabilities/fathom.capabilities.json b/packages/core/integrations/data-defined/capabilities/fathom.capabilities.json new file mode 100644 index 000000000..ee900d93e --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/fathom.capabilities.json @@ -0,0 +1,228 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "fathom", + "label": "Fathom", + "package": "@friggframework/api-module-fathom", + "version": "0.4.0", + "auth": "api_key", + "objects": { + "meeting": { + "label": "Meeting recording", + "fields": [ + { + "name": "recording_id", + "type": "id", + "readOnly": true + }, + { + "name": "title", + "type": "string", + "tag": "meeting.title" + }, + { + "name": "url", + "type": "string", + "readOnly": true + }, + { + "name": "share_url", + "type": "string", + "tag": "meeting.recordingUrl", + "readOnly": true + }, + { + "name": "scheduled_start_time", + "type": "datetime", + "tag": "meeting.scheduledStart" + }, + { + "name": "recording_start_time", + "type": "datetime", + "tag": "meeting.start" + }, + { + "name": "recording_end_time", + "type": "datetime", + "tag": "meeting.end" + }, + { + "name": "recorded_by.email", + "type": "email", + "tag": "record.owner" + }, + { + "name": "calendar_invitees", + "type": "json", + "tag": "meeting.attendees" + }, + { + "name": "default_summary.markdown_formatted", + "type": "text", + "tag": "meeting.summary" + }, + { + "name": "action_items", + "type": "json", + "tag": "meeting.actionItems" + }, + { + "name": "transcript_language", + "type": "string" + } + ] + } + }, + "events": { + "meeting.ready": { + "delivery": "webhook", + "object": "meeting", + "subscribe": "webhook.create", + "description": "Fired when a recording's summary, action items (and transcript, if asked for) are ready. The payload is the meeting.", + "payload": { + "recording_id": 123456789, + "title": "Acme / Initech renewal review", + "meeting_title": "Acme renewal", + "url": "https://fathom.video/calls/123456789", + "share_url": "https://fathom.video/share/zXv8m2KqQhJt", + "created_at": "2026-10-20T15:47:10Z", + "scheduled_start_time": "2026-10-20T15:00:00Z", + "recording_start_time": "2026-10-20T15:01:12Z", + "recording_end_time": "2026-10-20T15:46:30Z", + "transcript_language": "en", + "recorded_by": { + "name": "Alex Morgan", + "email": "alex@acme.example", + "team": "Sales" + }, + "calendar_invitees": [ + { + "name": "Alex Morgan", + "email": "alex@acme.example", + "is_external": false + }, + { + "name": "Priya Natarajan", + "email": "priya@acme.example", + "email_domain": "acme.example", + "is_external": true + } + ], + "default_summary": { + "template_name": "general", + "markdown_formatted": "## Summary\nAcme wants to renew at the Medium tier and add a support block.\n\n## Next steps\n- Send revised order form" + }, + "action_items": [ + { + "description": "Send revised order form with the support block", + "user_generated": false, + "completed": false, + "recording_timestamp": "00:31:05", + "recording_playback_url": "https://fathom.video/calls/123456789?timestamp=1865", + "assignee": { + "name": "Alex Morgan", + "email": "alex@acme.example" + } + }, + { + "description": "Confirm procurement contact", + "user_generated": false, + "completed": false, + "recording_timestamp": "00:38:40", + "assignee": { + "name": "Priya Natarajan", + "email": "priya@acme.example" + } + } + ] + }, + "idPath": "recording_id" + } + }, + "webhook": { + "scope": "installation", + "topic": "meeting.ready", + "eventId": "{{body.recording_id}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "meeting.list": { + "kind": "read", + "method": "listMeetings", + "input": { + "created_after": { + "type": "string" + }, + "calendar_invitees_domains": { + "type": "array" + }, + "include_summary": { + "type": "boolean" + }, + "include_action_items": { + "type": "boolean" + }, + "cursor": { + "type": "string" + } + }, + "output": "page" + }, + "meeting.summary": { + "kind": "read", + "method": "getRecordingSummary", + "input": { + "recordingId": { + "type": "string", + "required": true + } + }, + "output": "summary" + }, + "meeting.transcript": { + "kind": "read", + "method": "getRecordingTranscript", + "input": { + "recordingId": { + "type": "string", + "required": true + } + }, + "output": "transcript" + }, + "webhook.create": { + "kind": "subscribe", + "method": "createWebhook", + "input": { + "destination_url": { + "type": "string", + "required": true + }, + "triggered_for": { + "type": "array", + "required": true + }, + "include_summary": { + "type": "boolean" + }, + "include_action_items": { + "type": "boolean" + }, + "include_transcript": { + "type": "boolean" + } + }, + "output": "webhook" + }, + "webhook.delete": { + "kind": "delete", + "method": "deleteWebhook", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "void" + } + } +} diff --git a/packages/core/integrations/data-defined/capabilities/github.capabilities.json b/packages/core/integrations/data-defined/capabilities/github.capabilities.json new file mode 100644 index 000000000..a02d6b5dd --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/github.capabilities.json @@ -0,0 +1,563 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "github", + "label": "GitHub", + "package": "@friggframework/api-module-github", + "version": "1.8.0", + "auth": "oauth2", + "objects": { + "issue": { + "label": "Issue", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "number", + "type": "number", + "tag": "record.displayId", + "readOnly": true + }, + { + "name": "title", + "type": "string", + "tag": "task.title" + }, + { + "name": "body", + "type": "text", + "tag": "task.description" + }, + { + "name": "state", + "type": "enum", + "tag": "task.state", + "values": [ + "open", + "closed" + ] + }, + { + "name": "state_reason", + "type": "enum", + "values": [ + "completed", + "not_planned", + "reopened" + ] + }, + { + "name": "labels", + "type": "label[]", + "tag": "record.tags" + }, + { + "name": "labels[0].name", + "type": "string" + }, + { + "name": "assignee.login", + "type": "string", + "tag": "record.owner" + }, + { + "name": "html_url", + "type": "string", + "tag": "record.url", + "readOnly": true + }, + { + "name": "pull_request", + "type": "json", + "readOnly": true + }, + { + "name": "created_at", + "type": "datetime", + "tag": "record.createdAt", + "readOnly": true + }, + { + "name": "updated_at", + "type": "datetime", + "tag": "record.updatedAt", + "readOnly": true + } + ] + }, + "pullRequest": { + "label": "Pull request", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "number", + "type": "number", + "readOnly": true + }, + { + "name": "title", + "type": "string" + }, + { + "name": "body", + "type": "text" + }, + { + "name": "state", + "type": "enum", + "values": [ + "open", + "closed" + ] + }, + { + "name": "merged", + "type": "boolean", + "readOnly": true + }, + { + "name": "merged_at", + "type": "datetime", + "readOnly": true + }, + { + "name": "head.ref", + "type": "string", + "tag": "git.branch" + }, + { + "name": "base.ref", + "type": "string" + }, + { + "name": "html_url", + "type": "string", + "tag": "record.url", + "readOnly": true + } + ] + } + }, + "events": { + "issue.opened": { + "delivery": "webhook", + "object": "issue", + "description": "GitHub App webhook (issues/opened); app-level, routed by installation.id.", + "payload": { + "action": "opened", + "issue": { + "id": 2601234567, + "number": 318, + "title": "Webhook retries double-post comments", + "body": "When the queue retries a delivery, the comment is posted twice.", + "state": "open", + "state_reason": null, + "labels": [ + { + "id": 6012345, + "name": "bug" + }, + { + "id": 6012399, + "name": "P1" + } + ], + "assignee": { + "login": "octocat" + }, + "html_url": "https://github.com/acme/platform/issues/318", + "user": { + "login": "dana-r" + }, + "created_at": "2026-10-19T10:00:00Z", + "updated_at": "2026-10-20T08:00:00Z", + "closed_at": null + }, + "repository": { + "id": 712345678, + "full_name": "acme/platform", + "name": "platform", + "owner": { + "login": "acme" + } + }, + "installation": { + "id": 51234567 + }, + "sender": { + "login": "dana-r" + } + }, + "record": "issue", + "match": "issues.opened", + "headers": { + "x-github-event": "issues", + "x-github-delivery": "72d3162e-issue.opened" + } + }, + "issue.edited": { + "delivery": "webhook", + "object": "issue", + "payload": { + "action": "edited", + "issue": { + "id": 2601234567, + "number": 318, + "title": "Webhook retries double-post comments", + "body": "When the queue retries a delivery, the comment is posted twice.", + "state": "open", + "state_reason": null, + "labels": [ + { + "id": 6012345, + "name": "bug" + }, + { + "id": 6012399, + "name": "P1" + } + ], + "assignee": { + "login": "octocat" + }, + "html_url": "https://github.com/acme/platform/issues/318", + "user": { + "login": "dana-r" + }, + "created_at": "2026-10-19T10:00:00Z", + "updated_at": "2026-10-20T08:00:00Z", + "closed_at": null + }, + "changes": { + "title": { + "from": "Webhook retries double post" + } + }, + "repository": { + "id": 712345678, + "full_name": "acme/platform", + "name": "platform", + "owner": { + "login": "acme" + } + }, + "installation": { + "id": 51234567 + } + }, + "record": "issue", + "match": "issues.edited", + "headers": { + "x-github-event": "issues", + "x-github-delivery": "72d3162e-issue.edited" + } + }, + "issue.closed": { + "delivery": "webhook", + "object": "issue", + "payload": { + "action": "closed", + "issue": { + "id": 2601234567, + "number": 318, + "title": "Webhook retries double-post comments", + "body": "When the queue retries a delivery, the comment is posted twice.", + "state": "closed", + "state_reason": "completed", + "labels": [ + { + "id": 6012345, + "name": "bug" + }, + { + "id": 6012399, + "name": "P1" + } + ], + "assignee": { + "login": "octocat" + }, + "html_url": "https://github.com/acme/platform/issues/318", + "user": { + "login": "dana-r" + }, + "created_at": "2026-10-19T10:00:00Z", + "updated_at": "2026-10-20T08:00:00Z", + "closed_at": "2026-10-21T09:00:00Z" + }, + "repository": { + "id": 712345678, + "full_name": "acme/platform", + "name": "platform", + "owner": { + "login": "acme" + } + }, + "installation": { + "id": 51234567 + } + }, + "record": "issue", + "match": "issues.closed", + "headers": { + "x-github-event": "issues", + "x-github-delivery": "72d3162e-issue.closed" + } + }, + "issue.labeled": { + "delivery": "webhook", + "object": "issue", + "payload": { + "action": "labeled", + "issue": { + "id": 2601234567, + "number": 318, + "title": "Webhook retries double-post comments", + "body": "When the queue retries a delivery, the comment is posted twice.", + "state": "open", + "state_reason": null, + "labels": [ + { + "id": 6012345, + "name": "bug" + }, + { + "id": 6012399, + "name": "P1" + } + ], + "assignee": { + "login": "octocat" + }, + "html_url": "https://github.com/acme/platform/issues/318", + "user": { + "login": "dana-r" + }, + "created_at": "2026-10-19T10:00:00Z", + "updated_at": "2026-10-20T08:00:00Z", + "closed_at": null + }, + "label": { + "name": "P1" + }, + "repository": { + "id": 712345678, + "full_name": "acme/platform", + "name": "platform", + "owner": { + "login": "acme" + } + }, + "installation": { + "id": 51234567 + } + }, + "record": "issue", + "match": "issues.labeled", + "headers": { + "x-github-event": "issues", + "x-github-delivery": "72d3162e-issue.labeled" + } + }, + "pullRequest.closed": { + "delivery": "webhook", + "object": "pullRequest", + "payload": { + "action": "closed", + "number": 412, + "pull_request": { + "id": 2109876543, + "number": 412, + "title": "Fix duplicate comment on retry (ENG-142)", + "state": "closed", + "merged": true, + "merged_at": "2026-10-21T08:55:00Z", + "head": { + "ref": "dana/eng-142-dedupe-comments" + }, + "base": { + "ref": "main" + }, + "html_url": "https://github.com/acme/platform/pull/412", + "body": "Closes #318" + }, + "repository": { + "id": 712345678, + "full_name": "acme/platform", + "name": "platform", + "owner": { + "login": "acme" + } + }, + "installation": { + "id": 51234567 + } + }, + "record": "pull_request", + "match": "pull_request.closed", + "headers": { + "x-github-event": "pull_request", + "x-github-delivery": "72d3162e-pullRequest.closed" + } + } + }, + "webhook": { + "scope": "app", + "topic": "{{headers.x-github-event}}.{{body.action}}", + "eventId": "{{headers.x-github-delivery}}", + "account": "{{body.installation.id}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "issue.get": { + "kind": "read", + "method": "getIssue", + "input": { + "owner": { + "type": "string", + "required": true + }, + "repo": { + "type": "string", + "required": true + }, + "issue_number": { + "type": "number", + "required": true + } + }, + "output": "issue" + }, + "issue.list": { + "kind": "read", + "method": "listIssuesForRepo", + "input": { + "owner": { + "type": "string", + "required": true + }, + "repo": { + "type": "string", + "required": true + }, + "state": { + "type": "string" + }, + "since": { + "type": "string" + }, + "per_page": { + "type": "number" + }, + "page": { + "type": "number" + } + }, + "output": "issue[]" + }, + "issue.create": { + "kind": "write", + "method": "createIssue", + "input": { + "owner": { + "type": "string", + "required": true + }, + "repo": { + "type": "string", + "required": true + }, + "title": { + "type": "string", + "required": true + }, + "body": { + "type": "string" + }, + "labels": { + "type": "array" + }, + "assignees": { + "type": "array" + } + }, + "output": "issue" + }, + "issue.update": { + "kind": "write", + "method": "updateIssue", + "input": { + "owner": { + "type": "string", + "required": true + }, + "repo": { + "type": "string", + "required": true + }, + "issue_number": { + "type": "number", + "required": true + }, + "title": { + "type": "string" + }, + "body": { + "type": "string" + }, + "state": { + "type": "string" + }, + "state_reason": { + "type": "string" + }, + "labels": { + "type": "array" + } + }, + "output": "issue" + }, + "issue.comment": { + "kind": "write", + "method": "createIssueComment", + "input": { + "owner": { + "type": "string", + "required": true + }, + "repo": { + "type": "string", + "required": true + }, + "issue_number": { + "type": "number", + "required": true + }, + "body": { + "type": "string", + "required": true + } + }, + "output": "comment" + }, + "repo.list": { + "kind": "read", + "method": "listInstallationRepos", + "input": {}, + "output": "repository[]" + } + }, + "options": { + "repo": { + "capability": "repo.list", + "label": "{{item.full_name}}", + "value": "$.item.full_name" + } + } +} diff --git a/packages/core/integrations/data-defined/capabilities/hubspot.capabilities.json b/packages/core/integrations/data-defined/capabilities/hubspot.capabilities.json new file mode 100644 index 000000000..47caa1dcb --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/hubspot.capabilities.json @@ -0,0 +1,715 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "hubspot", + "label": "HubSpot", + "package": "@friggframework/api-module-hubspot", + "version": "1.4.0", + "auth": "oauth2", + "account": { + "description": "Values about the connected account, persisted by the module (apiPropertiesToPersist). Definitions read them as $.install.accounts...", + "fields": [ + { + "name": "portalId", + "type": "string", + "description": "HubSpot portal (hub) id" + } + ] + }, + "objects": { + "contact": { + "label": "Contact", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "properties.email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "properties.firstname", + "type": "string", + "tag": "person.givenName" + }, + { + "name": "properties.lastname", + "type": "string", + "tag": "person.familyName" + }, + { + "name": "properties.phone", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "properties.mobilephone", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "properties.company", + "type": "string", + "tag": "organization.name" + }, + { + "name": "properties.lifecyclestage", + "type": "enum", + "tag": "contact.lifecycleStage", + "values": [ + "subscriber", + "lead", + "marketingqualifiedlead", + "salesqualifiedlead", + "opportunity", + "customer" + ] + }, + { + "name": "properties.hubspot_owner_id", + "type": "string", + "tag": "record.owner" + }, + { + "name": "properties.notes_last_contacted", + "type": "datetime", + "tag": "activity.lastContactedAt", + "readOnly": true + }, + { + "name": "properties.phone_last_call", + "type": "datetime", + "custom": true + }, + { + "name": "properties.jobtitle", + "type": "string", + "tag": "person.jobTitle" + }, + { + "name": "properties.city", + "type": "string", + "tag": "address.locality" + }, + { + "name": "properties.hs_lead_status", + "type": "enum", + "tag": "contact.leadStatus", + "values": [ + "NEW", + "OPEN", + "IN_PROGRESS", + "OPEN_DEAL", + "UNQUALIFIED", + "ATTEMPTED_TO_CONTACT", + "CONNECTED", + "BAD_TIMING" + ] + }, + { + "name": "properties.hs_email_optout", + "type": "boolean", + "tag": "contact.emailOptOut", + "readOnly": true + } + ] + }, + "deal": { + "label": "Deal", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "properties.dealname", + "type": "string", + "tag": "deal.name" + }, + { + "name": "properties.amount", + "type": "number", + "tag": "deal.amount" + }, + { + "name": "properties.dealstage", + "type": "enum", + "tag": "deal.stage", + "values": "dealStage" + }, + { + "name": "properties.closedate", + "type": "date", + "tag": "deal.closeDate" + }, + { + "name": "properties.hubspot_owner_id", + "type": "string", + "tag": "record.owner" + }, + { + "name": "properties.pipeline", + "type": "string", + "tag": "deal.pipeline" + }, + { + "name": "properties.hs_next_step", + "type": "text", + "tag": "deal.nextStep" + } + ] + }, + "company": { + "label": "Company", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "properties.name", + "type": "string", + "tag": "organization.name" + }, + { + "name": "properties.domain", + "type": "string", + "tag": "organization.domain" + } + ] + }, + "task": { + "label": "Task", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "properties.hs_task_subject", + "type": "string", + "required": true + }, + { + "name": "properties.hs_task_body", + "type": "text" + }, + { + "name": "properties.hs_timestamp", + "type": "datetime", + "required": true + }, + { + "name": "properties.hubspot_owner_id", + "type": "string" + }, + { + "name": "properties.hs_task_status", + "type": "enum", + "tag": "task.status", + "values": [ + "NOT_STARTED", + "IN_PROGRESS", + "WAITING", + "COMPLETED", + "DEFERRED" + ] + }, + { + "name": "properties.hs_task_priority", + "type": "enum", + "tag": "task.priority", + "values": [ + "LOW", + "MEDIUM", + "HIGH" + ] + } + ] + }, + "call": { + "label": "Call engagement", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "properties.hs_call_title", + "type": "string" + }, + { + "name": "properties.hs_call_body", + "type": "text" + }, + { + "name": "properties.hs_call_duration", + "type": "number" + }, + { + "name": "properties.hs_call_direction", + "type": "enum", + "values": [ + "INBOUND", + "OUTBOUND" + ] + }, + { + "name": "properties.hs_call_recording_url", + "type": "string" + }, + { + "name": "properties.hs_timestamp", + "type": "datetime", + "required": true + } + ] + }, + "note": { + "label": "Note", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "properties.hs_note_body", + "type": "text", + "required": true + }, + { + "name": "properties.hs_timestamp", + "type": "datetime", + "required": true + } + ] + } + }, + "events": { + "contact.created": { + "delivery": "webhook", + "object": "contact", + "description": "App-level subscription; Frigg routes it to the installation by portalId.", + "payload": [ + { + "portalId": 12345678, + "objectId": "51", + "subscriptionType": "contact.creation", + "occurredAt": 1760800000000, + "eventId": 9001 + } + ], + "match": "contact.creation" + }, + "contact.propertyChange": { + "delivery": "webhook", + "object": "contact", + "payload": [ + { + "portalId": 12345678, + "objectId": "51", + "propertyName": "phone", + "propertyValue": "+15125550142", + "occurredAt": 1760800000000, + "eventId": 9002, + "subscriptionType": "contact.propertyChange" + } + ] + }, + "deal.propertyChange": { + "delivery": "webhook", + "object": "deal", + "payload": [ + { + "portalId": 12345678, + "objectId": "880", + "propertyName": "dealstage", + "propertyValue": "closedwon", + "occurredAt": 1760800000000, + "eventId": 9003, + "subscriptionType": "deal.propertyChange" + } + ] + }, + "deal.created": { + "delivery": "webhook", + "object": "deal", + "payload": [ + { + "portalId": 12345678, + "objectId": "881", + "occurredAt": 1760800000000, + "eventId": 9004, + "subscriptionType": "deal.creation" + } + ], + "match": "deal.creation" + } + }, + "webhook": { + "scope": "app", + "batch": "body", + "topic": "{{item.subscriptionType}}", + "eventId": "{{item.eventId}}", + "account": "{{item.portalId}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "contact.get": { + "kind": "read", + "method": "getContactById", + "input": { + "id": { + "type": "string", + "required": true + }, + "properties": { + "type": "array" + } + }, + "output": "contact" + }, + "contact.search": { + "kind": "search", + "method": "searchContacts", + "input": { + "email": { + "type": "string" + }, + "phone": { + "type": "string" + }, + "limit": { + "type": "number" + }, + "lastContactedBefore": { + "type": "string" + } + }, + "output": "contact[]" + }, + "contact.list": { + "kind": "read", + "method": "listContacts", + "input": { + "after": { + "type": "string" + }, + "limit": { + "type": "number" + }, + "updatedSince": { + "type": "string" + } + }, + "output": "page" + }, + "contact.create": { + "kind": "write", + "method": "createContact", + "input": { + "properties": { + "type": "object", + "required": true + } + }, + "output": "contact" + }, + "contact.update": { + "kind": "write", + "method": "updateContact", + "input": { + "id": { + "type": "string", + "required": true + }, + "properties": { + "type": "object", + "required": true + } + }, + "output": "contact" + }, + "deal.get": { + "kind": "read", + "method": "getDealById", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "deal" + }, + "deal.listForContact": { + "kind": "read", + "method": "listDealsForContact", + "input": { + "contactId": { + "type": "string", + "required": true + }, + "open": { + "type": "boolean" + } + }, + "output": "deal[]" + }, + "deal.contacts": { + "kind": "read", + "method": "listContactsForDeal", + "input": { + "dealId": { + "type": "string", + "required": true + } + }, + "output": "contact[]" + }, + "task.create": { + "kind": "write", + "method": "createTask", + "input": { + "properties": { + "type": "object", + "required": true + }, + "contactId": { + "type": "string" + } + }, + "output": "task" + }, + "call.create": { + "kind": "write", + "method": "createCallEngagement", + "input": { + "properties": { + "type": "object", + "required": true + }, + "contactId": { + "type": "string", + "required": true + } + }, + "output": "call" + }, + "note.create": { + "kind": "write", + "method": "createNote", + "input": { + "properties": { + "type": "object", + "required": true + }, + "contactId": { + "type": "string" + }, + "dealId": { + "type": "string" + } + }, + "output": "note" + }, + "owner.list": { + "kind": "read", + "method": "listOwners", + "input": {}, + "output": "owner[]" + }, + "pipeline.stages": { + "kind": "read", + "method": "listDealStages", + "input": { + "pipelineId": { + "type": "string" + } + }, + "output": "stage[]" + }, + "contact.upsertByEmail": { + "kind": "write", + "method": "upsertContactByEmail", + "label": "Batch upsert with idProperty=email", + "input": { + "email": { + "type": "string", + "required": true + }, + "properties": { + "type": "object", + "required": true + } + }, + "output": "contact" + }, + "deal.update": { + "kind": "write", + "method": "updateDeal", + "input": { + "id": { + "type": "string", + "required": true + }, + "properties": { + "type": "object", + "required": true + } + }, + "output": "deal" + }, + "deal.search": { + "kind": "search", + "method": "searchDeals", + "input": { + "open": { + "type": "boolean" + }, + "closeDateBefore": { + "type": "string" + }, + "closeDateAfter": { + "type": "string" + }, + "minAmount": { + "type": "number" + }, + "limit": { + "type": "number" + } + }, + "output": "deal[]" + }, + "task.get": { + "kind": "read", + "method": "getTaskById", + "input": { + "id": { + "type": "string", + "required": true + }, + "properties": { + "type": "array" + } + }, + "output": "task" + }, + "task.update": { + "kind": "write", + "method": "updateTask", + "input": { + "id": { + "type": "string", + "required": true + }, + "properties": { + "type": "object", + "required": true + } + }, + "output": "task" + }, + "task.search": { + "kind": "search", + "method": "searchTasks", + "input": { + "dueBefore": { + "type": "string" + }, + "status": { + "type": "string" + }, + "ownerId": { + "type": "string" + }, + "limit": { + "type": "number" + } + }, + "output": "task[]" + }, + "owner.get": { + "kind": "read", + "method": "getOwnerById", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "owner" + }, + "owner.findByEmail": { + "kind": "search", + "method": "getOwnerByEmail", + "input": { + "email": { + "type": "string", + "required": true + } + }, + "output": "owner" + }, + "subscription.definitions": { + "kind": "read", + "method": "listSubscriptionDefinitions", + "input": {}, + "output": "subscriptionDefinition[]" + }, + "subscription.unsubscribe": { + "kind": "write", + "method": "unsubscribeEmail", + "label": "Communication preferences API; hs_email_optout itself is read-only", + "input": { + "emailAddress": { + "type": "string", + "required": true + }, + "subscriptionId": { + "type": "string", + "required": true + }, + "legalBasis": { + "type": "string" + }, + "legalBasisExplanation": { + "type": "string" + } + }, + "output": "subscriptionStatus" + } + }, + "options": { + "owner": { + "capability": "owner.list", + "label": "{{item.firstName}} {{item.lastName}}", + "value": "$.item.id" + }, + "dealStage": { + "capability": "pipeline.stages", + "label": "$.item.label", + "value": "$.item.id" + }, + "subscriptionType": { + "capability": "subscription.definitions", + "label": "{{item.name}}", + "value": "$.item.id" + } + }, + "ui": { + "placements": [ + "crm.record.sidebar", + "crm.record.tab", + "crm.record.preview" + ], + "renderer": "left-hook-hubspot-app/frigg-card" + } +} diff --git a/packages/core/integrations/data-defined/capabilities/jira.capabilities.json b/packages/core/integrations/data-defined/capabilities/jira.capabilities.json new file mode 100644 index 000000000..f390dc563 --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/jira.capabilities.json @@ -0,0 +1,376 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "jira", + "label": "Jira Cloud", + "package": "@friggframework/api-module-jira", + "version": "2.2.0", + "auth": "oauth2", + "account": { + "description": "Values about the connected account, persisted by the module (apiPropertiesToPersist). Definitions read them as $.install.accounts...", + "fields": [ + { + "name": "site", + "type": "string", + "description": "Atlassian site host (acme.atlassian.net)" + }, + { + "name": "cloudId", + "type": "string", + "description": "Atlassian cloud id used in API URLs" + } + ] + }, + "objects": { + "issue": { + "label": "Issue", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "key", + "type": "string", + "tag": "record.displayId", + "readOnly": true + }, + { + "name": "fields.summary", + "type": "string", + "tag": "ticket.subject", + "required": true + }, + { + "name": "fields.description", + "type": "text", + "tag": "ticket.description" + }, + { + "name": "fields.issuetype.id", + "type": "string" + }, + { + "name": "fields.project.key", + "type": "string" + }, + { + "name": "fields.priority.name", + "type": "enum", + "tag": "ticket.priority", + "values": [ + "Highest", + "High", + "Medium", + "Low", + "Lowest" + ] + }, + { + "name": "fields.labels", + "type": "string[]", + "tag": "record.tags" + }, + { + "name": "fields.assignee.accountId", + "type": "string", + "tag": "record.owner" + }, + { + "name": "fields.status.name", + "type": "string", + "tag": "ticket.status", + "readOnly": true + }, + { + "name": "fields.status.statusCategory.key", + "type": "enum", + "values": [ + "new", + "indeterminate", + "done" + ], + "readOnly": true + }, + { + "name": "fields.resolution.name", + "type": "string", + "readOnly": true + }, + { + "name": "fields.created", + "type": "datetime", + "tag": "record.createdAt", + "readOnly": true + }, + { + "name": "fields.updated", + "type": "datetime", + "tag": "record.updatedAt", + "readOnly": true + } + ] + } + }, + "events": { + "issue.updated": { + "delivery": "webhook", + "object": "issue", + "subscribe": "webhook.register", + "description": "jira:issue_updated from a dynamic webhook registered by the OAuth app. Dynamic webhooks expire after 30 days unless refreshed.", + "payload": { + "timestamp": 1760958061000, + "webhookEvent": "jira:issue_updated", + "issue_event_type_name": "issue_generic", + "matchedWebhookIds": [ + 104 + ], + "user": { + "accountId": "5b10ac8d82e05b22cc7d4ef5", + "displayName": "Lee Wong" + }, + "issue": { + "id": "10071", + "key": "SUP-482", + "self": "https://acme.atlassian.net/rest/api/2/issue/10071", + "fields": { + "summary": "[ZD-35436] Checkout fails with 500 for EU customers", + "priority": { + "name": "Highest" + }, + "labels": [ + "zendesk" + ], + "status": { + "name": "Done", + "statusCategory": { + "key": "done" + } + }, + "resolution": { + "name": "Fixed" + }, + "updated": "2026-10-20T11:01:01.000+0000" + } + }, + "changelog": { + "id": "10155", + "items": [ + { + "field": "status", + "fieldtype": "jira", + "fromString": "In Progress", + "toString": "Done" + } + ] + } + }, + "record": "issue", + "match": "jira:issue_updated" + }, + "issue.created": { + "delivery": "webhook", + "object": "issue", + "subscribe": "webhook.register", + "payload": { + "timestamp": 1760950000000, + "webhookEvent": "jira:issue_created", + "issue": { + "id": "10071", + "key": "SUP-482", + "fields": { + "summary": "[ZD-35436] Checkout fails with 500 for EU customers", + "labels": [ + "zendesk" + ] + } + } + }, + "record": "issue", + "match": "jira:issue_created" + } + }, + "webhook": { + "scope": "installation", + "topic": "{{body.webhookEvent}}", + "eventId": "{{body.issue.id}}:{{body.timestamp}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "issue.create": { + "kind": "write", + "method": "createIssue", + "input": { + "fields": { + "type": "object", + "required": true + } + }, + "output": "issue" + }, + "issue.get": { + "kind": "read", + "method": "getIssue", + "input": { + "issueIdOrKey": { + "type": "string", + "required": true + }, + "fields": { + "type": "array" + } + }, + "output": "issue" + }, + "issue.update": { + "kind": "write", + "method": "editIssue", + "input": { + "issueIdOrKey": { + "type": "string", + "required": true + }, + "fields": { + "type": "object", + "required": true + } + }, + "output": "void" + }, + "issue.addComment": { + "kind": "write", + "method": "addComment", + "input": { + "issueIdOrKey": { + "type": "string", + "required": true + }, + "body": { + "type": "string", + "required": true + } + }, + "output": "comment" + }, + "issue.transition": { + "kind": "write", + "method": "transitionIssue", + "input": { + "issueIdOrKey": { + "type": "string", + "required": true + }, + "transitionId": { + "type": "string", + "required": true + } + }, + "output": "void" + }, + "issue.search": { + "kind": "search", + "method": "searchJql", + "input": { + "jql": { + "type": "string", + "required": true + }, + "maxResults": { + "type": "number" + }, + "fields": { + "type": "array" + } + }, + "output": "issue[]" + }, + "remoteLink.create": { + "kind": "write", + "method": "createRemoteLink", + "input": { + "issueIdOrKey": { + "type": "string", + "required": true + }, + "url": { + "type": "string", + "required": true + }, + "title": { + "type": "string", + "required": true + } + }, + "output": "remoteLink" + }, + "project.list": { + "kind": "read", + "method": "listProjects", + "input": {}, + "output": "project[]" + }, + "issueType.list": { + "kind": "read", + "method": "listIssueTypes", + "input": { + "projectKey": { + "type": "string" + } + }, + "output": "issueType[]" + }, + "webhook.register": { + "kind": "subscribe", + "method": "registerDynamicWebhooks", + "input": { + "url": { + "type": "string", + "required": true + }, + "events": { + "type": "array", + "required": true + }, + "jqlFilter": { + "type": "string", + "required": true + } + }, + "output": "webhook[]" + }, + "webhook.refresh": { + "kind": "write", + "method": "refreshWebhooks", + "input": { + "webhookIds": { + "type": "array", + "required": true + } + }, + "output": "void" + }, + "webhook.delete": { + "kind": "delete", + "method": "deleteWebhooks", + "input": { + "webhookIds": { + "type": "array", + "required": true + } + }, + "output": "void" + } + }, + "options": { + "project": { + "capability": "project.list", + "label": "{{item.name}} ({{item.key}})", + "value": "$.item.key" + }, + "issueType": { + "capability": "issueType.list", + "label": "{{item.name}}", + "value": "$.item.id" + } + } +} diff --git a/packages/core/integrations/data-defined/capabilities/klaviyo.capabilities.json b/packages/core/integrations/data-defined/capabilities/klaviyo.capabilities.json new file mode 100644 index 000000000..363f0f0cc --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/klaviyo.capabilities.json @@ -0,0 +1,475 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "klaviyo", + "label": "Klaviyo", + "package": "@friggframework/api-module-klaviyo", + "version": "1.1.0", + "auth": "oauth2", + "objects": { + "profile": { + "label": "Profile", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "attributes.email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "attributes.phone_number", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "attributes.first_name", + "type": "string", + "tag": "person.givenName" + }, + { + "name": "attributes.last_name", + "type": "string", + "tag": "person.familyName" + }, + { + "name": "attributes.organization", + "type": "string", + "tag": "organization.name" + }, + { + "name": "attributes.title", + "type": "string", + "tag": "person.jobTitle" + }, + { + "name": "attributes.external_id", + "type": "string", + "tag": "record.externalId" + }, + { + "name": "attributes.location.city", + "type": "string", + "tag": "address.locality" + }, + { + "name": "attributes.location.region", + "type": "string", + "tag": "address.region" + }, + { + "name": "attributes.location.country", + "type": "string", + "tag": "address.country" + }, + { + "name": "attributes.location.zip", + "type": "string", + "tag": "address.postalCode" + }, + { + "name": "attributes.subscriptions.email.marketing.consent", + "type": "enum", + "tag": "contact.emailConsent", + "values": [ + "SUBSCRIBED", + "UNSUBSCRIBED", + "NEVER_SUBSCRIBED" + ], + "readOnly": true + }, + { + "name": "attributes.properties.lifetime_value", + "type": "number", + "tag": "customer.lifetimeValue" + }, + { + "name": "attributes.properties.shopify_tags", + "type": "string", + "tag": "record.tags" + }, + { + "name": "attributes.properties.hubspot_lifecycle_stage", + "type": "string", + "tag": "contact.lifecycleStage" + }, + { + "name": "attributes.created", + "type": "datetime", + "tag": "record.createdAt", + "readOnly": true + }, + { + "name": "attributes.updated", + "type": "datetime", + "tag": "record.updatedAt", + "readOnly": true + } + ] + }, + "list": { + "label": "List", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "attributes.name", + "type": "string", + "tag": "list.name" + }, + { + "name": "attributes.created", + "type": "datetime", + "readOnly": true + } + ] + }, + "event": { + "label": "Event", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "attributes.metric_name", + "type": "string" + }, + { + "name": "attributes.properties", + "type": "json" + }, + { + "name": "attributes.value", + "type": "number" + }, + { + "name": "attributes.time", + "type": "datetime" + }, + { + "name": "attributes.unique_id", + "type": "string" + } + ] + } + }, + "events": { + "list.subscribed": { + "delivery": "webhook", + "object": "profile", + "subscribe": "webhook.create", + "description": "Webhook topic event:klaviyo.subscribed_to_list. $.record is the profile; the list is in data.attributes.event_properties.", + "payload": { + "id": "01JAEXAMPLE7Q9VK3D6M2N8P4R", + "topic": "event:klaviyo.subscribed_to_list", + "created_at": "2026-10-18T16:02:11Z", + "data": { + "type": "event", + "id": "4vRpBT", + "attributes": { + "metric_name": "Subscribed to List", + "datetime": "2026-10-18T16:02:10Z", + "event_properties": { + "list_id": "Y6nRLr", + "List": "VIP Customers", + "method": "Shopify" + } + }, + "profile": { + "type": "profile", + "id": "01HV6N2Y4K9JQ3ZB8XWQ7T5R2M", + "email": "dana.reyes@example.com", + "phone_number": "+15125550142", + "first_name": "Dana", + "last_name": "Reyes", + "organization": "Reyes Coffee Co", + "external_id": "7012345678901" + } + } + }, + "record": "data.profile", + "match": "event:klaviyo.subscribed_to_list" + }, + "profile.unsubscribed": { + "delivery": "webhook", + "object": "profile", + "subscribe": "webhook.create", + "description": "Webhook topic event:klaviyo.unsubscribed (email marketing). Global unsubscribe, not one list.", + "payload": { + "id": "01JAEXAMPLE7Q9VK3D6M2N8P4R", + "topic": "event:klaviyo.unsubscribed", + "created_at": "2026-10-18T16:02:11Z", + "data": { + "type": "event", + "id": "4vRpBT", + "attributes": { + "metric_name": "Unsubscribed", + "datetime": "2026-10-18T16:02:10Z", + "event_properties": { + "channel": "email", + "method": "Unsubscribe link" + } + }, + "profile": { + "type": "profile", + "id": "01HV6N2Y4K9JQ3ZB8XWQ7T5R2M", + "email": "dana.reyes@example.com", + "phone_number": "+15125550142", + "first_name": "Dana", + "last_name": "Reyes", + "organization": "Reyes Coffee Co", + "external_id": "7012345678901" + } + } + }, + "record": "data.profile", + "match": "event:klaviyo.unsubscribed" + }, + "profile.updated": { + "delivery": "poll", + "object": "profile", + "description": "Polled from GET /api/profiles?filter=greater-than(updated,).", + "payload": { + "type": "profile", + "id": "01HV6N2Y4K9JQ3ZB8XWQ7T5R2M", + "attributes": { + "email": "dana.reyes@example.com", + "first_name": "Dana", + "last_name": "Reyes", + "updated": "2026-10-18T17:00:00Z" + } + } + } + }, + "webhook": { + "scope": "installation", + "topic": "{{body.topic}}", + "eventId": "{{body.id}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "profile.get": { + "kind": "read", + "method": "getProfile", + "input": { + "id": { + "type": "string", + "required": true + }, + "additionalFields": { + "type": "array" + } + }, + "output": "profile" + }, + "profile.search": { + "kind": "search", + "method": "getProfiles", + "input": { + "email": { + "type": "string" + }, + "phone_number": { + "type": "string" + }, + "external_id": { + "type": "string" + } + }, + "output": "profile[]" + }, + "profile.list": { + "kind": "read", + "method": "listProfiles", + "input": { + "updatedSince": { + "type": "string" + }, + "pageCursor": { + "type": "string" + }, + "pageSize": { + "type": "number" + } + }, + "output": "page" + }, + "profile.create": { + "kind": "write", + "method": "createProfile", + "input": { + "attributes": { + "type": "object", + "required": true + } + }, + "output": "profile" + }, + "profile.update": { + "kind": "write", + "method": "updateProfile", + "input": { + "id": { + "type": "string", + "required": true + }, + "attributes": { + "type": "object", + "required": true + } + }, + "output": "profile" + }, + "profile.upsert": { + "kind": "write", + "method": "createOrUpdateProfile", + "input": { + "attributes": { + "type": "object", + "required": true + } + }, + "output": "profile" + }, + "profile.lists": { + "kind": "read", + "method": "getProfileLists", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "list[]" + }, + "profile.subscribe": { + "kind": "write", + "method": "subscribeProfiles", + "input": { + "listId": { + "type": "string", + "required": true + }, + "email": { + "type": "string" + }, + "phoneNumber": { + "type": "string" + }, + "channels": { + "type": "array" + } + }, + "output": "void" + }, + "profile.unsubscribe": { + "kind": "write", + "method": "unsubscribeProfiles", + "input": { + "email": { + "type": "string", + "required": true + }, + "listId": { + "type": "string" + } + }, + "output": "void" + }, + "list.list": { + "kind": "read", + "method": "getLists", + "input": {}, + "output": "list[]" + }, + "list.addProfiles": { + "kind": "write", + "method": "addProfilesToList", + "input": { + "listId": { + "type": "string", + "required": true + }, + "profileIds": { + "type": "array", + "required": true + } + }, + "output": "void" + }, + "event.create": { + "kind": "write", + "method": "createEvent", + "input": { + "metric": { + "type": "string", + "required": true + }, + "profile": { + "type": "object", + "required": true + }, + "properties": { + "type": "object" + }, + "value": { + "type": "number" + }, + "time": { + "type": "string" + }, + "uniqueId": { + "type": "string" + } + }, + "output": "void" + }, + "webhook.create": { + "kind": "subscribe", + "method": "createWebhook", + "input": { + "name": { + "type": "string", + "required": true + }, + "endpointUrl": { + "type": "string", + "required": true + }, + "topics": { + "type": "array", + "required": true + }, + "secretKey": { + "type": "string" + } + }, + "output": "webhook" + }, + "webhook.delete": { + "kind": "delete", + "method": "deleteWebhook", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "void" + } + }, + "options": { + "list": { + "capability": "list.list", + "label": "{{item.attributes.name}}", + "value": "$.item.id" + } + } +} diff --git a/packages/core/integrations/data-defined/capabilities/linear.capabilities.json b/packages/core/integrations/data-defined/capabilities/linear.capabilities.json new file mode 100644 index 000000000..c93db82e8 --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/linear.capabilities.json @@ -0,0 +1,324 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "linear", + "label": "Linear", + "package": "@friggframework/api-module-linear", + "version": "1.1.0", + "auth": "oauth2", + "objects": { + "issue": { + "label": "Issue", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "identifier", + "type": "string", + "tag": "record.displayId", + "readOnly": true + }, + { + "name": "title", + "type": "string", + "tag": "task.title", + "required": true + }, + { + "name": "description", + "type": "text", + "tag": "task.description" + }, + { + "name": "priority", + "type": "enum", + "tag": "task.priority", + "values": [ + 0, + 1, + 2, + 3, + 4 + ] + }, + { + "name": "state.name", + "type": "string", + "tag": "task.state" + }, + { + "name": "state.type", + "type": "enum", + "values": [ + "triage", + "backlog", + "unstarted", + "started", + "completed", + "canceled" + ], + "readOnly": true + }, + { + "name": "labelIds", + "type": "string[]", + "tag": "record.tags" + }, + { + "name": "assignee.email", + "type": "email", + "tag": "record.owner" + }, + { + "name": "team.key", + "type": "string", + "readOnly": true + }, + { + "name": "attachments[0].url", + "type": "string", + "tag": "record.externalUrl" + }, + { + "name": "url", + "type": "string", + "tag": "record.url", + "readOnly": true + }, + { + "name": "createdAt", + "type": "datetime", + "tag": "record.createdAt", + "readOnly": true + }, + { + "name": "updatedAt", + "type": "datetime", + "tag": "record.updatedAt", + "readOnly": true + } + ] + } + }, + "events": { + "issue.created": { + "delivery": "webhook", + "object": "issue", + "description": "OAuth-app webhook configured once on the application; routed by organizationId. No per-install subscription.", + "payload": { + "action": "create", + "type": "Issue", + "createdAt": "2026-10-19T10:01:00.000Z", + "organizationId": "b0c1d2e3-f4a5-4b6c-8d7e-9f0a1b2c3d4e", + "webhookTimestamp": 1760868060000, + "url": "https://linear.app/acme/issue/ENG-142", + "data": { + "id": "5f1a2b3c-4d5e-4f60-8a7b-9c0d1e2f3a4b", + "identifier": "ENG-142", + "title": "Webhook retries double-post comments", + "priority": 2, + "state": { + "name": "Todo", + "type": "unstarted" + }, + "team": { + "key": "ENG" + } + } + }, + "record": "data", + "match": "Issue.create" + }, + "issue.updated": { + "delivery": "webhook", + "object": "issue", + "payload": { + "action": "update", + "type": "Issue", + "createdAt": "2026-10-20T12:00:00.000Z", + "organizationId": "b0c1d2e3-f4a5-4b6c-8d7e-9f0a1b2c3d4e", + "webhookTimestamp": 1760961600000, + "data": { + "id": "5f1a2b3c-4d5e-4f60-8a7b-9c0d1e2f3a4b", + "identifier": "ENG-142", + "title": "Webhook retries double-post comments", + "priority": 2, + "state": { + "name": "In Progress", + "type": "started" + } + }, + "updatedFrom": { + "stateId": "1d2c3b4a-5e6f-4a7b-8c9d-0e1f2a3b4c5d" + } + }, + "record": "data", + "match": "Issue.update" + } + }, + "webhook": { + "scope": "app", + "topic": "{{body.type}}.{{body.action}}", + "eventId": [ + "{{headers.linear-delivery}}", + "{{body.webhookId}}:{{body.webhookTimestamp}}" + ], + "account": "{{body.organizationId}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "issue.get": { + "kind": "read", + "method": "issue", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "issue" + }, + "issue.byIdentifier": { + "kind": "read", + "method": "issueByIdentifier", + "input": { + "identifier": { + "type": "string", + "required": true + } + }, + "output": "issue" + }, + "issue.search": { + "kind": "search", + "method": "searchIssues", + "input": { + "teamId": { + "type": "string" + }, + "stateType": { + "type": "string" + }, + "updatedBefore": { + "type": "string" + }, + "first": { + "type": "number" + } + }, + "output": "issue[]" + }, + "issue.create": { + "kind": "write", + "method": "issueCreate", + "input": { + "teamId": { + "type": "string", + "required": true + }, + "title": { + "type": "string", + "required": true + }, + "description": { + "type": "string" + }, + "priority": { + "type": "number" + }, + "stateName": { + "type": "string" + }, + "labelIds": { + "type": "array" + } + }, + "output": "issue" + }, + "issue.update": { + "kind": "write", + "method": "issueUpdate", + "input": { + "id": { + "type": "string", + "required": true + }, + "input": { + "type": "object", + "required": true + } + }, + "output": "issue" + }, + "comment.create": { + "kind": "write", + "method": "commentCreate", + "input": { + "issueId": { + "type": "string", + "required": true + }, + "body": { + "type": "string", + "required": true + } + }, + "output": "comment" + }, + "attachment.link": { + "kind": "write", + "method": "attachmentLinkURL", + "input": { + "issueId": { + "type": "string", + "required": true + }, + "url": { + "type": "string", + "required": true + }, + "title": { + "type": "string" + } + }, + "output": "attachment" + }, + "team.list": { + "kind": "read", + "method": "teams", + "input": {}, + "output": "team[]" + }, + "workflowState.list": { + "kind": "read", + "method": "workflowStates", + "input": { + "teamId": { + "type": "string", + "required": true + } + }, + "output": "workflowState[]" + } + }, + "options": { + "team": { + "capability": "team.list", + "label": "{{item.name}} ({{item.key}})", + "value": "$.item.id" + }, + "workflowState": { + "capability": "workflowState.list", + "label": "{{item.name}}", + "value": "$.item.name", + "inputs": { + "teamId": { + "type": "string", + "required": true + } + } + } + } +} diff --git a/packages/core/integrations/data-defined/capabilities/netsuite.capabilities.json b/packages/core/integrations/data-defined/capabilities/netsuite.capabilities.json new file mode 100644 index 000000000..c05e5f92b --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/netsuite.capabilities.json @@ -0,0 +1,491 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "netsuite", + "label": "NetSuite", + "package": "@friggframework/api-module-netsuite", + "version": "1.2.0", + "auth": "oauth2", + "objects": { + "customer": { + "label": "Customer", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "entityId", + "type": "string", + "tag": "record.displayId", + "readOnly": true + }, + { + "name": "externalId", + "type": "string", + "tag": "record.externalId" + }, + { + "name": "isPerson", + "type": "boolean" + }, + { + "name": "firstName", + "type": "string", + "tag": "person.givenName" + }, + { + "name": "lastName", + "type": "string", + "tag": "person.familyName" + }, + { + "name": "companyName", + "type": "string", + "tag": "organization.name" + }, + { + "name": "email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "phone", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "subsidiary.id", + "type": "string" + }, + { + "name": "dateCreated", + "type": "datetime", + "tag": "record.createdAt", + "readOnly": true + }, + { + "name": "lastModifiedDate", + "type": "datetime", + "tag": "record.updatedAt", + "readOnly": true + } + ] + }, + "salesOrder": { + "label": "Sales order", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "tranId", + "type": "string", + "tag": "record.displayId", + "readOnly": true + }, + { + "name": "externalId", + "type": "string", + "tag": "record.externalId" + }, + { + "name": "otherRefNum", + "type": "string", + "tag": "order.number" + }, + { + "name": "entity.id", + "type": "string" + }, + { + "name": "subsidiary.id", + "type": "string" + }, + { + "name": "location.id", + "type": "string" + }, + { + "name": "tranDate", + "type": "date", + "tag": "order.createdAt" + }, + { + "name": "memo", + "type": "text" + }, + { + "name": "email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "currency.refName", + "type": "string", + "tag": "order.currency" + }, + { + "name": "subtotal", + "type": "number", + "tag": "order.subtotal", + "readOnly": true + }, + { + "name": "taxTotal", + "type": "number", + "tag": "order.tax", + "readOnly": true + }, + { + "name": "total", + "type": "number", + "tag": "order.total", + "readOnly": true + }, + { + "name": "orderStatus", + "type": "enum", + "tag": "order.status", + "values": [ + "A", + "B", + "C", + "D", + "E", + "F", + "G", + "H" + ] + }, + { + "name": "item.items", + "type": "lineItem[]", + "tag": "order.lineItems" + } + ] + }, + "itemFulfillment": { + "label": "Item fulfillment", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "tranId", + "type": "string", + "tag": "record.displayId", + "readOnly": true + }, + { + "name": "createdFrom.id", + "type": "string" + }, + { + "name": "tranDate", + "type": "date" + }, + { + "name": "shipStatus", + "type": "enum", + "values": [ + "_picked", + "_packed", + "_shipped" + ] + }, + { + "name": "package.packages", + "type": "json", + "tag": "shipment.packages" + }, + { + "name": "memo", + "type": "text" + } + ] + }, + "creditMemo": { + "label": "Credit memo", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "tranId", + "type": "string", + "tag": "record.displayId", + "readOnly": true + }, + { + "name": "externalId", + "type": "string", + "tag": "record.externalId" + }, + { + "name": "entity.id", + "type": "string" + }, + { + "name": "createdFrom.id", + "type": "string" + }, + { + "name": "tranDate", + "type": "date" + }, + { + "name": "memo", + "type": "text" + }, + { + "name": "total", + "type": "number", + "readOnly": true + }, + { + "name": "item.items", + "type": "lineItem[]" + } + ] + } + }, + "events": { + "itemFulfillment.created": { + "delivery": "poll", + "object": "itemFulfillment", + "description": "Polled with SuiteQL on transaction.lastModifiedDate. NetSuite has no outbound webhooks without a deployed SuiteScript.", + "payload": { + "id": "88120", + "tranId": "IF-10391", + "createdFrom": { + "id": "87904", + "refName": "Sales Order #SO-20455" + }, + "tranDate": "2026-10-19", + "shipStatus": "_shipped", + "package": { + "packages": [ + { + "packageTrackingNumber": "1Z999AA10123456784", + "packageWeight": 2.1, + "packageDescr": "Box 1 of 1" + } + ] + }, + "lastModifiedDate": "2026-10-19T19:02:11Z" + } + }, + "customer.updated": { + "delivery": "poll", + "object": "customer", + "description": "Polled with SuiteQL on customer.lastModifiedDate.", + "payload": { + "id": "40211", + "entityId": "CUST-40211", + "email": "dana.reyes@example.com", + "firstName": "Dana", + "lastName": "Reyes", + "companyName": "Reyes Coffee Co", + "lastModifiedDate": "2026-10-18T15:00:00Z" + } + } + }, + "capabilities": { + "customer.get": { + "kind": "read", + "method": "getCustomer", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "customer" + }, + "customer.search": { + "kind": "search", + "method": "searchCustomers", + "input": { + "email": { + "type": "string" + }, + "externalId": { + "type": "string" + } + }, + "output": "customer[]" + }, + "customer.list": { + "kind": "read", + "method": "listCustomers", + "input": { + "lastModifiedAfter": { + "type": "string" + }, + "limit": { + "type": "number" + }, + "offset": { + "type": "number" + } + }, + "output": "page" + }, + "customer.create": { + "kind": "write", + "method": "createCustomer", + "input": { + "record": { + "type": "object", + "required": true + } + }, + "output": "customer" + }, + "customer.update": { + "kind": "write", + "method": "updateCustomer", + "input": { + "id": { + "type": "string", + "required": true + }, + "record": { + "type": "object", + "required": true + } + }, + "output": "customer" + }, + "salesOrder.create": { + "kind": "write", + "method": "createSalesOrder", + "input": { + "record": { + "type": "object", + "required": true + } + }, + "output": "salesOrder" + }, + "salesOrder.get": { + "kind": "read", + "method": "getSalesOrder", + "input": { + "id": { + "type": "string", + "required": true + }, + "expandSubResources": { + "type": "boolean" + } + }, + "output": "salesOrder" + }, + "salesOrder.update": { + "kind": "write", + "method": "updateSalesOrder", + "input": { + "id": { + "type": "string", + "required": true + }, + "record": { + "type": "object", + "required": true + } + }, + "output": "salesOrder" + }, + "salesOrder.search": { + "kind": "search", + "method": "searchSalesOrders", + "input": { + "otherRefNum": { + "type": "string" + }, + "externalId": { + "type": "string" + } + }, + "output": "salesOrder[]" + }, + "creditMemo.create": { + "kind": "write", + "method": "createCreditMemo", + "input": { + "record": { + "type": "object", + "required": true + } + }, + "output": "creditMemo" + }, + "itemFulfillment.get": { + "kind": "read", + "method": "getItemFulfillment", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "itemFulfillment" + }, + "itemFulfillment.list": { + "kind": "read", + "method": "listItemFulfillments", + "input": { + "modifiedAfter": { + "type": "string" + }, + "limit": { + "type": "number" + } + }, + "output": "itemFulfillment[]" + }, + "item.search": { + "kind": "search", + "method": "searchItems", + "input": { + "sku": { + "type": "string", + "required": true + } + }, + "output": "item[]" + }, + "subsidiary.list": { + "kind": "read", + "method": "listSubsidiaries", + "input": {}, + "output": "subsidiary[]" + }, + "location.list": { + "kind": "read", + "method": "listLocations", + "input": {}, + "output": "location[]" + } + }, + "options": { + "subsidiary": { + "capability": "subsidiary.list", + "label": "{{item.name}}", + "value": "$.item.id" + }, + "location": { + "capability": "location.list", + "label": "{{item.name}}", + "value": "$.item.id" + } + } +} \ No newline at end of file diff --git a/packages/core/integrations/data-defined/capabilities/notion.capabilities.json b/packages/core/integrations/data-defined/capabilities/notion.capabilities.json new file mode 100644 index 000000000..49d6be2a6 --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/notion.capabilities.json @@ -0,0 +1,208 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "notion", + "label": "Notion", + "package": "@friggframework/api-module-notion", + "version": "1.3.0", + "auth": "oauth2", + "objects": { + "page": { + "label": "Page", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "parent.database_id", + "type": "string" + }, + { + "name": "properties", + "type": "json" + }, + { + "name": "url", + "type": "string", + "tag": "record.url", + "readOnly": true + }, + { + "name": "created_time", + "type": "datetime", + "tag": "record.createdAt", + "readOnly": true + }, + { + "name": "last_edited_time", + "type": "datetime", + "tag": "record.updatedAt", + "readOnly": true + }, + { + "name": "archived", + "type": "boolean" + } + ] + }, + "database": { + "label": "Database", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "title", + "type": "richText" + }, + { + "name": "properties", + "type": "json", + "readOnly": true + }, + { + "name": "url", + "type": "string", + "readOnly": true + } + ] + } + }, + "events": { + "page.created": { + "delivery": "webhook", + "object": "page", + "description": "Integration webhook (configured once on the public integration); routed by workspace_id.", + "payload": { + "id": "7f2c3b8a-91c5-4b1d-8e0f-2a6f1d3c9e41", + "timestamp": "2026-10-20T15:50:02.000Z", + "workspace_id": "13950b26-c203-4f3b-b97d-93ec06319565", + "subscription_id": "29d75c0d-5546-4414-8459-7b7a92f1fc4b", + "integration_id": "0ef104cd-477e-4c15-a1e4-5c1bd3dbd0a0", + "type": "page.created", + "entity": { + "id": "153104cd-477e-809d-8dc4-ff2d96ae3090", + "type": "page" + }, + "data": { + "parent": { + "id": "13950b26-c203-4f3b-b97d-93ec06319565", + "type": "database" + } + } + }, + "record": "entity" + } + }, + "webhook": { + "scope": "app", + "topic": "{{body.type}}", + "eventId": "{{body.id}}", + "account": "{{body.workspace_id}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "page.create": { + "kind": "write", + "method": "createPage", + "input": { + "parent": { + "type": "object", + "required": true + }, + "properties": { + "type": "object", + "required": true + }, + "children": { + "type": "array" + }, + "icon": { + "type": "object" + } + }, + "output": "page" + }, + "page.get": { + "kind": "read", + "method": "retrievePage", + "input": { + "pageId": { + "type": "string", + "required": true + } + }, + "output": "page" + }, + "page.update": { + "kind": "write", + "method": "updatePage", + "input": { + "pageId": { + "type": "string", + "required": true + }, + "properties": { + "type": "object", + "required": true + } + }, + "output": "page" + }, + "block.append": { + "kind": "write", + "method": "appendBlockChildren", + "input": { + "blockId": { + "type": "string", + "required": true + }, + "children": { + "type": "array", + "required": true + } + }, + "output": "block[]" + }, + "database.query": { + "kind": "search", + "method": "queryDatabase", + "input": { + "databaseId": { + "type": "string", + "required": true + }, + "filter": { + "type": "object" + }, + "sorts": { + "type": "array" + }, + "pageSize": { + "type": "number" + } + }, + "output": "page[]" + }, + "database.list": { + "kind": "read", + "method": "searchDatabases", + "input": { + "query": { + "type": "string" + } + }, + "output": "database[]" + } + }, + "options": { + "database": { + "capability": "database.list", + "label": "{{item.title[0].plain_text}}", + "value": "$.item.id" + } + } +} diff --git a/packages/core/integrations/data-defined/capabilities/phone.capabilities.json b/packages/core/integrations/data-defined/capabilities/phone.capabilities.json new file mode 100644 index 000000000..958699dc2 --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/phone.capabilities.json @@ -0,0 +1,470 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "phone", + "label": "Phone", + "package": "@friggframework/api-module-phone", + "version": "2.1.0", + "auth": "api_key", + "objects": { + "contact": { + "label": "Contact", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "defaultFields.firstName", + "type": "string", + "tag": "person.givenName" + }, + { + "name": "defaultFields.lastName", + "type": "string", + "tag": "person.familyName" + }, + { + "name": "defaultFields.company", + "type": "string", + "tag": "organization.name" + }, + { + "name": "defaultFields.emails[0].value", + "type": "email", + "tag": "contact.email" + }, + { + "name": "defaultFields.phoneNumbers[0].value", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "externalId", + "type": "string", + "tag": "record.externalId" + }, + { + "name": "source", + "type": "string" + } + ] + }, + "call": { + "label": "Call", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "direction", + "type": "enum", + "values": [ + "incoming", + "outgoing" + ] + }, + { + "name": "status", + "type": "enum", + "values": [ + "completed", + "missed", + "voicemail", + "busy" + ] + }, + { + "name": "participants", + "type": "phone[]", + "tag": "contact.phone" + }, + { + "name": "duration", + "type": "number" + }, + { + "name": "createdAt", + "type": "datetime", + "readOnly": true + }, + { + "name": "answeredAt", + "type": "datetime" + }, + { + "name": "userId", + "type": "string", + "tag": "record.owner" + } + ] + }, + "message": { + "label": "Text message", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "from", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "to", + "type": "phone[]", + "tag": "contact.phone" + }, + { + "name": "text", + "type": "text" + }, + { + "name": "direction", + "type": "enum", + "values": [ + "incoming", + "outgoing" + ] + }, + { + "name": "createdAt", + "type": "datetime" + } + ] + }, + "summary": { + "label": "Call summary", + "fields": [ + { + "name": "callId", + "type": "id" + }, + { + "name": "summary", + "type": "text[]" + }, + { + "name": "nextSteps", + "type": "text[]" + } + ] + } + }, + "events": { + "call.completed": { + "delivery": "webhook", + "object": "call", + "subscribe": "webhook.create", + "payload": { + "id": "EVc0a1", + "type": "call.completed", + "data": { + "object": { + "id": "AC9x", + "direction": "incoming", + "status": "completed", + "from": "+15125550142", + "to": "+15125550100", + "participants": [ + "+15125550142" + ], + "duration": 312, + "userId": "US1", + "answeredAt": "2026-10-18T14:02:00Z", + "completedAt": "2026-10-18T14:07:12Z" + } + } + }, + "record": "data.object" + }, + "call.missed": { + "delivery": "webhook", + "object": "call", + "subscribe": "webhook.create", + "payload": { + "id": "EVc0a2", + "type": "call.completed", + "data": { + "object": { + "id": "AC9y", + "direction": "incoming", + "status": "missed", + "from": "+15125550142", + "participants": [ + "+15125550142" + ], + "userId": "US1", + "createdAt": "2026-10-18T15:00:00Z" + } + } + }, + "record": "data.object", + "match": "call.completed", + "when": "$.record.status == 'missed' || $.record.status == 'voicemail'", + "description": "Phone reports a missed call as call.completed with status missed (or voicemail); this event is that subset." + }, + "call.summary.completed": { + "delivery": "webhook", + "object": "summary", + "subscribe": "webhook.create", + "payload": { + "id": "EVs01", + "type": "call.summary.completed", + "data": { + "object": { + "callId": "AC9x", + "status": "completed", + "summary": [ + "Asked about renewal pricing" + ], + "nextSteps": [ + "Send the renewal quote by Friday" + ] + } + } + }, + "record": "data.object" + }, + "message.received": { + "delivery": "webhook", + "object": "message", + "subscribe": "webhook.create", + "payload": { + "id": "EVm01", + "type": "message.received", + "data": { + "object": { + "id": "ACm1", + "from": "+15125550142", + "to": [ + "+15125550100" + ], + "text": "Can we move tomorrow to 3pm?", + "direction": "incoming", + "createdAt": "2026-10-18T16:00:00Z" + } + } + }, + "record": "data.object" + }, + "contact.updated": { + "delivery": "webhook", + "object": "contact", + "subscribe": "webhook.create", + "payload": { + "id": "EVk01", + "type": "contact.updated", + "data": { + "object": { + "id": "CT7", + "defaultFields": { + "firstName": "Dana", + "phoneNumbers": [ + { + "value": "+15125550142" + } + ] + } + } + } + }, + "record": "data.object" + } + }, + "webhook": { + "scope": "installation", + "topic": "{{body.type}}", + "eventId": "{{body.id}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "contact.get": { + "kind": "read", + "method": "getContact", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "contact" + }, + "contact.search": { + "kind": "search", + "method": "searchContacts", + "input": { + "phone": { + "type": "string" + }, + "email": { + "type": "string" + } + }, + "output": "contact[]" + }, + "contact.list": { + "kind": "read", + "method": "listContacts", + "input": { + "pageToken": { + "type": "string" + }, + "maxResults": { + "type": "number" + } + }, + "output": "page" + }, + "contact.create": { + "kind": "write", + "method": "createContact", + "input": { + "defaultFields": { + "type": "object", + "required": true + }, + "externalId": { + "type": "string" + }, + "source": { + "type": "string" + } + }, + "output": "contact" + }, + "contact.update": { + "kind": "write", + "method": "updateContact", + "input": { + "id": { + "type": "string", + "required": true + }, + "defaultFields": { + "type": "object" + } + }, + "output": "contact" + }, + "call.list": { + "kind": "read", + "method": "listCalls", + "input": { + "participant": { + "type": "string", + "required": true + }, + "maxResults": { + "type": "number" + } + }, + "output": "call[]" + }, + "call.recording": { + "kind": "read", + "method": "getCallRecordings", + "input": { + "callId": { + "type": "string", + "required": true + } + }, + "output": "recording[]" + }, + "message.send": { + "kind": "write", + "method": "sendMessage", + "input": { + "to": { + "type": "array", + "required": true + }, + "from": { + "type": "string", + "required": true + }, + "content": { + "type": "string", + "required": true + } + }, + "output": "message" + }, + "message.list": { + "kind": "read", + "method": "listMessages", + "input": { + "participants": { + "type": "array", + "required": true + }, + "maxResults": { + "type": "number" + } + }, + "output": "message[]" + }, + "phoneNumber.list": { + "kind": "read", + "method": "listPhoneNumbers", + "input": {}, + "output": "phoneNumber[]" + }, + "user.list": { + "kind": "read", + "method": "listUsers", + "input": {}, + "output": "user[]" + }, + "webhook.create": { + "kind": "subscribe", + "method": "createWebhook", + "input": { + "url": { + "type": "string", + "required": true + }, + "events": { + "type": "array", + "required": true + } + }, + "output": "webhook" + }, + "webhook.delete": { + "kind": "delete", + "method": "deleteWebhook", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "void" + } + }, + "options": { + "phoneNumber": { + "capability": "phoneNumber.list", + "label": "{{item.name}} {{item.number}}", + "value": "$.item.number" + }, + "user": { + "capability": "user.list", + "label": "{{item.firstName}} {{item.lastName}}", + "value": "$.item.id" + } + }, + "ui": { + "placements": [ + "conversation.sidebar", + "contact.panel" + ], + "renderer": "phone-integrations-panel" + } +} diff --git a/packages/core/integrations/data-defined/capabilities/pipedrive.capabilities.json b/packages/core/integrations/data-defined/capabilities/pipedrive.capabilities.json new file mode 100644 index 000000000..25ac96150 --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/pipedrive.capabilities.json @@ -0,0 +1,388 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "pipedrive", + "label": "Pipedrive", + "package": "@friggframework/api-module-pipedrive", + "version": "2.1.0", + "auth": "oauth2", + "objects": { + "person": { + "label": "Person", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "name", + "type": "string", + "tag": "person.fullName" + }, + { + "name": "first_name", + "type": "string", + "tag": "person.givenName" + }, + { + "name": "last_name", + "type": "string", + "tag": "person.familyName" + }, + { + "name": "emails[0].value", + "type": "email", + "tag": "contact.email" + }, + { + "name": "phones[0].value", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "org_id", + "type": "string" + }, + { + "name": "owner_id", + "type": "string", + "tag": "record.owner" + }, + { + "name": "add_time", + "type": "datetime", + "tag": "record.createdAt", + "readOnly": true + }, + { + "name": "update_time", + "type": "datetime", + "tag": "record.updatedAt", + "readOnly": true + } + ] + }, + "activity": { + "label": "Activity", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "subject", + "type": "string", + "required": true + }, + { + "name": "type", + "type": "string" + }, + { + "name": "done", + "type": "boolean" + }, + { + "name": "due_date", + "type": "date" + }, + { + "name": "due_time", + "type": "string" + }, + { + "name": "duration", + "type": "string" + }, + { + "name": "person_id", + "type": "string" + }, + { + "name": "deal_id", + "type": "string" + }, + { + "name": "owner_id", + "type": "string", + "tag": "record.owner" + }, + { + "name": "note", + "type": "text" + } + ] + } + }, + "events": { + "person.created": { + "delivery": "webhook", + "object": "person", + "subscribe": "webhook.create", + "payload": { + "meta": { + "action": "create", + "entity": "person", + "entity_id": "1842", + "id": "9a5e6a4f-0a1b-4c55-9d1b-6f4b2a1e8c33", + "company_id": "7521439", + "user_id": "13293848", + "timestamp": "2026-10-18T16:01:00.412Z", + "type": "general", + "version": "2.0" + }, + "data": { + "id": 1842, + "name": "Dana Reyes", + "first_name": "Dana", + "last_name": "Reyes", + "emails": [ + { + "value": "dana.reyes@example.com", + "primary": true, + "label": "work" + } + ], + "phones": [ + { + "value": "+15125550142", + "primary": true, + "label": "mobile" + } + ], + "org_id": 311, + "owner_id": 13293848, + "add_time": "2026-08-02T15:10:00Z", + "update_time": "2026-10-18T16:01:00Z" + }, + "previous": null + }, + "record": "data", + "idPath": "meta.id", + "match": "person.create" + }, + "person.updated": { + "delivery": "webhook", + "object": "person", + "subscribe": "webhook.create", + "payload": { + "meta": { + "action": "change", + "entity": "person", + "entity_id": "1842", + "id": "9a5e6a4f-0a1b-4c55-9d1b-6f4b2a1e8c33", + "company_id": "7521439", + "user_id": "13293848", + "timestamp": "2026-10-18T16:01:00.412Z", + "type": "general", + "version": "2.0" + }, + "data": { + "id": 1842, + "name": "Dana Reyes", + "first_name": "Dana", + "last_name": "Reyes", + "emails": [ + { + "value": "dana.reyes@example.com", + "primary": true, + "label": "work" + } + ], + "phones": [ + { + "value": "+15125550199", + "primary": true, + "label": "mobile" + } + ], + "org_id": 311, + "owner_id": 13293848, + "add_time": "2026-08-02T15:10:00Z", + "update_time": "2026-10-18T16:01:00Z" + }, + "previous": null + }, + "record": "data", + "idPath": "meta.id", + "match": "person.change" + } + }, + "webhook": { + "scope": "installation", + "topic": "{{body.meta.entity}}.{{body.meta.action}}", + "eventId": [ + "{{body.meta.id}}", + "{{body.meta.entity}}:{{body.data.id}}:{{body.meta.timestamp}}" + ], + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "person.get": { + "kind": "read", + "method": "getPerson", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "person" + }, + "person.search": { + "kind": "search", + "method": "searchPersons", + "input": { + "term": { + "type": "string", + "required": true + }, + "fields": { + "type": "string" + }, + "exact_match": { + "type": "boolean" + } + }, + "output": "person[]" + }, + "person.list": { + "kind": "read", + "method": "listPersons", + "input": { + "cursor": { + "type": "string" + }, + "limit": { + "type": "number" + }, + "updated_since": { + "type": "string" + } + }, + "output": "page" + }, + "person.create": { + "kind": "write", + "method": "addPerson", + "input": { + "person": { + "type": "object", + "required": true + } + }, + "output": "person" + }, + "person.update": { + "kind": "write", + "method": "updatePerson", + "input": { + "id": { + "type": "string", + "required": true + }, + "person": { + "type": "object", + "required": true + } + }, + "output": "person" + }, + "activity.create": { + "kind": "write", + "method": "addActivity", + "input": { + "activity": { + "type": "object", + "required": true + } + }, + "output": "activity" + }, + "activity.list": { + "kind": "read", + "method": "listActivities", + "input": { + "person_id": { + "type": "string" + }, + "done": { + "type": "boolean" + }, + "limit": { + "type": "number" + } + }, + "output": "activity[]" + }, + "deal.listForPerson": { + "kind": "read", + "method": "listPersonDeals", + "input": { + "personId": { + "type": "string", + "required": true + }, + "status": { + "type": "string" + } + }, + "output": "deal[]" + }, + "user.list": { + "kind": "read", + "method": "getUsers", + "input": {}, + "output": "user[]" + }, + "webhook.create": { + "kind": "subscribe", + "method": "addWebhook", + "input": { + "subscription_url": { + "type": "string", + "required": true + }, + "event_action": { + "type": "string", + "required": true + }, + "event_object": { + "type": "string", + "required": true + }, + "version": { + "type": "string" + } + }, + "output": "webhook" + }, + "webhook.delete": { + "kind": "delete", + "method": "deleteWebhook", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "void" + } + }, + "options": { + "user": { + "capability": "user.list", + "label": "{{item.name}}", + "value": "$.item.id" + } + }, + "ui": { + "placements": [ + "person.details.panel", + "deal.details.panel", + "organization.details.panel" + ], + "renderer": "frigg-pipedrive-app-extension" + } +} diff --git a/packages/core/integrations/data-defined/capabilities/quickbooks.capabilities.json b/packages/core/integrations/data-defined/capabilities/quickbooks.capabilities.json new file mode 100644 index 000000000..6b77f320e --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/quickbooks.capabilities.json @@ -0,0 +1,615 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "quickbooks", + "label": "QuickBooks Online", + "package": "@friggframework/api-module-quickbooks", + "version": "2.0.1", + "auth": "oauth2", + "account": { + "description": "Values about the connected account, persisted by the module (apiPropertiesToPersist). Definitions read them as $.install.accounts...", + "fields": [ + { + "name": "realmId", + "type": "string", + "description": "QuickBooks company id the connection was authorized for" + } + ] + }, + "objects": { + "customer": { + "label": "Customer", + "fields": [ + { + "name": "Id", + "type": "id", + "readOnly": true + }, + { + "name": "SyncToken", + "type": "string", + "readOnly": true + }, + { + "name": "DisplayName", + "type": "string", + "tag": "record.displayName", + "required": true + }, + { + "name": "GivenName", + "type": "string", + "tag": "person.givenName" + }, + { + "name": "FamilyName", + "type": "string", + "tag": "person.familyName" + }, + { + "name": "CompanyName", + "type": "string", + "tag": "organization.name" + }, + { + "name": "PrimaryEmailAddr.Address", + "type": "email", + "tag": "contact.email" + }, + { + "name": "PrimaryPhone.FreeFormNumber", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "Mobile.FreeFormNumber", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "BillAddr.Line1", + "type": "string", + "tag": "address.line1" + }, + { + "name": "BillAddr.City", + "type": "string", + "tag": "address.locality" + }, + { + "name": "BillAddr.CountrySubDivisionCode", + "type": "string", + "tag": "address.region" + }, + { + "name": "BillAddr.PostalCode", + "type": "string", + "tag": "address.postalCode" + }, + { + "name": "BillAddr.Country", + "type": "string", + "tag": "address.country" + }, + { + "name": "Notes", + "type": "text" + }, + { + "name": "Active", + "type": "boolean" + }, + { + "name": "Balance", + "type": "number", + "tag": "customer.balance", + "readOnly": true + }, + { + "name": "MetaData.LastUpdatedTime", + "type": "datetime", + "tag": "record.updatedAt", + "readOnly": true + } + ] + }, + "salesReceipt": { + "label": "Sales receipt", + "fields": [ + { + "name": "Id", + "type": "id", + "readOnly": true + }, + { + "name": "DocNumber", + "type": "string", + "tag": "record.displayId" + }, + { + "name": "TxnDate", + "type": "date" + }, + { + "name": "CustomerRef.value", + "type": "string" + }, + { + "name": "DepositToAccountRef.value", + "type": "string" + }, + { + "name": "PaymentRefNum", + "type": "string", + "tag": "record.externalId" + }, + { + "name": "PrivateNote", + "type": "text" + }, + { + "name": "BillEmail.Address", + "type": "email", + "tag": "contact.email" + }, + { + "name": "CurrencyRef.value", + "type": "string" + }, + { + "name": "Line", + "type": "lineItem[]" + }, + { + "name": "Line[0].Amount", + "type": "number" + }, + { + "name": "Line[0].Description", + "type": "string" + }, + { + "name": "Line[0].DetailType", + "type": "string" + }, + { + "name": "Line[0].SalesItemLineDetail.ItemRef.value", + "type": "string" + }, + { + "name": "TotalAmt", + "type": "number", + "readOnly": true + } + ] + }, + "refundReceipt": { + "label": "Refund receipt", + "fields": [ + { + "name": "Id", + "type": "id", + "readOnly": true + }, + { + "name": "DocNumber", + "type": "string" + }, + { + "name": "TxnDate", + "type": "date" + }, + { + "name": "CustomerRef.value", + "type": "string" + }, + { + "name": "DepositToAccountRef.value", + "type": "string" + }, + { + "name": "PaymentRefNum", + "type": "string" + }, + { + "name": "PrivateNote", + "type": "text" + }, + { + "name": "Line", + "type": "lineItem[]" + }, + { + "name": "TotalAmt", + "type": "number", + "readOnly": true + } + ] + }, + "invoice": { + "label": "Invoice", + "fields": [ + { + "name": "Id", + "type": "id", + "readOnly": true + }, + { + "name": "DocNumber", + "type": "string", + "tag": "record.displayId" + }, + { + "name": "TxnDate", + "type": "date" + }, + { + "name": "DueDate", + "type": "date" + }, + { + "name": "CustomerRef.value", + "type": "string" + }, + { + "name": "BillEmail.Address", + "type": "email", + "tag": "contact.email" + }, + { + "name": "CustomerMemo.value", + "type": "text" + }, + { + "name": "PrivateNote", + "type": "text" + }, + { + "name": "Line", + "type": "lineItem[]" + }, + { + "name": "TotalAmt", + "type": "number", + "tag": "invoice.total", + "readOnly": true + }, + { + "name": "Balance", + "type": "number", + "tag": "invoice.balance", + "readOnly": true + }, + { + "name": "EmailStatus", + "type": "enum", + "values": [ + "NotSet", + "NeedToSend", + "EmailSent" + ] + } + ] + }, + "payment": { + "label": "Payment", + "fields": [ + { + "name": "Id", + "type": "id", + "readOnly": true + }, + { + "name": "TxnDate", + "type": "date" + }, + { + "name": "TotalAmt", + "type": "number" + }, + { + "name": "CustomerRef.value", + "type": "string" + }, + { + "name": "PaymentRefNum", + "type": "string" + }, + { + "name": "Line", + "type": "json" + } + ] + }, + "deposit": { + "label": "Deposit", + "fields": [ + { + "name": "Id", + "type": "id", + "readOnly": true + }, + { + "name": "TxnDate", + "type": "date" + }, + { + "name": "DepositToAccountRef.value", + "type": "string" + }, + { + "name": "PrivateNote", + "type": "text" + }, + { + "name": "Line", + "type": "json" + }, + { + "name": "TotalAmt", + "type": "number", + "readOnly": true + } + ] + } + }, + "events": { + "payment.created": { + "delivery": "webhook", + "object": "payment", + "description": "Intuit webhooks are app-level and carry only entity references (name, id, operation); Frigg routes by realmId. Fetch the entity to read it.", + "payload": { + "eventNotifications": [ + { + "realmId": "9130354628571476", + "dataChangeEvent": { + "entities": [ + { + "name": "Payment", + "id": "1871", + "operation": "Create", + "lastUpdated": "2026-10-20T18:22:41.000Z" + } + ] + } + } + ] + }, + "match": "Payment.Create" + }, + "customer.updated": { + "delivery": "webhook", + "object": "customer", + "payload": { + "eventNotifications": [ + { + "realmId": "9130354628571476", + "dataChangeEvent": { + "entities": [ + { + "name": "Customer", + "id": "58", + "operation": "Update", + "lastUpdated": "2026-10-20T18:22:41.000Z" + } + ] + } + } + ] + }, + "match": "Customer.Update" + }, + "customer.created": { + "delivery": "webhook", + "object": "customer", + "payload": { + "eventNotifications": [ + { + "realmId": "9130354628571476", + "dataChangeEvent": { + "entities": [ + { + "name": "Customer", + "id": "59", + "operation": "Create", + "lastUpdated": "2026-10-20T18:22:41.000Z" + } + ] + } + } + ] + }, + "match": "Customer.Create" + } + }, + "webhook": { + "scope": "app", + "batch": [ + "body.eventNotifications", + "dataChangeEvent.entities" + ], + "topic": "{{item.name}}.{{item.operation}}", + "eventId": "{{parent.realmId}}:{{item.name}}:{{item.id}}:{{item.lastUpdated}}", + "account": "{{parent.realmId}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "customer.get": { + "kind": "read", + "method": "getCustomer", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "customer" + }, + "customer.query": { + "kind": "search", + "method": "queryCustomers", + "input": { + "where": { + "type": "string", + "required": true + } + }, + "output": "customer[]" + }, + "customer.list": { + "kind": "read", + "method": "listCustomers", + "input": { + "changedSince": { + "type": "string" + }, + "startPosition": { + "type": "number" + }, + "maxResults": { + "type": "number" + } + }, + "output": "page" + }, + "customer.create": { + "kind": "write", + "method": "createCustomer", + "input": { + "customer": { + "type": "object", + "required": true + } + }, + "output": "customer" + }, + "customer.update": { + "kind": "write", + "method": "updateCustomer", + "label": "Sparse update; needs Id and SyncToken", + "input": { + "customer": { + "type": "object", + "required": true + } + }, + "output": "customer" + }, + "salesReceipt.create": { + "kind": "write", + "method": "createSalesReceipt", + "input": { + "salesReceipt": { + "type": "object", + "required": true + } + }, + "output": "salesReceipt" + }, + "refundReceipt.create": { + "kind": "write", + "method": "createRefundReceipt", + "input": { + "refundReceipt": { + "type": "object", + "required": true + } + }, + "output": "refundReceipt" + }, + "invoice.create": { + "kind": "write", + "method": "createInvoice", + "input": { + "invoice": { + "type": "object", + "required": true + } + }, + "output": "invoice" + }, + "invoice.get": { + "kind": "read", + "method": "getInvoice", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "invoice" + }, + "invoice.send": { + "kind": "write", + "method": "sendInvoice", + "input": { + "id": { + "type": "string", + "required": true + }, + "sendTo": { + "type": "string" + } + }, + "output": "invoice" + }, + "payment.get": { + "kind": "read", + "method": "getPayment", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "payment" + }, + "deposit.create": { + "kind": "write", + "method": "createDeposit", + "input": { + "deposit": { + "type": "object", + "required": true + } + }, + "output": "deposit" + }, + "item.list": { + "kind": "read", + "method": "listItems", + "input": { + "type": { + "type": "string" + } + }, + "output": "item[]" + }, + "account.list": { + "kind": "read", + "method": "listAccounts", + "input": { + "accountType": { + "type": "string" + } + }, + "output": "account[]" + } + }, + "options": { + "item": { + "capability": "item.list", + "label": "{{item.Name}}", + "value": "$.item.Id" + }, + "account": { + "capability": "account.list", + "label": "{{item.Name}} ({{item.AccountType}})", + "value": "$.item.Id" + }, + "customer": { + "capability": "customer.list", + "label": "{{item.DisplayName}}", + "value": "$.item.Id" + } + } +} diff --git a/packages/core/integrations/data-defined/capabilities/salesforce.capabilities.json b/packages/core/integrations/data-defined/capabilities/salesforce.capabilities.json new file mode 100644 index 000000000..c66d9094d --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/salesforce.capabilities.json @@ -0,0 +1,507 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "salesforce", + "label": "Salesforce", + "package": "@friggframework/api-module-salesforce", + "version": "3.0.2", + "auth": "oauth2", + "objects": { + "lead": { + "label": "Lead", + "fields": [ + { + "name": "Id", + "type": "id", + "readOnly": true + }, + { + "name": "FirstName", + "type": "string", + "tag": "person.givenName" + }, + { + "name": "LastName", + "type": "string", + "tag": "person.familyName", + "required": true + }, + { + "name": "Email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "Phone", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "Company", + "type": "string", + "tag": "organization.name", + "required": true + }, + { + "name": "Title", + "type": "string", + "tag": "person.jobTitle" + }, + { + "name": "LeadSource", + "type": "string", + "tag": "marketing.source" + }, + { + "name": "Status", + "type": "enum", + "tag": "contact.leadStatus", + "values": [ + "Open - Not Contacted", + "Working - Contacted", + "Closed - Converted", + "Closed - Not Converted" + ] + }, + { + "name": "OwnerId", + "type": "string", + "tag": "record.owner" + }, + { + "name": "Description", + "type": "text" + }, + { + "name": "IsConverted", + "type": "boolean", + "readOnly": true + } + ] + }, + "contact": { + "label": "Contact", + "fields": [ + { + "name": "Id", + "type": "id", + "readOnly": true + }, + { + "name": "FirstName", + "type": "string", + "tag": "person.givenName" + }, + { + "name": "LastName", + "type": "string", + "tag": "person.familyName", + "required": true + }, + { + "name": "Email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "Phone", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "AccountId", + "type": "string" + }, + { + "name": "OwnerId", + "type": "string", + "tag": "record.owner" + } + ] + }, + "event": { + "label": "Event", + "fields": [ + { + "name": "Id", + "type": "id", + "readOnly": true + }, + { + "name": "Subject", + "type": "string", + "tag": "meeting.title" + }, + { + "name": "StartDateTime", + "type": "datetime", + "tag": "meeting.start" + }, + { + "name": "EndDateTime", + "type": "datetime", + "tag": "meeting.end" + }, + { + "name": "Location", + "type": "string", + "tag": "meeting.joinUrl" + }, + { + "name": "Description", + "type": "text" + }, + { + "name": "WhoId", + "type": "string" + }, + { + "name": "WhatId", + "type": "string" + }, + { + "name": "OwnerId", + "type": "string", + "tag": "record.owner" + }, + { + "name": "ShowAs", + "type": "enum", + "values": [ + "Busy", + "OutOfOffice", + "Free" + ] + }, + { + "name": "Calendly_Event_URI__c", + "type": "string", + "tag": "record.externalId", + "custom": true + } + ] + }, + "opportunity": { + "label": "Opportunity", + "fields": [ + { + "name": "Id", + "type": "id", + "readOnly": true + }, + { + "name": "Name", + "type": "string", + "tag": "deal.name" + }, + { + "name": "StageName", + "type": "enum", + "tag": "deal.stage", + "values": [ + "Prospecting", + "Qualification", + "Needs Analysis", + "Value Proposition", + "Proposal/Price Quote", + "Negotiation/Review", + "Closed Won", + "Closed Lost" + ] + }, + { + "name": "Amount", + "type": "number", + "tag": "deal.amount" + }, + { + "name": "CloseDate", + "type": "date", + "tag": "deal.closeDate" + }, + { + "name": "NextStep", + "type": "string" + }, + { + "name": "AccountId", + "type": "string" + }, + { + "name": "OwnerId", + "type": "string", + "tag": "record.owner" + } + ] + }, + "task": { + "label": "Task", + "fields": [ + { + "name": "Id", + "type": "id", + "readOnly": true + }, + { + "name": "Subject", + "type": "string", + "required": true + }, + { + "name": "Status", + "type": "enum", + "values": [ + "Not Started", + "In Progress", + "Completed", + "Waiting on someone else", + "Deferred" + ] + }, + { + "name": "Priority", + "type": "enum", + "values": [ + "High", + "Normal", + "Low" + ] + }, + { + "name": "ActivityDate", + "type": "date" + }, + { + "name": "WhoId", + "type": "string" + }, + { + "name": "WhatId", + "type": "string" + }, + { + "name": "OwnerId", + "type": "string", + "tag": "record.owner" + }, + { + "name": "Description", + "type": "text" + } + ] + } + }, + "events": { + "opportunity.changed": { + "delivery": "webhook", + "object": "opportunity", + "description": "Change Data Capture (OpportunityChangeEvent) relayed through the Pub/Sub API subscriber; routed by organization id.", + "payload": { + "id": "0WS8c000000abcdGAA", + "schema": "IeRuaY6cbI_HsV8Rv1Mc5g", + "payload": { + "ChangeEventHeader": { + "entityName": "Opportunity", + "recordIds": [ + "0068c00000ABCDeAAP" + ], + "changeType": "UPDATE", + "changedFields": [ + "StageName" + ], + "commitTimestamp": 1760958061000 + }, + "StageName": "Negotiation/Review" + } + }, + "record": "payload", + "match": "Opportunity.UPDATE" + } + }, + "webhook": { + "scope": "installation", + "topic": "{{body.payload.ChangeEventHeader.entityName}}.{{body.payload.ChangeEventHeader.changeType}}", + "eventId": "{{body.id}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "lead.get": { + "kind": "read", + "method": "getLead", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "lead" + }, + "lead.create": { + "kind": "write", + "method": "createLead", + "input": { + "fields": { + "type": "object", + "required": true + } + }, + "output": "lead" + }, + "lead.update": { + "kind": "write", + "method": "updateLead", + "input": { + "id": { + "type": "string", + "required": true + }, + "fields": { + "type": "object", + "required": true + } + }, + "output": "void" + }, + "person.searchByEmail": { + "kind": "search", + "method": "searchPeopleByEmail", + "label": "SOSL across Lead and Contact", + "input": { + "email": { + "type": "string", + "required": true + } + }, + "output": "{ leads: lead[], contacts: contact[] }" + }, + "event.create": { + "kind": "write", + "method": "createEvent", + "input": { + "fields": { + "type": "object", + "required": true + } + }, + "output": "event" + }, + "event.update": { + "kind": "write", + "method": "updateEvent", + "input": { + "id": { + "type": "string", + "required": true + }, + "fields": { + "type": "object", + "required": true + } + }, + "output": "void" + }, + "opportunity.get": { + "kind": "read", + "method": "getOpportunity", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "opportunity" + }, + "opportunity.update": { + "kind": "write", + "method": "updateOpportunity", + "input": { + "id": { + "type": "string", + "required": true + }, + "fields": { + "type": "object", + "required": true + } + }, + "output": "void" + }, + "task.create": { + "kind": "write", + "method": "createTask", + "input": { + "fields": { + "type": "object", + "required": true + } + }, + "output": "task" + }, + "file.attach": { + "kind": "write", + "method": "createContentVersion", + "input": { + "title": { + "type": "string", + "required": true + }, + "pathOnClient": { + "type": "string", + "required": true + }, + "versionData": { + "type": "string", + "required": true + }, + "firstPublishLocationId": { + "type": "string", + "required": true + } + }, + "output": "contentVersion" + }, + "note.create": { + "kind": "write", + "method": "createContentNote", + "input": { + "title": { + "type": "string", + "required": true + }, + "content": { + "type": "string", + "required": true + }, + "linkedEntityId": { + "type": "string", + "required": true + } + }, + "output": "contentNote" + }, + "user.list": { + "kind": "read", + "method": "listUsers", + "input": { + "activeOnly": { + "type": "boolean" + } + }, + "output": "user[]" + } + }, + "options": { + "user": { + "capability": "user.list", + "label": "{{item.Name}}", + "value": "$.item.Id" + } + }, + "ui": { + "placements": [ + "lightning.record.sidebar", + "lightning.record.tab", + "lightning.utility-bar" + ], + "renderer": "frigg-lwc-host" + } +} diff --git a/packages/core/integrations/data-defined/capabilities/shopify.capabilities.json b/packages/core/integrations/data-defined/capabilities/shopify.capabilities.json new file mode 100644 index 000000000..90e5d6e76 --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/shopify.capabilities.json @@ -0,0 +1,1082 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "shopify", + "label": "Shopify", + "package": "@friggframework/api-module-shopify", + "version": "3.1.0", + "auth": "oauth2", + "objects": { + "customer": { + "label": "Customer", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "first_name", + "type": "string", + "tag": "person.givenName" + }, + { + "name": "last_name", + "type": "string", + "tag": "person.familyName" + }, + { + "name": "phone", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "tags", + "type": "string", + "tag": "record.tags" + }, + { + "name": "note", + "type": "text" + }, + { + "name": "email_marketing_consent.state", + "type": "enum", + "tag": "contact.emailConsent", + "values": [ + "subscribed", + "not_subscribed", + "pending", + "unsubscribed", + "redacted", + "invalid" + ] + }, + { + "name": "default_address.company", + "type": "string", + "tag": "organization.name" + }, + { + "name": "default_address.address1", + "type": "string", + "tag": "address.line1" + }, + { + "name": "default_address.city", + "type": "string", + "tag": "address.locality" + }, + { + "name": "default_address.province_code", + "type": "string", + "tag": "address.region" + }, + { + "name": "default_address.zip", + "type": "string", + "tag": "address.postalCode" + }, + { + "name": "default_address.country_code", + "type": "string", + "tag": "address.country" + }, + { + "name": "orders_count", + "type": "number", + "tag": "customer.orderCount", + "readOnly": true + }, + { + "name": "total_spent", + "type": "number", + "tag": "customer.lifetimeValue", + "readOnly": true + }, + { + "name": "created_at", + "type": "datetime", + "tag": "record.createdAt", + "readOnly": true + }, + { + "name": "updated_at", + "type": "datetime", + "tag": "record.updatedAt", + "readOnly": true + } + ] + }, + "order": { + "label": "Order", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "name", + "type": "string", + "tag": "order.number", + "readOnly": true + }, + { + "name": "order_number", + "type": "number", + "readOnly": true + }, + { + "name": "email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "created_at", + "type": "datetime", + "tag": "order.createdAt", + "readOnly": true + }, + { + "name": "processed_at", + "type": "datetime" + }, + { + "name": "currency", + "type": "string", + "tag": "order.currency" + }, + { + "name": "financial_status", + "type": "enum", + "tag": "order.paymentStatus", + "values": [ + "pending", + "authorized", + "partially_paid", + "paid", + "partially_refunded", + "refunded", + "voided" + ] + }, + { + "name": "fulfillment_status", + "type": "enum", + "tag": "order.fulfillmentStatus", + "values": [ + "unfulfilled", + "partial", + "fulfilled", + "restocked" + ] + }, + { + "name": "subtotal_price", + "type": "number", + "tag": "order.subtotal" + }, + { + "name": "total_discounts", + "type": "number", + "tag": "order.discount" + }, + { + "name": "total_tax", + "type": "number", + "tag": "order.tax" + }, + { + "name": "total_price", + "type": "number", + "tag": "order.total" + }, + { + "name": "total_shipping_price_set.shop_money.amount", + "type": "number", + "tag": "order.shipping" + }, + { + "name": "customer.id", + "type": "string" + }, + { + "name": "line_items", + "type": "lineItem[]", + "tag": "order.lineItems" + }, + { + "name": "discount_codes", + "type": "json" + }, + { + "name": "shipping_address.name", + "type": "string", + "tag": "person.fullName" + }, + { + "name": "shipping_address.address1", + "type": "string", + "tag": "address.line1" + }, + { + "name": "shipping_address.city", + "type": "string", + "tag": "address.locality" + }, + { + "name": "shipping_address.province_code", + "type": "string", + "tag": "address.region" + }, + { + "name": "shipping_address.zip", + "type": "string", + "tag": "address.postalCode" + }, + { + "name": "shipping_address.country_code", + "type": "string", + "tag": "address.country" + }, + { + "name": "note", + "type": "text" + }, + { + "name": "tags", + "type": "string", + "tag": "record.tags" + }, + { + "name": "source_name", + "type": "string" + } + ] + }, + "fulfillment": { + "label": "Fulfillment", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "order_id", + "type": "string" + }, + { + "name": "status", + "type": "enum", + "values": [ + "pending", + "open", + "success", + "cancelled", + "error", + "failure" + ] + }, + { + "name": "tracking_company", + "type": "string", + "tag": "shipment.carrier" + }, + { + "name": "tracking_number", + "type": "string", + "tag": "shipment.trackingNumber" + }, + { + "name": "tracking_url", + "type": "string", + "tag": "shipment.trackingUrl" + }, + { + "name": "created_at", + "type": "datetime", + "readOnly": true + } + ] + }, + "refund": { + "label": "Refund", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "order_id", + "type": "string" + }, + { + "name": "created_at", + "type": "datetime", + "readOnly": true + }, + { + "name": "note", + "type": "text" + }, + { + "name": "refund_line_items", + "type": "json" + }, + { + "name": "transactions", + "type": "json" + } + ] + }, + "checkout": { + "label": "Abandoned checkout", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "token", + "type": "string" + }, + { + "name": "email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "abandoned_checkout_url", + "type": "string" + }, + { + "name": "total_price", + "type": "number", + "tag": "order.total" + }, + { + "name": "currency", + "type": "string", + "tag": "order.currency" + }, + { + "name": "line_items", + "type": "lineItem[]", + "tag": "order.lineItems" + }, + { + "name": "customer.id", + "type": "string" + }, + { + "name": "created_at", + "type": "datetime", + "readOnly": true + }, + { + "name": "completed_at", + "type": "datetime", + "readOnly": true + } + ] + }, + "inventoryLevel": { + "label": "Inventory level", + "fields": [ + { + "name": "inventory_item_id", + "type": "id" + }, + { + "name": "location_id", + "type": "string" + }, + { + "name": "available", + "type": "number", + "tag": "inventory.available" + }, + { + "name": "updated_at", + "type": "datetime", + "readOnly": true + } + ] + } + }, + "events": { + "order.created": { + "delivery": "webhook", + "subscribe": "webhook.create", + "description": "Shopify topic orders/create. The payload is the resource itself.", + "object": "order", + "payload": { + "id": 5612789014562, + "admin_graphql_api_id": "gid://shopify/Order/5612789014562", + "name": "#1042", + "order_number": 1042, + "email": "dana.reyes@example.com", + "created_at": "2026-10-18T09:14:22-05:00", + "processed_at": "2026-10-18T09:14:21-05:00", + "currency": "USD", + "financial_status": "paid", + "fulfillment_status": null, + "subtotal_price": "128.00", + "total_discounts": "12.80", + "total_tax": "9.50", + "total_price": "139.70", + "total_shipping_price_set": { + "shop_money": { + "amount": "15.00", + "currency_code": "USD" + } + }, + "discount_codes": [ + { + "code": "FALL10", + "amount": "12.80", + "type": "percentage" + } + ], + "note": "Gift wrap please", + "tags": "wholesale", + "source_name": "web", + "customer": { + "id": 7012345678901, + "email": "dana.reyes@example.com", + "first_name": "Dana", + "last_name": "Reyes" + }, + "line_items": [ + { + "id": 13579246801, + "variant_id": 44012345678, + "product_id": 8123456789, + "sku": "TW-MUG-12", + "title": "Acme Mug", + "variant_title": "12 oz / Slate", + "quantity": 2, + "price": "32.00", + "total_discount": "6.40" + }, + { + "id": 13579246802, + "variant_id": 44012345690, + "product_id": 8123456790, + "sku": "TW-PRESS-01", + "title": "Pour-over Press", + "variant_title": null, + "quantity": 1, + "price": "64.00", + "total_discount": "6.40" + } + ], + "shipping_address": { + "name": "Dana Reyes", + "address1": "501 Congress Ave", + "city": "Austin", + "province_code": "TX", + "zip": "78701", + "country_code": "US", + "phone": "+15125550142" + } + }, + "match": "orders/create", + "headers": { + "x-shopify-topic": "orders/create", + "x-shopify-webhook-id": "b54557e4-order-created", + "x-shopify-shop-domain": "acme.myshopify.com" + } + }, + "order.paid": { + "delivery": "webhook", + "subscribe": "webhook.create", + "description": "Shopify topic orders/paid. The payload is the resource itself.", + "object": "order", + "payload": { + "id": 5612789014562, + "admin_graphql_api_id": "gid://shopify/Order/5612789014562", + "name": "#1042", + "order_number": 1042, + "email": "dana.reyes@example.com", + "created_at": "2026-10-18T09:14:22-05:00", + "processed_at": "2026-10-18T09:14:21-05:00", + "currency": "USD", + "financial_status": "paid", + "fulfillment_status": null, + "subtotal_price": "128.00", + "total_discounts": "12.80", + "total_tax": "9.50", + "total_price": "139.70", + "total_shipping_price_set": { + "shop_money": { + "amount": "15.00", + "currency_code": "USD" + } + }, + "discount_codes": [ + { + "code": "FALL10", + "amount": "12.80", + "type": "percentage" + } + ], + "note": "Gift wrap please", + "tags": "wholesale", + "source_name": "web", + "customer": { + "id": 7012345678901, + "email": "dana.reyes@example.com", + "first_name": "Dana", + "last_name": "Reyes" + }, + "line_items": [ + { + "id": 13579246801, + "variant_id": 44012345678, + "product_id": 8123456789, + "sku": "TW-MUG-12", + "title": "Acme Mug", + "variant_title": "12 oz / Slate", + "quantity": 2, + "price": "32.00", + "total_discount": "6.40" + }, + { + "id": 13579246802, + "variant_id": 44012345690, + "product_id": 8123456790, + "sku": "TW-PRESS-01", + "title": "Pour-over Press", + "variant_title": null, + "quantity": 1, + "price": "64.00", + "total_discount": "6.40" + } + ], + "shipping_address": { + "name": "Dana Reyes", + "address1": "501 Congress Ave", + "city": "Austin", + "province_code": "TX", + "zip": "78701", + "country_code": "US", + "phone": "+15125550142" + } + }, + "match": "orders/paid", + "headers": { + "x-shopify-topic": "orders/paid", + "x-shopify-webhook-id": "b54557e4-order-paid", + "x-shopify-shop-domain": "acme.myshopify.com" + } + }, + "order.updated": { + "delivery": "webhook", + "subscribe": "webhook.create", + "description": "Shopify topic orders/updated. The payload is the resource itself.", + "object": "order", + "payload": { + "id": 5612789014562, + "admin_graphql_api_id": "gid://shopify/Order/5612789014562", + "name": "#1042", + "order_number": 1042, + "email": "dana.reyes@example.com", + "created_at": "2026-10-18T09:14:22-05:00", + "processed_at": "2026-10-18T09:14:21-05:00", + "currency": "USD", + "financial_status": "paid", + "fulfillment_status": null, + "subtotal_price": "128.00", + "total_discounts": "12.80", + "total_tax": "9.50", + "total_price": "139.70", + "total_shipping_price_set": { + "shop_money": { + "amount": "15.00", + "currency_code": "USD" + } + }, + "discount_codes": [ + { + "code": "FALL10", + "amount": "12.80", + "type": "percentage" + } + ], + "note": "Gift wrap please", + "tags": "wholesale, gift", + "source_name": "web", + "customer": { + "id": 7012345678901, + "email": "dana.reyes@example.com", + "first_name": "Dana", + "last_name": "Reyes" + }, + "line_items": [ + { + "id": 13579246801, + "variant_id": 44012345678, + "product_id": 8123456789, + "sku": "TW-MUG-12", + "title": "Acme Mug", + "variant_title": "12 oz / Slate", + "quantity": 2, + "price": "32.00", + "total_discount": "6.40" + }, + { + "id": 13579246802, + "variant_id": 44012345690, + "product_id": 8123456790, + "sku": "TW-PRESS-01", + "title": "Pour-over Press", + "variant_title": null, + "quantity": 1, + "price": "64.00", + "total_discount": "6.40" + } + ], + "shipping_address": { + "name": "Dana Reyes", + "address1": "501 Congress Ave", + "city": "Austin", + "province_code": "TX", + "zip": "78701", + "country_code": "US", + "phone": "+15125550142" + } + }, + "match": "orders/updated", + "headers": { + "x-shopify-topic": "orders/updated", + "x-shopify-webhook-id": "b54557e4-order-updated", + "x-shopify-shop-domain": "acme.myshopify.com" + } + }, + "customer.created": { + "delivery": "webhook", + "subscribe": "webhook.create", + "description": "Shopify topic customers/create. The payload is the resource itself.", + "object": "customer", + "payload": { + "id": 7012345678901, + "email": "dana.reyes@example.com", + "first_name": "Dana", + "last_name": "Reyes", + "phone": "+15125550142", + "tags": "wholesale, vip", + "note": null, + "orders_count": 7, + "total_spent": "1240.55", + "created_at": "2025-03-02T11:20:00-05:00", + "updated_at": "2026-10-18T09:14:23-05:00", + "email_marketing_consent": { + "state": "subscribed", + "opt_in_level": "confirmed_opt_in", + "consent_updated_at": "2025-03-02T11:20:00-05:00" + }, + "default_address": { + "company": "Reyes Coffee Co", + "address1": "501 Congress Ave", + "city": "Austin", + "province_code": "TX", + "zip": "78701", + "country_code": "US" + } + }, + "match": "customers/create", + "headers": { + "x-shopify-topic": "customers/create", + "x-shopify-webhook-id": "b54557e4-customer-created", + "x-shopify-shop-domain": "acme.myshopify.com" + } + }, + "customer.updated": { + "delivery": "webhook", + "subscribe": "webhook.create", + "description": "Shopify topic customers/update. The payload is the resource itself.", + "object": "customer", + "payload": { + "id": 7012345678901, + "email": "dana.reyes@example.com", + "first_name": "Dana", + "last_name": "Reyes", + "phone": "+15125550142", + "tags": "wholesale, vip, net30", + "note": null, + "orders_count": 7, + "total_spent": "1240.55", + "created_at": "2025-03-02T11:20:00-05:00", + "updated_at": "2026-10-18T09:14:23-05:00", + "email_marketing_consent": { + "state": "subscribed", + "opt_in_level": "confirmed_opt_in", + "consent_updated_at": "2025-03-02T11:20:00-05:00" + }, + "default_address": { + "company": "Reyes Coffee Co", + "address1": "501 Congress Ave", + "city": "Austin", + "province_code": "TX", + "zip": "78701", + "country_code": "US" + } + }, + "match": "customers/update", + "headers": { + "x-shopify-topic": "customers/update", + "x-shopify-webhook-id": "b54557e4-customer-updated", + "x-shopify-shop-domain": "acme.myshopify.com" + } + }, + "fulfillment.created": { + "delivery": "webhook", + "subscribe": "webhook.create", + "description": "Shopify topic fulfillments/create. The payload is the resource itself.", + "object": "fulfillment", + "payload": { + "id": 4987654321098, + "order_id": 5612789014562, + "status": "success", + "tracking_company": "UPS", + "tracking_number": "1Z999AA10123456784", + "tracking_url": "https://www.ups.com/track?tracknum=1Z999AA10123456784", + "created_at": "2026-10-19T14:02:00-05:00", + "line_items": [ + { + "id": 13579246801, + "quantity": 2 + } + ] + }, + "match": "fulfillments/create", + "headers": { + "x-shopify-topic": "fulfillments/create", + "x-shopify-webhook-id": "b54557e4-fulfillment-created", + "x-shopify-shop-domain": "acme.myshopify.com" + } + }, + "refund.created": { + "delivery": "webhook", + "subscribe": "webhook.create", + "description": "Shopify topic refunds/create. The payload is the resource itself.", + "object": "refund", + "payload": { + "id": 929361462, + "order_id": 5612789014562, + "created_at": "2026-10-21T10:30:00-05:00", + "note": "Mug arrived chipped", + "refund_line_items": [ + { + "id": 104689539, + "line_item_id": 13579246801, + "quantity": 1, + "restock_type": "return", + "subtotal": 28.8, + "total_tax": 2.38, + "line_item": { + "sku": "TW-MUG-12", + "title": "Acme Mug" + } + } + ], + "transactions": [ + { + "id": 389404469, + "kind": "refund", + "gateway": "shopify_payments", + "status": "success", + "amount": "31.18", + "currency": "USD" + } + ] + }, + "match": "refunds/create", + "headers": { + "x-shopify-topic": "refunds/create", + "x-shopify-webhook-id": "b54557e4-refund-created", + "x-shopify-shop-domain": "acme.myshopify.com" + } + }, + "inventory.updated": { + "delivery": "webhook", + "subscribe": "webhook.create", + "description": "Shopify topic inventory_levels/update. The payload is the resource itself.", + "object": "inventoryLevel", + "payload": { + "inventory_item_id": 46012345678, + "location_id": 61234567890, + "available": 3, + "updated_at": "2026-10-18T09:14:23-05:00", + "admin_graphql_api_id": "gid://shopify/InventoryLevel/61234567890?inventory_item_id=46012345678" + }, + "match": "inventory_levels/update", + "headers": { + "x-shopify-topic": "inventory_levels/update", + "x-shopify-webhook-id": "b54557e4-inventory-updated", + "x-shopify-shop-domain": "acme.myshopify.com" + } + }, + "checkout.abandoned": { + "delivery": "poll", + "object": "checkout", + "description": "Polled from GET /checkouts.json. Shopify sends no webhook when a checkout is abandoned; a checkout with an email and no completed_at after an hour counts as abandoned.", + "payload": { + "id": 35112233445566, + "token": "example-checkout-token", + "email": "dana.reyes@example.com", + "abandoned_checkout_url": "https://acme-goods.myshopify.com/58912/checkouts/b1946ac9/recover?key=7c2f", + "total_price": "64.00", + "currency": "USD", + "customer": { + "id": 7012345678901 + }, + "line_items": [ + { + "variant_id": 44012345690, + "sku": "TW-PRESS-01", + "title": "Pour-over Press", + "quantity": 1, + "price": "64.00" + } + ], + "created_at": "2026-10-18T20:41:00-05:00", + "completed_at": null + } + } + }, + "webhook": { + "scope": "installation", + "topic": "{{headers.x-shopify-topic}}", + "eventId": "{{headers.x-shopify-webhook-id}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "order.get": { + "kind": "read", + "method": "getOrder", + "input": { + "id": { + "type": "string", + "required": true + }, + "fields": { + "type": "string" + } + }, + "output": "order" + }, + "order.list": { + "kind": "read", + "method": "listOrders", + "input": { + "status": { + "type": "string" + }, + "created_at_min": { + "type": "string" + }, + "updated_at_min": { + "type": "string" + }, + "limit": { + "type": "number" + }, + "page_info": { + "type": "string" + } + }, + "output": "page" + }, + "order.update": { + "kind": "write", + "method": "updateOrder", + "input": { + "id": { + "type": "string", + "required": true + }, + "order": { + "type": "object", + "required": true + } + }, + "output": "order" + }, + "order.fulfillmentOrders": { + "kind": "read", + "method": "listFulfillmentOrders", + "input": { + "orderId": { + "type": "string", + "required": true + } + }, + "output": "fulfillmentOrder[]" + }, + "fulfillment.create": { + "kind": "write", + "method": "createFulfillment", + "input": { + "fulfillmentOrderId": { + "type": "string", + "required": true + }, + "trackingInfo": { + "type": "object" + }, + "notifyCustomer": { + "type": "boolean" + } + }, + "output": "fulfillment" + }, + "customer.get": { + "kind": "read", + "method": "getCustomer", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "customer" + }, + "customer.list": { + "kind": "read", + "method": "listCustomers", + "input": { + "updated_at_min": { + "type": "string" + }, + "limit": { + "type": "number" + }, + "page_info": { + "type": "string" + } + }, + "output": "page" + }, + "customer.search": { + "kind": "search", + "method": "searchCustomers", + "input": { + "query": { + "type": "string", + "required": true + } + }, + "output": "customer[]" + }, + "customer.create": { + "kind": "write", + "method": "createCustomer", + "input": { + "customer": { + "type": "object", + "required": true + } + }, + "output": "customer" + }, + "customer.update": { + "kind": "write", + "method": "updateCustomer", + "input": { + "id": { + "type": "string", + "required": true + }, + "customer": { + "type": "object", + "required": true + } + }, + "output": "customer" + }, + "checkout.list": { + "kind": "read", + "method": "listAbandonedCheckouts", + "input": { + "created_at_min": { + "type": "string" + }, + "limit": { + "type": "number" + } + }, + "output": "checkout[]" + }, + "variant.byInventoryItem": { + "kind": "read", + "method": "getVariantByInventoryItemId", + "input": { + "inventoryItemId": { + "type": "string", + "required": true + } + }, + "output": "variant" + }, + "location.list": { + "kind": "read", + "method": "listLocations", + "input": {}, + "output": "location[]" + }, + "webhook.create": { + "kind": "subscribe", + "method": "createWebhook", + "input": { + "topic": { + "type": "string", + "required": true + }, + "address": { + "type": "string", + "required": true + }, + "format": { + "type": "string" + } + }, + "output": "webhook" + }, + "webhook.delete": { + "kind": "delete", + "method": "deleteWebhook", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "void" + } + }, + "options": { + "location": { + "capability": "location.list", + "label": "{{item.name}}", + "value": "$.item.id" + } + }, + "ui": { + "placements": [ + "admin.customer-details.block", + "admin.order-details.block", + "admin.product-details.block" + ], + "renderer": "frigg-shopify-admin-extension" + } +} diff --git a/packages/core/integrations/data-defined/capabilities/slack.capabilities.json b/packages/core/integrations/data-defined/capabilities/slack.capabilities.json new file mode 100644 index 000000000..f419c04d6 --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/slack.capabilities.json @@ -0,0 +1,363 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "slack", + "label": "Slack", + "package": "@friggframework/api-module-slack", + "version": "2.0.0", + "auth": "oauth2", + "objects": { + "message": { + "label": "Message", + "fields": [ + { + "name": "ts", + "type": "id", + "readOnly": true + }, + { + "name": "channel", + "type": "string" + }, + { + "name": "text", + "type": "text" + }, + { + "name": "user", + "type": "string", + "tag": "record.owner" + }, + { + "name": "thread_ts", + "type": "string" + }, + { + "name": "blocks", + "type": "json" + }, + { + "name": "metadata.event_type", + "type": "string" + }, + { + "name": "metadata.event_payload", + "type": "json" + } + ] + }, + "channel": { + "label": "Channel", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "name", + "type": "string", + "tag": "channel.name" + }, + { + "name": "is_private", + "type": "boolean" + }, + { + "name": "is_archived", + "type": "boolean", + "readOnly": true + }, + { + "name": "num_members", + "type": "number", + "readOnly": true + } + ] + }, + "user": { + "label": "User", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "real_name", + "type": "string", + "tag": "person.fullName" + }, + { + "name": "profile.display_name", + "type": "string" + }, + { + "name": "profile.email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "profile.phone", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "tz", + "type": "string", + "tag": "person.timezone" + }, + { + "name": "is_bot", + "type": "boolean", + "readOnly": true + } + ] + } + }, + "events": { + "interaction.block_actions": { + "delivery": "webhook", + "object": "message", + "description": "Interactivity request (block_actions) from a button in a message the app posted. App-level: Frigg routes it to the installation by team.id; no per-install subscription.", + "payload": { + "type": "block_actions", + "trigger_id": "8171092340291.5283910.ab4c1f", + "team": { + "id": "T09B191G5HA", + "domain": "acme" + }, + "user": { + "id": "U04A8KX2P1Q", + "username": "dana", + "name": "dana" + }, + "channel": { + "id": "D06QX0B7Z3L", + "name": "directmessage" + }, + "message": { + "type": "message", + "ts": "1760800000.001200", + "channel": "D06QX0B7Z3L", + "text": "Due today: Send renewal quote to Acme", + "metadata": { + "event_type": "hubspot_task", + "event_payload": { + "taskId": "94512" + } + } + }, + "actions": [ + { + "action_id": "frigg.mark-task-done", + "block_id": "frigg", + "value": "94512", + "type": "button", + "action_ts": "1760800042.112233" + } + ] + }, + "record": "message", + "idPath": "trigger_id" + }, + "app_mention": { + "delivery": "webhook", + "object": "message", + "description": "Events API app_mention. App-level, routed by team_id.", + "payload": { + "token": "example-verification-token", + "team_id": "T09B191G5HA", + "api_app_id": "A0AS07TUM6H", + "event": { + "type": "app_mention", + "user": "U04A8KX2P1Q", + "text": "<@U0LAN0Z89> what closes this week?", + "ts": "1760800100.000300", + "channel": "C08SALES01", + "event_ts": "1760800100.000300" + }, + "type": "event_callback", + "event_id": "Ev08MFMKH6", + "event_time": 1760800100 + }, + "record": "event", + "idPath": "event_id" + } + }, + "webhook": { + "scope": "app", + "unwrap": "payload", + "topic": [ + "{{body.event.type}}", + "interaction.{{body.type}}" + ], + "eventId": [ + "{{body.event_id}}", + "{{body.trigger_id}}" + ], + "account": [ + "{{body.team_id}}", + "{{body.team.id}}" + ], + "verify": "verifyWebhookSignature", + "handshake": { + "when": "$.body.type == 'url_verification'", + "status": 200, + "body": { + "challenge": "$.body.challenge" + } + } + }, + "capabilities": { + "message.post": { + "kind": "write", + "method": "postMessage", + "label": "chat.postMessage", + "input": { + "channel": { + "type": "string", + "required": true + }, + "text": { + "type": "string", + "required": true + }, + "blocks": { + "type": "array" + }, + "thread_ts": { + "type": "string" + }, + "metadata": { + "type": "object" + }, + "unfurl_links": { + "type": "boolean" + } + }, + "output": "message" + }, + "message.update": { + "kind": "write", + "method": "updateMessage", + "label": "chat.update", + "input": { + "channel": { + "type": "string", + "required": true + }, + "ts": { + "type": "string", + "required": true + }, + "text": { + "type": "string", + "required": true + }, + "blocks": { + "type": "array" + } + }, + "output": "message" + }, + "message.postEphemeral": { + "kind": "write", + "method": "postEphemeral", + "label": "chat.postEphemeral", + "input": { + "channel": { + "type": "string", + "required": true + }, + "user": { + "type": "string", + "required": true + }, + "text": { + "type": "string", + "required": true + }, + "blocks": { + "type": "array" + } + }, + "output": "message" + }, + "message.permalink": { + "kind": "read", + "method": "getPermalink", + "label": "chat.getPermalink", + "input": { + "channel": { + "type": "string", + "required": true + }, + "message_ts": { + "type": "string", + "required": true + } + }, + "output": "string" + }, + "channel.list": { + "kind": "read", + "method": "listConversations", + "label": "conversations.list", + "input": { + "types": { + "type": "string" + }, + "exclude_archived": { + "type": "boolean" + }, + "limit": { + "type": "number" + } + }, + "output": "channel[]" + }, + "user.list": { + "kind": "read", + "method": "listUsers", + "label": "users.list", + "input": { + "limit": { + "type": "number" + } + }, + "output": "user[]" + }, + "user.lookupByEmail": { + "kind": "read", + "method": "lookupUserByEmail", + "label": "users.lookupByEmail", + "input": { + "email": { + "type": "string", + "required": true + } + }, + "output": "user" + } + }, + "options": { + "channel": { + "capability": "channel.list", + "label": "#{{item.name}}", + "value": "$.item.id" + }, + "user": { + "capability": "user.list", + "label": "{{item.real_name}}", + "value": "$.item.id" + } + }, + "ui": { + "placements": [ + "message.actions", + "app.home", + "modal" + ], + "renderer": "frigg-slack-blocks" + } +} diff --git a/packages/core/integrations/data-defined/capabilities/stripe.capabilities.json b/packages/core/integrations/data-defined/capabilities/stripe.capabilities.json new file mode 100644 index 000000000..9a68d9457 --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/stripe.capabilities.json @@ -0,0 +1,580 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "stripe", + "label": "Stripe", + "package": "@friggframework/api-module-stripe", + "version": "2.3.0", + "auth": "oauth2", + "objects": { + "customer": { + "label": "Customer", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "name", + "type": "string", + "tag": "person.fullName" + }, + { + "name": "phone", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "description", + "type": "text" + }, + { + "name": "address.line1", + "type": "string", + "tag": "address.line1" + }, + { + "name": "address.city", + "type": "string", + "tag": "address.locality" + }, + { + "name": "address.state", + "type": "string", + "tag": "address.region" + }, + { + "name": "address.postal_code", + "type": "string", + "tag": "address.postalCode" + }, + { + "name": "address.country", + "type": "string", + "tag": "address.country" + }, + { + "name": "currency", + "type": "string" + }, + { + "name": "metadata.quickbooks_id", + "type": "string", + "tag": "record.externalId" + }, + { + "name": "created", + "type": "timestamp", + "tag": "record.createdAt", + "readOnly": true + } + ] + }, + "paymentIntent": { + "label": "Payment intent", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "amount", + "type": "integer", + "tag": "payment.amountMinor" + }, + { + "name": "amount_received", + "type": "integer", + "readOnly": true + }, + { + "name": "currency", + "type": "string", + "tag": "payment.currency" + }, + { + "name": "customer", + "type": "string" + }, + { + "name": "description", + "type": "text" + }, + { + "name": "receipt_email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "status", + "type": "enum", + "tag": "payment.status", + "values": [ + "requires_payment_method", + "requires_confirmation", + "requires_action", + "processing", + "requires_capture", + "canceled", + "succeeded" + ], + "readOnly": true + }, + { + "name": "latest_charge", + "type": "string", + "readOnly": true + }, + { + "name": "metadata", + "type": "json" + }, + { + "name": "created", + "type": "timestamp", + "tag": "record.createdAt", + "readOnly": true + } + ] + }, + "refund": { + "label": "Refund", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "amount", + "type": "integer", + "tag": "payment.amountMinor" + }, + { + "name": "currency", + "type": "string", + "tag": "payment.currency" + }, + { + "name": "charge", + "type": "string" + }, + { + "name": "payment_intent", + "type": "string" + }, + { + "name": "reason", + "type": "enum", + "values": [ + "duplicate", + "fraudulent", + "requested_by_customer", + "expired_uncaptured_charge" + ] + }, + { + "name": "status", + "type": "enum", + "values": [ + "pending", + "requires_action", + "succeeded", + "failed", + "canceled" + ], + "readOnly": true + }, + { + "name": "created", + "type": "timestamp", + "tag": "record.createdAt", + "readOnly": true + } + ] + }, + "payout": { + "label": "Payout", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "amount", + "type": "integer", + "tag": "payment.amountMinor" + }, + { + "name": "currency", + "type": "string", + "tag": "payment.currency" + }, + { + "name": "arrival_date", + "type": "timestamp" + }, + { + "name": "status", + "type": "enum", + "values": [ + "paid", + "pending", + "in_transit", + "canceled", + "failed" + ], + "readOnly": true + }, + { + "name": "automatic", + "type": "boolean" + }, + { + "name": "balance_transaction", + "type": "string" + }, + { + "name": "statement_descriptor", + "type": "string" + } + ] + }, + "balanceTransaction": { + "label": "Balance transaction", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "amount", + "type": "integer" + }, + { + "name": "fee", + "type": "integer" + }, + { + "name": "net", + "type": "integer" + }, + { + "name": "currency", + "type": "string" + }, + { + "name": "type", + "type": "enum", + "values": [ + "charge", + "payment", + "refund", + "adjustment", + "stripe_fee", + "payout", + "application_fee" + ] + }, + { + "name": "reporting_category", + "type": "string" + }, + { + "name": "source", + "type": "string" + }, + { + "name": "created", + "type": "timestamp", + "readOnly": true + } + ] + } + }, + "events": { + "payment_intent.succeeded": { + "delivery": "webhook", + "object": "paymentIntent", + "description": "Connect webhook: one app-level endpoint receives events for every connected account; Frigg routes by `account`. No per-install subscription.", + "payload": { + "id": "evt_1QBz8kLkdIwHu7ix3XyZ9aQe", + "object": "event", + "account": "acct_1Nv0FGQ9RKHgCVdK", + "api_version": "2024-06-20", + "created": 1760800000, + "livemode": true, + "type": "payment_intent.succeeded", + "data": { + "object": { + "id": "pi_3QBz8jLkdIwHu7ix1Zt0nXyQ", + "object": "payment_intent", + "amount": 13970, + "amount_received": 13970, + "currency": "usd", + "customer": "cus_QzX1fL0aB2cD3e", + "description": "Invoice INV-2026-0142", + "receipt_email": "dana.reyes@example.com", + "status": "succeeded", + "latest_charge": "ch_3QBz8jLkdIwHu7ix1Kk4mPqR", + "metadata": { + "order_id": "1042" + }, + "created": 1760799990 + } + } + }, + "record": "data.object" + }, + "refund.created": { + "delivery": "webhook", + "object": "refund", + "payload": { + "id": "evt_1QBz8kLkdIwHu7ix3XyZ9aQe", + "object": "event", + "account": "acct_1Nv0FGQ9RKHgCVdK", + "api_version": "2024-06-20", + "created": 1760800000, + "livemode": true, + "type": "refund.created", + "data": { + "object": { + "id": "re_3QBz8jLkdIwHu7ix1aBcDeFg", + "object": "refund", + "amount": 3118, + "currency": "usd", + "charge": "ch_3QBz8jLkdIwHu7ix1Kk4mPqR", + "payment_intent": "pi_3QBz8jLkdIwHu7ix1Zt0nXyQ", + "reason": "requested_by_customer", + "status": "succeeded", + "created": 1760990000 + } + } + }, + "record": "data.object" + }, + "customer.created": { + "delivery": "webhook", + "object": "customer", + "payload": { + "id": "evt_1QBz8kLkdIwHu7ix3XyZ9aQe", + "object": "event", + "account": "acct_1Nv0FGQ9RKHgCVdK", + "api_version": "2024-06-20", + "created": 1760800000, + "livemode": true, + "type": "customer.created", + "data": { + "object": { + "id": "cus_QzX1fL0aB2cD3e", + "object": "customer", + "email": "dana.reyes@example.com", + "name": "Dana Reyes", + "phone": "+15125550142", + "address": { + "line1": "501 Congress Ave", + "city": "Austin", + "state": "TX", + "postal_code": "78701", + "country": "US" + }, + "metadata": {}, + "created": 1760700000 + } + } + }, + "record": "data.object" + }, + "customer.updated": { + "delivery": "webhook", + "object": "customer", + "payload": { + "id": "evt_1QBz8kLkdIwHu7ix3XyZ9aQe", + "object": "event", + "account": "acct_1Nv0FGQ9RKHgCVdK", + "api_version": "2024-06-20", + "created": 1760800000, + "livemode": true, + "type": "customer.updated", + "data": { + "object": { + "id": "cus_QzX1fL0aB2cD3e", + "object": "customer", + "email": "dana@reyescoffee.com", + "name": "Dana Reyes", + "metadata": {} + } + } + }, + "record": "data.object" + }, + "payout.paid": { + "delivery": "webhook", + "object": "payout", + "payload": { + "id": "evt_1QBz8kLkdIwHu7ix3XyZ9aQe", + "object": "event", + "account": "acct_1Nv0FGQ9RKHgCVdK", + "api_version": "2024-06-20", + "created": 1760800000, + "livemode": true, + "type": "payout.paid", + "data": { + "object": { + "id": "po_1QC1aBLkdIwHu7ixq2W3e4R5", + "object": "payout", + "amount": 482211, + "currency": "usd", + "arrival_date": 1760918400, + "status": "paid", + "automatic": true, + "balance_transaction": "txn_1QC1aBLkdIwHu7ixAbCdEfGh" + } + } + }, + "record": "data.object" + } + }, + "webhook": { + "scope": "app", + "topic": "{{body.type}}", + "eventId": "{{body.id}}", + "account": "{{body.account}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "customer.get": { + "kind": "read", + "method": "retrieveCustomer", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "customer" + }, + "customer.list": { + "kind": "read", + "method": "listCustomers", + "input": { + "created_gte": { + "type": "number" + }, + "limit": { + "type": "number" + }, + "starting_after": { + "type": "string" + } + }, + "output": "page" + }, + "customer.search": { + "kind": "search", + "method": "searchCustomers", + "input": { + "query": { + "type": "string", + "required": true + } + }, + "output": "customer[]" + }, + "customer.create": { + "kind": "write", + "method": "createCustomer", + "input": { + "params": { + "type": "object", + "required": true + } + }, + "output": "customer" + }, + "customer.update": { + "kind": "write", + "method": "updateCustomer", + "input": { + "id": { + "type": "string", + "required": true + }, + "params": { + "type": "object", + "required": true + } + }, + "output": "customer" + }, + "paymentIntent.get": { + "kind": "read", + "method": "retrievePaymentIntent", + "input": { + "id": { + "type": "string", + "required": true + }, + "expand": { + "type": "array" + } + }, + "output": "paymentIntent" + }, + "refund.get": { + "kind": "read", + "method": "retrieveRefund", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "refund" + }, + "payout.list": { + "kind": "read", + "method": "listPayouts", + "input": { + "status": { + "type": "string" + }, + "arrivedAfter": { + "type": "string" + }, + "arrivedBefore": { + "type": "string" + }, + "limit": { + "type": "number" + } + }, + "output": "payout[]" + }, + "balanceTransaction.list": { + "kind": "read", + "method": "listBalanceTransactions", + "input": { + "payout": { + "type": "string", + "required": true + }, + "type": { + "type": "string" + }, + "limit": { + "type": "number" + } + }, + "output": "balanceTransaction[]" + } + } +} diff --git a/packages/core/integrations/data-defined/capabilities/webflow.capabilities.json b/packages/core/integrations/data-defined/capabilities/webflow.capabilities.json new file mode 100644 index 000000000..0134d78fc --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/webflow.capabilities.json @@ -0,0 +1,196 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "webflow", + "label": "Webflow", + "package": "@friggframework/api-module-webflow", + "version": "2.0.0", + "auth": "oauth2", + "objects": { + "formSubmission": { + "label": "Form submission", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "name", + "type": "string", + "tag": "form.name" + }, + { + "name": "formId", + "type": "string" + }, + { + "name": "siteId", + "type": "string" + }, + { + "name": "publishedPath", + "type": "string", + "tag": "web.pagePath" + }, + { + "name": "submittedAt", + "type": "datetime", + "tag": "record.createdAt" + }, + { + "name": "data.Name", + "type": "string", + "tag": "person.fullName" + }, + { + "name": "data.Email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "data.Phone", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "data.Company", + "type": "string", + "tag": "organization.name" + }, + { + "name": "data.Interest", + "type": "enum", + "values": [ + "Sales", + "Support", + "Partnerships" + ] + }, + { + "name": "data.Message", + "type": "text" + }, + { + "name": "data.Website", + "type": "string", + "help": "Hidden honeypot field; humans leave it empty." + } + ] + } + }, + "events": { + "form.submitted": { + "delivery": "webhook", + "object": "formSubmission", + "subscribe": "webhook.create", + "description": "triggerType form_submission, registered per site.", + "payload": { + "triggerType": "form_submission", + "payload": { + "name": "Contact Us", + "siteId": "65427cf400e02b306eaa049c", + "formId": "65429eadebe8a9f3a30f62d0", + "formElementId": "4e038d2c-6a1e-4953-7be9-a59a2b453177", + "pageId": "65427cf400e02b306eaa04a0", + "publishedPath": "/contact", + "id": "6321ca84df3949bfc6752327", + "submittedAt": "2026-10-20T17:12:00.000Z", + "data": { + "Name": "Priya Natarajan", + "Email": "priya@acme.example", + "Phone": "512 555 0177", + "Company": "Acme Corp", + "Interest": "Sales", + "Message": "We need a Salesforce \u2194 NetSuite integration by Q1.", + "Website": "" + } + } + }, + "record": "payload", + "idPath": "payload.id", + "match": "form_submission" + } + }, + "webhook": { + "scope": "installation", + "topic": "{{body.triggerType}}", + "eventId": "{{body.payload.id}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "site.list": { + "kind": "read", + "method": "listSites", + "input": {}, + "output": "site[]" + }, + "form.list": { + "kind": "read", + "method": "listForms", + "input": { + "siteId": { + "type": "string", + "required": true + } + }, + "output": "form[]" + }, + "form.submissions": { + "kind": "read", + "method": "listFormSubmissions", + "input": { + "formId": { + "type": "string", + "required": true + }, + "offset": { + "type": "number" + }, + "limit": { + "type": "number" + } + }, + "output": "formSubmission[]" + }, + "webhook.create": { + "kind": "subscribe", + "method": "createWebhook", + "input": { + "siteId": { + "type": "string", + "required": true + }, + "triggerType": { + "type": "string", + "required": true + }, + "url": { + "type": "string", + "required": true + }, + "filter": { + "type": "object" + } + }, + "output": "webhook" + }, + "webhook.delete": { + "kind": "delete", + "method": "deleteWebhook", + "input": { + "webhookId": { + "type": "string", + "required": true + } + }, + "output": "void" + } + }, + "options": { + "site": { + "capability": "site.list", + "label": "{{item.displayName}}", + "value": "$.item.id" + } + } +} diff --git a/packages/core/integrations/data-defined/capabilities/zendesk.capabilities.json b/packages/core/integrations/data-defined/capabilities/zendesk.capabilities.json new file mode 100644 index 000000000..73b03540d --- /dev/null +++ b/packages/core/integrations/data-defined/capabilities/zendesk.capabilities.json @@ -0,0 +1,409 @@ +{ + "$schema": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "module": "zendesk", + "label": "Zendesk Support", + "package": "@friggframework/api-module-zendesk", + "version": "1.6.0", + "auth": "oauth2", + "account": { + "description": "Values about the connected account, persisted by the module (apiPropertiesToPersist). Definitions read them as $.install.accounts...", + "fields": [ + { + "name": "subdomain", + "type": "string", + "description": "Zendesk subdomain (acme in acme.zendesk.com)" + } + ] + }, + "objects": { + "ticket": { + "label": "Ticket", + "fields": [ + { + "name": "id", + "type": "id", + "tag": "record.displayId", + "readOnly": true + }, + { + "name": "subject", + "type": "string", + "tag": "ticket.subject" + }, + { + "name": "description", + "type": "text", + "tag": "ticket.description", + "readOnly": true + }, + { + "name": "status", + "type": "enum", + "tag": "ticket.status", + "values": [ + "new", + "open", + "pending", + "hold", + "solved", + "closed" + ] + }, + { + "name": "priority", + "type": "enum", + "tag": "ticket.priority", + "values": [ + "low", + "normal", + "high", + "urgent" + ] + }, + { + "name": "type", + "type": "enum", + "values": [ + "problem", + "incident", + "question", + "task" + ] + }, + { + "name": "requester_id", + "type": "string" + }, + { + "name": "assignee_id", + "type": "string", + "tag": "record.owner" + }, + { + "name": "group_id", + "type": "string" + }, + { + "name": "organization_id", + "type": "string" + }, + { + "name": "tags", + "type": "string[]", + "tag": "record.tags" + }, + { + "name": "external_id", + "type": "string", + "tag": "record.externalId" + }, + { + "name": "url", + "type": "string", + "readOnly": true + }, + { + "name": "created_at", + "type": "datetime", + "tag": "record.createdAt", + "readOnly": true + }, + { + "name": "updated_at", + "type": "datetime", + "tag": "record.updatedAt", + "readOnly": true + } + ] + }, + "user": { + "label": "User", + "fields": [ + { + "name": "id", + "type": "id", + "readOnly": true + }, + { + "name": "name", + "type": "string", + "tag": "person.fullName" + }, + { + "name": "email", + "type": "email", + "tag": "contact.email" + }, + { + "name": "phone", + "type": "phone", + "tag": "contact.phone" + }, + { + "name": "role", + "type": "enum", + "values": [ + "end-user", + "agent", + "admin" + ] + }, + { + "name": "organization_id", + "type": "string" + } + ] + } + }, + "events": { + "ticket.tagsChanged": { + "delivery": "webhook", + "object": "ticket", + "subscribe": "webhook.create", + "description": "Event-subscription webhook zen:event-type:ticket.tags_changed. $.record is the ticket (detail); added/removed tags are in event.", + "payload": { + "account_id": 22129848, + "detail": { + "id": 35436, + "url": "https://acmehelp.zendesk.com/api/v2/tickets/35436.json", + "subject": "Checkout fails with 500 for EU customers", + "description": "Since this morning every EU checkout returns a 500 at the payment step.", + "status": "open", + "priority": "urgent", + "type": "incident", + "requester_id": 20978392, + "assignee_id": 235323, + "group_id": 98738, + "organization_id": 509974, + "tags": [ + "checkout", + "eu", + "escalate_jira" + ], + "external_id": null, + "created_at": "2026-10-20T08:12:44Z", + "updated_at": "2026-10-20T09:01:02Z" + }, + "event": { + "tags_added": [ + "escalate_jira" + ], + "tags_removed": [] + }, + "id": "cbe4028c-7239-495d-b020-f22348516046", + "subject": "zen:ticket:35436", + "time": "2026-10-20T09:01:02.882Z", + "type": "zen:event-type:ticket.tags_changed", + "zendesk_event_version": "2022-11-06" + }, + "record": "detail", + "match": "zen:event-type:ticket.tags_changed" + }, + "ticket.priorityChanged": { + "delivery": "webhook", + "object": "ticket", + "subscribe": "webhook.create", + "payload": { + "account_id": 22129848, + "detail": { + "id": 35436, + "url": "https://acmehelp.zendesk.com/api/v2/tickets/35436.json", + "subject": "Checkout fails with 500 for EU customers", + "description": "Since this morning every EU checkout returns a 500 at the payment step.", + "status": "open", + "priority": "urgent", + "type": "incident", + "requester_id": 20978392, + "assignee_id": 235323, + "group_id": 98738, + "organization_id": 509974, + "tags": [ + "checkout", + "eu", + "escalate_jira" + ], + "external_id": null, + "created_at": "2026-10-20T08:12:44Z", + "updated_at": "2026-10-20T09:01:02Z" + }, + "event": { + "current": "urgent", + "previous": "high" + }, + "id": "cbe4028c-7239-495d-b020-f22348516046", + "subject": "zen:ticket:35436", + "time": "2026-10-20T09:01:02.882Z", + "type": "zen:event-type:ticket.priority_changed", + "zendesk_event_version": "2022-11-06" + }, + "record": "detail", + "match": "zen:event-type:ticket.priority_changed" + }, + "ticket.statusChanged": { + "delivery": "webhook", + "object": "ticket", + "subscribe": "webhook.create", + "payload": { + "account_id": 22129848, + "detail": { + "id": 35436, + "url": "https://acmehelp.zendesk.com/api/v2/tickets/35436.json", + "subject": "Checkout fails with 500 for EU customers", + "description": "Since this morning every EU checkout returns a 500 at the payment step.", + "status": "solved", + "priority": "urgent", + "type": "incident", + "requester_id": 20978392, + "assignee_id": 235323, + "group_id": 98738, + "organization_id": 509974, + "tags": [ + "checkout", + "eu", + "escalate_jira" + ], + "external_id": null, + "created_at": "2026-10-20T08:12:44Z", + "updated_at": "2026-10-20T09:01:02Z" + }, + "event": { + "current": "SOLVED", + "previous": "OPEN" + }, + "id": "cbe4028c-7239-495d-b020-f22348516046", + "subject": "zen:ticket:35436", + "time": "2026-10-20T09:01:02.882Z", + "type": "zen:event-type:ticket.status_changed", + "zendesk_event_version": "2022-11-06" + }, + "record": "detail", + "match": "zen:event-type:ticket.status_changed" + } + }, + "webhook": { + "scope": "installation", + "topic": "{{body.type}}", + "eventId": "{{body.id}}", + "verify": "verifyWebhookSignature" + }, + "capabilities": { + "ticket.get": { + "kind": "read", + "method": "showTicket", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "ticket" + }, + "ticket.update": { + "kind": "write", + "method": "updateTicket", + "input": { + "id": { + "type": "string", + "required": true + }, + "ticket": { + "type": "object", + "required": true + } + }, + "output": "ticket" + }, + "ticket.addComment": { + "kind": "write", + "method": "addTicketComment", + "input": { + "id": { + "type": "string", + "required": true + }, + "body": { + "type": "string", + "required": true + }, + "public": { + "type": "boolean" + } + }, + "output": "ticket" + }, + "ticket.search": { + "kind": "search", + "method": "searchTickets", + "input": { + "query": { + "type": "string", + "required": true + } + }, + "output": "ticket[]" + }, + "user.get": { + "kind": "read", + "method": "showUser", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "user" + }, + "group.list": { + "kind": "read", + "method": "listGroups", + "input": {}, + "output": "group[]" + }, + "webhook.create": { + "kind": "subscribe", + "method": "createWebhook", + "input": { + "name": { + "type": "string", + "required": true + }, + "endpoint": { + "type": "string", + "required": true + }, + "subscriptions": { + "type": "array", + "required": true + } + }, + "output": "webhook" + }, + "webhook.delete": { + "kind": "delete", + "method": "deleteWebhook", + "input": { + "id": { + "type": "string", + "required": true + } + }, + "output": "void" + } + }, + "options": { + "group": { + "capability": "group.list", + "label": "{{item.name}}", + "value": "$.item.id" + } + }, + "ui": { + "placements": [ + "ticket_sidebar", + "new_ticket_sidebar", + "user_sidebar", + "organization_sidebar" + ], + "renderer": "frigg-zaf-app" + } +} diff --git a/packages/core/integrations/data-defined/compile.js b/packages/core/integrations/data-defined/compile.js new file mode 100644 index 000000000..ba9d4a98d --- /dev/null +++ b/packages/core/integrations/data-defined/compile.js @@ -0,0 +1,77 @@ +/** + * Compile a definition + one installation into what Frigg runs: an events map shaped exactly like a hand-written + * integration's `this.events`, plus the routing tables the runtime needs. Pure; no I/O. + * + * Only what the installation has enabled is compiled, so a disabled workflow has no event, no route and no schedule. + * + * Event names: SYNC__BACKFILL (USER_ACTION) · SYNC__PAGE (QUEUE) · SYNC__ONGOING (WEBHOOK|CRON) + * WF_ (WEBHOOK|CRON|USER_ACTION|QUEUE) · Q_ (ROUTE) · UI_ (ROUTE) + */ +const eventName = (prefix, id, suffix) => [prefix, id.toUpperCase().replace(/-/g, '_'), suffix].filter(Boolean).join('_'); + +function isOn(item, chosen) { + if (chosen && typeof chosen.enabled === 'boolean') return chosen.enabled; + return (item.default || 'on') === 'on'; +} + +function compile(def, installation = {}) { + const events = {}; + const webhooks = {}; // moduleKey -> topic -> [eventName] + const schedules = []; // { event, cron, timezone } + const userActions = []; // { event, label, from } + const routes = []; // { method, path, event } + const add = (name, e) => { events[name] = e; return name; }; + const route = (mod, topic, name) => { ((webhooks[mod] ||= {})[topic] ||= []).push(name); }; + + for (const s of def.syncs || []) { + if (!isOn(s, installation.syncs?.[s.id])) continue; + if (s.backfill?.trigger === 'userAction') { + const n = add(eventName('SYNC', s.id, 'BACKFILL'), { type: 'USER_ACTION', userActionType: 'SYNC_ACTION', title: `Sync existing ${s.label.toLowerCase()}`, ref: { kind: 'sync', id: s.id, phase: 'backfill' } }); + userActions.push({ event: n, label: events[n].title }); + } + add(eventName('SYNC', s.id, 'PAGE'), { type: 'QUEUE', ref: { kind: 'sync', id: s.id, phase: 'page' } }); + const mode = s.ongoing?.mode || 'none'; + if (mode === 'webhook') { + const n = add(eventName('SYNC', s.id, 'ONGOING'), { type: 'WEBHOOK', ref: { kind: 'sync', id: s.id, phase: 'ongoing' } }); + for (const side of [s.source, ...(s.direction === 'two_way' ? [s.target] : [])]) { + const [mod, obj] = side.split('.'); + route(mod, `${obj}.*`, n); + } + } else if (mode === 'poll') { + const n = add(eventName('SYNC', s.id, 'ONGOING'), { type: 'CRON', ref: { kind: 'sync', id: s.id, phase: 'ongoing' } }); + schedules.push({ event: n, every: s.ongoing.every || '15m' }); + } + } + + for (const w of def.workflows || []) { + if (!isOn(w, installation.workflows?.[w.id])) continue; + const t = w.trigger; + const type = { webhook: 'WEBHOOK', schedule: 'CRON', userAction: 'USER_ACTION', syncRecord: 'QUEUE' }[t.type]; + const n = add(eventName('WF', w.id), { type, ...(type === 'USER_ACTION' ? { userActionType: 'DATA_ACTION', title: t.label } : {}), ref: { kind: 'workflow', id: w.id } }); + if (t.type === 'webhook') route(t.module, t.topic, n); + if (t.type === 'schedule') schedules.push({ event: n, cron: t.cron, timezone: t.timezone || 'UTC' }); + if (t.type === 'userAction') userActions.push({ event: n, label: t.label, from: t.from || null }); + if (t.type === 'syncRecord') route('$sync', `${t.sync}:${(t.on || []).join('|')}`, n); + } + + for (const q of def.queries || []) add(eventName('Q', q.id), { type: 'ROUTE', ref: { kind: 'query', id: q.id } }); + for (const u of def.ui || []) { + if (!isOn(u, installation.ui?.[u.id])) continue; + const n = add(eventName('UI', u.id), { type: 'ROUTE', ref: { kind: 'ui', id: u.id } }); + routes.push({ method: 'GET', path: `/ui/${u.id}`, event: n }); + } + + return { events, webhooks, schedules, userActions, routes }; +} + +/** Which compiled events a delivered webhook goes to. Topic patterns may end in ".*". */ +function routeWebhook(compiled, moduleKey, topic) { + const table = compiled.webhooks[moduleKey] || {}; + const out = new Set(table[topic] || []); + for (const [pattern, names] of Object.entries(table)) { + if (pattern.endsWith('.*') && topic.startsWith(pattern.slice(0, -1))) names.forEach((n) => out.add(n)); + } + return [...out]; +} + +module.exports = { compile, routeWebhook, eventName }; diff --git a/packages/core/integrations/data-defined/config-form.js b/packages/core/integrations/data-defined/config-form.js new file mode 100644 index 000000000..d82bc9575 --- /dev/null +++ b/packages/core/integrations/data-defined/config-form.js @@ -0,0 +1,110 @@ +/** + * The configure form for a data-defined installation, generated from its definition: what GET_CONFIG_OPTIONS and + * REFRESH_CONFIG_OPTIONS return on Frigg's existing, authenticated /api/integrations/:id/config/options routes. + * + * The form's data has exactly the shape of Integration.config, so what the management UI PATCHes back is the config: + * { settings: { … }, syncs: { : { enabled } }, workflows: { : { enabled, settings: { … } } }, ui: { : { enabled } } } + * + * Option settings become enums filled by `options(path, values)` (resolveOptions underneath). A list whose inputs are + * not chosen yet comes back empty with a help line saying what to choose first; REFRESH_CONFIG_OPTIONS, called with + * the form's current values, fills it. + */ +const SETTING_SCHEMA = { + string: () => ({ type: 'string' }), + text: () => ({ type: 'string' }), + number: () => ({ type: 'number' }), + boolean: () => ({ type: 'boolean' }), + duration: () => ({ type: 'string', pattern: '^\\d+(m|h|d)$' }), + enum: (s) => ({ type: 'string', enum: s.enum }), +}; + +const dependsOn = (s) => (s && typeof s.options === 'object' ? Object.values(s.options.with || {}).map((v) => String(v).replace(/^\$\.(settings|install)\./, '')) : []); + +async function settingsSchema(settings, pathPrefix, values, options, integrationSettings) { + const properties = {}, ui = {}, required = []; + for (const [key, s] of Object.entries(settings || {})) { + let schema; + if (s.type === 'option') { + const choices = await options(`${pathPrefix}.${key}`, values, integrationSettings); + schema = choices.length + ? { type: 'string', oneOf: choices.map((c) => ({ const: String(c.value), title: c.label })) } + : { type: 'string', enum: [] }; + if (!choices.length && dependsOn(s).length) ui[key] = { 'ui:help': `Choose ${dependsOn(s).join(' and ')} first.` }; + } else schema = (SETTING_SCHEMA[s.type] || SETTING_SCHEMA.string)(s); + properties[key] = { ...schema, title: s.label, ...(s.default !== undefined ? { default: s.default } : {}) }; + if (s.help) ui[key] = { ...(ui[key] || {}), 'ui:help': s.help }; + if (s.required && s.default === undefined) required.push(key); + } + return { schema: { type: 'object', properties, ...(required.length ? { required } : {}) }, ui }; +} + +const toggle = (title, description, enabledByDefault) => ({ type: 'boolean', title, ...(description ? { description } : {}), default: enabledByDefault }); +const isOn = (item, chosen) => (typeof chosen?.enabled === 'boolean' ? chosen.enabled : (item.default || 'on') === 'on'); + +/** + * @param {object} def + * @param {object} config the installation's current config (or the form's current values on refresh) + * @param {(path: string, values: object, integrationSettings: object) => Promise>} options + * @returns {Promise<{ jsonSchema: object, uiSchema: object, data: object }>} + */ +async function buildConfigForm(def, config = {}, options = async () => []) { + const jsonSchema = { type: 'object', title: def.display?.label, properties: {} }; + const uiSchema = {}; + const data = { settings: { ...(config.settings || {}) }, syncs: {}, workflows: {}, ui: {} }; + + if (Object.keys(def.settings || {}).length) { + const s = await settingsSchema(def.settings, 'settings', config.settings || {}, options, config.settings || {}); + jsonSchema.properties.settings = { title: 'Settings', ...s.schema }; + uiSchema.settings = s.ui; + } + + if ((def.syncs || []).length) { + jsonSchema.properties.syncs = { type: 'object', title: 'Syncs', properties: {} }; + for (const s of def.syncs) { + jsonSchema.properties.syncs.properties[s.id] = { type: 'object', title: s.label, properties: { enabled: toggle('On', s.description, (s.default || 'on') === 'on') } }; + data.syncs[s.id] = { enabled: isOn(s, config.syncs?.[s.id]) }; + } + } + + if ((def.workflows || []).length) { + jsonSchema.properties.workflows = { type: 'object', title: 'Workflows', properties: {} }; + uiSchema.workflows = {}; + for (const w of def.workflows) { + const chosen = config.workflows?.[w.id] || {}; + const node = { type: 'object', title: w.label, properties: { enabled: toggle('On', w.description, (w.default || 'on') === 'on') } }; + data.workflows[w.id] = { enabled: isOn(w, chosen), settings: { ...(chosen.settings || {}) } }; + if (Object.keys(w.settings || {}).length) { + // A workflow setting's option inputs may read integration settings, which it sees as $.install. + const s = await settingsSchema(w.settings, `workflows.${w.id}.settings`, chosen.settings || {}, options, config.settings || {}); + node.properties.settings = { title: 'Settings', ...s.schema }; + uiSchema.workflows[w.id] = { settings: s.ui }; + // Settings only matter while the workflow is on: required only then. + node.dependencies = { enabled: { oneOf: [{ properties: { enabled: { const: false } } }, { properties: { enabled: { const: true } }, required: ['settings'] }] } }; + } + jsonSchema.properties.workflows.properties[w.id] = node; + } + } + + if ((def.ui || []).length) { + jsonSchema.properties.ui = { type: 'object', title: 'Cards and panels', properties: {} }; + for (const u of def.ui) { + jsonSchema.properties.ui.properties[u.id] = { type: 'object', title: u.label || u.id, properties: { enabled: toggle('Show', `${u.host} · ${u.placement}`, (u.default || 'on') === 'on') } }; + data.ui[u.id] = { enabled: isOn(u, config.ui?.[u.id]) }; + } + } + + return { jsonSchema, uiSchema, data }; +} + +/** Is anything required still missing? (NEEDS_CONFIG) */ +function missingRequired(def, config = {}) { + const missing = []; + for (const [k, s] of Object.entries(def.settings || {})) if (s.required && s.default === undefined && (config.settings?.[k] == null || config.settings[k] === '')) missing.push(`settings.${k}`); + for (const w of def.workflows || []) { + if (!isOn(w, config.workflows?.[w.id])) continue; + for (const [k, s] of Object.entries(w.settings || {})) if (s.required && s.default === undefined && (config.workflows?.[w.id]?.settings?.[k] == null || config.workflows[w.id].settings[k] === '')) missing.push(`workflows.${w.id}.settings.${k}`); + } + return missing; +} + +module.exports = { buildConfigForm, missingRequired }; diff --git a/packages/core/integrations/data-defined/data-defined-integration.js b/packages/core/integrations/data-defined/data-defined-integration.js new file mode 100644 index 000000000..bc5e2fd79 --- /dev/null +++ b/packages/core/integrations/data-defined/data-defined-integration.js @@ -0,0 +1,366 @@ +/** + * DataDefinedIntegration: one IntegrationBase subclass that runs any published definition. + * + * Frigg registers integration classes at startup (appDefinition.integrations). For data-defined integrations the app + * registers one generated class per definition id: + * + * const { defineIntegrationClass } = require('@friggframework/core/integrations/data-defined'); + * integrations: [ defineIntegrationClass({ id: 'phone-hubspot', definitions, manifests, modules: { phone: PhoneModule, hubspot: HubSpotModule } }) ] + * + * The class's static Definition (name, modules) is what the infrastructure needs, and it is the same for every + * version, so publishing a new version deploys nothing. Each installation loads the exact version it pins + * (record.config.definition.version) when it is hydrated, compiles it with that installation's choices, and gets an + * ordinary `this.events` map. Everything after that is the existing Frigg machinery: the event dispatcher, the + * queue worker, the scheduler and the mapping repository. + * + * Ports used: definitions.get(id, version) -> definition doc (immutable; cache freely) + * runCode(ref, args) -> registered code steps (optional) + */ +const { compile, routeWebhook } = require('./compile'); +const { makePorts, runWorkflow, runQuery, renderSurface, runLifecycle, resolveOptions } = require('./runtime'); +const { eventsIn, handshakeFor, modulesFor } = require('./webhooks'); +const { buildConfigForm, missingRequired } = require('./config-form'); + +/** + * Shape A — one class per definition id (defineIntegrationClass): the class name and modules come from the + * definition. Publishing a new VERSION deploys nothing; a new definition ID is a new class (a deploy, and its own + * functions and queue). Right for a product team with a handful of integrations. + * + * Shape B — one class for every definition (defineSharedIntegrationClass): registered once as "data-defined", its + * static modules are every API module the app has installed, and each installation names its definition id and + * version in config. A new integration is a row, not a deploy, and every installation shares one set of functions + * and one queue. Right for a multi-tenant product, where definitions accumulate without bound. + */ +function defineIntegrationClass({ id, definitions, manifests, modules, runCode, IntegrationBase, lookup, runs }) { + const latest = definitions.latest(id); + if (!latest || typeof latest.then === 'function') throw new Error(`defineIntegrationClass needs the definitions of "${id}" synchronously at registration (bundled files); a database store goes with defineSharedIntegrationClass`); + return makeClass({ + name: latest.id, defaultId: id, definitions, manifests, runCode, IntegrationBase, lookup, runs, + modules: Object.fromEntries(Object.keys(latest.modules).map((key) => [key, { definition: modules[key] }])), + display: { label: latest.display.label, description: latest.display.description, category: latest.display.category }, + version: latest.version, + }); +} + +function defineSharedIntegrationClass({ name = 'data-defined', definitions, manifests, modules, runCode, IntegrationBase, lookup, runs }) { + return makeClass({ + name, defaultId: null, definitions, manifests, runCode, IntegrationBase, lookup, runs, + modules: Object.fromEntries(Object.entries(modules).map(([key, definition]) => [key, { definition }])), + display: { label: 'Data-defined integration', description: 'Runs any published integration definition.', category: 'Custom' }, + version: '0.1.0', + }); +} + +/** + * App-level webhooks (one URL for every account: HubSpot, Stripe Connect, GitHub Apps) arrive with no integration id. + * The default lookup finds the installations of THIS class connected to that account, using Frigg's commands: the + * entity reverse lookup, then each candidate's record, because one account can be used by integrations of other + * types too, and the worker would hydrate those as their own class. + */ +function friggLookup(typeName) { + return async (externalId, moduleName) => { + const { createIntegrationCommands } = require('../../application/commands/integration-commands'); + const commands = createIntegrationCommands(); + const ids = await commands.listIntegrationsByEntityExternalId(externalId, moduleName); + const out = []; + for (const integrationId of ids) { + const found = await commands.loadIntegrationContextById(integrationId); + if (found?.context?.record?.config?.type === typeName) out.push(integrationId); + } + return out; + }; +} + +function makeClass({ name, defaultId, definitions, manifests, runCode, IntegrationBase, modules, display, version, lookup, runs }) { + const Base = IntegrationBase || require('../integration-base').IntegrationBase; + const id = name; + const findInstallations = lookup || friggLookup(name); + // Where workflow runs are kept: the WorkflowRun table by default (loaded lazily, so tests and tools without a + // database can pass their own store or `false` to keep none). + let runStore = runs; + const runRepository = () => (runStore === undefined ? (runStore = require('./repositories/workflow-run-repository').createWorkflowRunRepository()) : runStore); + const publicBase = () => `${process.env.FRIGG_PUBLIC_URL || ''}/api/${id}-integration/webhooks`; + + class DataDefinedIntegration extends Base { + static Definition = { name, version, modules, webhooks: { enabled: true }, display }; // version is display only; installations pin their own + + /** How the routers find data-defined classes and the store and manifests they run on. */ + static DataDefined = { definitions, manifests, shared: !defaultId, runs: () => runRepository() }; + + /** + * Frigg's use cases construct an integration from its record and call initialize() before anything else + * (create, update, delete, webhooks, user actions, config options). So this is where the pinned definition is + * loaded: every real entry point hydrates through here. A dry instance (no record, e.g. an app-level webhook + * before its installation is known) has no pin and loads nothing. + */ + async initialize() { + if (this.config?.definition || defaultId) await this.loadDefinition(); + return super.initialize(); + } + + async loadDefinition() { + let pin = this.config?.definition || (defaultId ? { id: defaultId } : null); + if (!pin?.id) throw new Error('This installation does not name a definition (config.definition.id)'); + if (!pin.version) { + // Installed by id alone: pin the latest published version now (onCreate persists the pin). + const latest = await definitions.latest(pin.id); + if (!latest) throw new Error(`No published version of "${pin.id}"`); + pin = { id: pin.id, version: latest.version, checksum: await definitions.checksumOf?.(pin.id, latest.version) || undefined }; + if (this.config?.definition) this.config = { ...this.config, definition: pin }; + } + this.def = await definitions.get(pin.id, pin.version, { checksum: pin.checksum }); + if (!this.def) throw new Error(`Definition ${pin.id}@${pin.version} is not published`); + this.compiled = compile(this.def, this.config || {}); + this.ports = this.portsFor(this.def); + // Replace, not merge: a workflow switched off since the last compile must lose its event. + for (const n of this.compiledEventNames || []) delete this.events[n]; + this.compiledEventNames = Object.keys(this.compiled.events); + for (const [n, e] of Object.entries(this.compiled.events)) { + this.events[n] = { ...e, handler: (payload) => this.dispatch(e.ref, payload) }; + } + this.registerEventHandlers(); + return this.def; + } + + /** Ports for a definition: Api calls through the manifest, and links/dedupe over IntegrationMapping. */ + portsFor(def) { + return makePorts({ + def, manifests, runCode, + // The shared class registers modules by module name; a definition may name them by its own keys. + callApi: (moduleKey, method, args) => this.moduleFor(moduleKey).api[method](args), + store: { + get: async (key) => (await this.getMapping(key))?.mapping || null, + put: (key, value) => this.upsertMapping(key, value), + claim: async (key, value) => { try { await this.createMapping(key, value); return true; } catch (e) { if (e.name === 'MappingAlreadyExistsError') return false; throw e; } }, + release: (key) => this.deleteMapping(key), + }, + }); + } + + moduleFor(key) { + const m = this[key] || this[this.def?.modules?.[key]?.module]; + if (!m?.api) throw new Error(`Module "${key}" is not connected to this installation`); + return m; + } + + async dispatch(ref, payload = {}) { + const msg = payload.data || payload; + // A message queued under an older version runs on that version (an upgrade drains, it never re-interprets). + const def = msg.definitionVersion && msg.definitionVersion !== this.def.version + ? await definitions.get(this.def.id, msg.definitionVersion) || this.def + : this.def; + const installation = this.installation(); + if (ref.kind === 'workflow') return this.recordRun(await runWorkflow({ def, installation, workflowId: ref.id, event: msg.body ?? msg, eventId: msg.eventId, record: msg.record, ports: this.ports, manifests })); + if (ref.kind === 'query') return runQuery({ def, installation, queryId: ref.id, record: msg.record, ports: this.ports }); + if (ref.kind === 'ui') return renderSurface({ def, installation, surfaceId: ref.id, record: payload.req?.body?.record || msg.record, ports: this.ports }); + if (ref.kind === 'sync') return this.runSync(ref, msg); + throw new Error(`Unknown event ref ${JSON.stringify(ref)}`); + } + + /** + * Receipt (no database): answer a provider handshake synchronously (Slack url_verification, Asana + * X-Hook-Secret), otherwise queue the delivery as Frigg always does. The URL names the module (?module=). + */ + async onWebhookReceived({ req, res }) { + const manifest = manifests[req.query?.module]; + const h = manifest && handshakeFor(manifest, { body: req.body, headers: req.headers, query: req.query }); + if (h) { + for (const [k, v] of Object.entries(h.headers || {})) res.set(k, String(v)); + return typeof h.body === 'object' ? res.status(h.status).json(h.body) : res.status(h.status).send(h.body); + } + return super.onWebhookReceived({ req, res }); + } + + /** + * Worker side of a raw delivery. Without an installation (an app-level webhook), find the installations + * connected to each account the delivery is about and queue it for each. With one: verify it, split it into + * module events, and queue one message per enabled workflow listening for each event. + */ + async onWebhook({ data = {} }) { + if (data.module && data.topic) return this.routeEvents(data.module, [{ topic: data.topic, event: data.body, eventId: data.eventId || data.body?.id || null }]); + const delivery = { body: data.body, headers: data.headers || {}, query: data.query || {} }; + if (!this.def) return this.fanOutAppWebhook(delivery); + + const keys = modulesFor(this.def, manifests, delivery); + if (keys.length !== 1) { this.logWebhook('webhook.unrouted', { module: delivery.query.module || null, candidates: keys }); return { routed: 0 }; } + const key = keys[0]; + const manifest = manifests[this.def.modules[key].module]; + if (manifest.webhook?.verify) { + let ok = false; + try { ok = await this.moduleFor(key).api[manifest.webhook.verify]({ headers: delivery.headers, body: delivery.body, query: delivery.query }); } catch { ok = false; } + if (ok !== true) { this.logWebhook('webhook.rejected', { module: key }); return { routed: 0, rejected: true }; } + } + const own = this.accountOf(key); + const events = eventsIn(manifest, delivery).filter((e) => manifest.webhook?.scope !== 'app' || !own || !e.account || String(e.account) === String(own)); + return this.routeEvents(key, events); + } + + async routeEvents(moduleKey, events) { + let routed = 0; + for (const e of events) { + for (const n of routeWebhook(this.compiled, moduleKey, e.topic)) { + await this.queueWebhook({ event: n, integrationId: this.id, module: moduleKey, topic: e.topic, body: e.event, eventId: e.eventId, definitionVersion: this.def.version }); + routed += 1; + } + } + return { routed }; + } + + async fanOutAppWebhook(delivery) { + const moduleName = delivery.query.module; + const manifest = manifests[moduleName]; + if (!manifest?.webhook?.account) { this.logWebhook('webhook.unrouted', { module: moduleName || null, reason: 'no installation and no account to look it up by' }); return { queued: 0 }; } + const accounts = new Set(eventsIn(manifest, delivery).map((e) => e.account).filter(Boolean)); + let queued = 0; + for (const account of accounts) { + for (const integrationId of await findInstallations(account, moduleName)) { + await this.queueWebhook({ integrationId, body: delivery.body, headers: delivery.headers, query: delivery.query }); + queued += 1; + } + } + return { queued }; + } + + /** The connected account's id for a module (its entity externalId): what app-level events are matched on. */ + accountOf(key) { + const m = this[key] || this[this.def?.modules?.[key]?.module] || this.modules?.[this.def?.modules?.[key]?.module]; + return m?.entity?.externalId ?? null; + } + + logWebhook(eventName, fields) { + try { require('../../logs').getLogger('frigg.data-defined').info('Webhook not routed', { eventName: `frigg.data-defined.${eventName}`, integrationId: this.id || null, ...fields }); } catch { /* logging must never break delivery */ } + } + + async onCreate(params) { + if (!this.def) await this.loadDefinition(); + const out = await runLifecycle({ def: this.def, installation: this.installation(), hook: 'onInstall', ports: this.ports }); + // Every onInstall step with an id keeps its result, so onUninstall can undo exactly what install did + // ($.install.runtime.installed.). Saved even when empty, so a re-install replaces stale ids. + const installed = Object.fromEntries((this.def.lifecycle?.onInstall || []).filter((st) => st.id).map((st) => [st.id, out.steps?.[st.id] ?? null])); + await this.patchConfig({ ...(this.config?.definition ? { definition: this.config.definition } : {}), runtime: { ...(this.config?.runtime || {}), installed } }); + return super.onCreate(params); + } + + async onUpdate(params) { + await super.onUpdate(params); // merges config (enabled workflows, settings, mapping overrides) + return this.loadDefinition(); // recompile: enabling a workflow adds its event, disabling removes it + } + + async onDelete(params) { + if (!this.def) await this.loadDefinition(); + await runLifecycle({ def: this.def, installation: this.installation(), hook: 'onUninstall', ports: this.ports }); + return super.onDelete(params); + } + + /** The configure form (GET /api/integrations/:id/config/options), generated from the definition. */ + async getConfigOptions() { + return buildConfigForm(this.def, this.config || {}, (path, values, integrationSettings) => this.optionsFor(path, values, integrationSettings)); + } + + /** The same form re-filled from the values chosen so far (POST …/config/options/refresh): dependent dropdowns. */ + async refreshConfigOptions(params = {}) { + const values = params.config || params.formData || params.data || params; + return buildConfigForm(this.def, { ...(this.config || {}), ...values }, (path, v, integrationSettings) => this.optionsFor(path, v, integrationSettings)); + } + + async optionsFor(path, values, integrationSettings) { + try { return await resolveOptions({ def: this.def, manifests, ports: this.ports, path, values, integrationSettings }); } catch { return []; } + } + + /** Required settings come from the definition; checked without calling any API (the base class renders the form). */ + async validateConfig() { + return missingRequired(this.def, this.config || {}).length > 0; + } + + /** The input form for a button workflow (POST …/actions/:id/options): the inputs of the UI buttons that run it. */ + async getActionOptions(actionId) { + const ref = this.compiled?.events?.[actionId]?.ref; + const inputs = {}; + const visit = (node) => { if (!node) return; const a = node.props?.action; if (a?.type === 'userAction' && a.workflow === ref?.id) Object.assign(inputs, a.input || {}); (node.children || []).forEach(visit); }; + if (ref?.kind === 'workflow') (this.def.ui || []).forEach((u) => visit(u.view)); + const properties = Object.fromEntries(Object.entries(inputs).map(([k, f]) => [k, { type: f.type === 'number' ? 'number' : 'string', title: f.label || k }])); + return { jsonSchema: Object.keys(properties).length ? { type: 'object', properties: { input: { type: 'object', properties } } } : {}, uiSchema: {} }; + } + + installation() { + const base = publicBase(); + const webhookUrls = Object.fromEntries(Object.entries(this.def?.modules || {}).map(([key, m]) => [key, manifests[m.module]?.webhook?.scope === 'app' + ? `${base}?module=${encodeURIComponent(m.module)}` + : `${base}/${this.id}?module=${encodeURIComponent(m.module)}`])); + return { ...(this.config || {}), integrationId: this.id, accounts: this.accounts(), webhookUrl: `${base}/${this.id}`, webhookUrls }; + } + + /** $.install.accounts: per module key, the account fields its manifest declares, read from the module's Api + * (where apiPropertiesToPersist land after auth). An undeclared property is never exposed. */ + accounts() { + return Object.fromEntries(Object.entries(this.def?.modules || {}).map(([key, m]) => { + const fields = manifests[m.module]?.account?.fields || []; + let api = {}; + try { api = this.moduleFor(key).api || {}; } catch { api = {}; } + return [key, Object.fromEntries(fields.map((f) => [f.name, api[f.name] ?? null]))]; + })); + } + + /** + * Move this installation to another published version (default: the latest). With apply false, only the plan: + * what a person must supply first and what will happen. Applying refuses while anything is needed, and when + * install steps changed, undoes the old ones before running the new ones (it never leaves the old webhooks). + */ + async upgradeTo(toVersion, { apply = false } = {}) { + const { PlanInstallationUpgrade } = require('./use-cases/definition-use-cases'); + const plan = await new PlanInstallationUpgrade({ catalog: definitions }).execute({ config: this.config, toVersion }); + if (!apply || !plan.allowed) return { ...plan, applied: false }; + const reinstall = plan.actions.some((a) => a.kind === 'reinstall'); + let config = plan.config; + if (reinstall) { + await runLifecycle({ def: this.def, installation: this.installation(), hook: 'onUninstall', ports: this.ports }); + const next = await definitions.get(plan.config.definition.id, plan.config.definition.version); + const out = await runLifecycle({ def: next, installation: { ...this.installation(), ...config }, hook: 'onInstall', ports: this.portsFor(next) }); + config = { ...config, runtime: { installed: Object.fromEntries((next.lifecycle?.onInstall || []).filter((st) => st.id).map((st) => [st.id, out.steps?.[st.id] ?? null])) } }; + } + await this.patchConfig(config); + this.config = { ...this.config, ...config }; + await this.loadDefinition(); + return { ...plan, config, applied: true }; + } + + async recordRun(run) { + this.telemetry?.count?.('workflow.run', 1, { workflow: run.workflowId, status: run.status }); + // Run history must never decide the run's outcome: a failed write is logged and the run stands. + try { + const store = runRepository(); + if (store) await store.createRun({ ...run, integrationId: this.id, definitionId: run.definition?.id, definitionVersion: run.definition?.version, triggerType: run.trigger?.type, eventId: run.trigger?.eventId }); + } catch (e) { + try { require('../../logs').getLogger('frigg.data-defined').warn('Workflow run not recorded', { eventName: 'frigg.data-defined.run.not_recorded', integrationId: this.id, workflowId: run.workflowId, cause: e?.message }); } catch { /* ignore */ } + } + if (run.status === 'failed') { const e = new Error(run.error); e.run = run; throw e; } // the queue retries + return run; + } + + async runSync(ref) { + // The sync engine (backfill pages, ongoing deltas, two-way master rules) reuses Frigg's existing SYNC_PAGE + // pattern; it is specified in README.md and not built yet. + throw new Error(`Sync ${ref.id} (${ref.phase}) is not implemented yet`); + } + } + Object.defineProperty(DataDefinedIntegration, 'name', { value: `DataDefined_${id.replace(/-/g, '_')}` }); + return DataDefinedIntegration; +} + +/** In-memory definition store for tests and local dev. Production: the IntegrationDefinitionVersion table. */ +function memoryDefinitions(docs) { + const byKey = new Map(docs.map((d) => [`${d.id}@${d.version}`, d])); + const cmp = (a, b) => a.split('.').map(Number).reduce((r, n, i) => r || n - Number(b.split('.')[i]), 0); + return { + get: async (defId, version) => byKey.get(`${defId}@${version}`) || null, + checksumOf: async (defId, version) => { const doc = byKey.get(`${defId}@${version}`); return doc ? require('./canonical').checksum(doc) : null; }, + list: async () => { + const latestById = new Map(); + for (const doc of docs) if (!latestById.has(doc.id) || cmp(doc.version, latestById.get(doc.id).version) > 0) latestById.set(doc.id, doc); + return [...latestById.values()].map((doc) => ({ id: doc.id, version: doc.version, checksum: require('./canonical').checksum(doc), display: doc.display, modules: Object.fromEntries(Object.entries(doc.modules || {}).map(([k, m]) => [k, m.module])), counts: { syncs: (doc.syncs || []).length, links: (doc.links || []).length, workflows: (doc.workflows || []).length, ui: (doc.ui || []).length } })); + }, + latest: (id) => docs.filter((d) => d.id === id).sort((a, b) => cmp(b.version, a.version))[0] || null, + }; +} + +module.exports = { defineIntegrationClass, defineSharedIntegrationClass, memoryDefinitions }; diff --git a/packages/core/integrations/data-defined/examples/acme-erp-quickbooks.definition.json b/packages/core/integrations/data-defined/examples/acme-erp-quickbooks.definition.json new file mode 100644 index 000000000..c6dc6c342 --- /dev/null +++ b/packages/core/integrations/data-defined/examples/acme-erp-quickbooks.definition.json @@ -0,0 +1,467 @@ +{ + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "acme-erp-quickbooks", + "version": "1.3.0", + "display": { + "label": "Acme ERP \u2194 QuickBooks Online", + "description": "Finished jobs become QuickBooks invoices with every line item, customers stay matched both ways, and payments recorded in QuickBooks mark the job paid in Acme ERP.", + "category": "Accounting", + "tags": [ + "field-service", + "invoicing", + "accounting" + ] + }, + "modules": { + "acmeerp": { + "module": "acmeerp", + "version": "^0.9" + }, + "quickbooks": { + "module": "quickbooks", + "version": "^2.0" + } + }, + "settings": { + "fallbackItem": { + "type": "option", + "options": "quickbooks.item", + "label": "Product/service for line items with no matching QuickBooks item", + "required": true + }, + "terms": { + "type": "enum", + "label": "Invoice terms", + "enum": [ + "Due on receipt", + "Net 15", + "Net 30" + ], + "default": "Due on receipt" + } + }, + "syncs": [ + { + "id": "customers", + "label": "Customers", + "description": "One customer in both systems. Contact details follow whichever side changed last; QuickBooks owns the billing name.", + "source": "acmeerp.customer", + "target": "quickbooks.customer", + "direction": "two_way", + "identity": { + "match": [ + { + "source": "email", + "target": "PrimaryEmailAddr.Address", + "transform": "lowercase", + "key": "email" + }, + { + "source": "mobile_number", + "target": "Mobile.FreeFormNumber", + "transform": "to_e164_phone", + "key": "phone" + } + ], + "onAmbiguous": "skip" + }, + "mapping": [ + { + "source": "display_name", + "target": "DisplayName", + "master": "target" + }, + { + "source": "first_name", + "target": "GivenName", + "master": "newest" + }, + { + "source": "last_name", + "target": "FamilyName", + "master": "newest" + }, + { + "source": "company_name", + "target": "CompanyName", + "master": "newest" + }, + { + "source": "email", + "target": "PrimaryEmailAddr.Address", + "transform": "lowercase", + "master": "newest" + }, + { + "source": "mobile_number", + "target": "Mobile.FreeFormNumber", + "transform": "to_e164_phone", + "master": "newest" + }, + { + "source": "address.street", + "target": "BillAddr.Line1", + "master": "source" + }, + { + "source": "address.city", + "target": "BillAddr.City", + "master": "source" + }, + { + "source": "address.state", + "target": "BillAddr.CountrySubDivisionCode", + "master": "source" + }, + { + "source": "address.zip", + "target": "BillAddr.PostalCode", + "master": "source" + } + ], + "backfill": { + "trigger": "userAction", + "pageSize": 100 + }, + "ongoing": { + "mode": "webhook" + }, + "default": "on" + } + ], + "workflows": [ + { + "id": "job-invoice", + "label": "Invoice finished jobs", + "purpose": "write", + "default": "on", + "settings": { + "emailInvoice": { + "type": "boolean", + "label": "Email the invoice from QuickBooks", + "default": false + } + }, + "trigger": { + "type": "webhook", + "module": "acmeerp", + "topic": "job.finished", + "filter": "$.record.total_amount > 0 && $.record.external_invoice_id == null" + }, + "steps": [ + { + "id": "customer", + "use": "links.find", + "with": { + "sync": "customers", + "from": "acmeerp", + "id": "$.record.customer_id" + } + }, + { + "use": "stop", + "when": "$.steps.customer == null", + "with": { + "reason": "Customer is not in QuickBooks yet; the queue will retry after the customer sync" + } + }, + { + "id": "lineItems", + "use": "acmeerp.job.lineItems", + "with": { + "jobId": "$.record.id" + } + }, + { + "id": "lines", + "use": "code", + "with": { + "ref": "@acme/frigg-steps#quickbooksInvoiceLines@1.0.0", + "with": { + "lineItems": "$.steps.lineItems", + "fallbackItemId": "$.install.fallbackItem", + "taxAmount": "$.record.tax_amount" + } + } + }, + { + "id": "memo", + "use": "template", + "with": { + "id": "invoice-memo" + } + }, + { + "id": "invoice", + "use": "quickbooks.invoice.create", + "with": { + "invoice": { + "CustomerRef": { + "value": "$.steps.customer.quickbooks" + }, + "DocNumber": "TW-{{record.job_number}}", + "TxnDate": "$.record.finished_at", + "SalesTermRef": { + "name": "$.install.terms" + }, + "CustomerMemo": { + "value": "$.steps.memo" + }, + "PrivateNote": "Acme ERP job {{record.id}}", + "Line": "$.steps.lines.lines" + } + } + }, + { + "use": "acmeerp.job.update", + "with": { + "id": "$.record.id", + "job": { + "status": "invoiced", + "external_invoice_id": "$.steps.invoice.Id" + } + } + }, + { + "use": "quickbooks.invoice.send", + "when": "$.settings.emailInvoice == true", + "with": { + "id": "$.steps.invoice.Id" + }, + "onError": "continue" + } + ], + "retry": { + "max": 5, + "backoff": "exponential" + } + }, + { + "id": "invoice-paid", + "label": "Mark jobs paid when QuickBooks records a payment", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "quickbooks", + "topic": "payment.created" + }, + "steps": [ + { + "id": "payment", + "use": "quickbooks.payment.get", + "with": { + "id": "$.record.id" + } + }, + { + "id": "apply", + "use": "foreach", + "with": { + "over": "$.steps.payment.Line", + "max": 20 + }, + "do": [ + { + "id": "jobs", + "use": "acmeerp.job.search", + "when": "$.item.LinkedTxn[0].TxnType == 'Invoice'", + "with": { + "external_invoice_id": "$.item.LinkedTxn[0].TxnId" + } + }, + { + "use": "acmeerp.job.markPaid", + "when": "$.steps.jobs[0] exists", + "with": { + "id": "$.steps.jobs[0].id", + "amount": "{{item.Amount | to_cents}}", + "paid_at": "$.steps.payment.TxnDate", + "method": "quickbooks", + "reference": "QBO payment {{steps.payment.Id}}" + } + } + ] + } + ] + }, + { + "id": "invoice-note", + "label": "Note the invoice on the job", + "purpose": "write", + "default": "off", + "trigger": { + "type": "syncRecord", + "sync": "customers", + "on": [ + "linked" + ] + }, + "steps": [ + { + "id": "open", + "use": "acmeerp.job.search", + "with": { + "customer_id": "$.record.id" + } + }, + { + "id": "notes", + "use": "foreach", + "with": { + "over": "$.steps.open", + "max": 25 + }, + "do": [ + { + "use": "acmeerp.job.addNote", + "when": "$.item.status == 'finished'", + "with": { + "jobId": "$.item.id", + "body": "Customer is now linked to QuickBooks; finished jobs will be invoiced automatically." + } + } + ] + } + ] + } + ], + "queries": [ + { + "id": "job-invoice-status", + "description": "The QuickBooks invoice for this job.", + "steps": [ + { + "use": "stop", + "when": "$.record.external_invoice_id == null", + "with": { + "reason": "Not invoiced" + } + }, + { + "id": "invoice", + "use": "quickbooks.invoice.get", + "with": { + "id": "$.record.external_invoice_id" + } + } + ], + "returns": { + "invoice": "$.steps.invoice", + "realmId": "$.install.accounts.quickbooks.realmId" + } + } + ], + "ui": [ + { + "id": "invoice-panel", + "label": "QuickBooks invoice", + "host": "acmeerp", + "placement": "job.sidebar", + "objects": [ + "job" + ], + "data": { + "query": "job-invoice-status" + }, + "view": { + "component": "Stack", + "children": [ + { + "component": "Heading", + "props": { + "text": "QuickBooks" + } + }, + { + "component": "Empty", + "when": "!($.data.invoice exists)", + "props": { + "text": "Invoiced automatically when the job is finished." + } + }, + { + "component": "KeyValue", + "when": "$.data.invoice exists", + "props": { + "items": [ + { + "label": "Invoice", + "value": "{{data.invoice.DocNumber}}" + }, + { + "label": "Total", + "value": "${{data.invoice.TotalAmt}}" + }, + { + "label": "Balance", + "value": "${{data.invoice.Balance}}" + }, + { + "label": "Due", + "value": "{{data.invoice.DueDate | date}}" + } + ] + } + }, + { + "component": "Badge", + "when": "$.data.invoice.Balance == 0", + "props": { + "text": "Paid", + "tone": "success" + } + }, + { + "component": "Badge", + "when": "$.data.invoice.Balance > 0", + "props": { + "text": "Open", + "tone": "warning" + } + }, + { + "component": "Button", + "when": "$.data.invoice exists", + "props": { + "label": "Open in QuickBooks", + "action": { + "type": "open_url", + "url": "https://app.qbo.intuit.com/app/invoice?txnId={{data.invoice.Id}}" + } + } + } + ] + } + } + ], + "templates": { + "invoice-memo": "Thanks for choosing Acme! Job #{{record.job_number}}: {{record.description | truncate: 200}}" + }, + "lifecycle": { + "onInstall": [ + { + "id": "hook", + "use": "acmeerp.webhook.create", + "with": { + "url": "$.install.webhookUrls.acmeerp", + "events": [ + "job.finished", + "job.scheduled", + "customer.created", + "customer.updated" + ] + } + } + ], + "onUninstall": [ + { + "use": "acmeerp.webhook.delete", + "with": { + "id": "$.install.runtime.installed.hook.id" + } + } + ] + } +} diff --git a/packages/core/integrations/data-defined/examples/asana-slack.definition.json b/packages/core/integrations/data-defined/examples/asana-slack.definition.json new file mode 100644 index 000000000..e12e9f01f --- /dev/null +++ b/packages/core/integrations/data-defined/examples/asana-slack.definition.json @@ -0,0 +1,220 @@ +{ + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "asana-slack", + "version": "0.2.0", + "display": { + "label": "Asana \u2192 Slack nudges", + "description": "Each afternoon, everyone gets a DM listing their Asana tasks due tomorrow; completed tasks in a project are announced in its channel.", + "category": "Notifications", + "tags": [ + "tasks", + "slack", + "project-management" + ] + }, + "modules": { + "asana": { + "module": "asana", + "version": "^1.5" + }, + "slack": { + "module": "slack", + "version": "^2.0" + } + }, + "settings": { + "workspace": { + "type": "option", + "options": "asana.workspace", + "label": "Asana workspace", + "required": true + }, + "project": { + "type": "option", + "options": { + "source": "asana.project", + "with": { + "workspaceGid": "$.settings.workspace" + } + }, + "label": "Project to announce completed tasks from", + "required": true + } + }, + "workflows": [ + { + "id": "due-tomorrow-digest", + "label": "DM each assignee their tasks due tomorrow", + "purpose": "notify", + "default": "on", + "settings": { + "unassignedChannel": { + "type": "option", + "options": "slack.channel", + "label": "Post unassigned tasks to" + } + }, + "trigger": { + "type": "schedule", + "cron": "0 16 * * 1-5", + "timezone": "America/New_York" + }, + "steps": [ + { + "id": "tasks", + "use": "asana.task.search", + "with": { + "workspaceGid": "$.install.workspace", + "due_on": "+1d", + "completed": false, + "limit": 100 + } + }, + { + "use": "stop", + "when": "$.steps.tasks[0] == null", + "with": { + "reason": "Nothing due tomorrow" + } + }, + { + "id": "groups", + "use": "code", + "with": { + "ref": "@acme/asana-steps#groupByAssignee@1.0.0", + "with": { + "tasks": "$.steps.tasks" + } + } + }, + { + "id": "dms", + "use": "foreach", + "with": { + "over": "$.steps.groups.assigned", + "max": 100 + }, + "do": [ + { + "id": "slackUser", + "use": "slack.user.lookupByEmail", + "with": { + "email": "$.item.email" + }, + "onError": "continue" + }, + { + "id": "text", + "use": "template", + "with": { + "id": "assignee-digest" + } + }, + { + "use": "slack.message.post", + "when": "$.steps.slackUser.id exists", + "with": { + "channel": "$.steps.slackUser.id", + "text": "$.steps.text", + "unfurl_links": false + } + } + ] + }, + { + "id": "unassigned", + "use": "template", + "with": { + "id": "unassigned-digest" + } + }, + { + "use": "slack.message.post", + "when": "$.settings.unassignedChannel exists && $.steps.groups.unassigned[0] exists", + "with": { + "channel": "$.settings.unassignedChannel", + "text": "$.steps.unassigned" + } + } + ] + }, + { + "id": "completed-in-project", + "label": "Announce completed tasks", + "purpose": "notify", + "default": "on", + "settings": { + "channel": { + "type": "option", + "options": "slack.channel", + "label": "Announce in", + "required": true + } + }, + "trigger": { + "type": "webhook", + "module": "asana", + "topic": "task.changed", + "filter": "$.event.events[0].change.field == 'completed' && $.event.events[0].change.new_value == true" + }, + "steps": [ + { + "id": "task", + "use": "asana.task.get", + "with": { + "taskGid": "$.record.gid", + "opt_fields": "name,completed,completed_by.name,assignee.name,permalink_url" + } + }, + { + "use": "stop", + "when": "$.steps.task.completed != true", + "with": { + "reason": "Reopened before we got to it" + } + }, + { + "use": "slack.message.post", + "with": { + "channel": "$.settings.channel", + "text": ":white_check_mark: {{steps.task.completed_by.name | default: 'Someone'}} completed <{{steps.task.permalink_url}}|{{steps.task.name}}>" + } + } + ] + } + ], + "templates": { + "assignee-digest": ":spiral_calendar_pad: Hi {{item.firstName | default: item.name}}, you have {{item.count}} Asana task(s) due tomorrow:\n{{item.lines}}", + "unassigned-digest": ":warning: {{steps.groups.unassignedCount}} task(s) due tomorrow have no assignee:\n{{steps.groups.unassignedLines}}" + }, + "lifecycle": { + "onInstall": [ + { + "id": "hook", + "use": "asana.webhook.create", + "with": { + "resource": "$.install.project", + "target": "$.install.webhookUrls.asana", + "filters": [ + { + "resource_type": "task", + "action": "changed", + "fields": [ + "completed" + ] + } + ] + } + } + ], + "onUninstall": [ + { + "use": "asana.webhook.delete", + "with": { + "webhookGid": "$.install.runtime.installed.hook.gid" + } + } + ] + } +} diff --git a/packages/core/integrations/data-defined/examples/calendly-salesforce.definition.json b/packages/core/integrations/data-defined/examples/calendly-salesforce.definition.json new file mode 100644 index 000000000..2f5f0ac75 --- /dev/null +++ b/packages/core/integrations/data-defined/examples/calendly-salesforce.definition.json @@ -0,0 +1,459 @@ +{ + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "calendly-salesforce", + "version": "2.0.0", + "display": { + "label": "Calendly \u2192 Salesforce", + "description": "Booked meetings land in Salesforce as Events on the right Lead or Contact (creating a Lead when the invitee is new), cancellations update the Event, and the record page shows upcoming meetings.", + "category": "Scheduling", + "tags": [ + "meetings", + "crm", + "lead-capture" + ] + }, + "modules": { + "calendly": { + "module": "calendly", + "version": "^1.0" + }, + "salesforce": { + "module": "salesforce", + "version": "^3.0" + } + }, + "settings": { + "leadOwner": { + "type": "option", + "options": "salesforce.user", + "label": "Owner of new Leads and Events", + "required": true + }, + "leadSource": { + "type": "string", + "label": "Lead Source for new Leads", + "default": "Calendly" + } + }, + "links": [ + { + "id": "people", + "label": "Invitees and Leads", + "description": "Which Salesforce Lead or Contact each invitee booked as. Written by the booking workflow.", + "between": [ + "calendly.invitee", + "salesforce.lead" + ], + "identity": { + "match": [ + { + "key": "email", + "transform": "lowercase" + } + ] + } + }, + { + "id": "meetings", + "label": "Meetings and Events", + "description": "Each Calendly meeting is one Salesforce Event.", + "between": [ + "calendly.scheduledEvent", + "salesforce.event" + ], + "identity": { + "match": [ + { + "key": "eventUri" + } + ] + } + } + ], + "workflows": [ + { + "id": "booking-to-event", + "label": "Log bookings as Events on the Lead or Contact", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "calendly", + "topic": "invitee.created" + }, + "steps": [ + { + "id": "link", + "use": "links.find", + "with": { + "key": { + "email": "$.record.email" + }, + "link": "people" + } + }, + { + "id": "found", + "use": "salesforce.person.searchByEmail", + "when": "$.steps.link == null", + "with": { + "email": "$.record.email" + } + }, + { + "id": "who", + "use": "branch", + "cases": [ + { + "when": "$.steps.link != null", + "do": [ + { + "use": "set", + "with": { + "whoId": "$.steps.link.salesforce" + } + } + ] + }, + { + "when": "$.steps.found.contacts[0] exists", + "do": [ + { + "use": "set", + "with": { + "whoId": "$.steps.found.contacts[0].Id" + } + } + ] + }, + { + "when": "$.steps.found.leads[0] exists", + "do": [ + { + "use": "set", + "with": { + "whoId": "$.steps.found.leads[0].Id" + } + }, + { + "use": "links.upsert", + "with": { + "ids": { + "calendly": "$.record.uri", + "salesforce": "$.steps.found.leads[0].Id" + }, + "keys": { + "email": "$.record.email" + }, + "link": "people" + } + } + ] + } + ], + "else": [ + { + "id": "leadFields", + "use": "map", + "with": { + "record": "$.record", + "mapping": [ + { + "source": "name", + "target": "FirstName", + "transform": "split_full_name", + "options": { + "part": "first" + } + }, + { + "source": "name", + "target": "LastName", + "transform": "split_full_name", + "options": { + "part": "last" + } + }, + { + "source": "first_name", + "target": "FirstName" + }, + { + "source": "last_name", + "target": "LastName" + }, + { + "source": "email", + "target": "Email", + "transform": "lowercase" + }, + { + "source": "text_reminder_number", + "target": "Phone", + "transform": "to_e164_phone" + }, + { + "source": "tracking.utm_source", + "target": "LeadSource" + } + ] + } + }, + { + "id": "leadNote", + "use": "template", + "with": { + "id": "lead-description" + } + }, + { + "id": "lead", + "use": "salesforce.lead.create", + "with": { + "fields": { + "FirstName": "$.steps.leadFields.FirstName", + "LastName": "{{steps.leadFields.LastName | default: '(unknown)'}}", + "Email": "$.steps.leadFields.Email", + "Phone": "$.steps.leadFields.Phone", + "Company": "{{record.questions_and_answers[0].answer | default: '[not provided]'}}", + "LeadSource": "{{steps.leadFields.LeadSource | default: settings.leadSource}}", + "OwnerId": "$.install.leadOwner", + "Description": "$.steps.leadNote" + } + } + }, + { + "use": "links.upsert", + "with": { + "ids": { + "calendly": "$.record.uri", + "salesforce": "$.steps.lead.id" + }, + "keys": { + "email": "$.record.email" + }, + "link": "people" + } + }, + { + "use": "set", + "with": { + "whoId": "$.steps.lead.id" + } + } + ] + }, + { + "id": "eventNote", + "use": "template", + "with": { + "id": "event-description" + } + }, + { + "id": "event", + "use": "salesforce.event.create", + "with": { + "fields": { + "Subject": "{{record.scheduled_event.name}} with {{record.name}}", + "StartDateTime": "$.record.scheduled_event.start_time", + "EndDateTime": "$.record.scheduled_event.end_time", + "Location": "$.record.scheduled_event.location.join_url", + "Description": "$.steps.eventNote", + "WhoId": "$.vars.whoId", + "OwnerId": "$.install.leadOwner", + "Calendly_Event_URI__c": "$.record.scheduled_event.uri" + } + } + }, + { + "use": "links.upsert", + "with": { + "ids": { + "calendly": "$.record.scheduled_event.uri", + "salesforce": "$.steps.event.id" + }, + "keys": { + "eventUri": "$.record.scheduled_event.uri" + }, + "link": "meetings" + } + } + ] + }, + { + "id": "cancellation", + "label": "Mark canceled meetings on the Event", + "purpose": "write", + "default": "on", + "settings": { + "followUpTask": { + "type": "boolean", + "label": "Create a follow-up task to rebook", + "default": false + } + }, + "trigger": { + "type": "webhook", + "module": "calendly", + "topic": "invitee.canceled", + "filter": "$.record.rescheduled != true" + }, + "steps": [ + { + "id": "meeting", + "use": "links.find", + "with": { + "key": { + "eventUri": "$.record.scheduled_event.uri" + }, + "link": "meetings" + } + }, + { + "use": "stop", + "when": "$.steps.meeting == null", + "with": { + "reason": "Meeting was booked before the integration was installed" + } + }, + { + "use": "salesforce.event.update", + "with": { + "id": "$.steps.meeting.salesforce", + "fields": { + "Subject": "Canceled: {{record.scheduled_event.name}} with {{record.name}}", + "ShowAs": "Free", + "Description": "Canceled by {{record.cancellation.canceled_by}}: {{record.cancellation.reason | default: 'no reason given'}}" + } + } + }, + { + "id": "person", + "use": "links.find", + "when": "$.settings.followUpTask == true", + "with": { + "key": { + "email": "$.record.email" + }, + "link": "people" + } + }, + { + "use": "salesforce.task.create", + "when": "$.settings.followUpTask == true && $.steps.person != null", + "with": { + "fields": { + "Subject": "Rebook {{record.name}} ({{record.scheduled_event.name}} canceled)", + "Status": "Not Started", + "Priority": "Normal", + "ActivityDate": "$.record.scheduled_event.start_time", + "WhoId": "$.steps.person.salesforce", + "OwnerId": "$.install.leadOwner" + } + } + } + ] + } + ], + "queries": [ + { + "id": "upcoming-meetings", + "description": "Active Calendly meetings booked by this Lead or Contact's email, soonest first.", + "steps": [ + { + "use": "stop", + "when": "!($.record.Email exists)", + "with": { + "reason": "Record has no email" + } + }, + { + "id": "events", + "use": "calendly.event.list", + "with": { + "organization": "$.install.accounts.calendly.organization", + "invitee_email": "$.record.Email", + "min_start_time": "$.now", + "status": "active", + "count": 5 + } + } + ], + "returns": { + "events": "$.steps.events" + } + } + ], + "ui": [ + { + "id": "meetings-card", + "label": "Upcoming Calendly meetings", + "host": "salesforce", + "placement": "lightning.record.sidebar", + "objects": [ + "lead", + "contact" + ], + "data": { + "query": "upcoming-meetings" + }, + "view": { + "component": "Stack", + "children": [ + { + "component": "Heading", + "props": { + "text": "Upcoming meetings" + } + }, + { + "component": "List", + "props": { + "items": "$.data.events", + "title": "{{item.name}}", + "meta": "{{item.start_time | date}} \u00b7 {{item.location.type}}" + } + }, + { + "component": "Empty", + "when": "$.data.events[0] == null", + "props": { + "text": "No upcoming Calendly meetings with this person." + } + } + ] + } + } + ], + "templates": { + "lead-description": "Booked \"{{record.scheduled_event.name}}\" via Calendly on {{record.created_at | date}}.\nTopic: {{record.questions_and_answers[1].answer | default: 'not given'}}\nCampaign: {{record.tracking.utm_campaign | default: 'none'}}", + "event-description": "{{record.questions_and_answers[1].answer | default: 'No agenda given.'}}\n\nJoin: {{record.scheduled_event.location.join_url}}\nReschedule: {{record.reschedule_url}}\nCancel: {{record.cancel_url}}" + }, + "lifecycle": { + "onInstall": [ + { + "id": "hook", + "use": "calendly.webhook.create", + "with": { + "url": "$.install.webhookUrls.calendly", + "events": [ + "invitee.created", + "invitee.canceled" + ], + "organization": "$.install.accounts.calendly.organization", + "scope": "organization" + } + } + ], + "onUninstall": [ + { + "use": "calendly.webhook.delete", + "with": { + "uri": "$.install.runtime.installed.hook.resource.uri" + } + } + ] + } +} diff --git a/packages/core/integrations/data-defined/examples/docusign-salesforce.definition.json b/packages/core/integrations/data-defined/examples/docusign-salesforce.definition.json new file mode 100644 index 000000000..a311e2915 --- /dev/null +++ b/packages/core/integrations/data-defined/examples/docusign-salesforce.definition.json @@ -0,0 +1,305 @@ +{ + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "docusign-salesforce", + "version": "1.3.2", + "display": { + "label": "DocuSign \u2192 Salesforce", + "description": "Completed envelopes move the opportunity forward and attach the signed PDF; declined envelopes create a task for the owner; the opportunity page shows every envelope and its status.", + "category": "Contracts", + "tags": [ + "e-signature", + "crm", + "contracts" + ] + }, + "modules": { + "docusign": { + "module": "docusign", + "version": "^1.2" + }, + "salesforce": { + "module": "salesforce", + "version": "^3.0" + } + }, + "settings": { + "opportunityField": { + "type": "string", + "label": "Envelope custom field holding the Opportunity Id", + "default": "SalesforceOpportunityId", + "required": true + } + }, + "workflows": [ + { + "id": "completed-envelope", + "label": "Completed envelopes advance the opportunity", + "purpose": "write", + "default": "on", + "settings": { + "stage": { + "type": "enum", + "label": "Move the opportunity to", + "enum": [ + "Negotiation/Review", + "Closed Won" + ], + "default": "Closed Won" + }, + "attachPdf": { + "type": "boolean", + "label": "Attach the signed PDF", + "default": true + } + }, + "trigger": { + "type": "webhook", + "module": "docusign", + "topic": "envelope.completed" + }, + "steps": [ + { + "id": "opp", + "use": "code", + "with": { + "ref": "@acme/docusign-steps#customFieldValue@1.0.0", + "with": { + "fields": "$.record.customFields.textCustomFields", + "name": "$.install.opportunityField" + } + } + }, + { + "use": "stop", + "when": "$.steps.opp.value == null", + "with": { + "reason": "Envelope was not sent from a Salesforce opportunity" + } + }, + { + "use": "salesforce.opportunity.update", + "with": { + "id": "$.steps.opp.value", + "fields": { + "StageName": "$.settings.stage", + "NextStep": "Signed {{record.completedDateTime | date}}: kick off onboarding" + } + } + }, + { + "id": "pdf", + "use": "docusign.document.combined", + "when": "$.settings.attachPdf == true", + "with": { + "envelopeId": "$.record.envelopeId", + "certificate": true + } + }, + { + "use": "salesforce.file.attach", + "when": "$.settings.attachPdf == true", + "with": { + "title": "{{record.emailSubject}} (signed)", + "pathOnClient": "{{record.envelopeId}}.pdf", + "versionData": "$.steps.pdf.base64", + "firstPublishLocationId": "$.steps.opp.value" + } + }, + { + "id": "note", + "use": "template", + "with": { + "id": "completed-note" + } + }, + { + "use": "salesforce.note.create", + "with": { + "title": "Envelope completed", + "content": "$.steps.note", + "linkedEntityId": "$.steps.opp.value" + } + } + ], + "retry": { + "max": 3, + "backoff": "exponential" + } + }, + { + "id": "declined-envelope", + "label": "Declined envelopes create a task for the owner", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "docusign", + "topic": "envelope.declined" + }, + "steps": [ + { + "id": "opp", + "use": "code", + "with": { + "ref": "@acme/docusign-steps#customFieldValue@1.0.0", + "with": { + "fields": "$.record.customFields.textCustomFields", + "name": "$.install.opportunityField" + } + } + }, + { + "use": "stop", + "when": "$.steps.opp.value == null", + "with": { + "reason": "Envelope was not sent from a Salesforce opportunity" + } + }, + { + "id": "opportunity", + "use": "salesforce.opportunity.get", + "with": { + "id": "$.steps.opp.value" + } + }, + { + "id": "description", + "use": "template", + "with": { + "id": "declined-task" + } + }, + { + "use": "salesforce.task.create", + "with": { + "fields": { + "Subject": "Envelope declined by {{record.recipients.signers[0].name}}", + "Status": "Not Started", + "Priority": "High", + "ActivityDate": "{{record.declinedDateTime | date}}", + "WhatId": "$.steps.opp.value", + "OwnerId": "$.steps.opportunity.OwnerId", + "Description": "$.steps.description" + } + } + } + ] + } + ], + "queries": [ + { + "id": "opportunity-envelopes", + "description": "Envelopes sent for this opportunity in the last year.", + "steps": [ + { + "id": "envelopes", + "use": "docusign.envelope.list", + "with": { + "from_date": "-365d", + "custom_field": "{{install.opportunityField}}={{record.Id}}", + "count": 20 + } + } + ], + "returns": { + "envelopes": "$.steps.envelopes" + } + } + ], + "ui": [ + { + "id": "envelopes-card", + "label": "DocuSign envelopes", + "host": "salesforce", + "placement": "lightning.record.sidebar", + "objects": [ + "opportunity" + ], + "data": { + "query": "opportunity-envelopes" + }, + "view": { + "component": "Stack", + "children": [ + { + "component": "Heading", + "props": { + "text": "DocuSign" + } + }, + { + "component": "Table", + "props": { + "rows": "$.data.envelopes", + "columns": [ + { + "label": "Envelope", + "value": "{{row.emailSubject | truncate: 40}}" + }, + { + "label": "Status", + "value": "{{row.status}}" + }, + { + "label": "Sent", + "value": "{{row.sentDateTime | date}}" + }, + { + "label": "Completed", + "value": "{{row.completedDateTime | date}}" + } + ] + } + }, + { + "component": "Empty", + "when": "$.data.envelopes[0] == null", + "props": { + "text": "No envelopes sent for this opportunity." + } + }, + { + "component": "Button", + "props": { + "label": "Open DocuSign", + "action": { + "type": "open_url", + "url": "https://app.docusign.com/documents" + } + } + } + ] + } + } + ], + "templates": { + "completed-note": "{{record.emailSubject}} was completed on {{record.completedDateTime | date}}. Signers: {{record.recipients.signers[0].name}} <{{record.recipients.signers[0].email}}>. Envelope {{record.envelopeId}}.", + "declined-task": "{{record.recipients.signers[0].name}} ({{record.recipients.signers[0].email}}) declined \"{{record.emailSubject}}\" on {{record.declinedDateTime | date}}.\nReason: {{record.recipients.signers[0].declinedReason | default: 'none given'}}\nCorrect the documents and resend from DocuSign." + }, + "lifecycle": { + "onInstall": [ + { + "id": "connect", + "use": "docusign.connect.create", + "with": { + "name": "Frigg: Salesforce", + "urlToPublishTo": "$.install.webhookUrls.docusign", + "events": [ + "envelope-sent", + "envelope-completed", + "envelope-declined" + ] + } + } + ], + "onUninstall": [ + { + "use": "docusign.connect.delete", + "with": { + "connectId": "$.install.runtime.installed.connect.connectId" + } + } + ] + } +} diff --git a/packages/core/integrations/data-defined/examples/fathom-hubspot-notion.definition.json b/packages/core/integrations/data-defined/examples/fathom-hubspot-notion.definition.json new file mode 100644 index 000000000..b9ba685e8 --- /dev/null +++ b/packages/core/integrations/data-defined/examples/fathom-hubspot-notion.definition.json @@ -0,0 +1,575 @@ +{ + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "fathom-hubspot-notion", + "version": "0.4.0", + "display": { + "label": "Fathom call follow-through", + "description": "After a recorded customer call: the summary lands on the HubSpot deal, action items become tasks, the meeting gets a Notion page, and proposed deal-stage changes wait for approval in Slack.", + "category": "Sales productivity", + "tags": [ + "meetings", + "crm", + "notes", + "ai" + ] + }, + "modules": { + "fathom": { + "module": "fathom", + "version": "^0.4" + }, + "hubspot": { + "module": "hubspot", + "version": "^1.4" + }, + "notion": { + "module": "notion", + "version": "^1.3" + }, + "slack": { + "module": "slack", + "version": "^2.0" + } + }, + "settings": { + "meetingsDatabase": { + "type": "option", + "options": "notion.database", + "label": "Notion meetings database", + "required": true + } + }, + "workflows": [ + { + "id": "summary-note", + "label": "Call summary as a deal note", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "fathom", + "topic": "meeting.ready" + }, + "steps": [ + { + "id": "people", + "use": "code", + "with": { + "ref": "@acme/meeting-steps#externalInvitees@1.0.0", + "with": { + "invitees": "$.record.calendar_invitees" + } + } + }, + { + "use": "stop", + "when": "$.steps.people.primaryEmail == null", + "with": { + "reason": "No external attendees: an internal meeting" + } + }, + { + "id": "contacts", + "use": "hubspot.contact.search", + "with": { + "email": "$.steps.people.primaryEmail", + "limit": 1 + } + }, + { + "use": "stop", + "when": "$.steps.contacts[0] == null", + "with": { + "reason": "Attendee is not a HubSpot contact" + } + }, + { + "id": "deals", + "use": "hubspot.deal.listForContact", + "with": { + "contactId": "$.steps.contacts[0].id", + "open": true + } + }, + { + "id": "body", + "use": "template", + "with": { + "id": "summary-note" + } + }, + { + "use": "hubspot.note.create", + "with": { + "contactId": "$.steps.contacts[0].id", + "dealId": "$.steps.deals[0].id", + "properties": { + "hs_note_body": "$.steps.body", + "hs_timestamp": "$.record.recording_end_time" + } + } + } + ] + }, + { + "id": "action-item-tasks", + "label": "Action items become HubSpot tasks", + "purpose": "write", + "default": "on", + "settings": { + "fallbackOwner": { + "type": "option", + "options": "hubspot.owner", + "label": "Owner when the assignee is not a HubSpot user" + }, + "dueIn": { + "type": "duration", + "label": "Due in", + "default": "2d" + } + }, + "trigger": { + "type": "webhook", + "module": "fathom", + "topic": "meeting.ready", + "filter": "$.record.action_items[0] exists" + }, + "steps": [ + { + "id": "people", + "use": "code", + "with": { + "ref": "@acme/meeting-steps#externalInvitees@1.0.0", + "with": { + "invitees": "$.record.calendar_invitees" + } + } + }, + { + "use": "stop", + "when": "$.steps.people.primaryEmail == null", + "with": { + "reason": "No external attendees: an internal meeting" + } + }, + { + "id": "contacts", + "use": "hubspot.contact.search", + "with": { + "email": "$.steps.people.primaryEmail", + "limit": 1 + } + }, + { + "use": "stop", + "when": "$.steps.contacts[0] == null", + "with": { + "reason": "Attendee is not a HubSpot contact" + } + }, + { + "id": "tasks", + "use": "foreach", + "with": { + "over": "$.record.action_items", + "max": 25 + }, + "do": [ + { + "id": "assignee", + "use": "hubspot.owner.findByEmail", + "when": "$.item.assignee.email exists", + "with": { + "email": "$.item.assignee.email" + }, + "onError": "continue" + }, + { + "use": "hubspot.task.create", + "when": "$.item.completed != true", + "with": { + "contactId": "$.steps.contacts[0].id", + "properties": { + "hs_task_subject": "{{item.description | truncate: 120}}", + "hs_task_body": "From \"{{record.title}}\" at {{item.recording_timestamp}}: {{record.share_url}}", + "hs_timestamp": "$.now", + "hs_task_priority": "MEDIUM", + "hubspot_owner_id": "{{steps.assignee.id | default: settings.fallbackOwner}}" + } + } + } + ] + } + ] + }, + { + "id": "meeting-page", + "label": "Meeting page in Notion", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "fathom", + "topic": "meeting.ready" + }, + "steps": [ + { + "id": "blocks", + "use": "code", + "with": { + "ref": "@acme/notion-steps#markdownToBlocks@1.1.0", + "with": { + "markdown": "$.record.default_summary.markdown_formatted", + "actionItems": "$.record.action_items" + } + } + }, + { + "id": "attendees", + "use": "code", + "with": { + "ref": "@acme/meeting-steps#inviteeNames@1.0.0", + "with": { + "invitees": "$.record.calendar_invitees" + } + } + }, + { + "use": "notion.page.create", + "with": { + "parent": { + "database_id": "$.install.meetingsDatabase" + }, + "icon": { + "type": "emoji", + "emoji": "\ud83d\udcde" + }, + "properties": { + "Name": { + "title": [ + { + "text": { + "content": "$.record.title" + } + } + ] + }, + "Date": { + "date": { + "start": "$.record.recording_start_time", + "end": "$.record.recording_end_time" + } + }, + "Recording": { + "url": "$.record.share_url" + }, + "Attendees": { + "rich_text": [ + { + "text": { + "content": "$.steps.attendees.text" + } + } + ] + }, + "Recorded by": { + "email": "$.record.recorded_by.email" + } + }, + "children": "$.steps.blocks.blocks" + } + } + ] + }, + { + "id": "deal-updates", + "label": "Propose deal updates from the call", + "description": "A stage change proposed by the call goes to Slack for approval; a next step alone is written straight to the deal.", + "purpose": "write", + "default": "off", + "settings": { + "approvalChannel": { + "type": "option", + "options": "slack.channel", + "label": "Ask for approval in", + "required": true + } + }, + "trigger": { + "type": "webhook", + "module": "fathom", + "topic": "meeting.ready" + }, + "steps": [ + { + "id": "people", + "use": "code", + "with": { + "ref": "@acme/meeting-steps#externalInvitees@1.0.0", + "with": { + "invitees": "$.record.calendar_invitees" + } + } + }, + { + "use": "stop", + "when": "$.steps.people.primaryEmail == null", + "with": { + "reason": "No external attendees: an internal meeting" + } + }, + { + "id": "contacts", + "use": "hubspot.contact.search", + "with": { + "email": "$.steps.people.primaryEmail", + "limit": 1 + } + }, + { + "use": "stop", + "when": "$.steps.contacts[0] == null", + "with": { + "reason": "Attendee is not a HubSpot contact" + } + }, + { + "id": "deals", + "use": "hubspot.deal.listForContact", + "with": { + "contactId": "$.steps.contacts[0].id", + "open": true + } + }, + { + "use": "stop", + "when": "$.steps.deals[0] == null", + "with": { + "reason": "Contact has no open deal" + } + }, + { + "id": "proposal", + "use": "code", + "with": { + "ref": "@acme/meeting-steps#proposeDealUpdates@1.0.0", + "with": { + "summary": "$.record.default_summary.markdown_formatted", + "actionItems": "$.record.action_items", + "deal": "$.steps.deals[0]" + } + } + }, + { + "id": "route", + "use": "branch", + "cases": [ + { + "when": "$.steps.proposal.stage exists && $.steps.proposal.stage != $.steps.deals[0].properties.dealstage", + "do": [ + { + "id": "ask", + "use": "template", + "with": { + "id": "approval-request" + } + }, + { + "use": "slack.message.post", + "with": { + "channel": "$.settings.approvalChannel", + "text": "$.steps.ask", + "metadata": { + "event_type": "deal_stage_proposal", + "event_payload": { + "dealId": "$.steps.deals[0].id", + "stage": "$.steps.proposal.stage", + "nextStep": "$.steps.proposal.nextStep", + "recording": "$.record.share_url" + } + } + } + } + ] + }, + { + "when": "$.steps.proposal.nextStep exists", + "do": [ + { + "use": "hubspot.deal.update", + "with": { + "id": "$.steps.deals[0].id", + "properties": { + "hs_next_step": "$.steps.proposal.nextStep" + } + } + } + ] + } + ] + } + ] + }, + { + "id": "approve-stage", + "label": "Approve a proposed stage change", + "purpose": "respond", + "default": "on", + "trigger": { + "type": "userAction", + "label": "Approve", + "from": "stage-approval" + }, + "steps": [ + { + "use": "stop", + "when": "$.record.metadata.event_type != 'deal_stage_proposal'", + "with": { + "reason": "Not a stage proposal" + } + }, + { + "use": "hubspot.deal.update", + "with": { + "id": "$.record.metadata.event_payload.dealId", + "properties": { + "dealstage": "$.record.metadata.event_payload.stage", + "hs_next_step": "$.record.metadata.event_payload.nextStep" + } + } + }, + { + "use": "slack.message.update", + "with": { + "channel": "$.record.channel", + "ts": "$.record.ts", + "text": ":white_check_mark: Approved by <@{{event.user.id}}>. {{record.text}}" + } + } + ] + } + ], + "queries": [ + { + "id": "stage-proposal", + "description": "The deal a stage-proposal message is about, with the proposed stage.", + "steps": [ + { + "use": "stop", + "when": "$.record.metadata.event_type != 'deal_stage_proposal'", + "with": { + "reason": "Not a stage proposal" + } + }, + { + "id": "deal", + "use": "hubspot.deal.get", + "with": { + "id": "$.record.metadata.event_payload.dealId" + } + } + ], + "returns": { + "deal": "$.steps.deal", + "proposal": "$.record.metadata.event_payload" + } + } + ], + "ui": [ + { + "id": "stage-approval", + "label": "Deal stage proposal", + "host": "slack", + "placement": "message.actions", + "data": { + "query": "stage-proposal" + }, + "view": { + "component": "Stack", + "children": [ + { + "component": "Heading", + "props": { + "text": "{{data.deal.properties.dealname}}" + } + }, + { + "component": "KeyValue", + "props": { + "items": [ + { + "label": "Current stage", + "value": "{{data.deal.properties.dealstage}}" + }, + { + "label": "Proposed stage", + "value": "{{data.proposal.stage}}" + }, + { + "label": "Next step", + "value": "{{data.proposal.nextStep | default: '\u2014'}}" + } + ] + } + }, + { + "component": "Divider" + }, + { + "component": "Button", + "when": "$.data.deal.properties.dealstage != $.data.proposal.stage", + "props": { + "label": "Approve", + "style": "primary", + "action": { + "type": "userAction", + "workflow": "approve-stage" + } + } + }, + { + "component": "Button", + "props": { + "label": "Watch the call", + "action": { + "type": "open_url", + "url": "{{data.proposal.recording}}" + } + } + } + ] + } + } + ], + "templates": { + "summary-note": "{{record.title}} ({{record.recording_start_time | date}})
{{record.default_summary.markdown_formatted}}
Watch the recording", + "approval-request": ":telephone_receiver: After \"{{record.title}}\", Fathom suggests moving *{{steps.deals[0].properties.dealname}}* from `{{steps.deals[0].properties.dealstage}}` to `{{steps.proposal.stage}}`.\nNext step: {{steps.proposal.nextStep | default: 'none captured'}}\nUse the message menu \u2192 *Deal stage proposal* to approve." + }, + "lifecycle": { + "onInstall": [ + { + "id": "hook", + "use": "fathom.webhook.create", + "with": { + "destination_url": "$.install.webhookUrls.fathom", + "triggered_for": [ + "my_recordings", + "shared_external_recordings" + ], + "include_summary": true, + "include_action_items": true, + "include_transcript": false + } + } + ], + "onUninstall": [ + { + "use": "fathom.webhook.delete", + "with": { + "id": "$.install.runtime.installed.hook.id" + } + } + ] + } +} diff --git a/packages/core/integrations/data-defined/examples/github-linear.definition.json b/packages/core/integrations/data-defined/examples/github-linear.definition.json new file mode 100644 index 000000000..7465ba2f7 --- /dev/null +++ b/packages/core/integrations/data-defined/examples/github-linear.definition.json @@ -0,0 +1,316 @@ +{ + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "github-linear", + "version": "0.9.0", + "display": { + "label": "GitHub \u2194 Linear", + "description": "GitHub issues and Linear issues stay in sync with label and status mapping, merged PRs move the Linear issue named in the branch to Done, and stale in-progress issues are listed in Slack every morning.", + "category": "Engineering", + "tags": [ + "issues", + "developer-tools", + "slack" + ] + }, + "modules": { + "github": { + "module": "github", + "version": "^1.8" + }, + "linear": { + "module": "linear", + "version": "^1.1" + }, + "slack": { + "module": "slack", + "version": "^2.0" + } + }, + "settings": { + "repo": { + "type": "option", + "options": "github.repo", + "label": "GitHub repository", + "required": true + }, + "team": { + "type": "option", + "options": "linear.team", + "label": "Linear team", + "required": true + } + }, + "syncs": [ + { + "id": "issues", + "label": "Issues", + "description": "Each GitHub issue is one Linear issue; the Linear issue carries the GitHub URL as an attachment. Pull requests are not synced.", + "source": "github.issue", + "target": "linear.issue", + "direction": "two_way", + "identity": { + "match": [ + { + "source": "html_url", + "target": "attachments[0].url", + "key": "issueUrl" + } + ], + "onAmbiguous": "skip" + }, + "mapping": [ + { + "source": "title", + "target": "title", + "master": "newest" + }, + { + "source": "body", + "target": "description", + "master": "newest" + }, + { + "source": "state", + "target": "state.name", + "transform": "map_values", + "options": { + "values": { + "open": "Todo", + "closed": "Done" + } + }, + "direction": "to_target" + }, + { + "source": "state", + "target": "state.type", + "transform": "map_values", + "options": { + "values": { + "triage": "open", + "backlog": "open", + "unstarted": "open", + "started": "open", + "completed": "closed", + "canceled": "closed" + } + }, + "direction": "to_source" + }, + { + "source": "labels[0].name", + "target": "priority", + "transform": "map_values", + "options": { + "values": { + "P0": 1, + "P1": 2, + "P2": 3, + "P3": 4 + }, + "fallback": 0 + }, + "direction": "to_target" + }, + { + "source": "html_url", + "target": "attachments[0].url", + "direction": "to_target" + } + ], + "filter": "!($.record.pull_request exists)", + "backfill": { + "trigger": "userAction", + "pageSize": 100 + }, + "ongoing": { + "mode": "webhook" + }, + "default": "on" + } + ], + "workflows": [ + { + "id": "pr-merged-done", + "label": "Merged PRs complete the Linear issue in the branch name", + "description": "A branch like dana/eng-142-dedupe moves ENG-142 to the done state when its PR merges.", + "purpose": "write", + "default": "on", + "settings": { + "doneState": { + "type": "option", + "options": { + "source": "linear.workflowState", + "with": { + "teamId": "$.install.team" + } + }, + "label": "Move issues to", + "default": "Done" + }, + "commentWhenUnlinked": { + "type": "boolean", + "label": "Comment on PRs whose branch names no Linear issue", + "default": false + } + }, + "trigger": { + "type": "webhook", + "module": "github", + "topic": "pullRequest.closed", + "filter": "$.record.merged == true" + }, + "steps": [ + { + "id": "ids", + "use": "code", + "with": { + "ref": "@acme/github-steps#linearIdentifiers@1.0.0", + "with": { + "branch": "$.record.head.ref", + "title": "$.record.title", + "body": "$.record.body" + } + } + }, + { + "id": "route", + "use": "branch", + "cases": [ + { + "when": "$.steps.ids.identifiers[0] exists", + "do": [ + { + "id": "complete", + "use": "foreach", + "with": { + "over": "$.steps.ids.identifiers", + "max": 10 + }, + "do": [ + { + "id": "issue", + "use": "linear.issue.byIdentifier", + "with": { + "identifier": "$.item" + }, + "onError": "continue" + }, + { + "use": "linear.issue.update", + "when": "$.steps.issue != null && $.steps.issue.state.type != 'completed' && $.steps.issue.state.type != 'canceled'", + "with": { + "id": "$.steps.issue.id", + "input": { + "stateName": "$.settings.doneState" + } + } + }, + { + "use": "linear.comment.create", + "when": "$.steps.issue != null", + "with": { + "issueId": "$.steps.issue.id", + "body": "Completed by merged PR [#{{record.number}}]({{record.html_url}})." + } + } + ] + } + ] + }, + { + "when": "$.settings.commentWhenUnlinked == true", + "do": [ + { + "use": "github.issue.comment", + "with": { + "owner": "$.event.repository.owner.login", + "repo": "$.event.repository.name", + "issue_number": "$.record.number", + "body": "No Linear issue found in branch `{{record.head.ref}}`. Name branches like `eng-123-short-title` to link them." + } + } + ] + } + ] + } + ] + }, + { + "id": "stale-digest", + "label": "Daily stale-issue digest in Slack", + "purpose": "notify", + "default": "off", + "settings": { + "channel": { + "type": "option", + "options": "slack.channel", + "label": "Post to", + "required": true + }, + "staleAfter": { + "type": "enum", + "label": "Stale after no update for", + "enum": [ + "-7d", + "-14d", + "-30d" + ], + "default": "-14d" + } + }, + "trigger": { + "type": "schedule", + "cron": "30 9 * * 1-5", + "timezone": "Europe/London" + }, + "steps": [ + { + "id": "stale", + "use": "linear.issue.search", + "with": { + "teamId": "$.install.team", + "stateType": "started", + "updatedBefore": "$.settings.staleAfter", + "first": 50 + } + }, + { + "use": "stop", + "when": "$.steps.stale[0] == null", + "with": { + "reason": "Nothing is stale" + } + }, + { + "id": "build", + "use": "foreach", + "with": { + "over": "$.steps.stale", + "max": 50, + "collect": "\u2022 <{{item.url}}|{{item.identifier}}> {{item.title | truncate: 70}} ({{item.assignee.email | default: 'unassigned'}}, last update {{item.updatedAt | date}})" + } + }, + { + "id": "text", + "use": "template", + "with": { + "id": "stale-digest" + } + }, + { + "use": "slack.message.post", + "with": { + "channel": "$.settings.channel", + "text": "$.steps.text", + "unfurl_links": false + } + } + ] + } + ], + "templates": { + "stale-digest": ":hourglass_flowing_sand: *In progress with no update since {{settings.staleAfter}}* ({{steps.stale | size}} issues)\n{{steps.build | join: '\\n'}}" + } +} diff --git a/packages/core/integrations/data-defined/examples/hubspot-slack.definition.json b/packages/core/integrations/data-defined/examples/hubspot-slack.definition.json new file mode 100644 index 000000000..18fd91775 --- /dev/null +++ b/packages/core/integrations/data-defined/examples/hubspot-slack.definition.json @@ -0,0 +1,409 @@ +{ + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "hubspot-slack", + "version": "1.2.0", + "display": { + "label": "HubSpot \u2192 Slack alerts", + "description": "Deal-won and big-deal alerts, a Monday pipeline digest, and task reminders in DM that can be marked done from Slack.", + "category": "Notifications", + "tags": [ + "slack", + "crm", + "alerts", + "sales" + ] + }, + "modules": { + "hubspot": { + "module": "hubspot", + "version": "^1.4" + }, + "slack": { + "module": "slack", + "version": "^2.0" + } + }, + "workflows": [ + { + "id": "deal-won", + "label": "Celebrate closed-won deals", + "purpose": "notify", + "default": "on", + "settings": { + "channel": { + "type": "option", + "options": "slack.channel", + "label": "Post to", + "required": true + } + }, + "trigger": { + "type": "webhook", + "module": "hubspot", + "topic": "deal.propertyChange", + "filter": "$.record.propertyName == 'dealstage' && $.record.propertyValue == 'closedwon'" + }, + "steps": [ + { + "id": "deal", + "use": "hubspot.deal.get", + "with": { + "id": "$.record.objectId" + } + }, + { + "id": "owner", + "use": "hubspot.owner.get", + "with": { + "id": "$.steps.deal.properties.hubspot_owner_id" + }, + "onError": "continue" + }, + { + "id": "text", + "use": "template", + "with": { + "id": "deal-won" + } + }, + { + "use": "slack.message.post", + "with": { + "channel": "$.settings.channel", + "text": "$.steps.text" + } + } + ] + }, + { + "id": "high-value-deal", + "label": "Flag new high-value deals", + "purpose": "notify", + "default": "on", + "settings": { + "threshold": { + "type": "number", + "label": "Minimum amount", + "default": 25000 + }, + "channel": { + "type": "option", + "options": "slack.channel", + "label": "Post to", + "required": true + } + }, + "trigger": { + "type": "webhook", + "module": "hubspot", + "topic": "deal.created" + }, + "steps": [ + { + "id": "deal", + "use": "hubspot.deal.get", + "with": { + "id": "$.record.objectId" + } + }, + { + "use": "stop", + "when": "!($.steps.deal.properties.amount exists) || $.steps.deal.properties.amount < $.settings.threshold", + "with": { + "reason": "Below the threshold" + } + }, + { + "id": "text", + "use": "template", + "with": { + "id": "high-value" + } + }, + { + "use": "slack.message.post", + "with": { + "channel": "$.settings.channel", + "text": "$.steps.text" + } + } + ] + }, + { + "id": "weekly-digest", + "label": "Monday pipeline digest", + "description": "Every open deal closing in the next 7 days, one line each, in a single message.", + "purpose": "notify", + "default": "off", + "settings": { + "channel": { + "type": "option", + "options": "slack.channel", + "label": "Post to", + "required": true + }, + "minAmount": { + "type": "number", + "label": "Only deals of at least", + "default": 0 + } + }, + "trigger": { + "type": "schedule", + "cron": "45 7 * * 1", + "timezone": "America/New_York" + }, + "steps": [ + { + "id": "deals", + "use": "hubspot.deal.search", + "with": { + "open": true, + "closeDateBefore": "+7d", + "minAmount": "$.settings.minAmount", + "limit": 50 + } + }, + { + "use": "stop", + "when": "$.steps.deals[0] == null", + "with": { + "reason": "Nothing closes this week" + } + }, + { + "id": "build", + "use": "foreach", + "with": { + "over": "$.steps.deals", + "max": 50, + "collect": "$.steps.line" + }, + "do": [ + { + "id": "line", + "use": "template", + "with": { + "id": "digest-line" + } + } + ] + }, + { + "id": "text", + "use": "template", + "with": { + "id": "digest" + } + }, + { + "use": "slack.message.post", + "with": { + "channel": "$.settings.channel", + "text": "$.steps.text", + "unfurl_links": false + } + } + ] + }, + { + "id": "task-reminders", + "label": "DM owners their tasks due today", + "purpose": "notify", + "default": "on", + "trigger": { + "type": "schedule", + "cron": "55 8 * * 1-5", + "timezone": "America/New_York" + }, + "steps": [ + { + "id": "tasks", + "use": "hubspot.task.search", + "with": { + "dueBefore": "+1d", + "status": "NOT_STARTED", + "limit": 100 + } + }, + { + "id": "remind", + "use": "foreach", + "with": { + "over": "$.steps.tasks", + "max": 100 + }, + "do": [ + { + "id": "owner", + "use": "hubspot.owner.get", + "when": "$.item.properties.hubspot_owner_id exists", + "with": { + "id": "$.item.properties.hubspot_owner_id" + }, + "onError": "continue" + }, + { + "id": "slackUser", + "use": "slack.user.lookupByEmail", + "when": "$.steps.owner.email exists", + "with": { + "email": "$.steps.owner.email" + }, + "onError": "continue" + }, + { + "use": "slack.message.post", + "when": "$.steps.slackUser.id exists", + "with": { + "channel": "$.steps.slackUser.id", + "text": "Due today: {{item.properties.hs_task_subject}}", + "metadata": { + "event_type": "hubspot_task", + "event_payload": { + "taskId": "$.item.id" + } + } + } + } + ] + } + ] + }, + { + "id": "mark-task-done", + "label": "Mark a HubSpot task done from Slack", + "purpose": "respond", + "default": "on", + "trigger": { + "type": "userAction", + "label": "Mark done", + "from": "task-actions" + }, + "steps": [ + { + "use": "hubspot.task.update", + "with": { + "id": "$.record.metadata.event_payload.taskId", + "properties": { + "hs_task_status": "COMPLETED" + } + } + }, + { + "use": "slack.message.update", + "with": { + "channel": "$.record.channel", + "ts": "$.record.ts", + "text": ":white_check_mark: Done: {{record.text}}" + } + } + ] + } + ], + "queries": [ + { + "id": "task-detail", + "description": "The HubSpot task a reminder message points at (via its Slack message metadata).", + "steps": [ + { + "use": "stop", + "when": "$.record.metadata.event_type != 'hubspot_task'", + "with": { + "reason": "Not a task reminder" + } + }, + { + "id": "task", + "use": "hubspot.task.get", + "with": { + "id": "$.record.metadata.event_payload.taskId", + "properties": [ + "hs_task_subject", + "hs_task_status", + "hs_task_priority", + "hs_timestamp" + ] + } + } + ], + "returns": { + "task": "$.steps.task", + "portalId": "$.install.portalId" + } + } + ], + "ui": [ + { + "id": "task-actions", + "label": "HubSpot task", + "host": "slack", + "placement": "message.actions", + "data": { + "query": "task-detail" + }, + "view": { + "component": "Stack", + "children": [ + { + "component": "Heading", + "props": { + "text": "{{data.task.properties.hs_task_subject}}" + } + }, + { + "component": "KeyValue", + "props": { + "items": [ + { + "label": "Priority", + "value": "{{data.task.properties.hs_task_priority | default: 'NONE'}}" + }, + { + "label": "Due", + "value": "{{data.task.properties.hs_timestamp | date}}" + } + ] + } + }, + { + "component": "Badge", + "props": { + "text": "{{data.task.properties.hs_task_status}}", + "tone": "neutral" + } + }, + { + "component": "Button", + "when": "$.data.task.properties.hs_task_status != 'COMPLETED'", + "props": { + "label": "Mark done", + "style": "primary", + "action": { + "type": "userAction", + "workflow": "mark-task-done" + } + } + }, + { + "component": "Button", + "props": { + "label": "Open in HubSpot", + "action": { + "type": "open_url", + "url": "https://app.hubspot.com/tasks/{{data.portalId}}/view/all/task/{{data.task.id}}" + } + } + } + ] + } + } + ], + "templates": { + "deal-won": ":tada: *{{steps.deal.properties.dealname}}* closed won for ${{steps.deal.properties.amount | default: '?'}}. Nice work {{steps.owner.firstName | default: 'team'}}! ", + "high-value": ":moneybag: New deal over ${{settings.threshold}}: *{{steps.deal.properties.dealname}}* (${{steps.deal.properties.amount}}), closing {{steps.deal.properties.closedate | date}}. ", + "digest-line": "\u2022 : ${{item.properties.amount | default: 0}}, closes {{item.properties.closedate | date}}", + "digest": ":calendar: *Closing in the next 7 days* ({{steps.deals | size}} deals)\n{{steps.build | join: '\\n'}}" + } +} diff --git a/packages/core/integrations/data-defined/examples/klaviyo-hubspot.definition.json b/packages/core/integrations/data-defined/examples/klaviyo-hubspot.definition.json new file mode 100644 index 000000000..96dfaa318 --- /dev/null +++ b/packages/core/integrations/data-defined/examples/klaviyo-hubspot.definition.json @@ -0,0 +1,339 @@ +{ + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "klaviyo-hubspot", + "version": "0.3.1", + "display": { + "label": "Klaviyo \u2194 HubSpot", + "description": "Keep Klaviyo profiles and HubSpot contacts in step, move contacts to a lifecycle stage when they join a Klaviyo list, and honour unsubscribes in both directions.", + "category": "Marketing", + "tags": [ + "email", + "marketing", + "crm", + "consent" + ] + }, + "modules": { + "klaviyo": { + "module": "klaviyo", + "version": "^1.1" + }, + "hubspot": { + "module": "hubspot", + "version": "^1.4" + } + }, + "settings": { + "subscriptionType": { + "type": "option", + "options": "hubspot.subscriptionType", + "label": "HubSpot subscription type that mirrors Klaviyo email marketing", + "required": true + } + }, + "syncs": [ + { + "id": "profiles", + "label": "Profiles and contacts", + "description": "One person on both sides, matched by email. HubSpot owns the lifecycle stage; names and phone follow whichever side changed last.", + "source": "klaviyo.profile", + "target": "hubspot.contact", + "direction": "two_way", + "identity": { + "match": [ + { + "source": "attributes.email", + "target": "properties.email", + "transform": "lowercase", + "key": "email" + } + ], + "onAmbiguous": "skip" + }, + "mapping": [ + { + "source": "attributes.email", + "target": "properties.email", + "transform": "lowercase" + }, + { + "source": "attributes.first_name", + "target": "properties.firstname", + "master": "newest" + }, + { + "source": "attributes.last_name", + "target": "properties.lastname", + "master": "newest" + }, + { + "source": "attributes.phone_number", + "target": "properties.phone", + "transform": "to_e164_phone", + "master": "newest" + }, + { + "source": "attributes.organization", + "target": "properties.company", + "master": "target" + }, + { + "source": "attributes.title", + "target": "properties.jobtitle", + "master": "target" + }, + { + "source": "attributes.location.city", + "target": "properties.city", + "direction": "to_target" + }, + { + "source": "attributes.properties.hubspot_lifecycle_stage", + "target": "properties.lifecyclestage", + "direction": "to_source" + }, + { + "source": "attributes.external_id", + "target": "id", + "direction": "to_source" + } + ], + "backfill": { + "trigger": "onInstall", + "pageSize": 200 + }, + "ongoing": { + "mode": "webhook" + }, + "default": "on" + } + ], + "workflows": [ + { + "id": "list-lifecycle", + "label": "Set lifecycle stage when a profile joins a list", + "description": "HubSpot never moves a lifecycle stage backwards, so a customer who joins a newsletter list stays a customer.", + "purpose": "write", + "default": "off", + "settings": { + "list": { + "type": "option", + "options": "klaviyo.list", + "label": "Klaviyo list", + "required": true + }, + "stage": { + "type": "enum", + "label": "Set lifecycle stage to", + "enum": [ + "subscriber", + "lead", + "marketingqualifiedlead" + ], + "default": "subscriber" + } + }, + "trigger": { + "type": "webhook", + "module": "klaviyo", + "topic": "list.subscribed", + "filter": "$.event.data.attributes.event_properties.list_id == $.settings.list" + }, + "steps": [ + { + "id": "contact", + "use": "links.find", + "with": { + "sync": "profiles", + "key": { + "email": "$.record.email" + } + } + }, + { + "id": "write", + "use": "branch", + "cases": [ + { + "when": "$.steps.contact == null", + "do": [ + { + "id": "created", + "use": "hubspot.contact.upsertByEmail", + "with": { + "email": "$.record.email", + "properties": { + "email": "$.record.email", + "firstname": "$.record.first_name", + "lastname": "$.record.last_name", + "lifecyclestage": "$.settings.stage" + } + } + }, + { + "use": "links.upsert", + "with": { + "sync": "profiles", + "ids": { + "klaviyo": "$.record.id", + "hubspot": "$.steps.created.id" + }, + "keys": { + "email": "$.record.email" + } + } + } + ] + } + ], + "else": [ + { + "use": "hubspot.contact.update", + "with": { + "id": "$.steps.contact.hubspot", + "properties": { + "lifecyclestage": "$.settings.stage" + } + } + } + ] + } + ] + }, + { + "id": "unsubscribe-to-hubspot", + "label": "Klaviyo unsubscribes opt the contact out in HubSpot", + "purpose": "maintain", + "default": "on", + "trigger": { + "type": "webhook", + "module": "klaviyo", + "topic": "profile.unsubscribed" + }, + "steps": [ + { + "use": "hubspot.subscription.unsubscribe", + "with": { + "emailAddress": "$.record.email", + "subscriptionId": "$.install.subscriptionType", + "legalBasis": "CONSENT_WITH_NOTICE", + "legalBasisExplanation": "Unsubscribed in Klaviyo ({{event.data.attributes.event_properties.method}}) on {{event.data.attributes.datetime | date}}" + } + } + ] + }, + { + "id": "unsubscribe-to-klaviyo", + "label": "HubSpot opt-outs unsubscribe the Klaviyo profile", + "purpose": "maintain", + "default": "on", + "trigger": { + "type": "webhook", + "module": "hubspot", + "topic": "contact.propertyChange", + "filter": "$.record.propertyName == 'hs_email_optout' && $.record.propertyValue == 'true'" + }, + "steps": [ + { + "id": "contact", + "use": "hubspot.contact.get", + "with": { + "id": "$.record.objectId", + "properties": [ + "email" + ] + } + }, + { + "use": "stop", + "when": "!($.steps.contact.properties.email exists)", + "with": { + "reason": "Contact has no email address" + } + }, + { + "use": "klaviyo.profile.unsubscribe", + "with": { + "email": "$.steps.contact.properties.email" + } + } + ] + }, + { + "id": "won-to-customers-list", + "label": "Add closed-won contacts to a Klaviyo list", + "purpose": "write", + "default": "off", + "settings": { + "list": { + "type": "option", + "options": "klaviyo.list", + "label": "Customers list", + "required": true + } + }, + "trigger": { + "type": "webhook", + "module": "hubspot", + "topic": "deal.propertyChange", + "filter": "$.record.propertyName == 'dealstage' && $.record.propertyValue == 'closedwon'" + }, + "steps": [ + { + "id": "contacts", + "use": "hubspot.deal.contacts", + "with": { + "dealId": "$.record.objectId" + } + }, + { + "id": "subscribe", + "use": "foreach", + "with": { + "over": "$.steps.contacts", + "max": 25 + }, + "do": [ + { + "use": "klaviyo.profile.subscribe", + "when": "$.item.properties.email exists", + "with": { + "listId": "$.settings.list", + "email": "$.item.properties.email", + "channels": [ + "email" + ] + }, + "onError": "continue" + } + ] + } + ] + } + ], + "lifecycle": { + "onInstall": [ + { + "id": "hook", + "use": "klaviyo.webhook.create", + "with": { + "name": "Frigg: HubSpot sync", + "endpointUrl": "$.install.webhookUrls.klaviyo", + "topics": [ + "event:klaviyo.subscribed_to_list", + "event:klaviyo.unsubscribed" + ] + } + } + ], + "onUninstall": [ + { + "use": "klaviyo.webhook.delete", + "with": { + "id": "$.install.runtime.installed.hook.id" + } + } + ] + } +} diff --git a/packages/core/integrations/data-defined/examples/phone-hubspot.definition.json b/packages/core/integrations/data-defined/examples/phone-hubspot.definition.json new file mode 100644 index 000000000..d022b5e15 --- /dev/null +++ b/packages/core/integrations/data-defined/examples/phone-hubspot.definition.json @@ -0,0 +1,637 @@ +{ + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "phone-hubspot", + "version": "1.4.0", + "display": { + "label": "Phone \u2194 HubSpot", + "description": "Two-way contact sync, call and text logging, missed-call follow-up, closed-won thank-you texts, and Phone activity inside HubSpot.", + "category": "CRM", + "tags": [ + "calls", + "sms", + "crm" + ] + }, + "modules": { + "phone": { + "module": "phone", + "version": "^2.1" + }, + "hubspot": { + "module": "hubspot", + "version": "^1.4" + } + }, + "settings": { + "fromNumber": { + "type": "option", + "options": "phone.phoneNumber", + "label": "Phone number to send texts from", + "required": true + } + }, + "syncs": [ + { + "id": "contacts", + "label": "Contacts", + "description": "HubSpot contacts and Phone contacts stay the same person on both sides.", + "source": "hubspot.contact", + "target": "phone.contact", + "direction": "two_way", + "identity": { + "match": [ + { + "source": "properties.email", + "target": "defaultFields.emails[0].value", + "transform": "lowercase", + "key": "email" + }, + { + "source": "properties.phone", + "target": "defaultFields.phoneNumbers[0].value", + "transform": "to_e164_phone", + "key": "phone" + } + ], + "onAmbiguous": "skip" + }, + "mapping": [ + { + "source": "properties.firstname", + "target": "defaultFields.firstName" + }, + { + "source": "properties.lastname", + "target": "defaultFields.lastName" + }, + { + "source": "properties.company", + "target": "defaultFields.company" + }, + { + "source": "properties.email", + "target": "defaultFields.emails[0].value", + "transform": "lowercase" + }, + { + "source": "properties.phone", + "target": "defaultFields.phoneNumbers[0].value", + "transform": "to_e164_phone" + }, + { + "source": "id", + "target": "externalId", + "direction": "to_target" + } + ], + "backfill": { + "trigger": "userAction", + "pageSize": 100 + }, + "ongoing": { + "mode": "webhook" + }, + "default": "on" + } + ], + "workflows": [ + { + "id": "log-call", + "label": "Log calls on the contact", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "phone", + "topic": "call.completed" + }, + "steps": [ + { + "id": "contact", + "use": "links.find", + "with": { + "sync": "contacts", + "key": { + "phone": "$.record.participants[0]" + } + } + }, + { + "use": "stop", + "when": "$.steps.contact == null", + "with": { + "reason": "No linked HubSpot contact for this number" + } + }, + { + "id": "body", + "use": "template", + "with": { + "id": "call-note" + } + }, + { + "id": "engagement", + "use": "hubspot.call.create", + "with": { + "contactId": "$.steps.contact.hubspot", + "properties": { + "hs_call_title": "Phone call ({{record.direction}})", + "hs_call_body": "$.steps.body", + "hs_call_duration": "$.record.duration", + "hs_call_direction": "{{record.direction | upper}}", + "hs_timestamp": "$.record.completedAt" + } + } + } + ] + }, + { + "id": "call-summary", + "label": "Add Phone AI call summaries as notes", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "phone", + "topic": "call.summary.completed" + }, + "steps": [ + { + "id": "calls", + "use": "code", + "with": { + "ref": "@acme/phone-steps#participantsForCall@1.0.0", + "with": { + "callId": "$.record.callId" + } + } + }, + { + "id": "contact", + "use": "links.find", + "with": { + "sync": "contacts", + "key": { + "phone": "$.steps.calls.participants[0]" + } + } + }, + { + "use": "stop", + "when": "$.steps.contact == null", + "with": { + "reason": "No linked contact" + } + }, + { + "use": "hubspot.note.create", + "with": { + "contactId": "$.steps.contact.hubspot", + "properties": { + "hs_note_body": "{{record.summary}}\n\nNext steps: {{record.nextSteps}}", + "hs_timestamp": "$.now" + } + } + } + ] + }, + { + "id": "missed-call-task", + "label": "Follow-up task for missed calls", + "purpose": "write", + "default": "off", + "settings": { + "owner": { + "type": "option", + "options": "hubspot.owner", + "label": "Assign tasks to", + "help": "Leave empty to use the contact owner." + }, + "dueIn": { + "type": "duration", + "label": "Due in", + "default": "4h" + } + }, + "trigger": { + "type": "webhook", + "module": "phone", + "topic": "call.missed", + "filter": "$.record.direction == 'incoming'" + }, + "steps": [ + { + "id": "contact", + "use": "links.find", + "with": { + "sync": "contacts", + "key": { + "phone": "$.record.from" + } + } + }, + { + "use": "stop", + "when": "$.steps.contact == null", + "with": { + "reason": "Caller is not a known contact" + } + }, + { + "id": "full", + "use": "hubspot.contact.get", + "with": { + "id": "$.steps.contact.hubspot", + "properties": [ + "hubspot_owner_id", + "firstname" + ] + } + }, + { + "use": "hubspot.task.create", + "with": { + "contactId": "$.steps.contact.hubspot", + "properties": { + "hs_task_subject": "Call back {{steps.full.properties.firstname | default: 'caller'}} (missed Phone call)", + "hs_task_body": "Missed call from {{record.from}} at {{record.createdAt}}.", + "hs_timestamp": "$.record.createdAt", + "hubspot_owner_id": "{{settings.owner | default: steps.full.properties.hubspot_owner_id}}" + } + } + } + ] + }, + { + "id": "texts-timeline", + "label": "Texts on the contact timeline", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "phone", + "topic": "message.received" + }, + "steps": [ + { + "id": "contact", + "use": "links.find", + "with": { + "sync": "contacts", + "key": { + "phone": "$.record.from" + } + } + }, + { + "use": "stop", + "when": "$.steps.contact == null", + "with": { + "reason": "Sender is not a known contact" + } + }, + { + "use": "hubspot.note.create", + "with": { + "contactId": "$.steps.contact.hubspot", + "properties": { + "hs_note_body": "Text from {{record.from}}: {{record.text}}", + "hs_timestamp": "$.record.createdAt" + } + } + } + ] + }, + { + "id": "won-thank-you", + "label": "Thank-you text on closed won", + "purpose": "notify", + "default": "off", + "settings": { + "message": { + "type": "text", + "label": "Message", + "default": "Thanks for choosing us, {{vars.contact.properties.firstname}}! Text this number any time." + } + }, + "trigger": { + "type": "webhook", + "module": "hubspot", + "topic": "deal.propertyChange", + "filter": "$.record.propertyName == 'dealstage' && $.record.propertyValue == 'closedwon'" + }, + "steps": [ + { + "id": "contacts", + "use": "hubspot.deal.contacts", + "with": { + "dealId": "$.record.objectId" + } + }, + { + "use": "set", + "with": { + "contact": "$.steps.contacts[0]" + } + }, + { + "use": "stop", + "when": "!($.vars.contact.properties.phone exists)", + "with": { + "reason": "Deal has no contact with a phone number" + } + }, + { + "id": "text", + "use": "template", + "with": { + "text": "$.settings.message" + } + }, + { + "use": "phone.message.send", + "with": { + "from": "$.install.fromNumber", + "to": [ + "$.vars.contact.properties.phone" + ], + "content": "$.steps.text" + } + } + ] + }, + { + "id": "stale-contacts", + "label": "Flag contacts with no calls in 30 days", + "purpose": "maintain", + "default": "off", + "trigger": { + "type": "schedule", + "cron": "0 6 * * *", + "timezone": "America/New_York" + }, + "steps": [ + { + "id": "stale", + "use": "hubspot.contact.search", + "with": { + "lastContactedBefore": "-30d", + "limit": 100 + } + }, + { + "id": "each", + "use": "foreach", + "with": { + "over": "$.steps.stale", + "max": 100 + }, + "do": [ + { + "use": "hubspot.contact.update", + "with": { + "id": "$.item.id", + "properties": { + "phone_last_call": "$.item.properties.notes_last_contacted" + } + } + } + ] + } + ] + }, + { + "id": "text-from-card", + "label": "Text the contact from HubSpot", + "purpose": "respond", + "default": "on", + "trigger": { + "type": "userAction", + "label": "Send text", + "from": "phone-card" + }, + "steps": [ + { + "use": "phone.message.send", + "with": { + "from": "$.install.fromNumber", + "to": [ + "$.record.properties.phone" + ], + "content": "$.event.input.message" + } + } + ] + } + ], + "queries": [ + { + "id": "recent-activity", + "description": "The last five calls and texts with this contact's number.", + "steps": [ + { + "id": "calls", + "use": "phone.call.list", + "with": { + "participant": "$.record.properties.phone", + "maxResults": 5 + } + }, + { + "id": "texts", + "use": "phone.message.list", + "with": { + "participants": [ + "$.record.properties.phone" + ], + "maxResults": 5 + } + } + ], + "returns": { + "calls": "$.steps.calls", + "texts": "$.steps.texts" + } + }, + { + "id": "open-deals", + "description": "Open HubSpot deals for the contact in the current Phone conversation.", + "steps": [ + { + "id": "contact", + "use": "links.find", + "with": { + "sync": "contacts", + "key": { + "phone": "$.record.phoneNumber" + } + } + }, + { + "use": "stop", + "when": "$.steps.contact == null", + "with": { + "reason": "Not linked" + } + }, + { + "id": "deals", + "use": "hubspot.deal.listForContact", + "with": { + "contactId": "$.steps.contact.hubspot", + "open": true + } + } + ], + "returns": { + "contactId": "$.steps.contact.hubspot", + "deals": "$.steps.deals" + } + } + ], + "ui": [ + { + "id": "phone-card", + "label": "Phone activity", + "host": "hubspot", + "placement": "crm.record.sidebar", + "objects": [ + "contact" + ], + "data": { + "query": "recent-activity" + }, + "view": { + "component": "Stack", + "children": [ + { + "component": "Heading", + "props": { + "text": "Phone activity" + } + }, + { + "component": "List", + "props": { + "items": "$.data.calls", + "title": "{{item.direction}} call \u00b7 {{item.duration | default: 0}}s", + "meta": "{{item.createdAt | date}}" + } + }, + { + "component": "List", + "props": { + "items": "$.data.texts", + "title": "{{item.text | truncate: 60}}", + "meta": "{{item.createdAt | date}}" + } + }, + { + "component": "Empty", + "when": "$.data.calls == null", + "props": { + "text": "No Phone activity with this number yet." + } + }, + { + "component": "Button", + "props": { + "label": "Send text", + "action": { + "type": "userAction", + "workflow": "text-from-card", + "input": { + "message": { + "type": "text", + "label": "Message" + } + } + } + } + } + ] + } + }, + { + "id": "deals-panel", + "label": "HubSpot deals", + "host": "phone", + "placement": "conversation.sidebar", + "data": { + "query": "open-deals" + }, + "view": { + "component": "Stack", + "children": [ + { + "component": "Heading", + "props": { + "text": "Open deals" + } + }, + { + "component": "Table", + "props": { + "rows": "$.data.deals", + "columns": [ + { + "label": "Deal", + "value": "{{row.properties.dealname}}" + }, + { + "label": "Amount", + "value": "{{row.properties.amount}}" + }, + { + "label": "Stage", + "value": "{{row.properties.dealstage}}" + } + ] + } + }, + { + "component": "Button", + "props": { + "label": "Open in HubSpot", + "action": { + "type": "open_url", + "url": "https://app.hubspot.com/contacts/{{install.portalId}}/record/0-1/{{data.contactId}}" + } + } + } + ] + } + } + ], + "templates": { + "call-note": "Phone call with {{record.from}} \u00b7 {{record.duration}}s ({{record.status}})" + }, + "lifecycle": { + "onInstall": [ + { + "id": "hook", + "use": "phone.webhook.create", + "with": { + "url": "$.install.webhookUrls.phone", + "events": [ + "call.completed", + "call.ringing", + "call.summary.completed", + "message.received", + "contact.updated" + ] + } + } + ], + "onUninstall": [ + { + "use": "phone.webhook.delete", + "with": { + "id": "$.install.runtime.installed.hook.id" + } + } + ] + } +} diff --git a/packages/core/integrations/data-defined/examples/pipedrive-hubspot.definition.json b/packages/core/integrations/data-defined/examples/pipedrive-hubspot.definition.json new file mode 100644 index 000000000..24889284d --- /dev/null +++ b/packages/core/integrations/data-defined/examples/pipedrive-hubspot.definition.json @@ -0,0 +1,136 @@ +{ + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "pipedrive-hubspot", + "version": "1.0.0", + "display": { + "label": "Pipedrive \u21c4 HubSpot people", + "description": "Keeps Pipedrive people and HubSpot contacts the same person on both sides, matched by email then phone. Sync only: no workflows, no UI.", + "category": "CRM", + "tags": [ + "crm", + "contacts", + "sync" + ] + }, + "modules": { + "pipedrive": { + "module": "pipedrive", + "version": "^2.1" + }, + "hubspot": { + "module": "hubspot", + "version": "^3" + } + }, + "settings": {}, + "syncs": [ + { + "id": "people", + "label": "People and contacts", + "description": "A new or changed person on either side is created or updated on the other. A blank value never overwrites a filled one.", + "source": "pipedrive.person", + "target": "hubspot.contact", + "direction": "two_way", + "identity": { + "match": [ + { + "source": "emails[0].value", + "target": "properties.email", + "transform": "lowercase", + "key": "email" + }, + { + "source": "phones[0].value", + "target": "properties.phone", + "transform": "to_e164_phone", + "key": "phone" + } + ], + "onAmbiguous": "skip" + }, + "mapping": [ + { + "source": "first_name", + "target": "properties.firstname", + "transform": "trim" + }, + { + "source": "last_name", + "target": "properties.lastname", + "transform": "trim" + }, + { + "source": "emails[0].value", + "target": "properties.email", + "transform": "lowercase" + }, + { + "source": "phones[0].value", + "target": "properties.phone", + "transform": "to_e164_phone" + }, + { + "target": "properties.hs_lead_status", + "value": "NEW", + "direction": "to_target" + } + ], + "backfill": { + "trigger": "userAction", + "pageSize": 100 + }, + "ongoing": { + "mode": "webhook" + }, + "default": "on" + } + ], + "workflows": [], + "lifecycle": { + "onInstall": [ + { + "id": "personCreated", + "use": "pipedrive.webhook.create", + "with": { + "subscription_url": "$.install.webhookUrls.pipedrive", + "event_action": "create", + "event_object": "person", + "version": "2.0" + } + }, + { + "id": "personChanged", + "use": "pipedrive.webhook.create", + "with": { + "subscription_url": "$.install.webhookUrls.pipedrive", + "event_action": "change", + "event_object": "person", + "version": "2.0" + } + } + ], + "onUninstall": [ + { + "use": "foreach", + "with": { + "over": [ + "$.install.runtime.installed.personCreated.id", + "$.install.runtime.installed.personChanged.id" + ], + "max": 2, + "where": "$.item exists" + }, + "do": [ + { + "use": "pipedrive.webhook.delete", + "with": { + "id": "$.item" + }, + "onError": "continue" + } + ] + } + ] + } +} diff --git a/packages/core/integrations/data-defined/examples/pipedrive-phone.definition.json b/packages/core/integrations/data-defined/examples/pipedrive-phone.definition.json new file mode 100644 index 000000000..7aae49fc5 --- /dev/null +++ b/packages/core/integrations/data-defined/examples/pipedrive-phone.definition.json @@ -0,0 +1,340 @@ +{ + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "pipedrive-phone", + "version": "0.3.1", + "display": { + "label": "Phone \u2194 Pipedrive", + "description": "Two-way person sync, Phone calls logged as Pipedrive call activities, missed calls as call-back activities, and recent Phone activity on the person panel.", + "category": "CRM", + "tags": [ + "calls", + "sms", + "crm" + ] + }, + "modules": { + "phone": { + "module": "phone", + "version": "^2.1" + }, + "pipedrive": { + "module": "pipedrive", + "version": "^2.1" + } + }, + "syncs": [ + { + "id": "people", + "label": "People and contacts", + "source": "pipedrive.person", + "target": "phone.contact", + "direction": "two_way", + "identity": { + "match": [ + { + "source": "phones[0].value", + "target": "defaultFields.phoneNumbers[0].value", + "transform": "to_e164_phone", + "key": "phone" + }, + { + "source": "emails[0].value", + "target": "defaultFields.emails[0].value", + "transform": "lowercase", + "key": "email" + } + ], + "onAmbiguous": "skip" + }, + "mapping": [ + { + "source": "first_name", + "target": "defaultFields.firstName" + }, + { + "source": "last_name", + "target": "defaultFields.lastName" + }, + { + "source": "emails[0].value", + "target": "defaultFields.emails[0].value", + "transform": "lowercase" + }, + { + "source": "phones[0].value", + "target": "defaultFields.phoneNumbers[0].value", + "transform": "to_e164_phone" + }, + { + "source": "id", + "target": "externalId", + "direction": "to_target" + } + ], + "backfill": { + "trigger": "userAction", + "pageSize": 100 + }, + "ongoing": { + "mode": "webhook" + }, + "default": "on" + } + ], + "workflows": [ + { + "id": "log-call", + "label": "Log Phone calls as call activities", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "phone", + "topic": "call.completed" + }, + "steps": [ + { + "id": "person", + "use": "links.find", + "with": { + "sync": "people", + "key": { + "phone": "$.record.participants[0]" + } + } + }, + { + "use": "stop", + "when": "$.steps.person == null", + "with": { + "reason": "Caller is not a linked Pipedrive person" + } + }, + { + "use": "pipedrive.activity.create", + "with": { + "activity": { + "subject": "Phone call ({{record.direction}})", + "type": "call", + "done": true, + "due_date": "{{record.completedAt | date}}", + "person_id": "$.steps.person.pipedrive", + "note": "{{record.direction | upper}} call with {{record.participants[0]}}, {{record.duration}}s ({{record.status}})." + } + } + } + ] + }, + { + "id": "missed-call-activity", + "label": "Call-back activity for missed calls", + "purpose": "write", + "default": "off", + "settings": { + "owner": { + "type": "option", + "options": "pipedrive.user", + "label": "Assign to", + "help": "Leave empty to use the person's owner." + } + }, + "trigger": { + "type": "webhook", + "module": "phone", + "topic": "call.missed", + "filter": "$.record.direction == 'incoming'" + }, + "steps": [ + { + "id": "person", + "use": "links.find", + "with": { + "sync": "people", + "key": { + "phone": "$.record.from" + } + } + }, + { + "use": "stop", + "when": "$.steps.person == null", + "with": { + "reason": "Caller is not a known person" + } + }, + { + "id": "full", + "use": "pipedrive.person.get", + "with": { + "id": "$.steps.person.pipedrive" + } + }, + { + "use": "pipedrive.activity.create", + "with": { + "activity": { + "subject": "Call back {{steps.full.first_name | default: 'caller'}} (missed Phone call)", + "type": "call", + "done": false, + "due_date": "{{record.createdAt | date}}", + "person_id": "$.steps.person.pipedrive", + "owner_id": "{{settings.owner | default: steps.full.owner_id}}", + "note": "Missed call from {{record.from}} at {{record.createdAt}}." + } + } + } + ] + } + ], + "queries": [ + { + "id": "phone-activity", + "description": "Last five Phone calls and texts with this person's phone number.", + "steps": [ + { + "use": "stop", + "when": "!($.record.phones[0].value exists)", + "with": { + "reason": "Person has no phone number" + } + }, + { + "id": "calls", + "use": "phone.call.list", + "with": { + "participant": "$.record.phones[0].value", + "maxResults": 5 + } + }, + { + "id": "texts", + "use": "phone.message.list", + "with": { + "participants": [ + "$.record.phones[0].value" + ], + "maxResults": 5 + } + } + ], + "returns": { + "calls": "$.steps.calls", + "texts": "$.steps.texts" + } + } + ], + "ui": [ + { + "id": "phone-panel", + "label": "Phone", + "host": "pipedrive", + "placement": "person.details.panel", + "objects": [ + "person" + ], + "data": { + "query": "phone-activity" + }, + "view": { + "component": "Stack", + "children": [ + { + "component": "Heading", + "props": { + "text": "Phone activity" + } + }, + { + "component": "List", + "props": { + "items": "$.data.calls", + "title": "{{item.direction}} call \u00b7 {{item.duration | default: 0}}s", + "meta": "{{item.createdAt | date}}" + } + }, + { + "component": "List", + "props": { + "items": "$.data.texts", + "title": "{{item.text | truncate: 60}}", + "meta": "{{item.createdAt | date}}" + } + }, + { + "component": "Empty", + "when": "$.data.calls[0] == null && $.data.texts[0] == null", + "props": { + "text": "No Phone calls or texts with this person yet." + } + } + ] + } + } + ], + "lifecycle": { + "onInstall": [ + { + "id": "phoneHook", + "use": "phone.webhook.create", + "with": { + "url": "$.install.webhookUrls.phone", + "events": [ + "call.completed", + "call.ringing", + "contact.updated" + ] + } + }, + { + "id": "personCreated", + "use": "pipedrive.webhook.create", + "with": { + "subscription_url": "$.install.webhookUrls.pipedrive", + "event_action": "create", + "event_object": "person", + "version": "2.0" + } + }, + { + "id": "personChanged", + "use": "pipedrive.webhook.create", + "with": { + "subscription_url": "$.install.webhookUrls.pipedrive", + "event_action": "change", + "event_object": "person", + "version": "2.0" + } + } + ], + "onUninstall": [ + { + "use": "phone.webhook.delete", + "with": { + "id": "$.install.runtime.installed.phoneHook.id" + }, + "onError": "continue" + }, + { + "id": "unhook", + "use": "foreach", + "with": { + "over": [ + "$.install.runtime.installed.personCreated.id", + "$.install.runtime.installed.personChanged.id" + ], + "max": 5 + }, + "do": [ + { + "use": "pipedrive.webhook.delete", + "with": { + "id": "$.item" + }, + "onError": "continue" + } + ] + } + ] + } +} diff --git a/packages/core/integrations/data-defined/examples/records/phone-hubspot.definition-version.json b/packages/core/integrations/data-defined/examples/records/phone-hubspot.definition-version.json new file mode 100644 index 000000000..ab26da19c --- /dev/null +++ b/packages/core/integrations/data-defined/examples/records/phone-hubspot.definition-version.json @@ -0,0 +1,652 @@ +{ + "$comment": "IntegrationDefinitionVersion row: one per published version, never updated after publish (status may move published -> deprecated -> retired). See schema/records.md.", + "id": "idv_01JB7Q2M4XQ", + "definitionId": "phone-hubspot", + "version": "1.4.0", + "schemaVersion": "0.1", + "status": "published", + "checksum": "sha256:386c69e18a51a7cf061634309eca3e8267c1a59629a388498eddaad9eb44be59", + "changeLevel": "minor", + "previousVersion": "1.3.2", + "notes": "Adds the closed-won thank-you text (off by default) and the Phone deals panel.", + "createdBy": "usr_alex", + "createdAt": "2026-10-17T21:04:11Z", + "publishedAt": "2026-10-17T21:10:02Z", + "document": { + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "phone-hubspot", + "version": "1.4.0", + "display": { + "label": "Phone ↔ HubSpot", + "description": "Two-way contact sync, call and text logging, missed-call follow-up, closed-won thank-you texts, and Phone activity inside HubSpot.", + "category": "CRM", + "tags": [ + "calls", + "sms", + "crm" + ] + }, + "modules": { + "phone": { + "module": "phone", + "version": "^2.1" + }, + "hubspot": { + "module": "hubspot", + "version": "^1.4" + } + }, + "settings": { + "fromNumber": { + "type": "option", + "options": "phone.phoneNumber", + "label": "Phone number to send texts from", + "required": true + } + }, + "syncs": [ + { + "id": "contacts", + "label": "Contacts", + "description": "HubSpot contacts and Phone contacts stay the same person on both sides.", + "source": "hubspot.contact", + "target": "phone.contact", + "direction": "two_way", + "identity": { + "match": [ + { + "source": "properties.email", + "target": "defaultFields.emails[0].value", + "transform": "lowercase", + "key": "email" + }, + { + "source": "properties.phone", + "target": "defaultFields.phoneNumbers[0].value", + "transform": "to_e164_phone", + "key": "phone" + } + ], + "onAmbiguous": "skip" + }, + "mapping": [ + { + "source": "properties.firstname", + "target": "defaultFields.firstName" + }, + { + "source": "properties.lastname", + "target": "defaultFields.lastName" + }, + { + "source": "properties.company", + "target": "defaultFields.company" + }, + { + "source": "properties.email", + "target": "defaultFields.emails[0].value", + "transform": "lowercase" + }, + { + "source": "properties.phone", + "target": "defaultFields.phoneNumbers[0].value", + "transform": "to_e164_phone" + }, + { + "source": "id", + "target": "externalId", + "direction": "to_target" + } + ], + "backfill": { + "trigger": "userAction", + "pageSize": 100 + }, + "ongoing": { + "mode": "webhook" + }, + "default": "on" + } + ], + "workflows": [ + { + "id": "log-call", + "label": "Log calls on the contact", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "phone", + "topic": "call.completed" + }, + "steps": [ + { + "id": "contact", + "use": "links.find", + "with": { + "sync": "contacts", + "key": { + "phone": "$.record.participants[0]" + } + } + }, + { + "use": "stop", + "when": "$.steps.contact == null", + "with": { + "reason": "No linked HubSpot contact for this number" + } + }, + { + "id": "body", + "use": "template", + "with": { + "id": "call-note" + } + }, + { + "id": "engagement", + "use": "hubspot.call.create", + "with": { + "contactId": "$.steps.contact.hubspot", + "properties": { + "hs_call_title": "Phone call ({{record.direction}})", + "hs_call_body": "$.steps.body", + "hs_call_duration": "$.record.duration", + "hs_call_direction": "{{record.direction | upper}}", + "hs_timestamp": "$.record.completedAt" + } + } + } + ] + }, + { + "id": "call-summary", + "label": "Add Phone AI call summaries as notes", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "phone", + "topic": "call.summary.completed" + }, + "steps": [ + { + "id": "calls", + "use": "code", + "with": { + "ref": "@acme/phone-steps#participantsForCall@1.0.0", + "with": { + "callId": "$.record.callId" + } + } + }, + { + "id": "contact", + "use": "links.find", + "with": { + "sync": "contacts", + "key": { + "phone": "$.steps.calls.participants[0]" + } + } + }, + { + "use": "stop", + "when": "$.steps.contact == null", + "with": { + "reason": "No linked contact" + } + }, + { + "use": "hubspot.note.create", + "with": { + "contactId": "$.steps.contact.hubspot", + "properties": { + "hs_note_body": "{{record.summary}}\n\nNext steps: {{record.nextSteps}}", + "hs_timestamp": "$.now" + } + } + } + ] + }, + { + "id": "missed-call-task", + "label": "Follow-up task for missed calls", + "purpose": "write", + "default": "off", + "settings": { + "owner": { + "type": "option", + "options": "hubspot.owner", + "label": "Assign tasks to", + "help": "Leave empty to use the contact owner." + }, + "dueIn": { + "type": "duration", + "label": "Due in", + "default": "4h" + } + }, + "trigger": { + "type": "webhook", + "module": "phone", + "topic": "call.missed", + "filter": "$.record.direction == 'incoming'" + }, + "steps": [ + { + "id": "contact", + "use": "links.find", + "with": { + "sync": "contacts", + "key": { + "phone": "$.record.from" + } + } + }, + { + "use": "stop", + "when": "$.steps.contact == null", + "with": { + "reason": "Caller is not a known contact" + } + }, + { + "id": "full", + "use": "hubspot.contact.get", + "with": { + "id": "$.steps.contact.hubspot", + "properties": [ + "hubspot_owner_id", + "firstname" + ] + } + }, + { + "use": "hubspot.task.create", + "with": { + "contactId": "$.steps.contact.hubspot", + "properties": { + "hs_task_subject": "Call back {{steps.full.properties.firstname | default: 'caller'}} (missed Phone call)", + "hs_task_body": "Missed call from {{record.from}} at {{record.createdAt}}.", + "hs_timestamp": "$.record.createdAt", + "hubspot_owner_id": "{{settings.owner | default: steps.full.properties.hubspot_owner_id}}" + } + } + } + ] + }, + { + "id": "texts-timeline", + "label": "Texts on the contact timeline", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "phone", + "topic": "message.received" + }, + "steps": [ + { + "id": "contact", + "use": "links.find", + "with": { + "sync": "contacts", + "key": { + "phone": "$.record.from" + } + } + }, + { + "use": "stop", + "when": "$.steps.contact == null", + "with": { + "reason": "Sender is not a known contact" + } + }, + { + "use": "hubspot.note.create", + "with": { + "contactId": "$.steps.contact.hubspot", + "properties": { + "hs_note_body": "Text from {{record.from}}: {{record.text}}", + "hs_timestamp": "$.record.createdAt" + } + } + } + ] + }, + { + "id": "won-thank-you", + "label": "Thank-you text on closed won", + "purpose": "notify", + "default": "off", + "settings": { + "message": { + "type": "text", + "label": "Message", + "default": "Thanks for choosing us, {{vars.contact.properties.firstname}}! Text this number any time." + } + }, + "trigger": { + "type": "webhook", + "module": "hubspot", + "topic": "deal.propertyChange", + "filter": "$.record.propertyName == 'dealstage' && $.record.propertyValue == 'closedwon'" + }, + "steps": [ + { + "id": "contacts", + "use": "hubspot.deal.contacts", + "with": { + "dealId": "$.record.objectId" + } + }, + { + "use": "set", + "with": { + "contact": "$.steps.contacts[0]" + } + }, + { + "use": "stop", + "when": "!($.vars.contact.properties.phone exists)", + "with": { + "reason": "Deal has no contact with a phone number" + } + }, + { + "id": "text", + "use": "template", + "with": { + "text": "$.settings.message" + } + }, + { + "use": "phone.message.send", + "with": { + "from": "$.install.fromNumber", + "to": [ + "$.vars.contact.properties.phone" + ], + "content": "$.steps.text" + } + } + ] + }, + { + "id": "stale-contacts", + "label": "Flag contacts with no calls in 30 days", + "purpose": "maintain", + "default": "off", + "trigger": { + "type": "schedule", + "cron": "0 6 * * *", + "timezone": "America/New_York" + }, + "steps": [ + { + "id": "stale", + "use": "hubspot.contact.search", + "with": { + "lastContactedBefore": "-30d", + "limit": 100 + } + }, + { + "id": "each", + "use": "foreach", + "with": { + "over": "$.steps.stale", + "max": 100 + }, + "do": [ + { + "use": "hubspot.contact.update", + "with": { + "id": "$.item.id", + "properties": { + "phone_last_call": "$.item.properties.notes_last_contacted" + } + } + } + ] + } + ] + }, + { + "id": "text-from-card", + "label": "Text the contact from HubSpot", + "purpose": "respond", + "default": "on", + "trigger": { + "type": "userAction", + "label": "Send text", + "from": "phone-card" + }, + "steps": [ + { + "use": "phone.message.send", + "with": { + "from": "$.install.fromNumber", + "to": [ + "$.record.properties.phone" + ], + "content": "$.event.input.message" + } + } + ] + } + ], + "queries": [ + { + "id": "recent-activity", + "description": "The last five calls and texts with this contact's number.", + "steps": [ + { + "id": "calls", + "use": "phone.call.list", + "with": { + "participant": "$.record.properties.phone", + "maxResults": 5 + } + }, + { + "id": "texts", + "use": "phone.message.list", + "with": { + "participants": [ + "$.record.properties.phone" + ], + "maxResults": 5 + } + } + ], + "returns": { + "calls": "$.steps.calls", + "texts": "$.steps.texts" + } + }, + { + "id": "open-deals", + "description": "Open HubSpot deals for the contact in the current Phone conversation.", + "steps": [ + { + "id": "contact", + "use": "links.find", + "with": { + "sync": "contacts", + "key": { + "phone": "$.record.phoneNumber" + } + } + }, + { + "use": "stop", + "when": "$.steps.contact == null", + "with": { + "reason": "Not linked" + } + }, + { + "id": "deals", + "use": "hubspot.deal.listForContact", + "with": { + "contactId": "$.steps.contact.hubspot", + "open": true + } + } + ], + "returns": { + "contactId": "$.steps.contact.hubspot", + "deals": "$.steps.deals" + } + } + ], + "ui": [ + { + "id": "phone-card", + "label": "Phone activity", + "host": "hubspot", + "placement": "crm.record.sidebar", + "objects": [ + "contact" + ], + "data": { + "query": "recent-activity" + }, + "view": { + "component": "Stack", + "children": [ + { + "component": "Heading", + "props": { + "text": "Phone activity" + } + }, + { + "component": "List", + "props": { + "items": "$.data.calls", + "title": "{{item.direction}} call · {{item.duration | default: 0}}s", + "meta": "{{item.createdAt | date}}" + } + }, + { + "component": "List", + "props": { + "items": "$.data.texts", + "title": "{{item.text | truncate: 60}}", + "meta": "{{item.createdAt | date}}" + } + }, + { + "component": "Empty", + "when": "$.data.calls == null", + "props": { + "text": "No Phone activity with this number yet." + } + }, + { + "component": "Button", + "props": { + "label": "Send text", + "action": { + "type": "userAction", + "workflow": "text-from-card", + "input": { + "message": { + "type": "text", + "label": "Message" + } + } + } + } + } + ] + } + }, + { + "id": "deals-panel", + "label": "HubSpot deals", + "host": "phone", + "placement": "conversation.sidebar", + "data": { + "query": "open-deals" + }, + "view": { + "component": "Stack", + "children": [ + { + "component": "Heading", + "props": { + "text": "Open deals" + } + }, + { + "component": "Table", + "props": { + "rows": "$.data.deals", + "columns": [ + { + "label": "Deal", + "value": "{{row.properties.dealname}}" + }, + { + "label": "Amount", + "value": "{{row.properties.amount}}" + }, + { + "label": "Stage", + "value": "{{row.properties.dealstage}}" + } + ] + } + }, + { + "component": "Button", + "props": { + "label": "Open in HubSpot", + "action": { + "type": "open_url", + "url": "https://app.hubspot.com/contacts/{{install.portalId}}/record/0-1/{{data.contactId}}" + } + } + } + ] + } + } + ], + "templates": { + "call-note": "Phone call with {{record.from}} · {{record.duration}}s ({{record.status}})" + }, + "lifecycle": { + "onInstall": [ + { + "id": "hook", + "use": "phone.webhook.create", + "with": { + "url": "$.install.webhookUrls.phone", + "events": [ + "call.completed", + "call.ringing", + "call.summary.completed", + "message.received", + "contact.updated" + ] + } + } + ], + "onUninstall": [ + { + "use": "phone.webhook.delete", + "with": { + "id": "$.install.runtime.installed.hook.id" + } + } + ] + } + } +} diff --git a/packages/core/integrations/data-defined/examples/records/phone-hubspot.integration.json b/packages/core/integrations/data-defined/examples/records/phone-hubspot.integration.json new file mode 100644 index 000000000..261d567ba --- /dev/null +++ b/packages/core/integrations/data-defined/examples/records/phone-hubspot.integration.json @@ -0,0 +1,86 @@ +{ + "$comment": "Frigg Integration row (existing model). Only `config` is new in shape; everything else is what Frigg stores today.", + "id": "int_7f3a9c", + "userId": "usr_acme_admin", + "status": "ENABLED", + "version": "1.4.0", + "entities": [ + { + "id": "ent_phone_51", + "moduleName": "phone", + "externalId": "ws_9a1" + }, + { + "id": "ent_hs_88", + "moduleName": "hubspot", + "externalId": "12345678" + } + ], + "config": { + "definition": { + "id": "phone-hubspot", + "version": "1.4.0", + "checksum": "sha256:386c69e18a51a7cf061634309eca3e8267c1a59629a388498eddaad9eb44be59" + }, + "settings": { + "fromNumber": "+15125550100" + }, + "syncs": { + "contacts": { + "enabled": true + } + }, + "workflows": { + "log-call": { + "enabled": true + }, + "call-summary": { + "enabled": true + }, + "missed-call-task": { + "enabled": true, + "settings": { + "owner": "81273645", + "dueIn": "2h" + } + }, + "texts-timeline": { + "enabled": true + }, + "won-thank-you": { + "enabled": false + }, + "stale-contacts": { + "enabled": false + }, + "text-from-card": { + "enabled": true + } + }, + "ui": { + "phone-card": { + "enabled": true + }, + "deals-panel": { + "enabled": true + } + }, + "runtime": { + "installed": { + "hook": { + "id": "WHk2c9", + "url": "https://api.example.com/api/data-defined-integration/webhooks/int_7f3a9c?module=phone", + "events": [ + "call.completed", + "call.ringing", + "call.summary.completed", + "message.received", + "contact.updated" + ] + } + } + } + }, + "createdAt": "2026-09-02T15:20:00Z", + "updatedAt": "2026-10-18T09:12:44Z" +} diff --git a/packages/core/integrations/data-defined/examples/records/phone-hubspot.mappings.json b/packages/core/integrations/data-defined/examples/records/phone-hubspot.mappings.json new file mode 100644 index 000000000..75bf5e7a1 --- /dev/null +++ b/packages/core/integrations/data-defined/examples/records/phone-hubspot.mappings.json @@ -0,0 +1,39 @@ +{ + "$comment": "IntegrationMapping rows (existing table). link:* = the same record on both sides; idx:* = identity index (normalised email/phone); evt:* = an event already handled (insert-only, so a redelivery is a no-op).", + "rows": [ + { + "integrationId": "int_7f3a9c", + "sourceId": "idx:contacts:phone:+15125550142", + "mapping": { + "hubspot": "51", + "phone": "CT7", + "linkedAt": "2026-09-02T15:31:10Z" + } + }, + { + "integrationId": "int_7f3a9c", + "sourceId": "link:contacts:hubspot:51", + "mapping": { + "hubspot": "51", + "phone": "CT7", + "linkedAt": "2026-09-02T15:31:10Z" + } + }, + { + "integrationId": "int_7f3a9c", + "sourceId": "link:contacts:phone:CT7", + "mapping": { + "hubspot": "51", + "phone": "CT7", + "linkedAt": "2026-09-02T15:31:10Z" + } + }, + { + "integrationId": "int_7f3a9c", + "sourceId": "evt:phone-hubspot:log-call:EVc0a1", + "mapping": { + "at": "2026-10-18T16:08:10.523Z" + } + } + ] +} diff --git a/packages/core/integrations/data-defined/examples/records/phone-hubspot.workflow-run.json b/packages/core/integrations/data-defined/examples/records/phone-hubspot.workflow-run.json new file mode 100644 index 000000000..0147ea5b5 --- /dev/null +++ b/packages/core/integrations/data-defined/examples/records/phone-hubspot.workflow-run.json @@ -0,0 +1,62 @@ +{ + "$comment": "WorkflowRun row written by the runtime for one delivered Phone call.completed event. `outbound` is the captured HubSpot call (shown here for documentation; production stores a digest).", + "id": "run_01JB8A0K", + "integrationId": "int_7f3a9c", + "definition": { + "id": "phone-hubspot", + "version": "1.4.0" + }, + "workflowId": "log-call", + "trigger": { + "type": "webhook", + "eventId": "EVc0a1" + }, + "startedAt": "2026-10-18T14:07:13.037Z", + "status": "completed", + "endedAt": "2026-10-18T14:07:13.074Z", + "trace": [ + { + "id": "contact", + "use": "links.find", + "at": "steps[0]", + "status": "ok", + "ms": 0 + }, + { + "id": "stop#1", + "use": "stop", + "at": "steps[1]", + "status": "skipped" + }, + { + "id": "body", + "use": "template", + "at": "steps[2]", + "status": "ok", + "ms": 2 + }, + { + "id": "engagement", + "use": "hubspot.call.create", + "at": "steps[3]", + "status": "ok", + "ms": 0 + } + ], + "outbound": [ + { + "module": "hubspot", + "method": "createCallEngagement", + "args": { + "contactId": "51", + "properties": { + "hs_call_title": "Phone call (incoming)", + "hs_call_body": "Phone call with +15125550142 · 312s (completed)", + "hs_call_duration": 312, + "hs_call_direction": "INCOMING", + "hs_timestamp": "2026-10-18T14:07:12Z" + } + } + } + ] +} diff --git a/packages/core/integrations/data-defined/examples/shopify-klaviyo.definition.json b/packages/core/integrations/data-defined/examples/shopify-klaviyo.definition.json new file mode 100644 index 000000000..51854a03d --- /dev/null +++ b/packages/core/integrations/data-defined/examples/shopify-klaviyo.definition.json @@ -0,0 +1,488 @@ +{ + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "shopify-klaviyo", + "version": "1.5.0", + "display": { + "label": "Shopify \u2192 Klaviyo", + "description": "Customers become Klaviyo profiles, and orders, ordered products, abandoned checkouts and fulfillments become Klaviyo events for flows. The Shopify customer page shows the customer's Klaviyo lists.", + "category": "Marketing", + "tags": [ + "ecommerce", + "email", + "events" + ] + }, + "modules": { + "shopify": { + "module": "shopify", + "version": "^3.1" + }, + "klaviyo": { + "module": "klaviyo", + "version": "^1.1" + } + }, + "syncs": [ + { + "id": "customers", + "label": "Customers", + "source": "shopify.customer", + "target": "klaviyo.profile", + "direction": "one_way", + "identity": { + "match": [ + { + "source": "email", + "target": "attributes.email", + "transform": "lowercase", + "key": "email" + } + ] + }, + "mapping": [ + { + "source": "email", + "target": "attributes.email", + "transform": "lowercase" + }, + { + "source": "first_name", + "target": "attributes.first_name" + }, + { + "source": "last_name", + "target": "attributes.last_name" + }, + { + "source": "phone", + "target": "attributes.phone_number", + "transform": "to_e164_phone" + }, + { + "source": "default_address.company", + "target": "attributes.organization" + }, + { + "source": "default_address.city", + "target": "attributes.location.city" + }, + { + "source": "default_address.province_code", + "target": "attributes.location.region" + }, + { + "source": "default_address.country_code", + "target": "attributes.location.country" + }, + { + "source": "default_address.zip", + "target": "attributes.location.zip" + }, + { + "source": "total_spent", + "target": "attributes.properties.lifetime_value", + "transform": "number" + }, + { + "source": "tags", + "target": "attributes.properties.shopify_tags" + }, + { + "source": "id", + "target": "attributes.external_id" + } + ], + "filter": "$.record.email exists", + "backfill": { + "trigger": "onInstall", + "pageSize": 250 + }, + "ongoing": { + "mode": "webhook" + }, + "default": "on" + } + ], + "workflows": [ + { + "id": "placed-order", + "label": "Placed Order and Ordered Product events", + "description": "One \"Placed Order\" event per order and one \"Ordered Product\" per line item, the shape Klaviyo's ecommerce flows expect.", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "shopify", + "topic": "order.created", + "filter": "$.record.email exists" + }, + "steps": [ + { + "use": "klaviyo.event.create", + "with": { + "metric": "Placed Order", + "profile": { + "email": "$.record.email", + "external_id": "{{record.customer.id}}" + }, + "properties": { + "OrderId": "$.record.id", + "OrderNumber": "$.record.name", + "Items": "$.record.line_items", + "DiscountCodes": "$.record.discount_codes", + "SourceName": "$.record.source_name", + "ShippingCity": "$.record.shipping_address.city" + }, + "value": "{{record.total_price | number}}", + "time": "$.record.created_at", + "uniqueId": "order-{{record.id}}" + } + }, + { + "id": "products", + "use": "foreach", + "with": { + "over": "$.record.line_items", + "max": 100 + }, + "do": [ + { + "id": "price", + "use": "map", + "with": { + "record": "$.item", + "mapping": [ + { + "source": "price", + "target": "value", + "transform": "number" + } + ] + } + }, + { + "use": "klaviyo.event.create", + "with": { + "metric": "Ordered Product", + "profile": { + "email": "$.record.email", + "external_id": "{{record.customer.id}}" + }, + "properties": { + "OrderId": "$.record.id", + "ProductID": "$.item.product_id", + "SKU": "$.item.sku", + "ProductName": "$.item.title", + "VariantName": "$.item.variant_title", + "Quantity": "$.item.quantity" + }, + "value": "$.steps.price.value", + "time": "$.record.created_at", + "uniqueId": "order-{{record.id}}-line-{{item.id}}" + } + } + ] + } + ] + }, + { + "id": "abandoned-checkout", + "label": "Started Checkout events for abandoned checkouts", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "shopify", + "topic": "checkout.abandoned", + "filter": "$.record.email exists && !($.record.completed_at exists)" + }, + "steps": [ + { + "use": "klaviyo.event.create", + "with": { + "metric": "Started Checkout", + "profile": { + "email": "$.record.email", + "external_id": "{{record.customer.id}}" + }, + "properties": { + "CheckoutURL": "$.record.abandoned_checkout_url", + "Items": "$.record.line_items", + "Currency": "$.record.currency" + }, + "value": "{{record.total_price | number}}", + "time": "$.record.created_at", + "uniqueId": "checkout-{{record.token}}" + } + } + ] + }, + { + "id": "fulfilled-order", + "label": "Fulfilled Order events", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "shopify", + "topic": "fulfillment.created" + }, + "steps": [ + { + "id": "order", + "use": "shopify.order.get", + "with": { + "id": "$.record.order_id", + "fields": "id,name,email,customer,total_price" + } + }, + { + "use": "stop", + "when": "!($.steps.order.email exists)", + "with": { + "reason": "Order has no email" + } + }, + { + "use": "klaviyo.event.create", + "with": { + "metric": "Fulfilled Order", + "profile": { + "email": "$.steps.order.email", + "external_id": "{{steps.order.customer.id}}" + }, + "properties": { + "OrderId": "$.steps.order.id", + "OrderNumber": "$.steps.order.name", + "TrackingCompany": "$.record.tracking_company", + "TrackingNumber": "$.record.tracking_number", + "TrackingUrl": "$.record.tracking_url" + }, + "time": "$.record.created_at", + "uniqueId": "fulfillment-{{record.id}}" + } + } + ] + }, + { + "id": "add-to-list", + "label": "Subscribe the customer to a Klaviyo list", + "purpose": "respond", + "default": "on", + "settings": { + "list": { + "type": "option", + "options": "klaviyo.list", + "label": "List the button subscribes to", + "required": true + } + }, + "trigger": { + "type": "userAction", + "label": "Subscribe to list", + "from": "klaviyo-block" + }, + "steps": [ + { + "use": "stop", + "when": "!($.record.email exists)", + "with": { + "reason": "Customer has no email" + } + }, + { + "use": "klaviyo.profile.subscribe", + "with": { + "listId": "$.settings.list", + "email": "$.record.email", + "channels": [ + "email" + ] + } + } + ] + } + ], + "queries": [ + { + "id": "klaviyo-profile", + "description": "The Klaviyo profile for this Shopify customer and the lists it is on.", + "steps": [ + { + "use": "stop", + "when": "!($.record.email exists)", + "with": { + "reason": "Customer has no email" + } + }, + { + "id": "profiles", + "use": "klaviyo.profile.search", + "with": { + "email": "$.record.email" + } + }, + { + "use": "stop", + "when": "$.steps.profiles[0] == null", + "with": { + "reason": "No Klaviyo profile" + } + }, + { + "id": "lists", + "use": "klaviyo.profile.lists", + "with": { + "id": "$.steps.profiles[0].id" + } + } + ], + "returns": { + "profile": "$.steps.profiles[0]", + "lists": "$.steps.lists" + } + } + ], + "ui": [ + { + "id": "klaviyo-block", + "label": "Klaviyo", + "host": "shopify", + "placement": "admin.customer-details.block", + "objects": [ + "customer" + ], + "data": { + "query": "klaviyo-profile" + }, + "view": { + "component": "Stack", + "children": [ + { + "component": "Heading", + "props": { + "text": "Klaviyo" + } + }, + { + "component": "Empty", + "when": "!($.data.profile exists)", + "props": { + "text": "No Klaviyo profile for this customer yet." + } + }, + { + "component": "KeyValue", + "when": "$.data.profile exists", + "props": { + "items": [ + { + "label": "Email consent", + "value": "{{data.profile.attributes.subscriptions.email.marketing.consent | default: 'NEVER_SUBSCRIBED'}}" + }, + { + "label": "Lifetime value", + "value": "{{data.profile.attributes.properties.lifetime_value | default: 0}}" + } + ] + } + }, + { + "component": "List", + "props": { + "items": "$.data.lists", + "title": "{{item.attributes.name}}", + "meta": "since {{item.attributes.created | date}}" + } + }, + { + "component": "Button", + "props": { + "label": "Subscribe to list", + "action": { + "type": "userAction", + "workflow": "add-to-list" + } + } + }, + { + "component": "Button", + "when": "$.data.profile exists", + "props": { + "label": "Open in Klaviyo", + "action": { + "type": "open_url", + "url": "https://www.klaviyo.com/profile/{{data.profile.id}}" + } + } + } + ] + } + } + ], + "lifecycle": { + "onInstall": [ + { + "use": "shopify.webhook.create", + "with": { + "topic": "orders/create", + "address": "$.install.webhookUrls.shopify", + "format": "json" + }, + "id": "hook-orders-create" + }, + { + "use": "shopify.webhook.create", + "with": { + "topic": "fulfillments/create", + "address": "$.install.webhookUrls.shopify", + "format": "json" + }, + "id": "hook-fulfillments-create" + }, + { + "use": "shopify.webhook.create", + "with": { + "topic": "customers/create", + "address": "$.install.webhookUrls.shopify", + "format": "json" + }, + "id": "hook-customers-create" + }, + { + "use": "shopify.webhook.create", + "with": { + "topic": "customers/update", + "address": "$.install.webhookUrls.shopify", + "format": "json" + }, + "id": "hook-customers-update" + } + ], + "onUninstall": [ + { + "id": "unhook", + "use": "foreach", + "with": { + "over": [ + "$.install.runtime.installed.hook-orders-create.id", + "$.install.runtime.installed.hook-fulfillments-create.id", + "$.install.runtime.installed.hook-customers-create.id", + "$.install.runtime.installed.hook-customers-update.id" + ], + "max": 20 + }, + "do": [ + { + "use": "shopify.webhook.delete", + "with": { + "id": "$.item" + }, + "onError": "continue" + } + ] + } + ] + } +} diff --git a/packages/core/integrations/data-defined/examples/shopify-netsuite.definition.json b/packages/core/integrations/data-defined/examples/shopify-netsuite.definition.json new file mode 100644 index 000000000..933e0ed0a --- /dev/null +++ b/packages/core/integrations/data-defined/examples/shopify-netsuite.definition.json @@ -0,0 +1,622 @@ +{ + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "shopify-netsuite", + "version": "2.2.0", + "display": { + "label": "Shopify \u2192 NetSuite", + "description": "Paid Shopify orders become NetSuite sales orders, customers follow them, NetSuite fulfillments write tracking back to Shopify, refunds become credit memos, and low stock is announced in Slack.", + "category": "ERP", + "tags": [ + "ecommerce", + "erp", + "order-to-cash", + "inventory" + ] + }, + "modules": { + "shopify": { + "module": "shopify", + "version": "^3.1" + }, + "netsuite": { + "module": "netsuite", + "version": "^1.2" + }, + "slack": { + "module": "slack", + "version": "^2.0" + } + }, + "settings": { + "subsidiary": { + "type": "option", + "options": "netsuite.subsidiary", + "label": "NetSuite subsidiary for new customers and orders", + "required": true + }, + "location": { + "type": "option", + "options": "netsuite.location", + "label": "Fulfil Shopify orders from", + "required": true + } + }, + "syncs": [ + { + "id": "customers", + "label": "Customers", + "description": "Every Shopify customer exists in NetSuite as an individual customer, matched by email.", + "source": "shopify.customer", + "target": "netsuite.customer", + "direction": "one_way", + "identity": { + "match": [ + { + "source": "email", + "target": "email", + "transform": "lowercase", + "key": "email" + } + ], + "onAmbiguous": "first" + }, + "mapping": [ + { + "source": "first_name", + "target": "firstName" + }, + { + "source": "last_name", + "target": "lastName" + }, + { + "source": "email", + "target": "email", + "transform": "lowercase" + }, + { + "source": "phone", + "target": "phone", + "transform": "to_e164_phone" + }, + { + "source": "default_address.company", + "target": "companyName" + }, + { + "source": "id", + "target": "externalId", + "transform": "none" + } + ], + "filter": "$.record.email exists", + "backfill": { + "trigger": "userAction", + "pageSize": 250 + }, + "ongoing": { + "mode": "webhook" + }, + "default": "on" + }, + { + "id": "orders", + "label": "Orders", + "description": "Paid orders become sales orders. The header is mapped here; lines and the customer are filled in by the \"Sales order lines\" workflow.", + "source": "shopify.order", + "target": "netsuite.salesOrder", + "direction": "one_way", + "identity": { + "match": [ + { + "source": "name", + "target": "otherRefNum", + "key": "orderNumber" + } + ], + "onAmbiguous": "skip" + }, + "mapping": [ + { + "source": "name", + "target": "otherRefNum" + }, + { + "source": "id", + "target": "externalId" + }, + { + "source": "created_at", + "target": "tranDate", + "transform": "to_iso_date" + }, + { + "source": "email", + "target": "email", + "transform": "lowercase" + }, + { + "source": "currency", + "target": "currency.refName", + "transform": "uppercase" + }, + { + "source": "note", + "target": "memo", + "transform": "default", + "options": { + "value": "Shopify order" + } + } + ], + "filter": "$.record.financial_status == 'paid' || $.record.financial_status == 'partially_refunded'", + "backfill": { + "trigger": "userAction", + "pageSize": 50 + }, + "ongoing": { + "mode": "webhook" + }, + "default": "on" + } + ], + "workflows": [ + { + "id": "order-lines", + "label": "Sales order lines and customer", + "description": "After a sales order is created from a Shopify order, attach the NetSuite customer and turn Shopify line items, shipping and discounts into NetSuite item lines.", + "purpose": "write", + "default": "on", + "trigger": { + "type": "syncRecord", + "sync": "orders", + "on": [ + "created" + ] + }, + "steps": [ + { + "id": "order", + "use": "links.find", + "with": { + "sync": "orders", + "from": "shopify", + "id": "$.record.id" + } + }, + { + "id": "customer", + "use": "links.find", + "with": { + "sync": "customers", + "key": { + "email": "$.record.email" + } + } + }, + { + "use": "stop", + "when": "$.steps.order == null || $.steps.customer == null", + "with": { + "reason": "Order or customer is not linked yet; the queue will retry" + } + }, + { + "id": "lines", + "use": "code", + "with": { + "ref": "@acme/commerce-steps#netsuiteItemLines@1.2.0", + "with": { + "lineItems": "$.record.line_items", + "shipping": "$.record.total_shipping_price_set.shop_money.amount", + "discountCodes": "$.record.discount_codes", + "locationId": "$.install.location" + } + } + }, + { + "use": "netsuite.salesOrder.update", + "with": { + "id": "$.steps.order.netsuite", + "record": { + "entity": { + "id": "$.steps.customer.netsuite" + }, + "subsidiary": { + "id": "$.install.subsidiary" + }, + "location": { + "id": "$.install.location" + }, + "item": { + "items": "$.steps.lines.items" + } + } + } + } + ], + "retry": { + "max": 5, + "backoff": "exponential" + } + }, + { + "id": "fulfillment-to-shopify", + "label": "Write NetSuite fulfillments back to Shopify", + "description": "When NetSuite ships an item fulfillment, fulfil the matching Shopify fulfillment orders with the tracking number.", + "purpose": "write", + "default": "on", + "settings": { + "carrier": { + "type": "enum", + "label": "Carrier shown to customers", + "enum": [ + "UPS", + "USPS", + "FedEx", + "DHL Express", + "Other" + ], + "default": "UPS" + }, + "notifyCustomer": { + "type": "boolean", + "label": "Send Shopify's shipping confirmation email", + "default": true + } + }, + "trigger": { + "type": "webhook", + "module": "netsuite", + "topic": "itemFulfillment.created", + "filter": "$.record.shipStatus == '_shipped'" + }, + "steps": [ + { + "id": "order", + "use": "links.find", + "with": { + "sync": "orders", + "from": "netsuite", + "id": "$.record.createdFrom.id" + } + }, + { + "use": "stop", + "when": "$.steps.order == null", + "with": { + "reason": "Fulfillment is for a sales order that did not come from Shopify" + } + }, + { + "id": "fulfillmentOrders", + "use": "shopify.order.fulfillmentOrders", + "with": { + "orderId": "$.steps.order.shopify" + } + }, + { + "id": "fulfil", + "use": "foreach", + "with": { + "over": "$.steps.fulfillmentOrders", + "max": 10 + }, + "do": [ + { + "use": "shopify.fulfillment.create", + "when": "$.item.status == 'open' || $.item.status == 'in_progress'", + "with": { + "fulfillmentOrderId": "$.item.id", + "trackingInfo": { + "number": "$.record.package.packages[0].packageTrackingNumber", + "company": "$.settings.carrier" + }, + "notifyCustomer": "$.settings.notifyCustomer" + } + } + ] + } + ] + }, + { + "id": "low-stock-alert", + "label": "Low-stock alert in Slack", + "purpose": "notify", + "default": "off", + "settings": { + "threshold": { + "type": "number", + "label": "Alert when available stock falls to", + "default": 5 + }, + "channel": { + "type": "option", + "options": "slack.channel", + "label": "Post alerts to", + "required": true + } + }, + "trigger": { + "type": "webhook", + "module": "shopify", + "topic": "inventory.updated", + "filter": "$.record.available <= $.settings.threshold" + }, + "steps": [ + { + "id": "variant", + "use": "shopify.variant.byInventoryItem", + "with": { + "inventoryItemId": "$.record.inventory_item_id" + } + }, + { + "id": "text", + "use": "template", + "with": { + "id": "low-stock" + } + }, + { + "use": "slack.message.post", + "with": { + "channel": "$.settings.channel", + "text": "$.steps.text", + "unfurl_links": false + } + } + ] + }, + { + "id": "refund-credit-memo", + "label": "Refunds become credit memos", + "description": "A Shopify refund creates a NetSuite credit memo against the original sales order, for the refunded lines only.", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "shopify", + "topic": "refund.created" + }, + "steps": [ + { + "id": "order", + "use": "links.find", + "with": { + "sync": "orders", + "from": "shopify", + "id": "$.record.order_id" + } + }, + { + "use": "stop", + "when": "$.steps.order == null", + "with": { + "reason": "Refunded order was never synced to NetSuite" + } + }, + { + "id": "salesOrder", + "use": "netsuite.salesOrder.get", + "with": { + "id": "$.steps.order.netsuite" + } + }, + { + "id": "lines", + "use": "code", + "with": { + "ref": "@acme/commerce-steps#netsuiteRefundLines@1.2.0", + "with": { + "refundLineItems": "$.record.refund_line_items", + "transactions": "$.record.transactions", + "salesOrderLines": "$.steps.salesOrder.item.items" + } + } + }, + { + "id": "memo", + "use": "template", + "with": { + "id": "credit-memo" + } + }, + { + "use": "netsuite.creditMemo.create", + "with": { + "record": { + "externalId": "shopify-refund-{{record.id}}", + "entity": { + "id": "$.steps.salesOrder.entity.id" + }, + "createdFrom": { + "id": "$.steps.order.netsuite" + }, + "tranDate": "$.record.created_at", + "memo": "$.steps.memo", + "item": { + "items": "$.steps.lines.items" + } + } + } + } + ] + } + ], + "queries": [ + { + "id": "netsuite-order", + "description": "The NetSuite sales order behind this Shopify order.", + "steps": [ + { + "id": "order", + "use": "links.find", + "with": { + "sync": "orders", + "from": "shopify", + "id": "$.record.id" + } + }, + { + "use": "stop", + "when": "$.steps.order == null", + "with": { + "reason": "Not synced" + } + }, + { + "id": "salesOrder", + "use": "netsuite.salesOrder.get", + "with": { + "id": "$.steps.order.netsuite" + } + } + ], + "returns": { + "salesOrder": "$.steps.salesOrder" + } + } + ], + "ui": [ + { + "id": "netsuite-status", + "label": "NetSuite", + "host": "shopify", + "placement": "admin.order-details.block", + "objects": [ + "order" + ], + "data": { + "query": "netsuite-order" + }, + "view": { + "component": "Stack", + "children": [ + { + "component": "Heading", + "props": { + "text": "NetSuite sales order" + } + }, + { + "component": "Empty", + "when": "!($.data.salesOrder exists)", + "props": { + "text": "This order has not reached NetSuite yet. Only paid orders are synced." + } + }, + { + "component": "KeyValue", + "when": "$.data.salesOrder exists", + "props": { + "items": [ + { + "label": "Document", + "value": "{{data.salesOrder.tranId}}" + }, + { + "label": "Status", + "value": "{{data.salesOrder.orderStatus}}" + }, + { + "label": "Total", + "value": "{{data.salesOrder.total}} {{data.salesOrder.currency.refName}}" + } + ] + } + } + ] + } + } + ], + "templates": { + "low-stock": ":warning: Low stock: *{{steps.variant.product_title}}* ({{steps.variant.sku}}) has {{record.available}} left at location {{record.location_id}}.", + "credit-memo": "Shopify refund {{record.id}} on order {{steps.salesOrder.otherRefNum}}. {{record.note | default: 'No reason given.'}}" + }, + "lifecycle": { + "onInstall": [ + { + "use": "shopify.webhook.create", + "with": { + "topic": "orders/create", + "address": "$.install.webhookUrls.shopify", + "format": "json" + }, + "id": "hook-orders-create" + }, + { + "use": "shopify.webhook.create", + "with": { + "topic": "orders/paid", + "address": "$.install.webhookUrls.shopify", + "format": "json" + }, + "id": "hook-orders-paid" + }, + { + "use": "shopify.webhook.create", + "with": { + "topic": "customers/create", + "address": "$.install.webhookUrls.shopify", + "format": "json" + }, + "id": "hook-customers-create" + }, + { + "use": "shopify.webhook.create", + "with": { + "topic": "customers/update", + "address": "$.install.webhookUrls.shopify", + "format": "json" + }, + "id": "hook-customers-update" + }, + { + "use": "shopify.webhook.create", + "with": { + "topic": "refunds/create", + "address": "$.install.webhookUrls.shopify", + "format": "json" + }, + "id": "hook-refunds-create" + }, + { + "use": "shopify.webhook.create", + "with": { + "topic": "inventory_levels/update", + "address": "$.install.webhookUrls.shopify", + "format": "json" + }, + "id": "hook-inventory-levels-update" + } + ], + "onUninstall": [ + { + "id": "unhook", + "use": "foreach", + "with": { + "over": [ + "$.install.runtime.installed.hook-orders-create.id", + "$.install.runtime.installed.hook-orders-paid.id", + "$.install.runtime.installed.hook-customers-create.id", + "$.install.runtime.installed.hook-customers-update.id", + "$.install.runtime.installed.hook-refunds-create.id", + "$.install.runtime.installed.hook-inventory-levels-update.id" + ], + "max": 20 + }, + "do": [ + { + "use": "shopify.webhook.delete", + "with": { + "id": "$.item" + }, + "onError": "continue" + } + ] + } + ] + } +} diff --git a/packages/core/integrations/data-defined/examples/stripe-quickbooks.definition.json b/packages/core/integrations/data-defined/examples/stripe-quickbooks.definition.json new file mode 100644 index 000000000..34c0b0933 --- /dev/null +++ b/packages/core/integrations/data-defined/examples/stripe-quickbooks.definition.json @@ -0,0 +1,476 @@ +{ + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "stripe-quickbooks", + "version": "1.0.0", + "display": { + "label": "Stripe \u2192 QuickBooks Online", + "description": "Book every Stripe payment as a QuickBooks sales receipt, refunds as refund receipts, keep customers matched, and reconcile each night's payouts into deposits net of Stripe fees.", + "category": "Accounting", + "tags": [ + "payments", + "accounting", + "reconciliation" + ] + }, + "modules": { + "stripe": { + "module": "stripe", + "version": "^2.3" + }, + "quickbooks": { + "module": "quickbooks", + "version": "^2.0" + } + }, + "settings": { + "clearingAccount": { + "type": "option", + "options": "quickbooks.account", + "label": "Book Stripe payments to (a Stripe clearing account)", + "required": true, + "help": "Payouts move money from this account to your bank." + }, + "salesItem": { + "type": "option", + "options": "quickbooks.item", + "label": "Product/service for Stripe payments", + "required": true + }, + "guestCustomer": { + "type": "option", + "options": "quickbooks.customer", + "label": "Customer for payments without a Stripe customer", + "help": "For example \"Stripe guest checkout\"." + } + }, + "syncs": [ + { + "id": "customers", + "label": "Customers", + "source": "stripe.customer", + "target": "quickbooks.customer", + "direction": "one_way", + "identity": { + "match": [ + { + "source": "email", + "target": "PrimaryEmailAddr.Address", + "transform": "lowercase", + "key": "email" + } + ], + "onAmbiguous": "first" + }, + "mapping": [ + { + "source": "name", + "target": "DisplayName", + "transform": "default", + "options": { + "value": "Stripe customer" + } + }, + { + "source": "name", + "target": "GivenName", + "transform": "split_full_name", + "options": { + "part": "first" + } + }, + { + "source": "name", + "target": "FamilyName", + "transform": "split_full_name", + "options": { + "part": "last" + } + }, + { + "source": "email", + "target": "PrimaryEmailAddr.Address", + "transform": "lowercase" + }, + { + "source": "phone", + "target": "PrimaryPhone.FreeFormNumber", + "transform": "to_e164_phone" + }, + { + "source": "address.line1", + "target": "BillAddr.Line1" + }, + { + "source": "address.city", + "target": "BillAddr.City" + }, + { + "source": "address.state", + "target": "BillAddr.CountrySubDivisionCode" + }, + { + "source": "address.postal_code", + "target": "BillAddr.PostalCode" + }, + { + "source": "address.country", + "target": "BillAddr.Country" + }, + { + "source": "id", + "target": "Notes", + "transform": "none" + } + ], + "backfill": { + "trigger": "userAction", + "pageSize": 100 + }, + "ongoing": { + "mode": "webhook" + }, + "default": "on" + } + ], + "workflows": [ + { + "id": "payment-sales-receipt", + "label": "Book payments as sales receipts", + "description": "Each succeeded payment intent becomes one sales receipt, deposited to the Stripe clearing account.", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "stripe", + "topic": "payment_intent.succeeded", + "filter": "$.record.metadata.quickbooks != 'skip'" + }, + "steps": [ + { + "id": "customer", + "use": "links.find", + "with": { + "sync": "customers", + "from": "stripe", + "id": "$.record.customer" + } + }, + { + "id": "resolveCustomer", + "use": "branch", + "cases": [ + { + "when": "$.record.customer == null", + "do": [ + { + "use": "set", + "with": { + "customerId": "$.install.guestCustomer" + } + } + ] + }, + { + "when": "$.steps.customer == null", + "do": [ + { + "id": "stripeCustomer", + "use": "stripe.customer.get", + "with": { + "id": "$.record.customer" + } + }, + { + "id": "qbFields", + "use": "map", + "with": { + "sync": "customers", + "record": "$.steps.stripeCustomer" + } + }, + { + "id": "qbCustomer", + "use": "quickbooks.customer.create", + "with": { + "customer": "$.steps.qbFields" + } + }, + { + "use": "links.upsert", + "with": { + "sync": "customers", + "ids": { + "stripe": "$.steps.stripeCustomer.id", + "quickbooks": "$.steps.qbCustomer.Id" + }, + "keys": { + "email": "$.steps.stripeCustomer.email" + } + } + }, + { + "use": "set", + "with": { + "customerId": "$.steps.qbCustomer.Id" + } + } + ] + } + ], + "else": [ + { + "use": "set", + "with": { + "customerId": "$.steps.customer.quickbooks" + } + } + ] + }, + { + "use": "stop", + "when": "$.vars.customerId == null", + "with": { + "reason": "Guest payment and no guest customer is configured" + } + }, + { + "id": "receipt", + "use": "map", + "with": { + "record": { + "paymentId": "$.record.id", + "amount": "{{record.amount_received | from_minor_units: record.currency}}", + "date": "{{record.created | from_unix | date}}", + "currency": "$.record.currency", + "description": "$.record.description", + "email": "$.record.receipt_email", + "customerId": "$.vars.customerId", + "depositTo": "$.install.clearingAccount", + "itemId": "$.install.salesItem" + }, + "mapping": [ + { + "source": "customerId", + "target": "CustomerRef.value" + }, + { + "source": "date", + "target": "TxnDate" + }, + { + "source": "depositTo", + "target": "DepositToAccountRef.value" + }, + { + "source": "paymentId", + "target": "PaymentRefNum" + }, + { + "source": "currency", + "target": "CurrencyRef.value", + "transform": "uppercase" + }, + { + "source": "email", + "target": "BillEmail.Address", + "transform": "lowercase" + }, + { + "source": "description", + "target": "PrivateNote", + "transform": "default", + "options": { + "value": "Stripe payment" + } + }, + { + "target": "Line[0].DetailType", + "value": "SalesItemLineDetail" + }, + { + "source": "amount", + "target": "Line[0].Amount", + "transform": "number" + }, + { + "source": "description", + "target": "Line[0].Description" + }, + { + "source": "itemId", + "target": "Line[0].SalesItemLineDetail.ItemRef.value" + } + ] + } + }, + { + "use": "quickbooks.salesReceipt.create", + "with": { + "salesReceipt": "$.steps.receipt" + } + } + ], + "retry": { + "max": 6, + "backoff": "exponential" + } + }, + { + "id": "refund-receipt", + "label": "Book refunds as refund receipts", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "stripe", + "topic": "refund.created", + "filter": "$.record.status == 'succeeded' || $.record.status == 'pending'" + }, + "steps": [ + { + "id": "payment", + "use": "stripe.paymentIntent.get", + "with": { + "id": "$.record.payment_intent" + } + }, + { + "id": "customer", + "use": "links.find", + "with": { + "sync": "customers", + "from": "stripe", + "id": "$.steps.payment.customer" + } + }, + { + "id": "note", + "use": "template", + "with": { + "id": "refund-note" + } + }, + { + "use": "quickbooks.refundReceipt.create", + "with": { + "refundReceipt": { + "CustomerRef": { + "value": "{{steps.customer.quickbooks | default: install.guestCustomer}}" + }, + "TxnDate": "{{record.created | from_unix | date}}", + "DepositToAccountRef": { + "value": "$.install.clearingAccount" + }, + "PaymentRefNum": "$.record.id", + "PrivateNote": "$.steps.note", + "Line": [ + { + "Amount": "{{record.amount | from_minor_units: record.currency}}", + "DetailType": "SalesItemLineDetail", + "SalesItemLineDetail": { + "ItemRef": { + "value": "$.install.salesItem" + } + } + } + ] + } + } + } + ] + }, + { + "id": "payout-reconciliation", + "label": "Nightly payout reconciliation", + "description": "For each payout that arrived yesterday, record a deposit from the clearing account to the bank, with Stripe fees split out to an expense account.", + "purpose": "maintain", + "default": "on", + "settings": { + "bankAccount": { + "type": "option", + "options": "quickbooks.account", + "label": "Bank account payouts land in", + "required": true + }, + "feeAccount": { + "type": "option", + "options": "quickbooks.account", + "label": "Expense account for Stripe fees", + "required": true + } + }, + "trigger": { + "type": "schedule", + "cron": "30 2 * * *", + "timezone": "America/Chicago" + }, + "steps": [ + { + "id": "payouts", + "use": "stripe.payout.list", + "with": { + "status": "paid", + "arrivedAfter": "-1d", + "limit": 100 + } + }, + { + "use": "stop", + "when": "$.steps.payouts[0] == null", + "with": { + "reason": "No payouts arrived yesterday" + } + }, + { + "id": "deposits", + "use": "foreach", + "with": { + "over": "$.steps.payouts", + "max": 50 + }, + "do": [ + { + "id": "transactions", + "use": "stripe.balanceTransaction.list", + "with": { + "payout": "$.item.id", + "limit": 100 + } + }, + { + "id": "summary", + "use": "code", + "with": { + "ref": "@acme/finance-steps#payoutDepositLines@1.0.0", + "with": { + "payout": "$.item", + "transactions": "$.steps.transactions", + "clearingAccount": "$.install.clearingAccount", + "feeAccount": "$.settings.feeAccount" + } + } + }, + { + "id": "deposit", + "use": "quickbooks.deposit.create", + "with": { + "deposit": { + "TxnDate": "$.steps.summary.date", + "DepositToAccountRef": { + "value": "$.settings.bankAccount" + }, + "PrivateNote": "Stripe payout {{item.id}} ({{steps.summary.count}} transactions)", + "Line": "$.steps.summary.lines" + } + } + } + ] + } + ] + } + ], + "templates": { + "refund-note": "Stripe refund {{record.id}} for payment {{record.payment_intent}} ({{record.reason | default: 'no reason given'}})." + } +} diff --git a/packages/core/integrations/data-defined/examples/webflow-hubspot-slack.definition.json b/packages/core/integrations/data-defined/examples/webflow-hubspot-slack.definition.json new file mode 100644 index 000000000..82678dd56 --- /dev/null +++ b/packages/core/integrations/data-defined/examples/webflow-hubspot-slack.definition.json @@ -0,0 +1,262 @@ +{ + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "webflow-hubspot-slack", + "version": "1.0.0", + "display": { + "label": "Webflow forms \u2192 HubSpot + Slack", + "description": "Website form submissions become HubSpot contacts and are routed to the right Slack channel by what the visitor asked about. Honeypot and link spam is dropped before anything is written.", + "category": "Lead capture", + "tags": [ + "forms", + "website", + "crm", + "slack" + ] + }, + "modules": { + "webflow": { + "module": "webflow", + "version": "^2.0" + }, + "hubspot": { + "module": "hubspot", + "version": "^1.4" + }, + "slack": { + "module": "slack", + "version": "^2.0" + } + }, + "settings": { + "site": { + "type": "option", + "options": "webflow.site", + "label": "Webflow site", + "required": true + } + }, + "workflows": [ + { + "id": "contact-form", + "label": "Contact form \u2192 HubSpot contact + routed Slack alert", + "purpose": "write", + "default": "on", + "settings": { + "salesChannel": { + "type": "option", + "options": "slack.channel", + "label": "Sales enquiries go to", + "required": true + }, + "supportChannel": { + "type": "option", + "options": "slack.channel", + "label": "Support requests go to", + "required": true + }, + "defaultChannel": { + "type": "option", + "options": "slack.channel", + "label": "Everything else goes to", + "required": true + } + }, + "trigger": { + "type": "webhook", + "module": "webflow", + "topic": "form.submitted", + "filter": "$.record.name != 'Newsletter' && $.record.data.Email exists && !($.record.data.Website exists && $.record.data.Website != '') && !($.record.data.Message contains 'http://') && !($.record.data.Message contains 'crypto')" + }, + "steps": [ + { + "id": "fields", + "use": "map", + "with": { + "record": "$.record.data", + "mapping": [ + { + "source": "Email", + "target": "properties.email", + "transform": "lowercase" + }, + { + "source": "Name", + "target": "properties.firstname", + "transform": "split_full_name", + "options": { + "part": "first" + } + }, + { + "source": "Name", + "target": "properties.lastname", + "transform": "split_full_name", + "options": { + "part": "last" + } + }, + { + "source": "Company", + "target": "properties.company", + "transform": "trim" + }, + { + "source": "Phone", + "target": "properties.phone", + "transform": "to_e164_phone" + }, + { + "target": "properties.lifecyclestage", + "value": "lead" + } + ] + } + }, + { + "id": "contact", + "use": "hubspot.contact.upsertByEmail", + "with": { + "email": "$.record.data.Email", + "properties": "$.steps.fields.properties" + } + }, + { + "id": "alert", + "use": "template", + "with": { + "id": "lead-alert" + } + }, + { + "id": "route", + "use": "branch", + "cases": [ + { + "when": "$.record.data.Interest == 'Sales'", + "do": [ + { + "use": "slack.message.post", + "with": { + "channel": "$.settings.salesChannel", + "text": "$.steps.alert" + } + } + ] + }, + { + "when": "$.record.data.Interest == 'Support'", + "do": [ + { + "use": "slack.message.post", + "with": { + "channel": "$.settings.supportChannel", + "text": ":sos: {{steps.alert}}" + } + } + ] + } + ], + "else": [ + { + "use": "slack.message.post", + "with": { + "channel": "$.settings.defaultChannel", + "text": "$.steps.alert" + } + } + ] + } + ] + }, + { + "id": "newsletter-signup", + "label": "Newsletter sign-ups become subscribers", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "webflow", + "topic": "form.submitted", + "filter": "$.record.name == 'Newsletter' && $.record.data.Email exists && !($.record.data.Website exists && $.record.data.Website != '') && !($.record.data.Message contains 'http://') && !($.record.data.Message contains 'crypto')" + }, + "steps": [ + { + "id": "fields", + "use": "map", + "with": { + "record": "$.record.data", + "mapping": [ + { + "source": "Email", + "target": "properties.email", + "transform": "lowercase" + }, + { + "source": "Name", + "target": "properties.firstname", + "transform": "split_full_name", + "options": { + "part": "first" + } + }, + { + "source": "Name", + "target": "properties.lastname", + "transform": "split_full_name", + "options": { + "part": "last" + } + }, + { + "source": "Company", + "target": "properties.company", + "transform": "trim" + }, + { + "source": "Phone", + "target": "properties.phone", + "transform": "to_e164_phone" + }, + { + "target": "properties.lifecyclestage", + "value": "subscriber" + } + ] + } + }, + { + "use": "hubspot.contact.upsertByEmail", + "with": { + "email": "$.record.data.Email", + "properties": "$.steps.fields.properties" + } + } + ] + } + ], + "templates": { + "lead-alert": "*New {{record.data.Interest | default: 'website'}} enquiry* from {{record.data.Name | default: 'someone'}} ({{record.data.Company | default: 'no company'}}) on {{record.publishedPath}}\n> {{record.data.Message | truncate: 280}}\n \u00b7 {{record.data.Email}}" + }, + "lifecycle": { + "onInstall": [ + { + "id": "hook", + "use": "webflow.webhook.create", + "with": { + "siteId": "$.install.site", + "triggerType": "form_submission", + "url": "$.install.webhookUrls.webflow" + } + } + ], + "onUninstall": [ + { + "use": "webflow.webhook.delete", + "with": { + "webhookId": "$.install.runtime.installed.hook.id" + } + } + ] + } +} diff --git a/packages/core/integrations/data-defined/examples/zendesk-jira.definition.json b/packages/core/integrations/data-defined/examples/zendesk-jira.definition.json new file mode 100644 index 000000000..2a03ce31c --- /dev/null +++ b/packages/core/integrations/data-defined/examples/zendesk-jira.definition.json @@ -0,0 +1,604 @@ +{ + "$schema": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "schemaVersion": "0.1", + "id": "zendesk-jira", + "version": "2.0.0", + "display": { + "label": "Zendesk \u2194 Jira escalation", + "description": "Escalate a ticket to a Jira issue from the ticket sidebar or with a tag, keep priority in step, and post Jira status changes back to the ticket as internal notes.", + "category": "Support", + "tags": [ + "helpdesk", + "engineering", + "escalation" + ] + }, + "modules": { + "zendesk": { + "module": "zendesk", + "version": "^1.6" + }, + "jira": { + "module": "jira", + "version": "^2.2" + } + }, + "settings": { + "project": { + "type": "option", + "options": "jira.project", + "label": "Jira project for escalations", + "required": true + }, + "issueType": { + "type": "option", + "options": "jira.issueType", + "label": "Issue type", + "required": true + } + }, + "syncs": [ + { + "id": "escalations", + "label": "Escalated tickets", + "description": "Only tickets that have been escalated. The ticket's external_id holds the Jira key; summary and priority follow the ticket.", + "source": "zendesk.ticket", + "target": "jira.issue", + "direction": "one_way", + "identity": { + "match": [ + { + "source": "external_id", + "target": "key", + "key": "issueKey" + } + ], + "onAmbiguous": "skip" + }, + "mapping": [ + { + "source": "subject", + "target": "fields.summary", + "transform": "trim" + }, + { + "source": "priority", + "target": "fields.priority.name", + "transform": "map_values", + "options": { + "values": { + "urgent": "Highest", + "high": "High", + "normal": "Medium", + "low": "Low" + }, + "fallback": "Medium" + } + } + ], + "filter": "$.record.tags contains 'jira_escalated' || $.record.tags contains 'escalate_jira'", + "backfill": { + "trigger": "none" + }, + "ongoing": { + "mode": "webhook" + }, + "default": "on" + } + ], + "workflows": [ + { + "id": "escalate-on-tag", + "label": "Escalate when an agent adds the escalate_jira tag", + "description": "Lets Zendesk triggers and macros escalate by tagging.", + "purpose": "write", + "default": "on", + "trigger": { + "type": "webhook", + "module": "zendesk", + "topic": "ticket.tagsChanged", + "filter": "$.event.event.tags_added contains 'escalate_jira'" + }, + "steps": [ + { + "id": "existing", + "use": "links.find", + "with": { + "sync": "escalations", + "from": "zendesk", + "id": "$.record.id" + } + }, + { + "use": "stop", + "when": "$.steps.existing != null", + "with": { + "reason": "Ticket is already escalated" + } + }, + { + "id": "mapped", + "use": "map", + "with": { + "sync": "escalations", + "record": "$.record" + } + }, + { + "id": "description", + "use": "template", + "with": { + "id": "issue-description" + } + }, + { + "id": "issue", + "use": "jira.issue.create", + "with": { + "fields": { + "project": { + "key": "$.install.project" + }, + "issuetype": { + "id": "$.install.issueType" + }, + "summary": "$.steps.mapped.fields.summary", + "priority": { + "name": "$.steps.mapped.fields.priority.name" + }, + "description": "$.steps.description", + "labels": [ + "zendesk", + "zd-{{record.id}}" + ] + } + } + }, + { + "use": "links.upsert", + "with": { + "sync": "escalations", + "ids": { + "zendesk": "$.record.id", + "jira": "$.steps.issue.id" + }, + "keys": { + "issueKey": "$.steps.issue.key" + } + } + }, + { + "use": "jira.remoteLink.create", + "with": { + "issueIdOrKey": "$.steps.issue.key", + "url": "https://{{install.accounts.zendesk.subdomain}}.zendesk.com/agent/tickets/{{record.id}}", + "title": "Zendesk #{{record.id}}" + } + }, + { + "use": "zendesk.ticket.update", + "with": { + "id": "$.record.id", + "ticket": { + "external_id": "$.steps.issue.key", + "additional_tags": [ + "jira_escalated" + ], + "remove_tags": [ + "escalate_jira" + ] + } + } + }, + { + "use": "zendesk.ticket.addComment", + "with": { + "id": "$.record.id", + "public": false, + "body": "Escalated to Jira {{steps.issue.key}}." + } + } + ] + }, + { + "id": "escalate-from-sidebar", + "label": "Escalate from the ticket sidebar", + "purpose": "respond", + "default": "on", + "trigger": { + "type": "userAction", + "label": "Escalate to Jira", + "from": "jira-panel" + }, + "steps": [ + { + "id": "existing", + "use": "links.find", + "with": { + "sync": "escalations", + "from": "zendesk", + "id": "$.record.id" + } + }, + { + "use": "stop", + "when": "$.steps.existing != null", + "with": { + "reason": "Ticket is already escalated" + } + }, + { + "id": "mapped", + "use": "map", + "with": { + "sync": "escalations", + "record": "$.record" + } + }, + { + "id": "description", + "use": "template", + "with": { + "id": "issue-description" + } + }, + { + "id": "issue", + "use": "jira.issue.create", + "with": { + "fields": { + "project": { + "key": "$.install.project" + }, + "issuetype": { + "id": "$.install.issueType" + }, + "summary": "$.steps.mapped.fields.summary", + "priority": { + "name": "$.steps.mapped.fields.priority.name" + }, + "description": "$.steps.description", + "labels": [ + "zendesk", + "zd-{{record.id}}" + ] + } + } + }, + { + "use": "links.upsert", + "with": { + "sync": "escalations", + "ids": { + "zendesk": "$.record.id", + "jira": "$.steps.issue.id" + }, + "keys": { + "issueKey": "$.steps.issue.key" + } + } + }, + { + "use": "jira.remoteLink.create", + "with": { + "issueIdOrKey": "$.steps.issue.key", + "url": "https://{{install.accounts.zendesk.subdomain}}.zendesk.com/agent/tickets/{{record.id}}", + "title": "Zendesk #{{record.id}}" + } + }, + { + "use": "zendesk.ticket.update", + "with": { + "id": "$.record.id", + "ticket": { + "external_id": "$.steps.issue.key", + "additional_tags": [ + "jira_escalated" + ], + "remove_tags": [ + "escalate_jira" + ] + } + } + }, + { + "use": "zendesk.ticket.addComment", + "with": { + "id": "$.record.id", + "public": false, + "body": "Escalated to Jira {{steps.issue.key}}: {{event.input.reason}}" + } + } + ] + }, + { + "id": "status-to-ticket", + "label": "Jira status changes as internal notes", + "purpose": "write", + "default": "on", + "settings": { + "reopenOnDone": { + "type": "boolean", + "label": "Reopen the ticket when the issue is done", + "default": true, + "help": "So the agent can confirm the fix with the customer." + } + }, + "trigger": { + "type": "webhook", + "module": "jira", + "topic": "issue.updated", + "filter": "$.event.changelog.items[0].field == 'status'" + }, + "steps": [ + { + "id": "ticket", + "use": "links.find", + "with": { + "sync": "escalations", + "from": "jira", + "id": "$.record.id" + } + }, + { + "use": "stop", + "when": "$.steps.ticket == null", + "with": { + "reason": "Issue was not escalated from Zendesk" + } + }, + { + "id": "note", + "use": "template", + "with": { + "id": "status-note" + } + }, + { + "use": "zendesk.ticket.addComment", + "with": { + "id": "$.steps.ticket.zendesk", + "body": "$.steps.note", + "public": false + } + }, + { + "use": "zendesk.ticket.update", + "when": "$.settings.reopenOnDone == true && $.record.fields.status.statusCategory.key == 'done'", + "with": { + "id": "$.steps.ticket.zendesk", + "ticket": { + "status": "open" + } + } + } + ] + }, + { + "id": "refresh-jira-webhook", + "label": "Keep the Jira webhook alive", + "description": "Jira dynamic webhooks expire after 30 days; refresh weekly.", + "purpose": "maintain", + "default": "on", + "trigger": { + "type": "schedule", + "cron": "17 4 * * 0", + "timezone": "UTC" + }, + "steps": [ + { + "id": "jiraWebhookIds", + "use": "foreach", + "with": { + "over": "$.install.runtime.installed.jiraHook.webhookRegistrationResult", + "max": 10, + "collect": "$.item.createdWebhookId" + } + }, + { + "use": "jira.webhook.refresh", + "with": { + "webhookIds": "$.steps.jiraWebhookIds" + } + } + ] + } + ], + "queries": [ + { + "id": "linked-issue", + "description": "The Jira issue this ticket was escalated to, if any.", + "steps": [ + { + "id": "link", + "use": "links.find", + "with": { + "sync": "escalations", + "from": "zendesk", + "id": "$.record.id" + } + }, + { + "use": "stop", + "when": "$.steps.link == null", + "with": { + "reason": "Not escalated" + } + }, + { + "id": "issue", + "use": "jira.issue.get", + "with": { + "issueIdOrKey": "$.steps.link.jira", + "fields": [ + "summary", + "status", + "priority", + "assignee", + "updated" + ] + } + } + ], + "returns": { + "issue": "$.steps.issue", + "site": "$.install.accounts.jira.site" + } + } + ], + "ui": [ + { + "id": "jira-panel", + "label": "Jira", + "host": "zendesk", + "placement": "ticket_sidebar", + "objects": [ + "ticket" + ], + "data": { + "query": "linked-issue" + }, + "view": { + "component": "Stack", + "children": [ + { + "component": "Heading", + "props": { + "text": "Jira escalation" + } + }, + { + "component": "Stack", + "when": "$.data.issue exists", + "children": [ + { + "component": "KeyValue", + "props": { + "items": [ + { + "label": "Issue", + "value": "{{data.issue.key}}" + }, + { + "label": "Priority", + "value": "{{data.issue.fields.priority.name}}" + }, + { + "label": "Assignee", + "value": "{{data.issue.fields.assignee.displayName | default: 'Unassigned'}}" + }, + { + "label": "Updated", + "value": "{{data.issue.fields.updated | date}}" + } + ] + } + }, + { + "component": "Badge", + "props": { + "text": "{{data.issue.fields.status.name}}", + "tone": "{{data.issue.fields.status.statusCategory.key}}" + } + }, + { + "component": "Button", + "props": { + "label": "Open in Jira", + "action": { + "type": "open_url", + "url": "https://{{data.site}}/browse/{{data.issue.key}}" + } + } + } + ] + }, + { + "component": "Stack", + "when": "!($.data.issue exists)", + "children": [ + { + "component": "Empty", + "props": { + "text": "Not escalated yet." + } + }, + { + "component": "Button", + "props": { + "label": "Escalate to Jira", + "style": "primary", + "action": { + "type": "userAction", + "workflow": "escalate-from-sidebar", + "input": { + "reason": { + "type": "text", + "label": "What should engineering know?" + } + } + } + } + } + ] + } + ] + } + } + ], + "templates": { + "issue-description": "Escalated from Zendesk ticket #{{record.id}} ({{record.priority}}, {{record.type | default: 'question'}}).\n\n{{record.description}}\n\nTicket: https://{{install.accounts.zendesk.subdomain}}.zendesk.com/agent/tickets/{{record.id}}", + "status-note": "Jira {{record.key}} moved from \"{{event.changelog.items[0].fromString}}\" to \"{{event.changelog.items[0].toString}}\" (by {{event.user.displayName | default: 'Jira'}})." + }, + "lifecycle": { + "onInstall": [ + { + "id": "zendeskHook", + "use": "zendesk.webhook.create", + "with": { + "name": "Frigg: Jira escalation", + "endpoint": "$.install.webhookUrls.zendesk", + "subscriptions": [ + "zen:event-type:ticket.tags_changed", + "zen:event-type:ticket.priority_changed" + ] + } + }, + { + "id": "jiraHook", + "use": "jira.webhook.register", + "with": { + "url": "$.install.webhookUrls.jira", + "events": [ + "jira:issue_updated" + ], + "jqlFilter": "labels = zendesk" + } + } + ], + "onUninstall": [ + { + "use": "zendesk.webhook.delete", + "with": { + "id": "$.install.runtime.installed.zendeskHook.id" + }, + "onError": "continue" + }, + { + "id": "jiraWebhookIds", + "use": "foreach", + "with": { + "over": "$.install.runtime.installed.jiraHook.webhookRegistrationResult", + "max": 10, + "collect": "$.item.createdWebhookId" + } + }, + { + "use": "jira.webhook.delete", + "with": { + "webhookIds": "$.steps.jiraWebhookIds" + } + } + ] + } +} diff --git a/packages/core/integrations/data-defined/expressions.js b/packages/core/integrations/data-defined/expressions.js new file mode 100644 index 000000000..760c6a89d --- /dev/null +++ b/packages/core/integrations/data-defined/expressions.js @@ -0,0 +1,225 @@ +/** + * Expressions for data-defined integrations (v0.1). + * + * Three forms, and no others (nothing is ever passed to eval): + * 1. Reference "$.event.contact.id" A string that is exactly a path. Resolves to the value, keeping its type. + * 2. Template "Call with {{event.from}}" A string with {{path | filter}} parts. Resolves to a string, except + * that a string which is exactly one tag ("{{record.amount | to_cents}}") keeps the value's type, so + * a filter can produce a number or a list for an API argument. + * 3. Condition "$.steps.contact == null && $.event.duration > 30" Used by `when` and trigger filters. + * + * Paths are read from a context object: { event, steps, settings, install, vars, item, record, now }. + * A missing segment resolves to undefined; it never throws. + */ + +const PATH_RE = /^\$\.[A-Za-z_][\w-]*(\.[\w-]+|\[\d+\])*$/; + +function getPath(ctx, path) { + const clean = path.replace(/^\$\./, ''); + let cur = ctx; + for (const seg of clean.split(/\.|(?=\[)/)) { + if (cur == null) return undefined; + const m = /^\[(\d+)\]$/.exec(seg); + cur = m ? cur[Number(m[1])] : cur[seg]; + } + return cur; +} + +const num = (v) => (v == null || v === '' ? null : Number(v)); +const arith = (f) => (v, arg) => { const a = num(v), b = num(arg); return a == null || b == null || Number.isNaN(a) || Number.isNaN(b) ? null : f(a, b); }; + +/** + * Template filters. Each is a named, total function of (value, argument): no filter loops, calls out or throws on + * bad input (a non-number gives null, which renders as ''). Arithmetic is here, as named filters, instead of an + * expression language: {{item.amount | to_cents}}, {{record.created | from_unix}}. + */ +// ISO 4217 minor-unit exponents that are not 2 (Stripe's zero-decimal and three-decimal lists). +const MINOR_EXPONENT = { BIF: 0, CLP: 0, DJF: 0, GNF: 0, JPY: 0, KMF: 0, KRW: 0, MGA: 0, PYG: 0, RWF: 0, UGX: 0, VND: 0, VUV: 0, XAF: 0, XOF: 0, XPF: 0, BHD: 3, JOD: 3, KWD: 3, OMR: 3, TND: 3 }; +const exponentOf = (currency) => MINOR_EXPONENT[String(currency || '').toUpperCase()] ?? 2; +/** a * 10^e, rounded half-up in decimal (19.995 -> 1999.5 -> 2000; 19.995 * 100 is 1999.4999... in binary). */ +const shiftRound = (a, e) => Math.round(Number(`${a}e${e}`)); + +const FILTERS = { + upper: (v) => String(v ?? '').toUpperCase(), + lower: (v) => String(v ?? '').toLowerCase(), + minutes: (v) => (v == null ? '' : String(Math.round(Number(v) / 60000))), + seconds: (v) => (v == null ? '' : String(Math.round(Number(v) / 1000))), + date: (v) => (v ? new Date(v).toISOString().slice(0, 10) : ''), + json: (v) => JSON.stringify(v), + default: (v, arg) => (v == null || v === '' ? arg : v), + truncate: (v, arg) => { const s = String(v ?? ''); const n = Number(arg) || 80; return s.length > n ? s.slice(0, n - 1) + '…' : s; }, + // lists + join: (v, arg) => (Array.isArray(v) ? v.filter((x) => x != null && x !== '').join(arg ?? ', ') : v), + size: (v) => (Array.isArray(v) || typeof v === 'string' ? v.length : v && typeof v === 'object' ? Object.keys(v).length : 0), + first: (v) => (Array.isArray(v) ? v[0] : v), + // arithmetic + plus: arith((a, b) => a + b), + minus: arith((a, b) => a - b), + times: arith((a, b) => a * b), + divided_by: arith((a, b) => (b === 0 ? null : a / b)), + // Rounded in decimal-exponent form, so 1.005 | round: 2 is 1.01 (1.005 * 100 is 100.49999... in binary floating point). + round: (v, arg) => { const a = num(v); if (a == null || Number.isNaN(a)) return null; const d = Math.max(0, Math.min(10, Number(arg) || 0)); return Number(`${shiftRound(a, d)}e-${d}`); }, + abs: (v) => { const a = num(v); return a == null || Number.isNaN(a) ? null : Math.abs(a); }, + number: (v) => { const a = num(v); return a == null || Number.isNaN(a) ? null : a; }, + // money in minor units, rounded so 19.99 -> 1999 (19.99 * 100 is 1998.9999999999998) + to_cents: (v) => { const a = num(v); return a == null || Number.isNaN(a) ? null : shiftRound(a, 2); }, + from_cents: (v) => { const a = num(v); return a == null || Number.isNaN(a) ? null : Math.round(a) / 100; }, + // the same for any currency: {{record.amount | from_minor_units: record.currency}} (JPY has no minor unit, KWD three) + to_minor_units: (v, cur) => { const a = num(v); if (a == null || Number.isNaN(a)) return null; return shiftRound(a, exponentOf(cur)); }, + from_minor_units: (v, cur) => { const a = num(v); if (a == null || Number.isNaN(a)) return null; const e = exponentOf(cur); return Math.round(a) / 10 ** e; }, + // unix seconds <-> ISO 8601 + from_unix: (v) => { const a = num(v); return a == null || Number.isNaN(a) ? null : new Date(a * 1000).toISOString(); }, + to_unix: (v) => { const t = v ? Date.parse(v) : NaN; return Number.isNaN(t) ? null : Math.floor(t / 1000); }, +}; + +/** A filter argument is a quoted literal ('x'), a number, or a path read from the context (steps.full.owner). */ +function filterArg(raw, ctx) { + if (raw == null) return undefined; + const a = raw.trim(); + if (/^'.*'$/.test(a)) return a.slice(1, -1).replace(/\\n/g, '\n').replace(/\\t/g, '\t'); + if (/^-?\d+(\.\d+)?$/.test(a)) return a; + return getPath(ctx, '$.' + a.replace(/^\$\./, '')); +} + +const TAG_RE = /\{\{\s*([^}]+?)\s*\}\}/g; +const SINGLE_TAG_RE = /^\{\{\s*([^}]+?)\s*\}\}$/; + +/** Split one tag's expression into its path and filters: "item.amount | times: 100" -> { path, filters: [{ name, arg }] }. */ +function parseTag(expr) { + const [p, ...filters] = expr.split('|').map((x) => x.trim()); + return { path: p.replace(/^\$\./, ''), filters: filters.map((f) => { const m = /^(\w+)(?::\s*(.*))?$/.exec(f); return m ? { name: m[1], arg: m[2] } : { name: f, arg: undefined, bad: true }; }) }; +} + +function evalTag(expr, ctx) { + const { path, filters } = parseTag(expr); + let v = getPath(ctx, '$.' + path); + for (const f of filters) { + const fn = !f.bad && FILTERS[f.name]; + if (!fn) throw new Error(`Unknown template filter "${f.name}"`); + v = fn(v, filterArg(f.arg, ctx)); + } + return v; +} + +function renderTemplate(str, ctx) { + const single = SINGLE_TAG_RE.exec(str); + if (single) return evalTag(single[1], ctx) ?? ''; + return str.replace(TAG_RE, (_, expr) => { + const v = evalTag(expr, ctx); + return v == null ? '' : typeof v === 'object' ? JSON.stringify(v) : String(v); + }); +} + +/** Every tag in a value (deep), for validation: [{ path, filters }]. */ +function templateTags(value, acc = []) { + if (typeof value === 'string') for (const m of value.matchAll(TAG_RE)) acc.push(parseTag(m[1])); + else if (Array.isArray(value)) value.forEach((v) => templateTags(v, acc)); + else if (value && typeof value === 'object') Object.values(value).forEach((v) => templateTags(v, acc)); + return acc; +} + +/** Resolve every reference and template inside a value (deep). Non-string scalars pass through. */ +function resolve(value, ctx) { + if (typeof value === 'string') { + if (PATH_RE.test(value)) return getPath(ctx, value); + if (value.includes('{{')) return renderTemplate(value, ctx); + return value; + } + if (Array.isArray(value)) return value.map((v) => resolve(v, ctx)); + if (value && typeof value === 'object') { + const out = {}; + for (const [k, v] of Object.entries(value)) out[k] = resolve(v, ctx); + return out; + } + return value; +} + +// ---------- conditions ---------- +const TOKEN_RE = /\s*(\$\.[A-Za-z_][\w.\-[\]]*|'(?:[^'\\]|\\.)*'|-?\d+(?:\.\d+)?|==|!=|>=|<=|&&|\|\||[()!<>]|\bin\b|\bcontains\b|\bexists\b|\btrue\b|\bfalse\b|\bnull\b)/y; + +function tokenize(src) { + const out = []; + TOKEN_RE.lastIndex = 0; + let pos = 0; + while (pos < src.length) { + if (/^\s*$/.test(src.slice(pos))) break; + TOKEN_RE.lastIndex = pos; + const m = TOKEN_RE.exec(src); + if (!m) throw new Error(`Cannot parse condition near "${src.slice(pos, pos + 20)}"`); + out.push(m[1]); + pos = TOKEN_RE.lastIndex; + } + return out; +} + +/** Parse a condition into a small AST. Grammar: or := and ('||' and)* ; and := not ('&&' not)* ; not := '!' not | cmp ; cmp := atom (op atom | 'exists')? */ +function parseCondition(src) { + const t = tokenize(src); + let i = 0; + const peek = () => t[i]; + const next = () => t[i++]; + function atom() { + const tok = next(); + if (i > t.length) throw new Error(`Condition ends early: "${src}"`); + if (tok === '(') { const e = or(); if (next() !== ')') throw new Error(`Missing ) in "${src}"`); return e; } + if (tok.startsWith('$.')) return { ref: tok }; + if (tok.startsWith("'")) return { lit: tok.slice(1, -1).replace(/\\'/g, "'") }; + if (tok === 'true' || tok === 'false') return { lit: tok === 'true' }; + if (tok === 'null') return { lit: null }; + if (/^-?\d/.test(tok)) return { lit: Number(tok) }; + throw new Error(`Unexpected "${tok}" in "${src}"`); + } + function cmp() { + const left = atom(); + const op = peek(); + if (op === 'exists') { next(); return { op: 'exists', left }; } + if (['==', '!=', '>', '>=', '<', '<=', 'in', 'contains'].includes(op)) { next(); return { op, left, right: atom() }; } + return { op: 'truthy', left }; + } + function not() { if (peek() === '!') { next(); return { op: '!', left: not() }; } return cmp(); } + function and() { let l = not(); while (peek() === '&&') { next(); l = { op: '&&', left: l, right: not() }; } return l; } + function or() { let l = and(); while (peek() === '||') { next(); l = { op: '||', left: l, right: and() }; } return l; } + const ast = or(); + if (i < t.length) throw new Error(`Unexpected "${t[i]}" in "${src}"`); + return ast; +} + +function evalAst(n, ctx) { + if ('ref' in n) return getPath(ctx, n.ref); + if ('lit' in n) return n.lit; + const L = () => evalAst(n.left, ctx), R = () => evalAst(n.right, ctx); + switch (n.op) { + case 'truthy': return Boolean(L()); + case 'exists': return L() != null; + case '!': return !evalAst(n.left, ctx); + case '&&': return Boolean(L()) && Boolean(R()); + case '||': return Boolean(L()) || Boolean(R()); + case '==': return L() == R(); // eslint-disable-line eqeqeq -- null == undefined is intended + case '!=': return L() != R(); // eslint-disable-line eqeqeq + case '>': return L() > R(); + case '>=': return L() >= R(); + case '<': return L() < R(); + case '<=': return L() <= R(); + case 'in': { const r = R(); return Array.isArray(r) ? r.includes(L()) : typeof r === 'string' && r.includes(String(L())); } + case 'contains': { const l = L(); return Array.isArray(l) ? l.includes(R()) : String(l ?? '').includes(String(R())); } + default: throw new Error(`Unknown operator ${n.op}`); + } +} + +function evaluate(condition, ctx) { + if (condition == null) return true; + return Boolean(evalAst(parseCondition(condition), ctx)); +} + +/** Every $.root referenced by an expression-bearing value (for validation). */ +function referencedPaths(value, acc = []) { + if (typeof value === 'string') { + if (PATH_RE.test(value)) acc.push(value); + for (const m of value.matchAll(/\{\{\s*([^}|]+?)\s*(\|[^}]*)?\}\}/g)) acc.push('$.' + m[1].replace(/^\$\./, '')); + for (const m of value.matchAll(/\$\.[A-Za-z_][\w.\-[\]]*/g)) if (!acc.includes(m[0])) acc.push(m[0]); + } else if (Array.isArray(value)) value.forEach((v) => referencedPaths(v, acc)); + else if (value && typeof value === 'object') Object.values(value).forEach((v) => referencedPaths(v, acc)); + return acc; +} + +module.exports = { getPath, resolve, renderTemplate, evaluate, parseCondition, referencedPaths, templateTags, FILTERS }; diff --git a/packages/core/integrations/data-defined/index.js b/packages/core/integrations/data-defined/index.js new file mode 100644 index 000000000..930f29cdf --- /dev/null +++ b/packages/core/integrations/data-defined/index.js @@ -0,0 +1,39 @@ +/** + * Data-defined integrations, v0.1 (ADR-063). An integration as a document: modules, syncs, workflows, queries, + * UI surfaces and templates, run by one runtime class on the existing Frigg infrastructure. + */ +const fs = require('fs'); +const path = require('path'); +const { validate } = require('./validate'); +const { compile, routeWebhook, eventName } = require('./compile'); +const { execute } = require('./interpreter'); +const { makePorts, runWorkflow, runQuery, renderSurface, resolveView, runLifecycle, resolveOptions } = require('./runtime'); +const { diff, checkBump, planUpgrade } = require('./versioning'); +const { defineIntegrationClass, defineSharedIntegrationClass, memoryDefinitions } = require('./data-defined-integration'); +const { BUILTINS } = require('./steps'); +const { TRANSFORMS, applyMapping } = require('./transforms'); +const { FILTERS } = require('./expressions'); +const { validateManifest } = require('./manifests'); +const { checkDefinition, checkManifest, checkInstallation } = require('./schema-check'); +const useCases = require('./use-cases/definition-use-cases'); + +/** Load every capabilities/*.capabilities.json (bundled mocks; real modules ship their own). */ +function loadManifests(dir = path.join(__dirname, 'capabilities')) { + const out = {}; + for (const f of fs.readdirSync(dir)) if (f.endsWith('.capabilities.json')) { const m = JSON.parse(fs.readFileSync(path.join(dir, f), 'utf8')); out[m.module] = m; } + return out; +} +function loadExamples(dir = path.join(__dirname, 'examples')) { + return fs.readdirSync(dir).filter((f) => f.endsWith('.definition.json')).map((f) => JSON.parse(fs.readFileSync(path.join(dir, f), 'utf8'))); +} + +module.exports = { + SCHEMA_VERSION: '0.1', + validate, validateManifest, checkDefinition, checkManifest, checkInstallation, compile, routeWebhook, eventName, execute, + makePorts, runWorkflow, runQuery, renderSurface, resolveView, runLifecycle, + diff, checkBump, planUpgrade, + defineIntegrationClass, defineSharedIntegrationClass, memoryDefinitions, + BUILTINS, TRANSFORMS, FILTERS, applyMapping, resolveOptions, + loadManifests, loadExamples, + ...useCases, +}; diff --git a/packages/core/integrations/data-defined/interpreter.js b/packages/core/integrations/data-defined/interpreter.js new file mode 100644 index 000000000..2a28a39cf --- /dev/null +++ b/packages/core/integrations/data-defined/interpreter.js @@ -0,0 +1,89 @@ +/** + * Runs a list of steps against a context. Pure apart from the ports it is given: + * ports.call(moduleKey, capability, args) a module capability (the runtime maps it to Api.) + * ports.links.find / ports.links.upsert the link + identity store (IntegrationMapping rows) + * ports.runCode(ref, args) registered code steps (optional) + * + * Every step leaves a trace entry: { id, use, status, ms, error? }. A step's result is stored at $.steps.. + * onError per step: 'fail' (default: throw, so the queue retries the whole run), 'continue' (result null), 'stop'. + */ +const { resolve, evaluate } = require('./expressions'); +const { BUILTINS, FOREACH_HARD_CAP, isBuiltin, splitCapability } = require('./steps'); + +class StopRun extends Error { constructor(reason) { super(reason); this.name = 'StopRun'; this.reason = reason; } } + +async function runSteps(steps, env, path = 'steps') { + const { ctx, trace } = env; + for (let i = 0; i < (steps || []).length; i++) { + const step = steps[i]; + const id = step.id || `${step.use}#${i}`; + const at = `${path}[${i}]`; + if (step.when && !evaluate(step.when, ctx)) { trace.push({ id, use: step.use, at, status: 'skipped' }); continue; } + const t0 = Date.now(); + try { + let out; + if (step.use === 'foreach') { + const all = resolve(step.with?.over, ctx) ?? []; + if (!Array.isArray(all)) throw new Error('foreach: `over` is not an array'); + const max = Math.min(step.with?.max || 100, FOREACH_HARD_CAP); + const saved = { item: ctx.item, index: ctx.index }; + // `where` is checked per item, before the cap, so max bounds the work actually done. + const items = []; + for (const [n, item] of all.entries()) { + ctx.item = item; ctx.index = n; + if (!step.with?.where || evaluate(step.with.where, ctx)) items.push(item); + } + Object.assign(ctx, saved); + const collect = step.with && 'collect' in step.with; + out = []; + for (const [n, item] of items.slice(0, max).entries()) { + ctx.item = item; ctx.index = n; + await runSteps(step.do || [], env, `${at}.do#${n}`); + out.push(collect + ? resolve(step.with.collect, ctx) + : Object.fromEntries((step.do || []).filter((s) => s.id).map((s) => [s.id, ctx.steps[s.id]]))); + Object.assign(ctx, saved); + } + if (items.length > max) trace.push({ id, use: 'foreach', at, status: 'truncated', note: `${items.length - max} items over max ${max}` }); + } else if (step.use === 'branch') { + const hit = (step.cases || []).find((c) => evaluate(c.when, ctx)); + await runSteps(hit ? hit.do : step.else || [], env, `${at}.${hit ? 'case' : 'else'}`); + out = { matched: hit ? hit.when : null }; + } else if (isBuiltin(step.use)) { + out = await BUILTINS[step.use].run(resolve(step.with || {}, ctx), env); + if (out && out.stop) { ctx.steps[id] = out; trace.push({ id, use: step.use, at, status: 'stopped', ms: Date.now() - t0, reason: out.reason }); throw new StopRun(out.reason); } + } else { + const cap = splitCapability(step.use); + if (!cap) throw new Error(`Unknown step "${step.use}"`); + out = await env.ports.call(cap.module, cap.capability, resolve(step.with || {}, ctx)); + } + ctx.steps[id] = out === undefined ? null : out; + trace.push({ id, use: step.use, at, status: 'ok', ms: Date.now() - t0 }); + } catch (e) { + if (e instanceof StopRun) throw e; + const policy = step.onError || 'fail'; + trace.push({ id, use: step.use, at, status: 'error', ms: Date.now() - t0, error: String(e && e.message || e), policy }); + if (policy === 'continue') { ctx.steps[id] = null; continue; } + if (policy === 'stop') throw new StopRun(`step ${id} failed`); + throw e; + } + } +} + +/** + * Run steps and report the outcome instead of throwing for a normal stop. + * @returns {Promise<{status:'completed'|'stopped', reason?:string, steps:object, trace:Array}>} + */ +async function execute(steps, { def, installation, ctx, ports }) { + const env = { def, installation, ports, trace: [], ctx: { steps: {}, vars: {}, ...ctx } }; + try { + await runSteps(steps, env); + return { status: 'completed', steps: env.ctx.steps, trace: env.trace }; + } catch (e) { + if (e instanceof StopRun) return { status: 'stopped', reason: e.reason, steps: env.ctx.steps, trace: env.trace }; + e.trace = env.trace; + throw e; + } +} + +module.exports = { execute, runSteps, StopRun }; diff --git a/packages/core/integrations/data-defined/manifests.js b/packages/core/integrations/data-defined/manifests.js new file mode 100644 index 000000000..51c677381 --- /dev/null +++ b/packages/core/integrations/data-defined/manifests.js @@ -0,0 +1,125 @@ +/** + * Validate a capability manifest (v0.1): the module author's promise of what a definition may call. Definitions are + * validated against manifests, so a broken manifest would let a broken definition through; this checks the manifest + * itself. + * + * @param {object} m capabilities/.capabilities.json + * @returns {{ valid: boolean, errors: Array<{path:string,message:string}> }} + */ +const { templateTags, parseCondition, FILTERS } = require('./expressions'); +const { eventsIn } = require('./webhooks'); + +const PATH_RE = /^\$\.[A-Za-z_][\w-]*(\.[\w-]+|\[\d+\])*$/; +const KINDS = new Set(['read', 'search', 'write', 'delete', 'subscribe']); // schema/capability-manifest.v0.1.schema.json +const READS = new Set(['read', 'search']); + +function validateManifest(m) { + const errors = []; + const err = (path, message) => errors.push({ path, message }); + if (!m || typeof m !== 'object') return { valid: false, errors: [{ path: '', message: 'Manifest must be an object' }] }; + if (!/^[a-z][a-z0-9-]*$/.test(m.module || '')) err('module', 'Lowercase module name'); + if (!m.label) err('label', 'Required'); + + const objects = m.objects || {}; + for (const [o, def] of Object.entries(objects)) { + const names = new Set(); + (def.fields || []).forEach((f, i) => { + if (!f?.name) err(`objects.${o}.fields[${i}]`, 'Field needs a name'); + else if (names.has(f.name)) err(`objects.${o}.fields[${i}]`, `Duplicate field "${f.name}"`); + else names.add(f.name); + // An enum lists its values, or names one of this module's option lists when they differ per account + // (HubSpot deal stages are per portal). + if (f?.type === 'enum' && !Array.isArray(f.values) && !(typeof f.values === 'string' && m.options?.[f.values])) err(`objects.${o}.fields[${i}]`, 'An enum lists its values, or names an option list of this module'); + }); + } + + const caps = m.capabilities || {}; + for (const [name, c] of Object.entries(caps)) { + const p = `capabilities.${name}`; + if (!/^[A-Za-z][\w]*\.[A-Za-z][\w]*$/.test(name)) err(p, 'Capability names are "."'); + if (!KINDS.has(c?.kind)) err(`${p}.kind`, [...KINDS].join(', ')); + if (!/^[A-Za-z_$][\w$]*$/.test(c?.method || '')) err(`${p}.method`, 'Name of a method on the module\'s Api class'); + for (const [arg, spec] of Object.entries(c?.input || {})) if (!spec?.type) err(`${p}.input.${arg}`, 'Input needs a type'); + } + + for (const [topic, e] of Object.entries(m.events || {})) { + const p = `events.${topic}`; + if (e.object && !objects[e.object]) err(`${p}.object`, `"${e.object}" is not an object of this module`); + if (e.payload && (e.delivery || 'webhook') === 'webhook' && m.webhook) { + // The example is a real delivery (body, plus headers when the topic is in one): it must route to this event + // through the same code a live webhook goes through, and its record path must find something. + const found = eventsIn(m, { body: e.payload, headers: e.headers }).find((x) => x.topic === topic); + if (!found) err(`${p}.payload`, `The example delivery does not route to "${topic}" (check webhook.topic and this event's match)`); + else if (e.record && found.record === undefined) err(`${p}.record`, `"${e.record}" finds nothing in the example event`); + } else if (e.record && e.payload) { + let cur = e.payload; + for (const seg of e.record.split(/\.|(?=\[)/)) { const i = /^\[(\d+)\]$/.exec(seg); cur = cur == null ? undefined : i ? cur[Number(i[1])] : cur[seg]; } + if (cur === undefined) err(`${p}.record`, `"${e.record}" is not a path in the example payload`); + } + } + + // An option list: a read capability, its inputs, and how each item becomes { label, value } (one reference, or + // a template). A string that mixes "$.x" into text renders the path literally, so it is refused. + const checkItemExpr = (v, p) => { + if (typeof v !== 'string') { err(p, 'A reference ($.item.id) or a template ({{item.name}})'); return; } + if (!PATH_RE.test(v) && v.includes('$.')) err(p, `"${v}" mixes a $. reference into text, which renders literally; use {{item.x}} for every part`); + for (const t of templateTags(v)) { + if (t.path.split(/\.|\[/)[0] !== 'item') err(p, `{{${t.path}}} — an option label can only read item`); + for (const f of t.filters) if (!FILTERS[f.name]) err(p, `Unknown template filter "${f.name}"`); + } + if (PATH_RE.test(v) && !v.startsWith('$.item')) err(p, 'An option value can only read $.item'); + }; + for (const [name, o] of Object.entries(m.options || {})) { + const p = `options.${name}`; + const c = caps[o?.capability]; + if (!c) err(`${p}.capability`, `"${o?.capability}" is not a capability of this module`); + else if (!READS.has(c.kind)) err(`${p}.capability`, `An option list reads; ${o.capability} is a ${c.kind} capability`); + checkItemExpr(o?.label, `${p}.label`); + checkItemExpr(o?.value, `${p}.value`); + for (const [input, spec] of Object.entries(o?.inputs || {})) { + if (c && !(input in (c.input || {}))) err(`${p}.inputs.${input}`, `${o.capability} takes no argument "${input}"`); + if (!spec?.type) err(`${p}.inputs.${input}`, 'Input needs a type'); + } + if (c) for (const [arg, spec] of Object.entries(c.input || {})) if (spec.required && !(arg in (o.inputs || {}))) err(`${p}.inputs`, `${o.capability} requires "${arg}", so the option list must declare it as an input`); + } + + // Webhook delivery: how a POST from this provider becomes module events (webhooks.js). Templates may read body, + // headers (lower-case), query, item (one event of a batch) and parent (the outer item of a two-level batch). + const w = m.webhook; + const webhookEvents = Object.entries(m.events || {}).filter(([, e]) => (e.delivery || 'webhook') === 'webhook'); + if (webhookEvents.length && !w) err('webhook', 'This module has webhook events, so it needs a webhook block saying where the topic is'); + if (w) { + if (!['installation', 'app'].includes(w.scope)) err('webhook.scope', 'installation (a URL per installation) or app (one URL for every account)'); + const tpl = (v, p, roots = ['body', 'headers', 'query', 'item', 'parent']) => { + for (const t of (Array.isArray(v) ? v : [v])) { + if (typeof t !== 'string') { err(p, 'A template string, or a list of them'); continue; } + for (const tag of templateTags(t)) { + if (!roots.includes(tag.path.split(/\.|\[/)[0])) err(p, `{{${tag.path}}}: a webhook template can read ${roots.join(', ')}`); + for (const f of tag.filters) if (!FILTERS[f.name]) err(p, `Unknown template filter "${f.name}"`); + } + } + }; + if (!w.topic) err('webhook.topic', 'Required: where the provider puts the event type'); + else tpl(w.topic, 'webhook.topic'); + if (w.eventId) tpl(w.eventId, 'webhook.eventId'); else err('webhook.eventId', 'Required: the id that makes a redelivery a no-op'); + if (w.scope === 'app' && !w.account) err('webhook.account', 'An app-level webhook needs account: which connected account an event is about'); + if (w.account) tpl(w.account, 'webhook.account'); + const batch = w.batch == null ? [] : Array.isArray(w.batch) ? w.batch : [w.batch]; + if (batch.length > 2 || batch.some((b) => typeof b !== 'string')) err('webhook.batch', 'A path to the array of events, or [outer, inner] for two levels'); + if (w.verify != null && !/^[A-Za-z_$][\w$]*$/.test(w.verify)) err('webhook.verify', 'Name of an Api method that checks the signature'); + if (w.handshake) { + try { parseCondition(w.handshake.when); } catch (e) { err('webhook.handshake.when', e.message); } + if (w.handshake.status != null && !Number.isInteger(w.handshake.status)) err('webhook.handshake.status', 'An HTTP status code'); + } + if (w.unwrap != null && typeof w.unwrap !== 'string') err('webhook.unwrap', 'The name of a form field holding the JSON body'); + } + for (const [topic, e] of webhookEvents) { + if (e.match != null && typeof e.match !== 'string') err(`events.${topic}.match`, 'The provider\'s topic string for this event'); + if (e.when) { try { parseCondition(e.when); } catch (x) { err(`events.${topic}.when`, x.message); } } + } + + for (const pl of m.ui?.placements || []) if (typeof pl !== 'string') err('ui.placements', 'Placements are strings'); + return { valid: errors.length === 0, errors }; +} + +module.exports = { validateManifest }; diff --git a/packages/core/integrations/data-defined/repositories/definition-version-repository.js b/packages/core/integrations/data-defined/repositories/definition-version-repository.js new file mode 100644 index 000000000..19ad3e97c --- /dev/null +++ b/packages/core/integrations/data-defined/repositories/definition-version-repository.js @@ -0,0 +1,67 @@ +/** + * IntegrationDefinitionVersion persistence (ADR-063). Pure data access: one Prisma call per method, no business rules. + * Which version is "latest", whether a publish is allowed and what a status change means are decided by the use cases + * (../use-cases/definition-use-cases.js); this layer only stores and reads rows. + * + * Rows come back as plain objects with string ids on every database. `document` is the definition verbatim. + */ +const { prisma } = require('../../../database/prisma'); + +const FIELDS = ['definitionId', 'version', 'schemaVersion', 'status', 'checksum', 'document', 'changeLevel', 'previousVersion', 'notes', 'createdBy', 'publishedAt']; +const pick = (o) => Object.fromEntries(FIELDS.filter((k) => o[k] !== undefined).map((k) => [k, o[k]])); + +class DefinitionVersionRepositoryMongo { + constructor({ client } = {}) { this.prisma = client || prisma; } + toRow(r) { return r ? { ...r, id: String(r.id) } : null; } + + async findVersion(definitionId, version) { + return this.toRow(await this.prisma.integrationDefinitionVersion.findUnique({ where: { definitionId_version: { definitionId, version } } })); + } + + /** Every version of one definition, any status, unordered (semver order is the caller's business). */ + async listVersions(definitionId) { + return (await this.prisma.integrationDefinitionVersion.findMany({ where: { definitionId } })).map((r) => this.toRow(r)); + } + + /** Rows in the given statuses across all definitions, without documents (a catalogue listing). */ + async listByStatus(statuses) { + const rows = await this.prisma.integrationDefinitionVersion.findMany({ + where: { status: { in: statuses } }, + select: { id: true, definitionId: true, version: true, schemaVersion: true, status: true, checksum: true, changeLevel: true, publishedAt: true, createdAt: true }, + }); + return rows.map((r) => this.toRow(r)); + } + + async createVersion(data) { + return this.toRow(await this.prisma.integrationDefinitionVersion.create({ data: pick(data) })); + } + + /** + * Change a row only if it is still in `fromStatus` (the use case's precondition, checked by the database, so two + * concurrent publishes cannot both win). Returns the updated row, or null if it was not in that status. + */ + async updateIfStatus(definitionId, version, fromStatus, data) { + const { count } = await this.prisma.integrationDefinitionVersion.updateMany({ where: { definitionId, version, status: fromStatus }, data: pick(data) }); + return count ? this.findVersion(definitionId, version) : null; + } + + async deleteIfStatus(definitionId, version, status) { + const { count } = await this.prisma.integrationDefinitionVersion.deleteMany({ where: { definitionId, version, status } }); + return count > 0; + } +} + +class DefinitionVersionRepositoryPostgres extends DefinitionVersionRepositoryMongo {} +class DefinitionVersionRepositoryDocumentDB extends DefinitionVersionRepositoryMongo {} + +function createDefinitionVersionRepository() { + const { DB_TYPE } = require('../../../database/config'); + switch (DB_TYPE) { + case 'mongodb': return new DefinitionVersionRepositoryMongo(); + case 'postgresql': return new DefinitionVersionRepositoryPostgres(); + case 'documentdb': return new DefinitionVersionRepositoryDocumentDB(); + default: throw new Error(`Unsupported database type: ${DB_TYPE}. Supported values: 'mongodb', 'documentdb', 'postgresql'`); + } +} + +module.exports = { createDefinitionVersionRepository, DefinitionVersionRepositoryMongo, DefinitionVersionRepositoryPostgres, DefinitionVersionRepositoryDocumentDB }; diff --git a/packages/core/integrations/data-defined/repositories/workflow-run-repository.js b/packages/core/integrations/data-defined/repositories/workflow-run-repository.js new file mode 100644 index 000000000..840ffd45f --- /dev/null +++ b/packages/core/integrations/data-defined/repositories/workflow-run-repository.js @@ -0,0 +1,59 @@ +/** + * WorkflowRun persistence (ADR-063). Pure data access. A run row never holds request or response bodies: `trace` is + * step ids, statuses, timings and error messages (see runtime.js). + */ +const { prisma } = require('../../../database/prisma'); + +class WorkflowRunRepositoryMongo { + constructor({ client } = {}) { this.prisma = client || prisma; } + toId(id) { return String(id); } + toRow(r) { return r ? { ...r, id: String(r.id), integrationId: String(r.integrationId) } : null; } + + async createRun(run) { + return this.toRow(await this.prisma.workflowRun.create({ + data: { + integrationId: this.toId(run.integrationId), definitionId: run.definitionId, definitionVersion: run.definitionVersion, + workflowId: run.workflowId, triggerType: run.triggerType, eventId: run.eventId ?? null, status: run.status, + reason: run.reason ?? null, error: run.error ?? null, retryable: Boolean(run.retryable), trace: run.trace || [], + startedAt: new Date(run.startedAt), endedAt: run.endedAt ? new Date(run.endedAt) : null, + }, + })); + } + + /** Newest first. `before` pages by startedAt. */ + async listRuns(integrationId, { workflowId, status, before, limit = 50 } = {}) { + const rows = await this.prisma.workflowRun.findMany({ + where: { integrationId: this.toId(integrationId), ...(workflowId ? { workflowId } : {}), ...(status ? { status } : {}), ...(before ? { startedAt: { lt: new Date(before) } } : {}) }, + orderBy: { startedAt: 'desc' }, + take: Math.min(Math.max(1, limit), 500), + }); + return rows.map((r) => this.toRow(r)); + } + + /** Delete runs that started before `olderThan`, optionally only in some statuses. Returns the count. */ + async deleteRunsBefore(olderThan, { statuses } = {}) { + const { count } = await this.prisma.workflowRun.deleteMany({ where: { startedAt: { lt: new Date(olderThan) }, ...(statuses ? { status: { in: statuses } } : {}) } }); + return count; + } +} + +class WorkflowRunRepositoryPostgres extends WorkflowRunRepositoryMongo { + toId(id) { + const n = Number.parseInt(id, 10); + if (Number.isNaN(n)) throw new Error(`Invalid integration id: ${id}`); + return n; + } +} +class WorkflowRunRepositoryDocumentDB extends WorkflowRunRepositoryMongo {} + +function createWorkflowRunRepository() { + const { DB_TYPE } = require('../../../database/config'); + switch (DB_TYPE) { + case 'mongodb': return new WorkflowRunRepositoryMongo(); + case 'postgresql': return new WorkflowRunRepositoryPostgres(); + case 'documentdb': return new WorkflowRunRepositoryDocumentDB(); + default: throw new Error(`Unsupported database type: ${DB_TYPE}. Supported values: 'mongodb', 'documentdb', 'postgresql'`); + } +} + +module.exports = { createWorkflowRunRepository, WorkflowRunRepositoryMongo, WorkflowRunRepositoryPostgres, WorkflowRunRepositoryDocumentDB }; diff --git a/packages/core/integrations/data-defined/router.js b/packages/core/integrations/data-defined/router.js new file mode 100644 index 000000000..e03eda0ab --- /dev/null +++ b/packages/core/integrations/data-defined/router.js @@ -0,0 +1,97 @@ +/** + * HTTP routes for data-defined integrations, mounted by createIntegrationRouter when the app registers a + * data-defined class. Thin adapters: authenticate, call a use case, map the result to a status. + * + * Signed-in users (Frigg's usual authentication: bearer token, x-frigg headers or adopter JWT): + * GET /api/data-defined/definitions what can be installed: latest published version of each + * GET /api/integrations/:integrationId/upgrade?to= the upgrade plan for an installation (changes nothing) + * POST /api/integrations/:integrationId/upgrade { to? } apply it (refused while anything is needed) + * GET /api/integrations/:integrationId/runs workflow run history, newest first (?workflowId&status&before&limit) + * + * App admins (x-frigg-admin-api-key, the validateAdminApiKey middleware that /admin/scripts and /admin/db-migrate use): + * POST /admin/integrations/definitions { document, notes?, publish? = true, strict? } validate + publish + * GET /admin/integrations/definitions/:id every version of one definition + * POST /admin/integrations/definitions/:id/versions/:version/deprecate + * + * Installing needs no new route: POST /api/integrations with config { type: '', definition: { id } } + * pins the latest published version. + */ +const catchAsyncError = require('express-async-handler'); +const { validateAdminApiKey } = require('../../handlers/middleware/admin-auth'); +const { DefinitionError, PublishDefinitionVersion, DeprecateDefinitionVersion, GetDefinitionHistory } = require('./use-cases/definition-use-cases'); + +function sendError(res, e) { + if (e instanceof DefinitionError || e?.name === 'DefinitionError') return res.status(e.status || 400).json({ error: e.message, code: e.code, details: e.details }); + throw e; +} +const guard = (fn) => catchAsyncError(async (req, res) => { try { await fn(req, res); } catch (e) { sendError(res, e); } }); + +/** + * @param {import('express').Router} router + * @param {{ execute(req): Promise<{ getId(): string }> }} authenticateUser + * @param {{ getIntegrationInstance, integrationClasses: Function[] }} deps + */ +function setDataDefinedRoutes(router, authenticateUser, { getIntegrationInstance, integrationClasses }) { + const classes = integrationClasses.filter((c) => c.DataDefined); + if (!classes.length) return false; + const catalogClass = classes.find((c) => c.DataDefined.shared) || classes[0]; + const { definitions, manifests } = catalogClass.DataDefined; + + const installation = async (req) => { + const user = await authenticateUser.execute(req); + const integration = await getIntegrationInstance.execute(req.params.integrationId, user.getId()); + if (!integration?.constructor?.DataDefined) throw new DefinitionError('This integration is not data-defined', { status: 404, code: 'NOT_DATA_DEFINED' }); + return integration; + }; + + router.route('/api/data-defined/definitions').get(guard(async (req, res) => { + await authenticateUser.execute(req); + const lists = await Promise.all(classes.map((c) => (c.DataDefined.definitions.list ? c.DataDefined.definitions.list() : []))); + const seen = new Set(); + res.json({ definitions: lists.flat().filter((d) => (seen.has(d.id) ? false : seen.add(d.id))) }); + })); + + router.route('/api/integrations/:integrationId/upgrade') + .get(guard(async (req, res) => { + const integration = await installation(req); + res.json(await integration.upgradeTo(req.query.to || undefined, { apply: false })); + })) + .post(guard(async (req, res) => { + const integration = await installation(req); + const result = await integration.upgradeTo(req.body?.to || undefined, { apply: true }); + res.status(result.applied ? 200 : 409).json(result); + })); + + router.route('/api/integrations/:integrationId/runs').get(guard(async (req, res) => { + const integration = await installation(req); + const store = integration.constructor.DataDefined.runs(); + if (!store) return res.json({ runs: [] }); + const { workflowId, status, before, limit } = req.query; + res.json({ runs: await store.listRuns(integration.id, { workflowId, status, before, limit: limit ? Number(limit) : undefined }) }); + })); + + const admin = (path) => router.route(path).all(validateAdminApiKey); + const repository = () => { + const repo = definitions.repository; + if (!repo) throw new DefinitionError('This app runs bundled definitions; there is no store to publish to', { status: 501, code: 'NO_STORE' }); + return repo; + }; + + admin('/admin/integrations/definitions').post(guard(async (req, res) => { + const { document, notes, publish, strict, createdBy } = req.body || {}; + const out = await new PublishDefinitionVersion({ repository: repository(), manifests }).execute({ document, notes, createdBy, publish: publish !== false, strict: Boolean(strict) }); + const { document: _omit, ...version } = out.version; + res.status(out.created ? 201 : 200).json({ ...out, version }); + })); + admin('/admin/integrations/definitions/:definitionId').get(guard(async (req, res) => { + res.json({ versions: await new GetDefinitionHistory({ repository: repository() }).execute({ definitionId: req.params.definitionId }) }); + })); + admin('/admin/integrations/definitions/:definitionId/versions/:version/deprecate').post(guard(async (req, res) => { + const row = await new DeprecateDefinitionVersion({ repository: repository() }).execute(req.params); + const { document: _omit, ...version } = row; + res.json({ version }); + })); + return true; +} + +module.exports = { setDataDefinedRoutes }; diff --git a/packages/core/integrations/data-defined/runtime.js b/packages/core/integrations/data-defined/runtime.js new file mode 100644 index 000000000..31373bb9c --- /dev/null +++ b/packages/core/integrations/data-defined/runtime.js @@ -0,0 +1,208 @@ +/** + * The pieces DataDefinedIntegration wires to Frigg. Each takes its I/O as arguments, so the whole runtime can be + * driven in a test with fakes and the same code runs in Lambda. + * + * makePorts capability calls go through the module manifest to an Api method; links live in IntegrationMapping + * runWorkflow one delivered event -> one WorkflowRun (dedupe, filter, steps, trace) + * runQuery a UI surface's data + * renderSurface query + view -> a resolved component tree the host's generic renderer draws + * runLifecycle onInstall / onUninstall steps + */ +const { execute } = require('./interpreter'); +const { evaluate, resolve, getPath } = require('./expressions'); +const { TRANSFORMS } = require('./transforms'); + +/** + * @param {object} p + * @param {object} p.def + * @param {Record} p.manifests manifest name -> manifest + * @param {(moduleKey:string, method:string, args:object) => Promise} p.callApi e.g. (k, m, a) => this[k].api[m](a) + * @param {{ get(key:string): Promise, put(key:string, value:object): Promise, claim(key:string, value:object): Promise }} p.store + * Backed by IntegrationMapping: get = getMapping, put = upsertMapping, claim = createMapping (insert-only; false if it existed) + * @param {(ref:string, args:object) => Promise} [p.runCode] + */ +function makePorts({ def, manifests, callApi, store, runCode }) { + const manifestFor = (key) => manifests[def.modules[key]?.module]; + const normalise = (set, key, value) => { + const m = (set.identity?.match || []).find((x) => x.key === key); + const fn = m?.transform && TRANSFORMS[m.transform]; + return fn ? fn(value) : value; + }; + /** A sync or a link set (definition.links): both store links the same way, under their own id. */ + const linkSetOf = ({ link, sync }) => { + const id = link ?? sync; + const s = link != null ? (def.links || []).find((x) => x.id === id) : (def.syncs || []).find((x) => x.id === id); + if (!s) throw new Error(`Unknown ${link != null ? 'link set' : 'sync'} "${id}"`); + return s; + }; + return { + async call(moduleKey, capability, args) { + const c = manifestFor(moduleKey)?.capabilities?.[capability]; + if (!c) throw new Error(`${moduleKey}.${capability} is not declared by the module manifest`); + return callApi(moduleKey, c.method, args); + }, + links: { + /** By one side's id: link:::. By identity key: idx:::. */ + async find({ link, sync, from, id, key }) { + const s = linkSetOf({ link, sync }); + if (from && id != null) return (await store.get(`link:${s.id}:${from}:${id}`)) || null; + for (const [k, v] of Object.entries(key || {})) { + if (v == null || v === '') continue; + const hit = await store.get(`idx:${s.id}:${k}:${normalise(s, k, v)}`); + if (hit) return hit; + } + return null; + }, + async upsert({ link, sync, ids, keys }) { + const s = linkSetOf({ link, sync }); + const value = { ...ids, linkedAt: new Date().toISOString() }; + for (const [mod, id] of Object.entries(ids || {})) if (id != null) await store.put(`link:${s.id}:${mod}:${id}`, value); + for (const [k, v] of Object.entries(keys || {})) if (v != null && v !== '') await store.put(`idx:${s.id}:${k}:${normalise(s, k, v)}`, value); + return value; + }, + }, + dedupe: { + claim: (key) => store.claim(key, { at: new Date().toISOString() }), + /** A failed run gives its claim back, so the queue's retry runs the steps instead of reading as a duplicate. */ + release: (key) => (store.release ? store.release(key) : undefined), + }, + runCode, + }; +} + +/** The record an event is about, per the manifest's `record` path (phone: data.object; HubSpot: the payload itself). */ +function recordOf(manifests, def, moduleKey, topic, payload) { + const ev = manifests[def.modules[moduleKey]?.module]?.events?.[topic]; + return ev?.record ? getPath(payload, '$.' + ev.record) : payload; +} + +/** + * Run one workflow for one event. + * @returns {Promise} a WorkflowRun record (see schema/records.md) + */ +async function runWorkflow({ def, installation, workflowId, event = {}, eventId, record, ports, manifests, now = () => new Date(), runId }) { + const w = (def.workflows || []).find((x) => x.id === workflowId); + if (!w) throw new Error(`Unknown workflow "${workflowId}"`); + const t = w.trigger; + if (record === undefined && t.type === 'webhook') record = recordOf(manifests, def, t.module, t.topic, event); + const startedAt = now(); + const base = { + id: runId || `run_${startedAt.getTime().toString(36)}`, integrationId: installation.integrationId || null, + definition: { id: def.id, version: def.version }, workflowId, trigger: { type: t.type, eventId: eventId ?? event.id ?? null }, + startedAt: startedAt.toISOString(), + }; + const settings = { ...defaultsOf(w.settings), ...(installation.workflows?.[w.id]?.settings || {}) }; + const install = installContext(def, installation); + const ctx = { event, record, settings, install, now: startedAt.toISOString() }; + + if (t.filter && !evaluate(t.filter, ctx)) return { ...base, status: 'filtered', endedAt: now().toISOString(), trace: [] }; + // The provider's event id (from the manifest's webhook.eventId, else event.id) makes a redelivery a no-op. + const dedupeId = eventId ?? event.id; + const claimKey = dedupeId && ports.dedupe ? `evt:${def.id}:${w.id}:${dedupeId}` : null; + if (claimKey) { + const fresh = await ports.dedupe.claim(claimKey); + if (!fresh) return { ...base, status: 'duplicate', endedAt: now().toISOString(), trace: [] }; + } + try { + const out = await execute(w.steps, { def, installation, ctx, ports }); + return { ...base, status: out.status, reason: out.reason, endedAt: now().toISOString(), trace: out.trace }; + } catch (e) { + if (claimKey) await ports.dedupe.release?.(claimKey); + return { ...base, status: 'failed', error: String(e.message || e), endedAt: now().toISOString(), trace: e.trace || [], retryable: true }; + } +} + +async function runQuery({ def, installation, queryId, record, ports }) { + const q = (def.queries || []).find((x) => x.id === queryId); + if (!q) throw new Error(`Unknown query "${queryId}"`); + const install = installContext(def, installation); + const out = await execute(q.steps, { def, installation, ctx: { record, install, event: {} }, ports }); + if (out.status === 'stopped') return { stopped: out.reason }; + return resolve(q.returns, { steps: out.steps, record, install }); +} + +/** Resolve a view tree against data: drop nodes whose `when` is false, expand List/Table rows. */ +function resolveView(node, ctx) { + if (!node) return null; + if (node.when && !evaluate(node.when, ctx)) return null; + const props = { ...(node.props || {}) }; + if (node.component === 'List') { + const items = resolve(props.items, ctx) || []; + props.items = (Array.isArray(items) ? items : []).map((item) => ({ title: resolve(props.title, { ...ctx, item }), meta: resolve(props.meta, { ...ctx, item }) })); + delete props.title; delete props.meta; + } else if (node.component === 'Table') { + const rows = resolve(props.rows, ctx) || []; + props.rows = (Array.isArray(rows) ? rows : []).map((row) => (props.columns || []).map((c) => resolve(c.value, { ...ctx, row }))); + props.columns = (props.columns || []).map((c) => c.label); + } else { + for (const [k, v] of Object.entries(props)) if (k !== 'action') props[k] = resolve(v, ctx); + if (props.action?.url) props.action = { ...props.action, url: resolve(props.action.url, ctx) }; + } + const children = (node.children || []).map((c) => resolveView(c, ctx)).filter(Boolean); + return { component: node.component, props, ...(children.length ? { children } : {}) }; +} + +/** UI_ (POST /api/integrations/:id/actions/UI_) -> { surface, view }: what a host renderer draws. */ +async function renderSurface({ def, installation, surfaceId, record, ports }) { + const u = (def.ui || []).find((x) => x.id === surfaceId); + if (!u) throw new Error(`Unknown UI surface "${surfaceId}"`); + const data = await runQuery({ def, installation, queryId: u.data.query, record, ports }); + const install = installContext(def, installation); + return { surface: { id: u.id, host: u.host, placement: u.placement, definition: `${def.id}@${def.version}` }, view: resolveView(u.view, { data, record, install }) }; +} + +async function runLifecycle({ def, installation, hook, ports }) { + const steps = def.lifecycle?.[hook] || []; + if (!steps.length) return { status: 'completed', steps: {}, trace: [] }; + const install = installContext(def, installation); + return execute(steps, { def, installation, ctx: { install, event: {}, record: null }, ports }); +} + +/** + * The choices for an option setting, for the configure form. `options` is "module.list" or + * { source: "module.list", with: { : "$.settings." } } — the list's capability is called with + * those inputs, resolved from the values the user has chosen so far, so a later dropdown depends on an earlier one + * (Asana projects for the chosen workspace). Returns [] until every required input has a value. + * + * @param {{ def, manifests, ports, path: string, values?: object, integrationSettings?: object }} p + * path: "settings.project" or "workflows..settings."; values: the form's current values at that level; + * integrationSettings: the integration-level values, which a workflow setting's inputs read as $.install + * @returns {Promise>} + */ +async function resolveOptions({ def, manifests, ports, path, values = {}, integrationSettings = {} }) { + const segs = path.split('.'); + const setting = segs[0] === 'workflows' + ? (def.workflows || []).find((w) => w.id === segs[1])?.settings?.[segs[3]] + : def.settings?.[segs[1]]; + if (!setting || setting.type !== 'option') throw new Error(`"${path}" is not an option setting`); + const ref = typeof setting.options === 'string' ? { source: setting.options } : setting.options; + const [moduleKey, listName] = ref.source.split('.'); + const list = manifests[def.modules[moduleKey]?.module]?.options?.[listName]; + if (!list) throw new Error(`Module "${moduleKey}" has no option list "${listName}"`); + const ctx = { settings: values, install: installContext(def, { settings: segs[0] === 'settings' ? values : integrationSettings }) }; + const inputs = resolve(ref.with || {}, ctx); + for (const [name, spec] of Object.entries(list.inputs || {})) if (spec.required && (inputs[name] == null || inputs[name] === '')) return []; + const items = await ports.call(moduleKey, list.capability, inputs); + return (Array.isArray(items) ? items : []).map((item) => ({ label: String(resolve(list.label, { item }) ?? ''), value: resolve(list.value, { item }) })); +} + +/** + * $.install, the same everywhere a definition runs (workflows, queries, UI, lifecycle, option lists): the integration + * settings with their defaults, the installation's webhook URL and id, `accounts` (per module key, the account fields + * its manifest declares, e.g. $.install.accounts.quickbooks.realmId) and `runtime.installed` (each onInstall step's + * result, by step id, so onUninstall can undo it). + */ +function installContext(def, installation = {}) { + return { + ...defaultsOf(def.settings), ...(installation.settings || {}), + webhookUrl: installation.webhookUrl, integrationId: installation.integrationId, portalId: installation.portalId, + accounts: installation.accounts || {}, + runtime: { installed: installation.runtime?.installed || {} }, + }; +} + +function defaultsOf(settings) { + return Object.fromEntries(Object.entries(settings || {}).filter(([, s]) => s.default !== undefined).map(([k, s]) => [k, s.default])); +} + +module.exports = { makePorts, runWorkflow, runQuery, renderSurface, resolveView, runLifecycle, resolveOptions, installContext, recordOf, defaultsOf }; diff --git a/packages/core/integrations/data-defined/schema-check.js b/packages/core/integrations/data-defined/schema-check.js new file mode 100644 index 000000000..6dcdc1596 --- /dev/null +++ b/packages/core/integrations/data-defined/schema-check.js @@ -0,0 +1,58 @@ +/** + * Two layers of validation, run together everywhere (the CLI, the tests, anything that publishes): + * 1. JSON Schema (schema/*.schema.json) — the published shape. Needs ajv, which the monorepo installs for + * @friggframework/schemas; core's runtime stays dependency-free, so ajv is loaded only here. + * 2. References (validate.js / manifests.js) — what a schema cannot see: steps name declared capabilities, + * $.steps.x is an earlier step, view paths exist, option inputs are declared, and so on. + * A document is valid only when both pass. + */ +const fs = require('fs'); +const path = require('path'); +const { validate } = require('./validate'); +const { validateManifest } = require('./manifests'); + +const SCHEMA_DIR = path.join(__dirname, 'schema'); +let compiled; + +function loadAjv() { + try { return require('ajv/dist/2020'); } catch { + const e = new Error('ajv is not installed. Run npm install at the monorepo root (@friggframework/schemas depends on ajv@8).'); + e.code = 'NO_AJV'; + throw e; + } +} + +function schemas() { + if (compiled) return compiled; + const Ajv = loadAjv(); + const ajv = new Ajv({ allErrors: true, strict: false }); + const read = (f) => JSON.parse(fs.readFileSync(path.join(SCHEMA_DIR, f), 'utf8')); + compiled = { + definition: ajv.compile(read('integration-definition.v0.1.schema.json')), + manifest: ajv.compile(read('capability-manifest.v0.1.schema.json')), + installation: ajv.compile(read('installation.v0.1.schema.json')), + }; + return compiled; +} + +const schemaErrors = (check, doc) => (check(doc) ? [] : check.errors.map((e) => ({ layer: 'schema', path: e.instancePath.replace(/^\//, '').replace(/\//g, '.') || '(root)', message: e.message + (e.params?.allowedValues ? ` (${e.params.allowedValues.join(', ')})` : e.params?.additionalProperty ? ` ("${e.params.additionalProperty}")` : '') }))); + +/** @returns {{ valid: boolean, errors: Array<{layer,path,message}>, warnings: Array<{layer,path,message}> }} */ +function checkManifest(manifest) { + const errors = [...schemaErrors(schemas().manifest, manifest), ...validateManifest(manifest).errors.map((e) => ({ layer: 'references', ...e }))]; + return { valid: errors.length === 0, errors, warnings: [] }; +} + +/** A definition against the manifests of the modules it uses. Both layers run even when the first fails. */ +function checkDefinition(def, manifests) { + const v = validate(def, manifests); + const errors = [...schemaErrors(schemas().definition, def), ...v.errors.map((e) => ({ layer: 'references', ...e }))]; + return { valid: errors.length === 0, errors, warnings: v.warnings.map((w) => ({ layer: 'references', ...w })) }; +} + +function checkInstallation(config) { + const errors = schemaErrors(schemas().installation, config); + return { valid: errors.length === 0, errors, warnings: [] }; +} + +module.exports = { checkDefinition, checkManifest, checkInstallation, loadAjv }; diff --git a/packages/core/integrations/data-defined/schema/capability-manifest.v0.1.schema.json b/packages/core/integrations/data-defined/schema/capability-manifest.v0.1.schema.json new file mode 100644 index 000000000..fa4ddbc45 --- /dev/null +++ b/packages/core/integrations/data-defined/schema/capability-manifest.v0.1.schema.json @@ -0,0 +1,377 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://friggframework.org/schemas/capability-manifest/v0.1.json", + "title": "Frigg API module capability manifest v0.1", + "description": "What an API module can do, in terms a definition can reference: objects and their fields, events it emits, capabilities (each backed by one Api method) and option lists for settings. Shipped with the module.", + "type": "object", + "required": [ + "module", + "package", + "objects", + "capabilities" + ], + "additionalProperties": false, + "properties": { + "$schema": { + "type": "string" + }, + "module": { + "type": "string" + }, + "label": { + "type": "string" + }, + "package": { + "type": "string" + }, + "version": { + "type": "string" + }, + "auth": { + "enum": [ + "oauth2", + "api_key", + "basic" + ] + }, + "account": { + "type": "object", + "description": "Values about the connected account the module persists (apiPropertiesToPersist). Definitions read them as $.install.accounts..; nothing undeclared is exposed.", + "required": [ + "fields" + ], + "properties": { + "description": { + "type": "string" + }, + "fields": { + "type": "array", + "items": { + "type": "object", + "required": [ + "name", + "type" + ], + "additionalProperties": false, + "properties": { + "name": { + "type": "string", + "pattern": "^[A-Za-z_]\\w*$" + }, + "type": { + "type": "string" + }, + "description": { + "type": "string" + } + } + } + } + } + }, + "objects": { + "type": "object", + "additionalProperties": { + "type": "object", + "properties": { + "label": { + "type": "string" + }, + "fields": { + "type": "array", + "items": { + "type": "object", + "required": [ + "name", + "type" + ], + "properties": { + "name": { + "type": "string" + }, + "type": { + "type": "string" + }, + "tag": { + "type": "string" + }, + "required": { + "type": "boolean" + }, + "readOnly": { + "type": "boolean" + }, + "values": { + "description": "enum: the values, or the name of one of this module's option lists when they differ per account.", + "oneOf": [ + { + "type": "array" + }, + { + "type": "string" + } + ] + } + } + } + } + } + } + }, + "events": { + "type": "object", + "additionalProperties": { + "type": "object", + "required": [ + "delivery" + ], + "properties": { + "delivery": { + "enum": [ + "webhook", + "poll" + ] + }, + "object": { + "type": "string" + }, + "description": { + "type": "string" + }, + "payload": { + "type": [ + "object", + "array" + ], + "description": "Example delivery body, exactly as the provider sends it (HubSpot sends an array). With the manifest's webhook block it must route to this event; $.event is the event item." + }, + "subscribe": { + "type": "string", + "description": "Capability that registers the subscription on install." + }, + "record": { + "type": "string", + "description": "Path inside the payload to the record the event is about (exposed as $.record). Omit when the payload is the record." + }, + "idPath": { + "type": "string", + "default": "id", + "description": "Path to the provider's event id, used to run each event once." + }, + "match": { + "type": "string", + "description": "The provider's topic for this event (default: the event key)." + }, + "when": { + "type": "string", + "description": "Condition on $.record / $.event: a phone module reports a missed call as call.completed with status missed." + }, + "headers": { + "type": "object", + "description": "Headers of the example delivery, when the topic is in a header." + } + } + } + }, + "webhook": { + "type": "object", + "description": "How a POST from this provider becomes module events (webhooks.js). Templates read body, headers (lower-case), query, item (one event of a batch) and parent (the outer item of a two-level batch).", + "required": [ + "scope", + "topic", + "eventId" + ], + "additionalProperties": false, + "properties": { + "scope": { + "enum": [ + "installation", + "app" + ], + "description": "installation: a URL per installation. app: one URL for every account, routed by account." + }, + "topic": { + "oneOf": [ + { + "type": "string" + }, + { + "type": "array", + "items": { + "type": "string" + }, + "minItems": 1 + } + ], + "description": "The provider's event type. A list tries each template until one renders non-empty." + }, + "eventId": { + "oneOf": [ + { + "type": "string" + }, + { + "type": "array", + "items": { + "type": "string" + }, + "minItems": 1 + } + ], + "description": "The id that makes a redelivery a no-op." + }, + "account": { + "oneOf": [ + { + "type": "string" + }, + { + "type": "array", + "items": { + "type": "string" + }, + "minItems": 1 + } + ], + "description": "The connected account (entity externalId) an event is about. Required for app scope." + }, + "batch": { + "oneOf": [ + { + "type": "string" + }, + { + "type": "array", + "items": { + "type": "string" + }, + "minItems": 1, + "maxItems": 2 + } + ], + "description": "Path to the array of events; [outer, inner] for two levels." + }, + "verify": { + "type": "string", + "pattern": "^[A-Za-z_$][\\w$]*$", + "description": "Api method that checks the signature: ({ headers, body, query }) -> true." + }, + "unwrap": { + "type": "string", + "description": "A form field holding the JSON body (Slack interactivity)." + }, + "handshake": { + "type": "object", + "required": [ + "when" + ], + "additionalProperties": false, + "properties": { + "when": { + "type": "string" + }, + "status": { + "type": "integer" + }, + "headers": { + "type": "object" + }, + "body": {} + }, + "description": "Answered at receipt, before anything is queued (Slack url_verification, Asana X-Hook-Secret)." + } + } + }, + "capabilities": { + "type": "object", + "additionalProperties": { + "type": "object", + "required": [ + "kind", + "method" + ], + "properties": { + "kind": { + "enum": [ + "read", + "write", + "delete", + "subscribe", + "search" + ] + }, + "method": { + "type": "string", + "description": "Api class method that implements it." + }, + "label": { + "type": "string" + }, + "input": { + "type": "object", + "description": "Argument names -> {type, required}." + }, + "output": { + "type": "string" + } + } + } + }, + "options": { + "type": "object", + "additionalProperties": { + "type": "object", + "required": [ + "capability", + "label", + "value" + ], + "properties": { + "capability": { + "type": "string" + }, + "label": { + "type": "string", + "description": "One reference ($.item.name) or a template ({{item.firstName}} {{item.lastName}}); never a $. reference mixed into text." + }, + "value": { + "type": "string" + }, + "inputs": { + "type": "object", + "description": "Arguments the list capability needs, filled from a setting's options.with.", + "additionalProperties": { + "type": "object", + "required": [ + "type" + ], + "properties": { + "type": { + "type": "string" + }, + "required": { + "type": "boolean" + } + } + } + } + } + } + }, + "ui": { + "type": "object", + "properties": { + "placements": { + "type": "array", + "items": { + "type": "string" + } + }, + "renderer": { + "type": "string", + "description": "The generic renderer extension deployed once for this host." + } + } + } + } +} diff --git a/packages/core/integrations/data-defined/schema/installation.v0.1.schema.json b/packages/core/integrations/data-defined/schema/installation.v0.1.schema.json new file mode 100644 index 000000000..a0e05cfd0 --- /dev/null +++ b/packages/core/integrations/data-defined/schema/installation.v0.1.schema.json @@ -0,0 +1,83 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://friggframework.org/schemas/installation/v0.1.json", + "title": "Data-defined installation config v0.1", + "description": "What one customer chose, stored as Integration.config. The definition says what is possible; this says what is on.", + "type": "object", + "required": [ + "definition" + ], + "properties": { + "definition": { + "type": "object", + "required": [ + "id", + "version" + ], + "properties": { + "id": { + "type": "string" + }, + "version": { + "type": "string" + }, + "checksum": { + "type": "string" + } + } + }, + "settings": { + "type": "object" + }, + "syncs": { + "type": "object", + "additionalProperties": { + "type": "object", + "properties": { + "enabled": { + "type": "boolean" + }, + "mapping": { + "type": "array", + "description": "Full replacement mapping when the customer overrides the default." + } + } + } + }, + "workflows": { + "type": "object", + "additionalProperties": { + "type": "object", + "properties": { + "enabled": { + "type": "boolean" + }, + "settings": { + "type": "object" + } + } + } + }, + "ui": { + "type": "object", + "additionalProperties": { + "type": "object", + "properties": { + "enabled": { + "type": "boolean" + } + } + } + }, + "runtime": { + "type": "object", + "description": "Owned by the runtime, not the customer. installed: each onInstall step's result by step id (webhook subscriptions), read by onUninstall as $.install.runtime.installed.. Sync cursors join it with the sync engine.", + "additionalProperties": false, + "properties": { + "installed": { + "type": "object" + } + } + } + } +} diff --git a/packages/core/integrations/data-defined/schema/integration-definition.v0.1.schema.json b/packages/core/integrations/data-defined/schema/integration-definition.v0.1.schema.json new file mode 100644 index 000000000..4620b8577 --- /dev/null +++ b/packages/core/integrations/data-defined/schema/integration-definition.v0.1.schema.json @@ -0,0 +1,848 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://friggframework.org/schemas/integration-definition/v0.1.json", + "title": "Frigg integration definition v0.1", + "description": "An integration as data: what it connects, what it keeps in sync, which workflows it runs and which UI it shows. One document per published version; immutable once published (ADR-063).", + "type": "object", + "required": [ + "schemaVersion", + "id", + "version", + "display", + "modules" + ], + "additionalProperties": false, + "properties": { + "$schema": { + "type": "string" + }, + "schemaVersion": { + "const": "0.1" + }, + "id": { + "type": "string", + "pattern": "^[a-z][a-z0-9-]*$" + }, + "version": { + "type": "string", + "pattern": "^\\d+\\.\\d+\\.\\d+(-[\\w.]+)?$" + }, + "display": { + "type": "object", + "required": [ + "label" + ], + "properties": { + "label": { + "type": "string" + }, + "description": { + "type": "string" + }, + "category": { + "type": "string" + }, + "tags": { + "type": "array", + "items": { + "type": "string" + } + } + } + }, + "modules": { + "type": "object", + "minProperties": 1, + "additionalProperties": { + "type": "object", + "required": [ + "module" + ], + "properties": { + "module": { + "type": "string", + "description": "Capability manifest name." + }, + "version": { + "type": "string" + } + } + } + }, + "settings": { + "type": "object", + "additionalProperties": { + "$ref": "#/$defs/setting" + } + }, + "syncs": { + "type": "array", + "items": { + "type": "object", + "required": [ + "id", + "label", + "source", + "target", + "direction", + "identity", + "mapping" + ], + "additionalProperties": false, + "properties": { + "id": { + "type": "string", + "pattern": "^[a-z][a-z0-9-]*$" + }, + "label": { + "type": "string" + }, + "description": { + "type": "string" + }, + "source": { + "type": "string", + "pattern": "^[a-zA-Z]\\w*\\.[a-zA-Z]\\w*$", + "description": "." + }, + "target": { + "type": "string", + "pattern": "^[a-zA-Z]\\w*\\.[a-zA-Z]\\w*$" + }, + "direction": { + "enum": [ + "one_way", + "two_way" + ] + }, + "identity": { + "type": "object", + "required": [ + "match" + ], + "properties": { + "match": { + "type": "array", + "minItems": 1, + "items": { + "type": "object", + "required": [ + "key", + "source", + "target" + ], + "properties": { + "source": { + "type": "string" + }, + "target": { + "type": "string" + }, + "transform": { + "enum": [ + "none", + "trim", + "lowercase", + "uppercase", + "number", + "boolean", + "to_iso_date", + "to_e164_phone", + "split_full_name", + "join", + "map_values", + "default", + "units_to_cents", + "cents_to_units", + "from_unix_seconds", + "to_unix_seconds" + ] + }, + "key": { + "type": "string", + "pattern": "^[a-z][a-zA-Z0-9]*$", + "description": "Name of the identity key (email, phone). links.find { key: { phone } } looks records up by it." + } + } + } + }, + "onAmbiguous": { + "enum": [ + "skip", + "first", + "create" + ], + "default": "skip" + } + } + }, + "mapping": { + "type": "array", + "items": { + "$ref": "#/$defs/mappingRow" + } + }, + "filter": { + "type": "string", + "description": "Only records matching this condition are synced ($.record)." + }, + "backfill": { + "type": "object", + "properties": { + "trigger": { + "enum": [ + "userAction", + "onInstall", + "none" + ] + }, + "pageSize": { + "type": "integer", + "minimum": 1, + "maximum": 500 + } + } + }, + "ongoing": { + "type": "object", + "required": [ + "mode" + ], + "properties": { + "mode": { + "enum": [ + "webhook", + "poll", + "none" + ] + }, + "every": { + "type": "string" + } + } + }, + "default": { + "enum": [ + "on", + "off" + ], + "default": "on" + } + } + } + }, + "links": { + "type": "array", + "description": "Link sets: records on two modules known to be the same thing, with no sync behind them. links.find / links.upsert take { link: }.", + "items": { + "type": "object", + "required": [ + "id", + "between" + ], + "additionalProperties": false, + "properties": { + "id": { + "type": "string", + "pattern": "^[a-z][a-z0-9-]*$" + }, + "label": { + "type": "string" + }, + "description": { + "type": "string" + }, + "between": { + "type": "array", + "minItems": 2, + "maxItems": 2, + "items": { + "type": "string", + "pattern": "^[a-zA-Z]\\w*\\.[a-zA-Z]\\w*$" + }, + "description": "Two . on different modules." + }, + "identity": { + "type": "object", + "required": [ + "match" + ], + "additionalProperties": false, + "properties": { + "match": { + "type": "array", + "items": { + "type": "object", + "required": [ + "key" + ], + "additionalProperties": false, + "properties": { + "key": { + "type": "string", + "pattern": "^[a-z][a-zA-Z0-9]*$" + }, + "transform": { + "enum": [ + "none", + "trim", + "lowercase", + "uppercase", + "number", + "boolean", + "to_iso_date", + "to_e164_phone", + "split_full_name", + "join", + "map_values", + "default", + "units_to_cents", + "cents_to_units", + "from_unix_seconds", + "to_unix_seconds" + ] + } + } + } + } + } + } + } + } + }, + "workflows": { + "type": "array", + "items": { + "type": "object", + "required": [ + "id", + "label", + "trigger", + "steps" + ], + "additionalProperties": false, + "properties": { + "id": { + "type": "string", + "pattern": "^[a-z][a-z0-9-]*$" + }, + "label": { + "type": "string" + }, + "description": { + "type": "string" + }, + "purpose": { + "enum": [ + "notify", + "write", + "enrich", + "maintain", + "respond" + ], + "description": "What the workflow is for. Shown in the UI; not executed." + }, + "default": { + "enum": [ + "on", + "off" + ], + "default": "on" + }, + "settings": { + "type": "object", + "additionalProperties": { + "$ref": "#/$defs/setting" + } + }, + "trigger": { + "$ref": "#/$defs/trigger" + }, + "steps": { + "type": "array", + "minItems": 1, + "items": { + "$ref": "#/$defs/step" + } + }, + "retry": { + "type": "object", + "properties": { + "max": { + "type": "integer", + "minimum": 0, + "maximum": 10 + }, + "backoff": { + "enum": [ + "fixed", + "exponential" + ] + } + } + } + } + } + }, + "queries": { + "type": "array", + "items": { + "type": "object", + "required": [ + "id", + "steps", + "returns" + ], + "additionalProperties": false, + "properties": { + "id": { + "type": "string", + "pattern": "^[a-z][a-z0-9-]*$" + }, + "description": { + "type": "string" + }, + "steps": { + "type": "array", + "items": { + "$ref": "#/$defs/step" + } + }, + "returns": { + "description": "Expression for the query result." + } + } + } + }, + "ui": { + "type": "array", + "items": { + "type": "object", + "required": [ + "id", + "label", + "host", + "placement", + "data", + "view" + ], + "additionalProperties": false, + "properties": { + "id": { + "type": "string", + "pattern": "^[a-z][a-z0-9-]*$" + }, + "label": { + "type": "string" + }, + "host": { + "type": "string", + "description": "Module key of the app that shows it." + }, + "placement": { + "type": "string", + "description": "One of the host manifest's ui.placements." + }, + "objects": { + "type": "array", + "items": { + "type": "string" + } + }, + "data": { + "type": "object", + "required": [ + "query" + ], + "properties": { + "query": { + "type": "string" + }, + "with": { + "type": "object" + } + } + }, + "view": { + "$ref": "#/$defs/view" + }, + "default": { + "enum": [ + "on", + "off" + ], + "default": "on" + } + } + } + }, + "templates": { + "type": "object", + "additionalProperties": { + "type": "string" + } + }, + "lifecycle": { + "type": "object", + "additionalProperties": false, + "properties": { + "onInstall": { + "type": "array", + "items": { + "$ref": "#/$defs/step" + } + }, + "onUninstall": { + "type": "array", + "items": { + "$ref": "#/$defs/step" + } + } + } + } + }, + "$defs": { + "step": { + "type": "object", + "required": [ + "use" + ], + "additionalProperties": false, + "properties": { + "id": { + "type": "string", + "pattern": "^[a-zA-Z][\\w-]*$" + }, + "use": { + "type": "string", + "description": "A built-in (links.find, links.upsert, map, merge, template, set, stop, foreach, branch, code) or .. declared by that module's capability manifest." + }, + "with": { + "type": "object", + "description": "Arguments. Values may be expressions." + }, + "when": { + "type": "string", + "description": "Condition; the step is skipped when false." + }, + "onError": { + "enum": [ + "fail", + "continue", + "stop" + ], + "default": "fail" + }, + "do": { + "type": "array", + "items": { + "$ref": "#/$defs/step" + }, + "description": "foreach: steps run per item." + }, + "cases": { + "type": "array", + "items": { + "type": "object", + "required": [ + "when", + "do" + ], + "properties": { + "when": { + "type": "string" + }, + "do": { + "type": "array", + "items": { + "$ref": "#/$defs/step" + } + } + } + } + }, + "else": { + "type": "array", + "items": { + "$ref": "#/$defs/step" + } + } + } + }, + "setting": { + "type": "object", + "required": [ + "type", + "label" + ], + "additionalProperties": false, + "properties": { + "type": { + "enum": [ + "string", + "text", + "number", + "boolean", + "enum", + "option", + "duration" + ] + }, + "label": { + "type": "string" + }, + "help": { + "type": "string" + }, + "required": { + "type": "boolean" + }, + "default": {}, + "enum": { + "type": "array", + "items": { + "type": "string" + } + }, + "options": { + "description": "option settings: \".\" from the module manifest, or { source, with } when the list takes inputs read from other settings ($.settings.workspace; a workflow setting may also read $.install.).", + "oneOf": [ + { + "type": "string", + "pattern": "^[a-zA-Z]\\w*\\.[a-zA-Z]\\w*$" + }, + { + "type": "object", + "required": [ + "source" + ], + "additionalProperties": false, + "properties": { + "source": { + "type": "string", + "pattern": "^[a-zA-Z]\\w*\\.[a-zA-Z]\\w*$" + }, + "with": { + "type": "object", + "additionalProperties": { + "type": "string" + } + } + } + } + ] + } + } + }, + "trigger": { + "oneOf": [ + { + "type": "object", + "required": [ + "type", + "module", + "topic" + ], + "additionalProperties": false, + "properties": { + "type": { + "const": "webhook" + }, + "module": { + "type": "string" + }, + "topic": { + "type": "string" + }, + "filter": { + "type": "string" + } + } + }, + { + "type": "object", + "required": [ + "type", + "cron" + ], + "additionalProperties": false, + "properties": { + "type": { + "const": "schedule" + }, + "cron": { + "type": "string" + }, + "timezone": { + "type": "string" + } + } + }, + { + "type": "object", + "required": [ + "type", + "label" + ], + "additionalProperties": false, + "properties": { + "type": { + "const": "userAction" + }, + "label": { + "type": "string" + }, + "from": { + "type": "string", + "description": "A UI surface id when the action is a button on that surface." + } + } + }, + { + "type": "object", + "required": [ + "type", + "sync", + "on" + ], + "additionalProperties": false, + "properties": { + "type": { + "const": "syncRecord" + }, + "sync": { + "type": "string" + }, + "on": { + "type": "array", + "items": { + "enum": [ + "created", + "updated", + "linked" + ] + } + }, + "filter": { + "type": "string" + } + } + } + ] + }, + "view": { + "type": "object", + "required": [ + "component" + ], + "additionalProperties": false, + "properties": { + "component": { + "enum": [ + "Stack", + "Heading", + "Text", + "KeyValue", + "List", + "Table", + "Badge", + "Button", + "Empty", + "Divider" + ] + }, + "props": { + "type": "object" + }, + "children": { + "type": "array", + "items": { + "$ref": "#/$defs/view" + } + }, + "when": { + "type": "string" + } + } + }, + "mappingRow": { + "type": "object", + "required": [ + "target" + ], + "additionalProperties": false, + "properties": { + "source": { + "type": "string" + }, + "target": { + "type": "string" + }, + "transform": { + "enum": [ + "none", + "trim", + "lowercase", + "uppercase", + "number", + "boolean", + "to_iso_date", + "to_e164_phone", + "split_full_name", + "join", + "map_values", + "default", + "units_to_cents", + "cents_to_units", + "from_unix_seconds", + "to_unix_seconds" + ] + }, + "options": { + "type": "object" + }, + "master": { + "enum": [ + "source", + "target", + "newest" + ], + "default": "source" + }, + "direction": { + "enum": [ + "both", + "to_target", + "to_source" + ], + "default": "both" + }, + "value": { + "description": "A fixed value written to target instead of reading source (QuickBooks Line[0].DetailType)." + } + }, + "oneOf": [ + { + "required": [ + "source" + ], + "not": { + "required": [ + "value" + ] + } + }, + { + "required": [ + "value" + ], + "not": { + "required": [ + "source" + ] + } + } + ] + } + } +} diff --git a/packages/core/integrations/data-defined/schema/records.md b/packages/core/integrations/data-defined/schema/records.md new file mode 100644 index 000000000..0ec9803df --- /dev/null +++ b/packages/core/integrations/data-defined/schema/records.md @@ -0,0 +1,98 @@ +# Data-defined integrations: database records (v0.1) + +Three layers, stored separately. Worked examples for Phone ↔ HubSpot are in `examples/records/`. + +| Layer | What it holds | Table | Changes when | +|---|---|---|---| +| Definition | What the integration can do | `IntegrationDefinitionVersion` (new) | A version is published (then never again) | +| Installation | What one customer turned on and chose | `Integration.config` (existing column, new shape) | The customer configures, or an upgrade moves the pin | +| State | What has happened | `IntegrationMapping` (existing), `WorkflowRun` (new) | Every event | + +## IntegrationDefinitionVersion (new) + +One row per published version. `document` is the definition verbatim; `checksum` is sha256 of its canonical JSON +(sorted keys), so the same content always has the same checksum and an installation can prove what it runs. + +```prisma +model IntegrationDefinitionVersion { + id String @id @default(cuid()) + definitionId String // "phone-hubspot" + version String // "1.4.0" (semver) + schemaVersion String // "0.1" + status DefinitionStatus @default(DRAFT) + checksum String // "sha256:..." of canonical(document) + document Json // the definition, immutable once PUBLISHED + changeLevel String? // patch | minor | major, computed by versioning.diff at publish + previousVersion String? + notes String? + createdBy String? + createdAt DateTime @default(now()) + publishedAt DateTime? + + @@unique([definitionId, version]) + @@index([definitionId, status]) +} + +enum DefinitionStatus { DRAFT PUBLISHED DEPRECATED RETIRED } +``` + +Rules: a `DRAFT` may be edited; publishing runs `validate` (must be valid) and `checkBump` against the latest +published version (the declared bump must cover the computed change level), then freezes the row. `DEPRECATED` +versions keep running for installations pinned to them but cannot be newly installed. A version is `RETIRED` only +when no installation pins it and no queued message references it. + +## Integration.config (existing column) + +```jsonc +{ + "definition": { "id": "phone-hubspot", "version": "1.4.0", "checksum": "sha256:…" }, // the pin + "settings": { "fromNumber": "+15125550100" }, // integration-level settings + "syncs": { "contacts": { "enabled": true, "mapping": [ /* optional full override */ ] } }, + "workflows": { "missed-call-task": { "enabled": true, "settings": { "owner": "81273645" } } }, + "ui": { "phone-card": { "enabled": true } }, + "runtime": { "installed": { "hook": { "id": "WHk2c9", "url": "…", "events": [ … ] } } } // runtime-owned: onInstall results by step id +} +``` + +An absent `enabled` means "use the definition's default". The existing `Integration.version` column mirrors +`config.definition.version` so it can be queried and indexed. + +## IntegrationMapping (existing) — three kinds of row + +| `sourceId` | `mapping` | Written by | +|---|---|---| +| `link:::` | `{ "": id, …, linkedAt }` | `links.upsert` (both sides get a row) | +| `idx:::` | same as link | `links.upsert` with `keys` (email lowercased, phone E.164) | +| `evt:::` | `{ at }` | `runWorkflow`, insert-only (`createMapping`) so a redelivery is a no-op | + +No new table: these are the rows hand-written integrations already keep, under a fixed naming scheme. + +## WorkflowRun (new) + +One row per workflow execution. Carries the definition version it ran on, so a run is always explainable against the +exact document, even after an upgrade. + +```prisma +model WorkflowRun { + id String @id @default(cuid()) + integrationId String + definitionId String + definitionVersion String + workflowId String + triggerType String // webhook | schedule | userAction | syncRecord + eventId String? // provider event id (dedupe key) + status String // completed | stopped | filtered | duplicate | failed + reason String? // why it stopped + error String? + attempt Int @default(1) + trace Json // [{ id, use, status, ms, error? }] — no payload bodies + startedAt DateTime + endedAt DateTime? + + @@index([integrationId, workflowId, startedAt]) + @@index([integrationId, status]) +} +``` + +Retention: keep `failed` runs 30 days and the rest 7 by default. `trace` holds step ids, timings and error messages, +never request or response bodies (those can hold customer data; the existing logging rules apply). diff --git a/packages/core/integrations/data-defined/scripts/validate.js b/packages/core/integrations/data-defined/scripts/validate.js new file mode 100755 index 000000000..cafb634ac --- /dev/null +++ b/packages/core/integrations/data-defined/scripts/validate.js @@ -0,0 +1,66 @@ +#!/usr/bin/env node +/** + * Validate data-defined integration definitions and capability manifests: JSON Schema, then references. + * + * node scripts/validate.js every bundled manifest and example + * node scripts/validate.js my.definition.json one or more definitions (files or directories) + * node scripts/validate.js --manifests dir/ ... check against manifests in another directory + * node scripts/validate.js --strict ... warnings fail too + * node scripts/validate.js --json ... machine-readable result on stdout + * + * Exit codes: 0 valid · 1 invalid · 2 could not run (bad arguments, unreadable file, ajv missing). + * Manifests are always checked first, because a definition is only as trustworthy as the manifests it is checked + * against: an invalid manifest fails the run even if every definition passes. + */ +const fs = require('fs'); +const path = require('path'); + +const root = path.join(__dirname, '..'); +const args = process.argv.slice(2); +const flag = (name) => { const i = args.indexOf(name); if (i < 0) return false; args.splice(i, 1); return true; }; +const option = (name) => { const i = args.indexOf(name); if (i < 0) return null; const v = args[i + 1]; args.splice(i, 2); return v; }; + +const strict = flag('--strict'); +const asJson = flag('--json'); +const manifestDir = option('--manifests') || path.join(root, 'capabilities'); +const unknown = args.find((a) => a.startsWith('--')); +if (unknown) { console.error(`Unknown option ${unknown}`); process.exit(2); } + +const jsonFiles = (p) => { + if (!fs.existsSync(p)) { console.error(`No such file or directory: ${p}`); process.exit(2); } + return fs.statSync(p).isDirectory() ? fs.readdirSync(p).filter((f) => f.endsWith('.json')).map((f) => path.join(p, f)) : [p]; +}; +const readJson = (f) => { try { return JSON.parse(fs.readFileSync(f, 'utf8')); } catch (e) { console.error(`${f}: not valid JSON (${e.message})`); process.exit(2); } }; + +let check; +try { check = require('../schema-check'); check.loadAjv(); } catch (e) { console.error(e.message); process.exit(2); } + +const manifestFiles = jsonFiles(manifestDir).filter((f) => f.endsWith('.capabilities.json')); +const definitionFiles = (args.length ? args : [path.join(root, 'examples')]).flatMap(jsonFiles).filter((f) => !f.endsWith('.capabilities.json') && !f.includes(`${path.sep}records${path.sep}`)); + +const manifests = {}; +const results = []; +for (const f of manifestFiles) { + const m = readJson(f); + manifests[m.module] = m; + results.push({ kind: 'manifest', file: path.relative(process.cwd(), f), name: m.module, ...check.checkManifest(m) }); +} +for (const f of definitionFiles) { + const def = readJson(f); + results.push({ kind: 'definition', file: path.relative(process.cwd(), f), name: `${def.id}@${def.version}`, ...check.checkDefinition(def, manifests) }); +} + +const failed = results.filter((r) => !r.valid || (strict && r.warnings.length)); +if (asJson) { + process.stdout.write(JSON.stringify({ valid: failed.length === 0, strict, results }, null, 2) + '\n'); +} else { + for (const r of results) { + const bad = !r.valid || (strict && r.warnings.length); + console.log(`${bad ? 'FAIL' : 'ok '} ${r.kind.padEnd(10)} ${r.name}${r.warnings.length ? ` (${r.warnings.length} warning${r.warnings.length > 1 ? 's' : ''})` : ''}`); + for (const e of r.errors) console.log(` error [${e.layer}] ${e.path}: ${e.message}`); + for (const w of r.warnings) console.log(` warning [${w.layer}] ${w.path}: ${w.message}`); + } + const n = (k) => results.filter((r) => r.kind === k).length; + console.log(`\n${n('manifest')} manifests, ${n('definition')} definitions: ${failed.length ? `${failed.length} failed` : 'all valid'}${strict ? ' (strict)' : ''}`); +} +process.exit(failed.length ? 1 : 0); diff --git a/packages/core/integrations/data-defined/steps.js b/packages/core/integrations/data-defined/steps.js new file mode 100644 index 000000000..3d9c21cb2 --- /dev/null +++ b/packages/core/integrations/data-defined/steps.js @@ -0,0 +1,89 @@ +/** + * The v0.1 step catalogue. A step is { id?, use, with?, when?, onError? }. + * + * `use` is either a built-in below or "..": a capability the module's manifest declares + * (e.g. "hubspot.task.create" -> HubSpot Api.createTask). Nothing else can run: there is no http, eval or script step. + * The only way to run custom code is `code`, which calls a named, versioned function registered with the runtime. + */ +const { applyMapping, deepMerge } = require('./transforms'); +const { resolve } = require('./expressions'); + +const BUILTINS = { + 'links.find': { + summary: 'Find records already linked, by a sync or a link set (definition.links). By one side\'s id, or by an identity key it indexes (email, phone).', + args: { link: 'link set id (or sync: sync id)', from: 'module key, with id', id: 'that module\'s record id', key: '{ : value }' }, + run: (a, { ports }) => ports.links.find(a), + }, + 'links.upsert': { + summary: 'Record that records on two sides are the same thing, and index their identity keys.', + args: { link: 'link set id (or sync: sync id)', ids: '{ : id }', keys: '{ : value }' }, + run: (a, { ports }) => ports.links.upsert(a), + }, + map: { + summary: 'Apply a field mapping: a sync\'s mapping (with the installation\'s overrides) or an inline one.', + args: { sync: 'sync id', mapping: 'inline rows', record: 'the source record' }, + run: (a, { def, installation }) => { + let rows = a.mapping; + if (!rows && a.sync) { + const sync = (def.syncs || []).find((s) => s.id === a.sync); + if (!sync) throw new Error(`map: unknown sync "${a.sync}"`); + rows = installation?.syncs?.[a.sync]?.mapping || sync.mapping; + } + return applyMapping(rows, a.record || {}); + }, + }, + template: { + summary: 'Render a named template (definition.templates) or inline text.', + args: { id: 'template id', text: 'inline template' }, + run: (a, { def, ctx }) => { + const src = a.id ? def.templates?.[a.id] : a.text; + if (src == null) throw new Error(`template: unknown template "${a.id}"`); + return resolve(src, ctx); + }, + }, + merge: { + summary: 'Deep-merge objects into one; later objects win, lists are replaced.', + args: { objects: '[object, ...]' }, + run: (a) => deepMerge(...(Array.isArray(a.objects) ? a.objects : [])), + }, + set: { + summary: 'Store values in $.vars for later steps.', + args: { '': 'value' }, + run: (a, { ctx }) => { Object.assign(ctx.vars, a); return a; }, + }, + stop: { + summary: 'End the run early (a normal outcome, not an error). Usually guarded by `when`.', + args: { reason: 'string' }, + run: (a) => ({ stop: true, reason: a.reason || 'stopped' }), + }, + foreach: { + summary: 'Run nested steps once per item ($.item). Bounded by max (default 100, hard cap 1000). With `collect`, the step\'s result is the list of that value per item; `where` skips items.', + args: { over: 'array', max: 'number', where: 'condition on $.item', collect: 'value per item (may use $.item and this iteration\'s $.steps)', do: 'steps (optional when collect is set)' }, + nested: true, + }, + branch: { + summary: 'Run the first case whose `when` holds, else the `else` steps.', + args: { cases: '[{ when, do }]', else: 'steps' }, + nested: true, + }, + code: { + summary: 'Call a registered code step: a named, versioned function shipped in reviewed code ("pkg#fn@1.2.0").', + args: { ref: 'string', with: 'object' }, + run: (a, { ports }) => { + if (!ports.runCode) throw new Error('code: this runtime has no code steps registered'); + return ports.runCode(a.ref, a.with || {}); + }, + }, +}; + +const FOREACH_HARD_CAP = 1000; + +function isBuiltin(use) { return Object.prototype.hasOwnProperty.call(BUILTINS, use); } + +/** "hubspot.task.create" -> { module: 'hubspot', capability: 'task.create' } */ +function splitCapability(use) { + const i = use.indexOf('.'); + return i < 0 ? null : { module: use.slice(0, i), capability: use.slice(i + 1) }; +} + +module.exports = { BUILTINS, FOREACH_HARD_CAP, isBuiltin, splitCapability }; diff --git a/packages/core/integrations/data-defined/tests/data-defined.test.js b/packages/core/integrations/data-defined/tests/data-defined.test.js new file mode 100644 index 000000000..cb476cc83 --- /dev/null +++ b/packages/core/integrations/data-defined/tests/data-defined.test.js @@ -0,0 +1,226 @@ +const d = require('..'); +const { checksum } = require('../canonical'); + +const manifests = d.loadManifests(); +const base = () => JSON.parse(JSON.stringify(require('../examples/phone-hubspot.definition.json'))); +const phoneCall = () => JSON.parse(JSON.stringify(manifests.phone.events['call.completed'].payload)); + +function harness(seed = {}) { + const store = new Map(Object.entries(seed)); + const calls = []; + const ports = (def, respond = () => ({ id: 'out_1' })) => d.makePorts({ + def, manifests, + store: { get: async (k) => store.get(k) || null, put: async (k, v) => { store.set(k, v); }, claim: async (k, v) => { if (store.has(k)) return false; store.set(k, v); return true; }, release: async (k) => { store.delete(k); } }, + callApi: async (module, method, args) => { calls.push({ module, method, args }); return respond(module, method, args); }, + runCode: async (ref, args) => ({ participants: ['+15125550142'], ref, args }), + }); + return { store, calls, ports }; +} +const linked = { 'idx:contacts:phone:+15125550142': { hubspot: '51', phone: 'CT7' } }; + +describe('expressions', () => { + it('resolves references with their type, templates as strings, and conditions without eval', () => { + const ctx = { event: { n: 3, tags: ['vip'] }, steps: { c: null } }; + expect(d.execute).toBeDefined(); + const { resolve, evaluate } = require('../expressions'); + expect(resolve({ a: '$.event.n', b: 'n={{event.n}}' }, ctx)).toEqual({ a: 3, b: 'n=3' }); + expect(evaluate("$.steps.c == null && $.event.n > 2 && $.event.tags contains 'vip'", ctx)).toBe(true); + expect(() => evaluate('process.exit()', ctx)).toThrow(/Cannot parse/); + }); +}); + +describe('validate', () => { + it('accepts every bundled example against the bundled manifests', () => { + for (const def of d.loadExamples()) { + const r = d.validate(def, manifests); + expect({ id: def.id, errors: r.errors }).toEqual({ id: def.id, errors: [] }); + } + }); + + // Each mutation breaks exactly one rule, so each rule is shown to be enforced on its own. + const cases = [ + ['unknown capability', (x) => { x.workflows[0].steps[3].use = 'hubspot.call.teleport'; }, /no capability "call.teleport"/], + ['missing required argument', (x) => { delete x.workflows[0].steps[3].with.contactId; }, /contactId.*Required/s], + ['undeclared argument', (x) => { x.workflows[0].steps[3].with.color = 'red'; }, /takes no argument "color"/], + ['step referenced before it runs', (x) => { x.workflows[0].steps[0].with.key.phone = '$.steps.engagement.id'; }, /not an earlier step/], + ['undeclared workflow setting', (x) => { x.workflows[2].steps[3].with.properties.hs_task_body = '{{settings.nope}}'; }, /\$\.settings\.nope/], + ['unknown trigger topic', (x) => { x.workflows[0].trigger.topic = 'call.teleported'; }, /emits no event "call.teleported"/], + ['identity key not in the sync', (x) => { x.workflows[0].steps[0].with.key = { fax: '$.record.from' }; }, /"fax" is not an identity key/], + ['mapping field the object lacks', (x) => { x.syncs[0].mapping[0].target = 'defaultFields.nickname'; }, /not a field of phone\.contact/], + ['read-only target field', (x) => { x.syncs[0].mapping.push({ source: 'properties.email', target: 'id' }); }, /read-only/], + ['placement the host lacks', (x) => { x.ui[0].placement = 'crm.navbar'; }, /not a placement hubspot supports/], + ['button pointing at a non-user-action workflow', (x) => { x.ui[0].view.children[4].props.action.workflow = 'log-call'; }, /not triggered by a user action/], + ['bad condition', (x) => { x.workflows[2].trigger.filter = "$.record.direction = 'incoming'"; }, /Cannot parse/], + ['$.item outside foreach', (x) => { x.workflows[0].steps[3].with.properties.hs_call_title = '$.item.id'; }, /only available inside foreach/], + ['duplicate ids across kinds', (x) => { x.queries[0].id = 'log-call'; }, /Duplicate id "log-call"/], + ['unknown template', (x) => { x.workflows[0].steps[2].with.id = 'nope'; }, /Unknown template "nope"/], + ]; + it.each(cases)('rejects: %s', (_, mutate, pattern) => { + const x = base(); mutate(x); + const r = d.validate(x, manifests); + expect(r.valid).toBe(false); + expect(r.errors.map((e) => `${e.path}: ${e.message}`).join("\n")).toMatch(pattern); + }); +}); + +describe('compile', () => { + it('compiles only what the installation turned on, into ordinary Frigg events', () => { + const def = base(); + const c = d.compile(def, { workflows: { 'missed-call-task': { enabled: true }, 'texts-timeline': { enabled: false } } }); + expect(c.events.WF_LOG_CALL).toMatchObject({ type: 'WEBHOOK' }); + expect(c.events.WF_MISSED_CALL_TASK).toBeDefined(); // default off, turned on + expect(c.events.WF_TEXTS_TIMELINE).toBeUndefined(); // default on, turned off + expect(c.events.WF_WON_THANK_YOU).toBeUndefined(); // default off, left off + expect(c.events.SYNC_CONTACTS_BACKFILL).toMatchObject({ type: 'USER_ACTION', userActionType: 'SYNC_ACTION' }); + expect(c.events.WF_TEXT_FROM_CARD).toMatchObject({ type: 'USER_ACTION' }); + expect(c.schedules).toEqual([]); // stale-contacts is off + expect(d.routeWebhook(c, 'phone', 'call.completed')).toEqual(['WF_LOG_CALL']); + expect(d.routeWebhook(c, 'hubspot', 'contact.propertyChange')).toEqual(['SYNC_CONTACTS_ONGOING']); + expect(c.routes.map((r) => r.path)).toEqual(['/ui/phone-card', '/ui/deals-panel']); + }); +}); + +describe('runWorkflow', () => { + it('logs a Phone call on the linked HubSpot contact, through the manifest method, with resolved arguments', async () => { + const def = base(); const h = harness(linked); + const run = await d.runWorkflow({ def, installation: {}, workflowId: 'log-call', event: phoneCall(), ports: h.ports(def), manifests }); + expect(run.status).toBe('completed'); + expect(h.calls).toEqual([{ module: 'hubspot', method: 'createCallEngagement', args: { + contactId: '51', + properties: { hs_call_title: 'Phone call (incoming)', hs_call_body: 'Phone call with +15125550142 · 312s (completed)', hs_call_duration: 312, hs_call_direction: 'INCOMING', hs_timestamp: '2026-10-18T14:07:12Z' }, + } }]); + }); + + it('runs each provider event once: a redelivery writes nothing', async () => { + const def = base(); const h = harness(linked); const p = h.ports(def); + await d.runWorkflow({ def, installation: {}, workflowId: 'log-call', event: phoneCall(), ports: p, manifests }); + const again = await d.runWorkflow({ def, installation: {}, workflowId: 'log-call', event: phoneCall(), ports: p, manifests }); + expect(again.status).toBe('duplicate'); + expect(h.calls).toHaveLength(1); + }); + + it('a failed run gives its claim back, so the retry runs the steps instead of reading as a duplicate', async () => { + const def = base(); const h = harness(linked); + let fail = true; + const p = h.ports(def, () => { if (fail) throw new Error('HubSpot 503'); return { id: 'out_1' }; }); + const first = await d.runWorkflow({ def, installation: {}, workflowId: 'log-call', event: phoneCall(), ports: p, manifests }); + expect(first).toMatchObject({ status: 'failed', retryable: true }); + fail = false; + const retry = await d.runWorkflow({ def, installation: {}, workflowId: 'log-call', event: phoneCall(), ports: p, manifests }); + expect(retry.status).toBe('completed'); + expect(h.calls).toHaveLength(2); + }); + + it('stops (not fails) when there is no linked contact, and writes nothing', async () => { + const def = base(); const h = harness(); + const run = await d.runWorkflow({ def, installation: {}, workflowId: 'log-call', event: phoneCall(), ports: h.ports(def), manifests }); + expect(run).toMatchObject({ status: 'stopped', reason: 'No linked HubSpot contact for this number' }); + expect(h.calls).toEqual([]); + }); + + it('applies the trigger filter before claiming the event', async () => { + const def = base(); const h = harness(linked); + const ev = JSON.parse(JSON.stringify(manifests.phone.events['call.missed'].payload)); ev.data.object.direction = 'outgoing'; + const run = await d.runWorkflow({ def, installation: {}, workflowId: 'missed-call-task', event: ev, ports: h.ports(def), manifests }); + expect(run.status).toBe('filtered'); + expect([...h.store.keys()].some((k) => k.startsWith('evt:'))).toBe(false); + }); + + it('uses installation settings over defaults (missed-call owner)', async () => { + const def = base(); const h = harness(linked); + const respond = (m, method) => (method === 'getContactById' ? { id: '51', properties: { firstname: 'Dana', hubspot_owner_id: '999' } } : { id: 't1' }); + const run = await d.runWorkflow({ def, installation: { workflows: { 'missed-call-task': { enabled: true, settings: { owner: '81273645' } } } }, workflowId: 'missed-call-task', event: manifests.phone.events['call.missed'].payload, ports: h.ports(def, respond), manifests }); + expect(run.status).toBe('completed'); + const task = h.calls.find((c) => c.method === 'createTask'); + expect(task.args.properties).toMatchObject({ hs_task_subject: 'Call back Dana (missed Phone call)', hubspot_owner_id: '81273645' }); + }); + + it('reports a failed step as a retryable failed run with the trace up to the failure', async () => { + const def = base(); const h = harness(linked); + const run = await d.runWorkflow({ def, installation: {}, workflowId: 'log-call', event: phoneCall(), ports: h.ports(def, () => { throw new Error('HubSpot 503'); }), manifests }); + expect(run).toMatchObject({ status: 'failed', error: 'HubSpot 503', retryable: true }); + expect(run.trace.map((t) => t.status)).toEqual(['ok', 'skipped', 'ok', 'error']); + }); + + it('runs foreach with $.item and caps it', async () => { + const def = base(); const h = harness(); + const respond = (m, method) => (method === 'searchContacts' ? [{ id: 'a', properties: { notes_last_contacted: 'x' } }, { id: 'b', properties: {} }] : {}); + const run = await d.runWorkflow({ def, installation: {}, workflowId: 'stale-contacts', event: {}, ports: h.ports(def, respond), manifests }); + expect(run.status).toBe('completed'); + expect(h.calls.filter((c) => c.method === 'updateContact').map((c) => c.args.id)).toEqual(['a', 'b']); + }); +}); + +describe('renderSurface', () => { + it('runs the query and resolves the view into a static tree for the host renderer', async () => { + const def = base(); const h = harness(); + const respond = (m, method) => (method === 'listCalls' ? [{ direction: 'incoming', duration: 60, createdAt: '2026-10-18T14:00:00Z' }] : []); + const out = await d.renderSurface({ def, installation: {}, surfaceId: 'phone-card', record: { properties: { phone: '+15125550142' } }, ports: h.ports(def, respond) }); + expect(out.surface).toEqual({ id: 'phone-card', host: 'hubspot', placement: 'crm.record.sidebar', definition: 'phone-hubspot@1.4.0' }); + const list = out.view.children.find((c) => c.component === 'List'); + expect(list.props.items).toEqual([{ title: 'incoming call · 60s', meta: '2026-10-18' }]); + expect(out.view.children.some((c) => c.component === 'Empty')).toBe(false); + }); +}); + +describe('versioning', () => { + it('classifies changes and refuses a bump smaller than the change', () => { + const a = base(); const b = base(); b.version = '1.4.1'; + b.workflows[0].steps[3].with.properties.hs_call_title = 'Call'; + expect(d.checkBump(a, b)).toMatchObject({ ok: true, required: 'patch' }); + b.syncs[0].identity.match.pop(); + expect(d.checkBump(a, b)).toMatchObject({ ok: false, required: 'major', declared: 'patch' }); + }); + + it('plans an upgrade: required input first, then what the runtime does', () => { + const a = base(); const b = base(); b.version = '2.0.0'; + b.workflows = b.workflows.filter((w) => w.id !== 'texts-timeline'); + b.workflows.push({ id: 'voicemail', label: 'Voicemail notes', default: 'on', settings: { pipeline: { type: 'string', label: 'Pipeline', required: true } }, trigger: { type: 'webhook', module: 'phone', topic: 'call.missed' }, steps: [{ use: 'stop' }] }); + const plan = d.planUpgrade({ definition: { id: a.id, version: a.version }, settings: { fromNumber: '+1' }, workflows: { 'texts-timeline': { enabled: true } } }, a, b); + expect(plan.allowed).toBe(false); + expect(plan.needs).toEqual([expect.objectContaining({ path: 'workflows.voicemail.settings.pipeline' })]); + expect(plan.actions.map((x) => x.kind)).toEqual(['disable', 'enable', 'drain']); + expect(plan.config.definition.version).toBe('2.0.0'); + expect(plan.config.workflows['texts-timeline']).toBeUndefined(); + }); + + it('stores versions under a canonical checksum that ignores key order', () => { + expect(checksum({ a: 1, b: [1, { c: 2, d: 3 }] })).toBe(checksum({ b: [1, { d: 3, c: 2 }], a: 1 })); + }); +}); + +describe('defineIntegrationClass', () => { + class FakeBase { + constructor(p = {}) { this.events = {}; this.config = p.config; this.id = p.id; this.rows = new Map(); this.hubspot = { api: { createCallEngagement: jest.fn(async () => ({ id: 'e1' })) } }; this.phone = { api: {} }; } + registerEventHandlers() { this.on = { ...this.events }; } + async getMapping(k) { return this.rows.has(k) ? { mapping: this.rows.get(k) } : null; } + async upsertMapping(k, v) { this.rows.set(k, v); } + async createMapping(k, v) { if (this.rows.has(k)) { const e = new Error('exists'); e.name = 'MappingAlreadyExistsError'; throw e; } this.rows.set(k, v); } + } + + it('loads the pinned version, builds this.events, and a dispatched webhook event writes to HubSpot', async () => { + const v14 = base(); const v13 = { ...base(), version: '1.3.2' }; + const Cls = d.defineIntegrationClass({ id: 'phone-hubspot', definitions: d.memoryDefinitions([v13, v14]), manifests, modules: { phone: {}, hubspot: {} }, IntegrationBase: FakeBase, runs: false }); + expect(Cls.Definition.name).toBe('phone-hubspot'); + const inst = new Cls({ id: 'int_1', config: { definition: { id: 'phone-hubspot', version: '1.3.2' } } }); + await inst.loadDefinition(); + expect(inst.def.version).toBe('1.3.2'); // the pin, not the latest + expect(Object.keys(inst.on)).toContain('WF_LOG_CALL'); + inst.rows.set('idx:contacts:phone:+15125550142', { hubspot: '51', phone: 'CT7' }); + const run = await inst.on.WF_LOG_CALL.handler({ data: { body: phoneCall(), definitionVersion: '1.3.2' } }); + expect(run.status).toBe('completed'); + expect(inst.hubspot.api.createCallEngagement).toHaveBeenCalledWith(expect.objectContaining({ contactId: '51' })); + }); + + it('shared class: one registration runs installations of different definitions', async () => { + const qh = base(); const other = { ...base(), id: 'phone-other', version: '0.1.0' }; + const Cls = d.defineSharedIntegrationClass({ definitions: d.memoryDefinitions([qh, other]), manifests, modules: { phone: {}, hubspot: {} }, IntegrationBase: FakeBase, runs: false }); + expect(Cls.Definition.name).toBe('data-defined'); + expect(Object.keys(Cls.Definition.modules)).toEqual(['phone', 'hubspot']); + const a = new Cls({ id: 'int_a', config: { definition: { id: 'phone-hubspot', version: '1.4.0' } } }); + const b = new Cls({ id: 'int_b', config: { definition: { id: 'phone-other', version: '0.1.0' } } }); + await a.loadDefinition(); await b.loadDefinition(); + expect([a.def.id, b.def.id]).toEqual(['phone-hubspot', 'phone-other']); + await expect(new Cls({ id: 'int_c', config: {} }).loadDefinition()).rejects.toThrow(/does not name a definition/); + }); +}); diff --git a/packages/core/integrations/data-defined/tests/persistence.test.js b/packages/core/integrations/data-defined/tests/persistence.test.js new file mode 100644 index 000000000..afa390a54 --- /dev/null +++ b/packages/core/integrations/data-defined/tests/persistence.test.js @@ -0,0 +1,140 @@ +/** + * Publishing, the catalogue and run history against real databases, through the real repositories and use cases. + * + * Skipped unless a database URL is set (the same convention as core's query-parity tests): + * - DATA_DEFINED_POSTGRES_URL a PostgreSQL database migrated with prisma-postgresql/migrations + * - DATA_DEFINED_MONGO_URL a MongoDB replica set with prisma-mongodb pushed + */ +const d = require('..'); +const { DefinitionVersionRepositoryPostgres, DefinitionVersionRepositoryMongo } = require('../repositories/definition-version-repository'); +const { WorkflowRunRepositoryPostgres, WorkflowRunRepositoryMongo } = require('../repositories/workflow-run-repository'); + +const manifests = d.loadManifests(); +const example = (id) => JSON.parse(JSON.stringify(d.loadExamples().find((x) => x.id === id))); + +const LEGS = [ + ['postgresql', process.env.DATA_DEFINED_POSTGRES_URL, () => require('../../../generated/prisma-postgresql'), DefinitionVersionRepositoryPostgres, WorkflowRunRepositoryPostgres], + ['mongodb', process.env.DATA_DEFINED_MONGO_URL, () => require('../../../generated/prisma-mongodb'), DefinitionVersionRepositoryMongo, WorkflowRunRepositoryMongo], +].filter(([, url]) => url); + +const describeLegs = LEGS.length ? describe.each(LEGS) : describe.skip.each([['no database URL set', null]]); + +describeLegs('data-defined persistence on %s', (_, url, loadClient, DefRepo, RunRepo) => { + let client, repo, runs, integrationId; + const publish = (document, opts = {}) => new d.PublishDefinitionVersion({ repository: repo, manifests }).execute({ document, ...opts }); + + beforeAll(async () => { + const { PrismaClient } = loadClient(); + client = new PrismaClient({ datasourceUrl: url }); + await client.workflowRun.deleteMany({}); + await client.integrationDefinitionVersion.deleteMany({}); + repo = new DefRepo({ client }); + runs = new RunRepo({ client }); + integrationId = String((await client.integration.create({ data: { config: { type: 'data-defined' } } })).id); + }); + afterAll(async () => { await client?.$disconnect(); }); + + it('publishes a first version: PUBLISHED, checksummed, served by the catalogue and named latest', async () => { + const out = await publish(example('phone-hubspot'), { notes: 'first', createdBy: 'test' }); + expect(out).toMatchObject({ created: true, changeLevel: null }); + expect(out.version).toMatchObject({ definitionId: 'phone-hubspot', version: '1.4.0', status: 'PUBLISHED', previousVersion: null, notes: 'first' }); + expect(out.version.checksum).toBe(d.checksum ? d.checksum(example('phone-hubspot')) : require('../canonical').checksum(example('phone-hubspot'))); + const catalog = d.createDefinitionCatalog({ repository: repo }); + expect((await catalog.get('phone-hubspot', '1.4.0')).id).toBe('phone-hubspot'); + expect((await catalog.latest('phone-hubspot')).version).toBe('1.4.0'); + }); + + it('re-publishing the same document is a no-op; changing a published version is refused', async () => { + expect((await publish(example('phone-hubspot'))).created).toBe(false); + const changed = example('phone-hubspot'); changed.display.label = 'Changed'; + await expect(publish(changed)).rejects.toMatchObject({ status: 409, code: 'IMMUTABLE' }); + }); + + it('refuses a bump smaller than the change, accepts the right one, and records its level and predecessor', async () => { + const v = example('phone-hubspot'); v.version = '1.5.0'; v.syncs[0].identity.match[0].transform = 'trim'; // identity change: major + await expect(publish(v)).rejects.toMatchObject({ status: 409, code: 'BUMP_TOO_SMALL', details: { required: 'major' } }); + v.version = '2.0.0'; + const out = await publish(v); + expect(out.version).toMatchObject({ version: '2.0.0', changeLevel: 'major', previousVersion: '1.4.0' }); + const older = example('phone-hubspot'); older.version = '1.9.0'; + await expect(publish(older)).rejects.toMatchObject({ code: 'NOT_NEWER' }); + }); + + it('refuses an invalid document, reporting each layer', async () => { + const bad = example('phone-hubspot'); bad.version = '3.0.0'; bad.syncs[0].mapping[0].transform = 'shout'; + await expect(publish(bad)).rejects.toMatchObject({ code: 'INVALID', details: { errors: [expect.objectContaining({ layer: 'schema' }), expect.objectContaining({ layer: 'references' })] } }); + expect(await repo.findVersion('phone-hubspot', '3.0.0')).toBeNull(); + }); + + it('a draft is never served, can be rewritten, then published', async () => { + const draft = example('pipedrive-hubspot'); + expect((await publish(draft, { publish: false })).version.status).toBe('DRAFT'); + const catalog = d.createDefinitionCatalog({ repository: repo }); + expect(await catalog.get('pipedrive-hubspot', '1.0.0')).toBeNull(); + expect(await catalog.latest('pipedrive-hubspot')).toBeNull(); + draft.display.label = 'Pipedrive and HubSpot people'; + expect((await publish(draft, { publish: false })).version.document.display.label).toBe('Pipedrive and HubSpot people'); + expect((await publish(draft)).version.status).toBe('PUBLISHED'); + expect((await catalog.latest('pipedrive-hubspot')).display.label).toBe('Pipedrive and HubSpot people'); + }); + + it('two identical publishes racing: one creates it, the other gets the same row back', async () => { + const doc = example('hubspot-slack'); + const [a, b] = await Promise.all([publish(doc), publish(doc)]); + expect([a.created, b.created].sort()).toEqual([false, true]); + expect(a.version.id).toBe(b.version.id); + }); + + it('deprecating keeps the version running for its pins but takes it out of "latest"; only once', async () => { + const deprecate = new d.DeprecateDefinitionVersion({ repository: repo }); + expect((await deprecate.execute({ definitionId: 'phone-hubspot', version: '2.0.0' })).status).toBe('DEPRECATED'); + await expect(deprecate.execute({ definitionId: 'phone-hubspot', version: '2.0.0' })).rejects.toMatchObject({ code: 'WRONG_STATUS' }); + await expect(deprecate.execute({ definitionId: 'phone-hubspot', version: '9.9.9' })).rejects.toMatchObject({ code: 'NOT_FOUND' }); + const catalog = d.createDefinitionCatalog({ repository: repo }); + expect((await catalog.get('phone-hubspot', '2.0.0')).version).toBe('2.0.0'); + expect((await catalog.latest('phone-hubspot')).version).toBe('1.4.0'); + }); + + it('the status precondition is enforced by the database: a published row cannot be rewritten as if it were a draft', async () => { + expect(await repo.updateIfStatus('phone-hubspot', '1.4.0', 'DRAFT', { notes: 'rewritten' })).toBeNull(); + expect((await repo.findVersion('phone-hubspot', '1.4.0')).notes).toBe('first'); + }); + + it('verifies a pinned checksum', async () => { + const catalog = d.createDefinitionCatalog({ repository: repo }); + await expect(catalog.get('phone-hubspot', '1.4.0', { checksum: 'sha256:not-it' })).rejects.toMatchObject({ code: 'CHECKSUM_MISMATCH' }); + const sum = await catalog.checksumOf('phone-hubspot', '1.4.0'); + expect((await catalog.get('phone-hubspot', '1.4.0', { checksum: sum })).version).toBe('1.4.0'); + }); + + it('lists the latest published version of each definition, and the full history newest first', async () => { + const list = await d.createDefinitionCatalog({ repository: repo }).list(); + expect(list.map((x) => [x.id, x.version]).sort()).toEqual([['hubspot-slack', '1.2.0'], ['phone-hubspot', '1.4.0'], ['pipedrive-hubspot', '1.0.0']]); + const history = await new d.GetDefinitionHistory({ repository: repo }).execute({ definitionId: 'phone-hubspot' }); + expect(history.map((r) => [r.version, r.status])).toEqual([['2.0.0', 'DEPRECATED'], ['1.4.0', 'PUBLISHED']]); + expect(history[0].document).toBeUndefined(); + }); + + it('records workflow runs and lists them newest first, filtered and paged; old runs can be deleted', async () => { + const base = { integrationId, definitionId: 'phone-hubspot', definitionVersion: '1.4.0', triggerType: 'webhook', trace: [{ id: 'contact', use: 'links.find', at: 'steps[0]', status: 'ok', ms: 3 }] }; + await runs.createRun({ ...base, workflowId: 'log-call', status: 'completed', eventId: 'e1', startedAt: '2026-10-18T10:00:00Z', endedAt: '2026-10-18T10:00:01Z' }); + await runs.createRun({ ...base, workflowId: 'log-call', status: 'failed', error: 'HubSpot 503', retryable: true, startedAt: '2026-10-18T11:00:00Z' }); + await runs.createRun({ ...base, workflowId: 'missed-call-task', status: 'stopped', reason: 'Caller is not a known contact', startedAt: '2026-10-18T12:00:00Z' }); + const all = await runs.listRuns(integrationId); + expect(all.map((r) => r.status)).toEqual(['stopped', 'failed', 'completed']); + expect(all[0]).toMatchObject({ integrationId, workflowId: 'missed-call-task', reason: 'Caller is not a known contact' }); + expect(all[2].trace).toEqual(base.trace); + expect((await runs.listRuns(integrationId, { workflowId: 'log-call' })).length).toBe(2); + expect((await runs.listRuns(integrationId, { status: 'failed' }))[0]).toMatchObject({ error: 'HubSpot 503', retryable: true }); + expect((await runs.listRuns(integrationId, { before: '2026-10-18T12:00:00Z', limit: 1 })).map((r) => r.status)).toEqual(['failed']); + expect(await runs.deleteRunsBefore('2026-10-18T10:30:00Z')).toBe(1); + expect((await runs.listRuns(integrationId)).length).toBe(2); + }); + + it('deleting an installation deletes its run history', async () => { + const other = String((await client.integration.create({ data: { config: { type: 'data-defined' } } })).id); + await runs.createRun({ integrationId: other, definitionId: 'x', definitionVersion: '1.0.0', workflowId: 'w', triggerType: 'schedule', status: 'completed', startedAt: '2026-10-18T10:00:00Z' }); + await client.integration.delete({ where: { id: /^\d+$/.test(other) ? Number(other) : other } }); + expect(await runs.listRuns(other)).toEqual([]); + }); +}); diff --git a/packages/core/integrations/data-defined/tests/routes-and-upgrade.test.js b/packages/core/integrations/data-defined/tests/routes-and-upgrade.test.js new file mode 100644 index 000000000..7deead20f --- /dev/null +++ b/packages/core/integrations/data-defined/tests/routes-and-upgrade.test.js @@ -0,0 +1,177 @@ +/** + * The HTTP adapter (router.js) over a real express app, and the class's upgrade and run recording. Authentication + * and integration lookup are the router's injected dependencies; the use cases, catalogue logic and class are real. + */ +const express = require('express'); +const request = require('supertest'); +const d = require('..'); +const { setDataDefinedRoutes } = require('../router'); + +const manifests = d.loadManifests(); +const example = (id) => JSON.parse(JSON.stringify(d.loadExamples().find((x) => x.id === id))); + +/** An in-memory DefinitionVersion repository with the real repository's contract (incl. the status precondition). */ +function memoryRepository() { + const rows = new Map(); + const key = (id, v) => `${id}@${v}`; + let n = 0; + return { + rows, + async findVersion(id, v) { return rows.get(key(id, v)) || null; }, + async listVersions(id) { return [...rows.values()].filter((r) => r.definitionId === id); }, + async listByStatus(statuses) { return [...rows.values()].filter((r) => statuses.includes(r.status)).map(({ document, ...m }) => m); }, + async createVersion(data) { if (rows.has(key(data.definitionId, data.version))) { const e = new Error('unique'); e.code = 'P2002'; throw e; } const r = { id: String(++n), createdAt: new Date(), ...data }; rows.set(key(data.definitionId, data.version), r); return r; }, + async updateIfStatus(id, v, from, data) { const r = rows.get(key(id, v)); if (!r || r.status !== from) return null; Object.assign(r, data); return r; }, + }; +} + +class FakeBase { + constructor(p = {}) { this.events = {}; this.config = p.config; this.id = p.id; this.patched = []; this.rows = new Map(); } + async initialize() { this.registerEventHandlers(); } + registerEventHandlers() { this.on = { ...this.events }; } + async patchConfig(c) { this.patched.push(c); this.config = { ...this.config, ...c }; } + async onCreate() {} + async getMapping(k) { return this.rows.has(k) ? { mapping: this.rows.get(k) } : null; } + async upsertMapping(k, v) { this.rows.set(k, v); } + async createMapping(k, v) { if (this.rows.has(k)) { const e = new Error('exists'); e.name = 'MappingAlreadyExistsError'; throw e; } this.rows.set(k, v); } + async deleteMapping(k) { this.rows.delete(k); } +} + +async function setup({ adminKey = 'admin-secret' } = {}) { + process.env.ADMIN_API_KEY = adminKey; + const repository = memoryRepository(); + const catalog = d.createDefinitionCatalog({ repository }); + const runStore = { runs: [], async createRun(r) { this.runs.push(r); return r; }, async listRuns(id, q = {}) { return this.runs.filter((r) => r.integrationId === id && (!q.workflowId || r.workflowId === q.workflowId)); } }; + const Cls = d.defineSharedIntegrationClass({ definitions: catalog, manifests, modules: { phone: {}, hubspot: {} }, IntegrationBase: FakeBase, runs: runStore }); + const installations = new Map(); + const app = express(); + app.use(express.json()); + const users = { 'token-a': 'user-a', 'token-b': 'user-b' }; + const authenticateUser = { execute: async (req) => { const u = users[(req.headers.authorization || '').replace('Bearer ', '')]; if (!u) { const e = new Error('Unauthorized'); e.status = 401; throw e; } return { getId: () => u }; } }; + const getIntegrationInstance = { execute: async (id, userId) => { const i = installations.get(id); if (!i || i.userId !== userId) { const e = new Error('Not found'); e.status = 404; throw e; } await i.inst.initialize(); return i.inst; } }; + expect(setDataDefinedRoutes(app, authenticateUser, { getIntegrationInstance, integrationClasses: [Cls] })).toBe(true); + app.use((err, req, res, next) => res.status(err.status || 500).json({ error: err.message })); // eslint-disable-line no-unused-vars + const install = (id, userId, config, apis = {}) => { const inst = new Cls({ id, config: { type: 'data-defined', ...config } }); Object.assign(inst, apis); installations.set(id, { inst, userId }); return inst; }; + const publishAs = (document, key = adminKey) => request(app).post('/admin/integrations/definitions').set('x-frigg-admin-api-key', key).send({ document }); + return { app, repository, catalog, runStore, install, publishAs, Cls }; +} + +describe('admin routes: publishing', () => { + it('refuses without the admin key, publishes with it (201), and is idempotent on the same document (200)', async () => { + const { publishAs } = await setup(); + expect((await publishAs(example('phone-hubspot'), 'wrong')).status).toBe(401); + const first = await publishAs(example('phone-hubspot')); + expect(first.status).toBe(201); + expect(first.body.version).toMatchObject({ definitionId: 'phone-hubspot', version: '1.4.0', status: 'PUBLISHED' }); + expect(first.body.version.document).toBeUndefined(); + expect((await publishAs(example('phone-hubspot'))).status).toBe(200); + }); + + it('answers an invalid document with 400 and both layers\' errors; a too-small bump with 409 and the changes', async () => { + const { publishAs } = await setup(); + const bad = example('phone-hubspot'); bad.syncs[0].mapping[0].transform = 'shout'; + const r = await publishAs(bad); + expect(r.status).toBe(400); + expect(r.body.code).toBe('INVALID'); + expect(r.body.details.errors.map((e) => e.layer)).toEqual(['schema', 'references']); + await publishAs(example('phone-hubspot')); + const v = example('phone-hubspot'); v.version = '1.4.1'; v.workflows = v.workflows.filter((w) => w.id !== 'won-thank-you'); + const bump = await publishAs(v); + expect(bump.status).toBe(409); + expect(bump.body).toMatchObject({ code: 'BUMP_TOO_SMALL', details: { required: 'major', declared: 'patch' } }); + }); + + it('history and deprecation', async () => { + const { app, publishAs } = await setup(); + await publishAs(example('phone-hubspot')); + const h = await request(app).get('/admin/integrations/definitions/phone-hubspot').set('x-frigg-admin-api-key', 'admin-secret'); + expect(h.body.versions.map((x) => x.version)).toEqual(['1.4.0']); + const dep = await request(app).post('/admin/integrations/definitions/phone-hubspot/versions/1.4.0/deprecate').set('x-frigg-admin-api-key', 'admin-secret'); + expect(dep.body.version.status).toBe('DEPRECATED'); + expect((await request(app).post('/admin/integrations/definitions/phone-hubspot/versions/1.4.0/deprecate').set('x-frigg-admin-api-key', 'admin-secret')).status).toBe(409); + }); +}); + +describe('user routes: catalogue, upgrade, runs', () => { + it('lists what can be installed to a signed-in user only', async () => { + const { app, publishAs } = await setup(); + await publishAs(example('phone-hubspot')); + await publishAs(example('pipedrive-hubspot')); + expect((await request(app).get('/api/data-defined/definitions')).status).toBe(401); + const r = await request(app).get('/api/data-defined/definitions').set('authorization', 'Bearer token-a'); + expect(r.body.definitions.map((x) => x.id).sort()).toEqual(['phone-hubspot', 'pipedrive-hubspot']); + }); + + it('plans an upgrade without changing anything, then refuses to apply while a required setting is missing', async () => { + const { app, publishAs, install } = await setup(); + await publishAs(example('phone-hubspot')); + const v2 = example('phone-hubspot'); v2.version = '2.0.0'; v2.settings.region = { type: 'string', label: 'Region', required: true }; + expect((await publishAs(v2)).status).toBe(201); + const inst = install('int_1', 'user-a', { definition: { id: 'phone-hubspot', version: '1.4.0' }, settings: { fromNumber: '+15125550100' } }); + const plan = await request(app).get('/api/integrations/int_1/upgrade').set('authorization', 'Bearer token-a'); + expect(plan.status).toBe(200); + expect(plan.body).toMatchObject({ from: '1.4.0', to: '2.0.0', allowed: false, applied: false }); + expect(plan.body.needs).toEqual([expect.objectContaining({ kind: 'setting', path: 'settings.region' })]); + expect(inst.patched).toEqual([]); + const apply = await request(app).post('/api/integrations/int_1/upgrade').set('authorization', 'Bearer token-a').send({}); + expect(apply.status).toBe(409); + expect(inst.config.definition.version).toBe('1.4.0'); + expect((await request(app).get('/api/integrations/int_1/upgrade').set('authorization', 'Bearer token-b')).status).toBe(404); + }); + + it('applies an upgrade: undoes the old install steps, runs the new ones, moves the pin with its checksum', async () => { + const { app, publishAs, install, catalog } = await setup(); + await publishAs(example('phone-hubspot')); + const v15 = example('phone-hubspot'); v15.version = '1.5.0'; + v15.lifecycle.onInstall[0].with.events = [...v15.lifecycle.onInstall[0].with.events, 'message.delivered']; + expect((await publishAs(v15)).body.changeLevel).toBe('minor'); + const calls = []; + const inst = install('int_2', 'user-a', { definition: { id: 'phone-hubspot', version: '1.4.0' }, settings: { fromNumber: '+15125550100' }, runtime: { installed: { hook: { id: 'WH_old' } } } }, { + phone: { api: { createWebhook: async (a) => { calls.push(['create', a.events.length]); return { id: 'WH_new' }; }, deleteWebhook: async (a) => { calls.push(['delete', a.id]); } } }, + hubspot: { api: {} }, + }); + const r = await request(app).post('/api/integrations/int_2/upgrade').set('authorization', 'Bearer token-a').send({ to: '1.5.0' }); + expect(r.status).toBe(200); + expect(r.body.applied).toBe(true); + expect(calls).toEqual([['delete', 'WH_old'], ['create', 6]]); + expect(inst.config.definition).toEqual({ id: 'phone-hubspot', version: '1.5.0', checksum: await catalog.checksumOf('phone-hubspot', '1.5.0') }); + expect(inst.config.runtime.installed).toEqual({ hook: { id: 'WH_new' } }); + expect(inst.def.version).toBe('1.5.0'); + }); + + it('records each workflow run (a store failure never fails the run) and lists runs for the owner', async () => { + const { app, publishAs, install, runStore } = await setup(); + await publishAs(example('phone-hubspot')); + const createCallEngagement = jest.fn(async () => ({ id: 'e1' })); + const inst = install('int_3', 'user-a', { definition: { id: 'phone-hubspot', version: '1.4.0' } }, { phone: { api: {} }, hubspot: { api: { createCallEngagement } } }); + await inst.initialize(); + inst.rows.set('idx:contacts:phone:+15125550142', { hubspot: '51', phone: 'CT7' }); + const body = JSON.parse(JSON.stringify(manifests.phone.events['call.completed'].payload)); + await inst.on.WF_LOG_CALL.handler({ data: { body, eventId: 'EVc0a1' } }); + expect(runStore.runs).toEqual([expect.objectContaining({ integrationId: 'int_3', workflowId: 'log-call', status: 'completed', definitionId: 'phone-hubspot', definitionVersion: '1.4.0', triggerType: 'webhook', eventId: 'EVc0a1' })]); + const listed = await request(app).get('/api/integrations/int_3/runs?workflowId=log-call').set('authorization', 'Bearer token-a'); + expect(listed.body.runs).toHaveLength(1); + runStore.createRun = async () => { throw new Error('database down'); }; + const again = await inst.on.WF_LOG_CALL.handler({ data: { body: { ...body, id: 'x' }, eventId: 'EVc0a2' } }); + expect(again.status).toBe('completed'); + }); +}); + +describe('installing by id alone', () => { + it('pins the latest published version and its checksum, and persists the pin at install', async () => { + const { publishAs, install, catalog } = await setup(); + await publishAs(example('phone-hubspot')); + const inst = install('int_4', 'user-a', { definition: { id: 'phone-hubspot' } }, { phone: { api: { createWebhook: async () => ({ id: 'WH1' }) } }, hubspot: { api: {} } }); + await inst.initialize(); + expect(inst.config.definition).toEqual({ id: 'phone-hubspot', version: '1.4.0', checksum: await catalog.checksumOf('phone-hubspot', '1.4.0') }); + await inst.onCreate({}); + expect(inst.patched.at(-1).definition).toEqual(inst.config.definition); + }); + + it('refuses to load a version whose content no longer matches the pinned checksum', async () => { + const { publishAs, install } = await setup(); + await publishAs(example('phone-hubspot')); + const inst = install('int_5', 'user-a', { definition: { id: 'phone-hubspot', version: '1.4.0', checksum: 'sha256:tampered' } }); + await expect(inst.initialize()).rejects.toMatchObject({ code: 'CHECKSUM_MISMATCH' }); + }); +}); diff --git a/packages/core/integrations/data-defined/tests/schemas.test.js b/packages/core/integrations/data-defined/tests/schemas.test.js new file mode 100644 index 000000000..cba5f5e23 --- /dev/null +++ b/packages/core/integrations/data-defined/tests/schemas.test.js @@ -0,0 +1,92 @@ +/** + * The JSON Schemas are the published contract; validate.js and manifests.js are what the runtime enforces. Every + * bundled example and manifest must pass both layers, through the same schema-check.js the CLI uses. ajv comes from + * the monorepo (@friggframework/schemas depends on it); if it is missing this file fails, it never skips. + */ +const fs = require('fs'); +const path = require('path'); +const { execFileSync } = require('child_process'); +const os = require('os'); +const d = require('..'); +const { checkDefinition, checkManifest, loadAjv } = require('../schema-check'); + +loadAjv(); // throws "ajv is not installed ..." so a missing check never reads as a pass + +const manifests = d.loadManifests(); +const failures = (r) => r.errors.map((e) => `[${e.layer}] ${e.path} ${e.message}`); + +describe('both validation layers', () => { + it('every bundled manifest passes', () => { + for (const [name, m] of Object.entries(manifests)) expect([name, failures(checkManifest(m))]).toEqual([name, []]); + }); + + it('every bundled example passes', () => { + for (const def of d.loadExamples()) expect([def.id, failures(checkDefinition(def, manifests))]).toEqual([def.id, []]); + }); + + it('the schema refuses what validate refuses, and each layer reports it under its own name', () => { + const def = JSON.parse(JSON.stringify(d.loadExamples().find((x) => x.id === 'phone-hubspot'))); + def.syncs[0].mapping[0].transform = 'shout'; + const layers = checkDefinition(def, manifests).errors.map((e) => e.layer); + expect(layers).toEqual(['schema', 'references']); + const ok = JSON.parse(JSON.stringify(d.loadExamples().find((x) => x.id === 'phone-hubspot'))); + ok.syncs[0].mapping.push({ target: 'source', value: 'hubspot', direction: 'to_target' }); + expect(checkDefinition(ok, manifests).valid).toBe(true); + }); + + it('the schema\'s transform list is the runtime\'s transform list', () => { + const s = JSON.parse(fs.readFileSync(path.join(__dirname, '../schema/integration-definition.v0.1.schema.json'), 'utf8')); + expect(s.$defs.mappingRow.properties.transform.enum).toEqual(Object.keys(d.TRANSFORMS)); + }); + + it('the generated installation record matches the installation schema', () => { + const rec = JSON.parse(fs.readFileSync(path.join(__dirname, '../examples/records/phone-hubspot.integration.json'), 'utf8')); + expect(failures(require('../schema-check').checkInstallation(rec.config))).toEqual([]); + }); +}); + +describe('scripts/validate.js (the real CLI, as a child process)', () => { + const cli = path.join(__dirname, '../scripts/validate.js'); + const runCli = (...args) => { + try { return { code: 0, out: execFileSync(process.execPath, [cli, ...args], { encoding: 'utf8', env: process.env }) }; } catch (e) { return { code: e.status, out: `${e.stdout}${e.stderr}` }; } + }; + const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'dd-validate-')); + const write = (name, doc) => { const f = path.join(tmp, name); fs.writeFileSync(f, typeof doc === 'string' ? doc : JSON.stringify(doc)); return f; }; + const phone = () => JSON.parse(JSON.stringify(require('../examples/phone-hubspot.definition.json'))); + + it('exits 0 on every bundled manifest and example', () => { + const r = runCli(); + expect(r.code).toBe(0); + expect(r.out).toMatch(/21 manifests, 16 definitions: all valid/); + }); + + it('exits 1 on an invalid definition and names the layer, path and reason', () => { + const bad = phone(); bad.workflows[0].steps[0].use = 'hubspot.nope.make'; + const r = runCli(write('bad.json', bad)); + expect(r.code).toBe(1); + expect(r.out).toMatch(/error\s+\[references\] workflows\[0\]\.steps\[0\]\.use: Module "hubspot" \(hubspot\) has no capability "nope\.make"/); + }); + + it('a warning passes, unless --strict', () => { + const w = phone(); w.templates.orphan = 'x'; + const f = write('warn.json', w); + expect(runCli(f).code).toBe(0); + expect(runCli('--strict', f).code).toBe(1); + }); + + it('exits 2 when it cannot run: unreadable JSON, a missing file, an unknown option', () => { + expect(runCli(write('junk.json', '{nope')).code).toBe(2); + expect(runCli(path.join(tmp, 'missing.json')).code).toBe(2); + expect(runCli('--bogus').code).toBe(2); + }); + + it('an invalid manifest fails the run even when every definition passes', () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'dd-manifests-')); + for (const [name, m] of Object.entries(manifests)) fs.writeFileSync(path.join(dir, `${name}.capabilities.json`), JSON.stringify(m)); + const hs = JSON.parse(JSON.stringify(manifests.hubspot)); hs.options.owner.capability = 'deal.update'; + fs.writeFileSync(path.join(dir, 'hubspot.capabilities.json'), JSON.stringify(hs)); + const r = runCli('--manifests', dir, path.join(__dirname, '../examples/pipedrive-hubspot.definition.json')); + expect(r.code).toBe(1); + expect(r.out).toMatch(/FAIL manifest\s+hubspot/); + }); +}); diff --git a/packages/core/integrations/data-defined/tests/v01-gaps.test.js b/packages/core/integrations/data-defined/tests/v01-gaps.test.js new file mode 100644 index 000000000..cbd11ad59 --- /dev/null +++ b/packages/core/integrations/data-defined/tests/v01-gaps.test.js @@ -0,0 +1,354 @@ +/** + * The v0.1 gaps the 15 examples exposed, each closed inside v0.1: link sets, fixed values and merge, foreach collect, + * static checks of templates and views, option lists that take inputs, arithmetic filters, a sync-only definition, + * and lifecycle results and account values on $.install. + */ +const d = require('..'); +const { renderTemplate, resolve } = require('../expressions'); + +const manifests = d.loadManifests(); +const example = (id) => JSON.parse(JSON.stringify(d.loadExamples().find((x) => x.id === id))); +const errorsOf = (def) => d.validate(def, manifests).errors.map((e) => `${e.path}: ${e.message}`); + +function harness(def, respond = () => ({ id: 'out_1' }), seed = {}) { + const store = new Map(Object.entries(seed)); + const calls = []; + const ports = d.makePorts({ + def, manifests, + store: { get: async (k) => store.get(k) || null, put: async (k, v) => { store.set(k, v); }, claim: async (k, v) => { if (store.has(k)) return false; store.set(k, v); return true; }, release: async (k) => { store.delete(k); } }, + callApi: async (module, method, args) => { calls.push({ module, method, args }); return respond(module, method, args); }, + }); + return { store, calls, ports }; +} +const run = (def, steps, ctx = {}, ports) => d.execute(steps, { def, installation: {}, ctx: { event: {}, ...ctx }, ports }); + +describe('link sets (links without a sync)', () => { + const def = () => example('calendly-salesforce'); + + it('stores and finds links under the link set, normalising identity keys with its transform', async () => { + const h = harness(def()); + await run(def(), [{ use: 'links.upsert', with: { link: 'people', ids: { calendly: 'inv_1', salesforce: '00Q1' }, keys: { email: 'Dana@Acme.com' } } }], {}, h.ports); + expect(h.store.get('link:people:calendly:inv_1')).toMatchObject({ calendly: 'inv_1', salesforce: '00Q1' }); + expect(h.store.has('idx:people:email:dana@acme.com')).toBe(true); + const out = await run(def(), [{ id: 'hit', use: 'links.find', with: { link: 'people', key: { email: 'DANA@acme.com' } } }], {}, h.ports); + expect(out.steps.hit).toMatchObject({ salesforce: '00Q1' }); + }); + + it('the calendly example has no syncs any more: its links are link sets', () => { + expect(def().syncs).toBeUndefined(); + expect(def().links.map((l) => l.id)).toEqual(['people', 'meetings']); + expect(errorsOf(def())).toEqual([]); + }); + + it('refuses an unknown link set, naming both link and sync, a module the set does not join, and a key it does not index', () => { + const x = def(); + const steps = x.workflows[0].steps; + steps.unshift( + { use: 'links.find', with: { link: 'nope', key: { email: 'a' } } }, + { use: 'links.find', with: { link: 'people', sync: 'people', key: { email: 'a' } } }, + { use: 'links.upsert', with: { link: 'people', ids: { hubspot: '1' } } }, + { use: 'links.find', with: { link: 'people', key: { phone: '1' } } }, + ); + const e = errorsOf(x).join('\n'); + expect(e).toMatch(/Unknown link set "nope"/); + expect(e).toMatch(/names a link set \(link\) or a sync \(sync\), exactly one/); + expect(e).toMatch(/"hubspot" is not one of the modules link set "people" links/); + expect(e).toMatch(/"phone" is not an identity key of link set "people"/); + }); + + it('a link set joins two different, existing objects', () => { + const x = def(); + x.links[0].between = ['calendly.invitee', 'calendly.scheduledEvent']; + x.links[1].between = ['calendly.nope', 'salesforce.event']; + const e = errorsOf(x).join('\n'); + expect(e).toMatch(/links\[0\]\.between: A link set joins two different modules/); + expect(e).toMatch(/links\[1\]\.between\[0\]: Module "calendly" has no object "nope"/); + }); +}); + +describe('fixed values and merge', () => { + it('a mapping row can write a fixed value, which skips the source and still passes the transform', () => { + const out = d.applyMapping([{ source: 'n', target: 'name' }, { target: 'Line[0].DetailType', value: 'SalesItemLineDetail' }, { target: 'stage', value: 'Lead', transform: 'lowercase' }], { n: 'Acme' }); + expect(out).toEqual({ name: 'Acme', Line: [{ DetailType: 'SalesItemLineDetail' }], stage: 'lead' }); + }); + + it('a row has a source or a value, never both or neither', () => { + const x = example('stripe-quickbooks'); + x.workflows[0].steps.unshift({ use: 'map', with: { record: '$.record', mapping: [{ target: 'a' }, { target: 'b', source: 'x', value: 1 }] } }); + const e = errorsOf(x).join('\n'); + expect(e).toMatch(/mapping\[0\]: A row needs a source or a fixed value/); + expect(e).toMatch(/mapping\[1\]: A row has a source or a fixed value, not both/); + }); + + it('merge deep-merges, later objects win, lists are replaced, and inputs are untouched', async () => { + const a = { properties: { name: 'A', tags: ['x'] }, keep: 1 }; + const out = await run(example('phone-hubspot'), [{ id: 'm', use: 'merge', with: { objects: ['$.vars.a', { properties: { tags: ['y'], owner: '7' } }] } }], { vars: { a } }); + expect(out.steps.m).toEqual({ properties: { name: 'A', tags: ['y'], owner: '7' }, keep: 1 }); + expect(a).toEqual({ properties: { name: 'A', tags: ['x'] }, keep: 1 }); + }); +}); + +describe('foreach collect and where', () => { + const def = example('phone-hubspot'); + + it('collect makes the step result a list of one value per item; where skips items before the cap', async () => { + const items = [{ n: 'a', ok: true }, { n: 'b', ok: false }, { n: 'c', ok: true }, { n: 'd', ok: true }]; + const out = await run(def, [{ id: 'names', use: 'foreach', with: { over: '$.vars.items', where: '$.item.ok == true', max: 2, collect: '{{item.n | upper}}' } }], { vars: { items } }); + expect(out.steps.names).toEqual(['A', 'C']); + expect(out.trace.find((t) => t.status === 'truncated').note).toMatch(/1 items over max 2/); + }); + + it('collect can read the step results of its own iteration', async () => { + const out = await run(def, [{ id: 'lines', use: 'foreach', with: { over: '$.vars.items', collect: '$.steps.line' }, do: [{ id: 'line', use: 'template', with: { text: '#{{index}} {{item}}' } }] }], { vars: { items: ['x', 'y'] } }); + expect(out.steps.lines).toEqual(['#0 x', '#1 y']); + }); + + it('the digests build their message with collect and join, not repeated set', async () => { + const hs = example('hubspot-slack'); + const w = hs.workflows.find((x) => x.id === 'weekly-digest'); + expect(JSON.stringify(w.steps)).not.toMatch(/"use":"set"/); + const deals = [{ id: '1', properties: { dealname: 'Acme', amount: '1200' } }, { id: '2', properties: { dealname: 'Globex', amount: '450' } }]; + const h = harness(hs, (m, method) => (method === 'searchDeals' ? deals : { ok: true })); + const r = await d.runWorkflow({ def: hs, installation: { workflows: { 'weekly-digest': { settings: { channel: 'C1' } } } }, workflowId: 'weekly-digest', ports: h.ports, manifests }); + expect(r.status).toBe('completed'); + const post = h.calls.find((c) => c.method === 'postMessage'); + expect(post.args.text.split('\n')).toHaveLength(3); // heading + one line per deal + expect(post.args.text).toMatch(/\(2 deals\)/); + }); + + it('foreach needs do, collect or both, and max is 1..1000', () => { + const x = example('phone-hubspot'); + x.workflows[0].steps.unshift({ use: 'foreach', with: { over: '$.event.x' } }, { use: 'foreach', with: { over: '$.event.x', max: 5000, collect: '$.item' } }); + const e = errorsOf(x).join('\n'); + expect(e).toMatch(/foreach needs do steps, collect, or both/); + expect(e).toMatch(/with\.max: A whole number from 1 to 1000/); + }); +}); + +describe('static checks of templates and views', () => { + it('an unknown filter is an error wherever a template appears: steps, named templates, views', () => { + const x = example('phone-hubspot'); + x.templates['call-note'] += ' {{record.x | shout}}'; + x.workflows[0].steps.push({ use: 'set', with: { y: '{{event.id | yell}}' } }); + x.ui[0].view.children[0].props.text = '{{data.calls | whisper}}'; + const e = errorsOf(x).join('\n'); + expect(e).toMatch(/templates\.call-note: Unknown template filter "shout"/); + expect(e).toMatch(/Unknown template filter "yell"/); + expect(e).toMatch(/Unknown template filter "whisper"/); + }); + + it('a view can only read what its query returns, $.item inside a List row and $.row inside a Table column', () => { + const x = example('phone-hubspot'); + const kids = x.ui[0].view.children; + kids[0].props.text = '{{data.deals}}'; + kids.push({ component: 'Text', props: { text: '{{item.title}}' } }, { component: 'Text', props: { text: '{{steps.x}}' } }); + const e = errorsOf(x).join('\n'); + expect(e).toMatch(/\$\.data\.deals is not returned by query "recent-activity" \(calls, texts\)/); + expect(e).toMatch(/\$\.item is not available here/); + expect(e).toMatch(/\$\.steps is not available here/); + }); + + it('a named template no step renders is a warning', () => { + const x = example('phone-hubspot'); + x.templates.orphan = 'hi'; + expect(d.validate(x, manifests).warnings.map((w) => w.path)).toContain('templates.orphan'); + }); +}); + +describe('option lists that take inputs', () => { + const def = example('asana-slack'); + const projects = [{ gid: 'p1', name: 'Launch' }, { gid: 'p2', name: 'Ops' }]; + + it('calls the list with inputs read from earlier choices, and returns nothing until they are chosen', async () => { + const h = harness(def, (m, method) => (method === 'getProjects' ? projects : [])); + expect(await d.resolveOptions({ def, manifests, ports: h.ports, path: 'settings.project', values: {} })).toEqual([]); + expect(h.calls).toEqual([]); + const opts = await d.resolveOptions({ def, manifests, ports: h.ports, path: 'settings.project', values: { workspace: 'w9' } }); + expect(opts).toEqual([{ label: 'Launch', value: 'p1' }, { label: 'Ops', value: 'p2' }]); + expect(h.calls).toEqual([{ module: 'asana', method: 'getProjects', args: { workspaceGid: 'w9' } }]); + }); + + it('a workflow setting reads integration settings as $.install', async () => { + const gl = example('github-linear'); + const h = harness(gl, () => [{ id: 's1', name: 'Done' }]); + const opts = await d.resolveOptions({ def: gl, manifests, ports: h.ports, path: 'workflows.pr-merged-done.settings.doneState', values: {}, integrationSettings: { team: 'T1' } }); + expect(opts).toEqual([{ label: 'Done', value: 'Done' }]); + expect(h.calls[0].args).toEqual({ teamId: 'T1' }); + }); + + it('refuses a missing required input, an undeclared one, reading itself, and a loop', () => { + const x = example('asana-slack'); + x.settings.project.options = { source: 'asana.project', with: { colour: '$.settings.workspace' } }; + x.settings.workspace.options = { source: 'asana.workspace', with: {} }; + let e = errorsOf(x).join('\n'); + expect(e).toMatch(/settings\.project\.options\.with\.workspaceGid: Option list asana\.project needs input "workspaceGid"/); + expect(e).toMatch(/takes no input "colour"/); + x.settings.project.options = { source: 'asana.project', with: { workspaceGid: '$.settings.project' } }; + expect(errorsOf(x).join('\n')).toMatch(/cannot read its own setting/); + const loop = example('asana-slack'); + loop.settings.a = { type: 'option', label: 'A', options: { source: 'asana.project', with: { workspaceGid: '$.settings.b' } } }; + loop.settings.b = { type: 'option', label: 'B', options: { source: 'asana.project', with: { workspaceGid: '$.settings.a' } } }; + expect(errorsOf(loop).join('\n')).toMatch(/Option inputs form a loop/); + }); +}); + +describe('arithmetic filters and typed single-tag templates', () => { + it('money in minor units follows the currency, and rounds away float error', () => { + expect(renderTemplate('{{v | to_cents}}', { v: 19.99 })).toBe(1999); + expect(renderTemplate('{{v | to_cents}}', { v: 19.995 })).toBe(2000); + expect(d.TRANSFORMS.units_to_cents(19.995)).toBe(2000); + expect(renderTemplate('{{v | from_minor_units: c}}', { v: 1999, c: 'usd' })).toBe(19.99); + expect(renderTemplate('{{v | from_minor_units: c}}', { v: 1999, c: 'JPY' })).toBe(1999); + expect(renderTemplate('{{v | from_minor_units: c}}', { v: 1999, c: 'KWD' })).toBe(1.999); + expect(renderTemplate('{{v | to_minor_units: c}}', { v: 12.5, c: 'EUR' })).toBe(1250); + expect(renderTemplate('{{v | to_minor_units: c}}', { v: 1250, c: 'JPY' })).toBe(1250); + expect(renderTemplate('{{v | to_minor_units: c}}', { v: 1.5, c: 'KWD' })).toBe(1500); + }); + + it('unix seconds convert both ways, and chain with date', () => { + expect(renderTemplate('{{t | from_unix | date}}', { t: 1760796433 })).toBe('2025-10-18'); + expect(renderTemplate('{{t | to_unix}}', { t: '2026-10-18T14:07:12Z' })).toBe(1792332432); + }); + + it('a single tag keeps its type; text around a tag makes a string; bad input gives blank, never a throw', () => { + expect(resolve({ n: '{{v | times: 2}}', s: 'n={{v | times: 2}}', l: '{{list}}' }, { v: 4, list: [1, 2] })).toEqual({ n: 8, s: 'n=8', l: [1, 2] }); + expect(renderTemplate('{{v | divided_by: 0}}', { v: 4 })).toBe(''); + expect(renderTemplate('{{v | plus: 1}}', { v: 'abc' })).toBe(''); + expect(renderTemplate('{{v | round: 2}}', { v: 1.005 })).toBe(1.01); + expect(renderTemplate("{{v | join: '\\n'}}", { v: ['a', '', 'b'] })).toBe('a\nb'); + }); + + it('the Stripe receipt converts cents and unix time with filters, not a code step', async () => { + const st = example('stripe-quickbooks'); + const w = st.workflows.find((x) => x.id === 'payment-sales-receipt'); + expect(JSON.stringify(w.steps)).not.toMatch(/MinorUnits/); + const h = harness(st, (m, method) => ({ id: 'qb_1', method }), { 'link:customers:stripe:cus_1': { quickbooks: '58', stripe: 'cus_1' } }); + const event = JSON.parse(JSON.stringify(manifests.stripe.events[w.trigger.topic].payload)); + Object.assign(event.data.object, { amount_received: 125000, currency: 'jpy', customer: 'cus_1', created: 1760796433 }); + const r = await d.runWorkflow({ def: st, installation: { settings: { depositAccount: '35', incomeItem: '1' } }, workflowId: w.id, event, ports: h.ports, manifests }); + expect(r.status).toBe('completed'); + const receipt = h.calls.find((c) => c.method === 'createSalesReceipt').args; + expect(JSON.stringify(receipt)).toMatch(/"Amount":125000/); // JPY has no minor unit + expect(JSON.stringify(receipt)).toMatch(/"TxnDate":"2025-10-18"/); + }); +}); + +describe('a sync-only definition', () => { + it('validates and compiles to sync events only: no workflow, query or UI events', () => { + const ph = example('pipedrive-hubspot'); + expect(errorsOf(ph)).toEqual([]); + expect(Object.keys(d.compile(ph, {}).events).sort()).toEqual(['SYNC_PEOPLE_BACKFILL', 'SYNC_PEOPLE_ONGOING', 'SYNC_PEOPLE_PAGE']); + expect(d.compile(ph, {}).webhooks).toEqual({ pipedrive: { 'person.*': ['SYNC_PEOPLE_ONGOING'] }, hubspot: { 'contact.*': ['SYNC_PEOPLE_ONGOING'] } }); + }); +}); + +describe('lifecycle results and account values on $.install', () => { + class FakeBase { + constructor(p = {}) { this.events = {}; this.config = p.config; this.id = p.id; this.pipedrive = { api: {} }; this.phone = { api: {} }; this.calls = []; } + registerEventHandlers() {} + async patchConfig(patch) { this.config = { ...this.config, ...patch }; } + async onCreate() {} async onDelete() {} async onUpdate() {} + async getMapping() { return null; } async upsertMapping() {} async createMapping() {} + } + const make = (def, apis) => { + const Cls = d.defineSharedIntegrationClass({ definitions: d.memoryDefinitions([def]), manifests, modules: { pipedrive: {}, phone: {}, hubspot: {}, zendesk: {}, jira: {} }, IntegrationBase: FakeBase, runs: false }); + const inst = new Cls({ id: 'int_1', config: { definition: { id: def.id, version: def.version } } }); + Object.assign(inst, apis(inst)); + return inst; + }; + + it('install keeps every onInstall result by step id, and uninstall undoes exactly those', async () => { + const def = example('pipedrive-phone'); + let n = 0; + const inst = make(def, (self) => ({ + phone: { api: { createWebhook: async (a) => { self.calls.push(['phone.create', a]); return { id: 'WHq' }; }, deleteWebhook: async (a) => { self.calls.push(['phone.delete', a]); } } }, + pipedrive: { api: { addWebhook: async (a) => { self.calls.push(['pd.create', a]); n += 1; return { id: `pd${n}` }; }, deleteWebhook: async (a) => { self.calls.push(['pd.delete', a]); } } }, + })); + await inst.onCreate({}); + expect(inst.config.runtime.installed).toEqual({ phoneHook: { id: 'WHq' }, personCreated: { id: 'pd1' }, personChanged: { id: 'pd2' } }); + inst.calls.length = 0; + await inst.onDelete({}); + expect(inst.calls).toEqual([['phone.delete', { id: 'WHq' }], ['pd.delete', { id: 'pd1' }], ['pd.delete', { id: 'pd2' }]]); + }); + + it('in every example, uninstall makes one call per thing install created, and never with a blank id', async () => { + for (const def of d.loadExamples().filter((x) => x.lifecycle?.onInstall?.length)) { + let n = 0; const calls = []; + const created = (k) => ({ id: `id${k}`, gid: `id${k}`, connectId: `id${k}`, resource: { uri: `id${k}` }, webhookRegistrationResult: [{ createdWebhookId: k }] }); + const ports = d.makePorts({ def, manifests, store: { get: async () => null, put: async () => {}, claim: async () => true }, callApi: async (module, method, args) => { calls.push(args); n += 1; return created(n); } }); + const install = await d.runLifecycle({ def, installation: { webhookUrl: 'https://x/wh' }, hook: 'onInstall', ports }); + const installed = Object.fromEntries(def.lifecycle.onInstall.filter((st) => st.id).map((st) => [st.id, install.steps[st.id] ?? null])); + const made = calls.length; calls.length = 0; + await d.runLifecycle({ def, installation: { runtime: { installed } }, hook: 'onUninstall', ports }); + const blank = calls.filter((args) => Object.values(args).some((v) => v == null || (Array.isArray(v) && (!v.length || v.some((x) => x == null))))); + expect([def.id, calls.length, blank]).toEqual([def.id, made, []]); + } + }); + + it('$.install.accounts exposes only the account fields a manifest declares, in queries as in workflows', async () => { + const def = example('zendesk-jira'); + const inst = make(def, () => ({ zendesk: { api: { subdomain: 'acme', secretToken: 'never' } }, jira: { api: { site: 'acme.atlassian.net', cloudId: 'c1' } } })); + await inst.loadDefinition(); + expect(inst.installation().accounts).toEqual({ zendesk: { subdomain: 'acme' }, jira: { site: 'acme.atlassian.net', cloudId: 'c1' } }); + const q = def.queries[0]; + const h = harness(def, () => ({ key: 'SUP-1' }), { [`link:escalations:zendesk:42`]: { jira: '10001', zendesk: 42 } }); + const out = await d.runQuery({ def, installation: inst.installation(), queryId: q.id, record: { id: 42 }, ports: h.ports }); + expect(JSON.stringify(out)).toMatch(/acme\.atlassian\.net/); + }); + + it('warns when uninstall never undoes an install step, or an install step keeps no result', () => { + const x = example('pipedrive-phone'); + x.lifecycle.onUninstall = x.lifecycle.onUninstall.filter((st) => st.use !== 'phone.webhook.delete'); + x.lifecycle.onInstall.push({ use: 'phone.webhook.create', with: { url: '$.install.webhookUrl', events: ['call.completed'] } }); + const w = d.validate(x, manifests).warnings.map((e) => `${e.path}: ${e.message}`).join('\n'); + expect(w).toMatch(/onInstall\[0\]: onUninstall never reads \$\.install\.runtime\.installed\.phoneHook/); + expect(w).toMatch(/onInstall\[3\]: This install step has no id/); + expect(d.validate(example('pipedrive-phone'), manifests).warnings).toEqual([]); + }); + + it('refuses an undeclared account field, an unknown install step, and install steps reading their own results', () => { + const x = example('zendesk-jira'); + x.workflows[0].steps.push({ use: 'set', with: { a: '$.install.accounts.zendesk.password', b: '$.install.runtime.installed.nope', c: '$.install.runtime.subscriptions' } }); + x.lifecycle.onInstall[1].with.url = '$.install.runtime.installed.zendeskHook.id'; + const e = errorsOf(x).join('\n'); + expect(e).toMatch(/zendesk declares no account field "password"/); + expect(e).toMatch(/no onInstall step has id "nope"/); + expect(e).toMatch(/runtime only holds "installed"/); + expect(e).toMatch(/Install steps cannot read \$\.install\.runtime\.installed/); + }); +}); + +describe('manifest validation', () => { + it('every bundled manifest is valid', () => { + for (const [name, m] of Object.entries(manifests)) expect([name, d.validateManifest(m).errors]).toEqual([name, []]); + }); + + it('refuses an option label that mixes $. into text, an option list on a write capability, and an unlisted enum', () => { + const m = JSON.parse(JSON.stringify(manifests.hubspot)); + m.options.owner.label = '$.item.firstName {{item.lastName}}'; + m.options.dealStage.capability = 'deal.update'; + m.objects.deal.fields.find((f) => f.name === 'properties.dealstage').values = undefined; + const e = d.validateManifest(m).errors.map((x) => `${x.path}: ${x.message}`).join('\n'); + expect(e).toMatch(/options\.owner\.label: .* renders literally/); + expect(e).toMatch(/options\.dealStage\.capability: An option list reads; deal\.update is a write capability/); + expect(e).toMatch(/An enum lists its values, or names an option list/); + }); +}); + +describe('versioning of the new shapes', () => { + it('a new link set is minor; changing its identity is major and the upgrade says lookups by key will miss', () => { + const a = example('calendly-salesforce'); + const b = { ...example('calendly-salesforce'), version: '2.1.0', links: [...a.links, { id: 'extra', between: ['calendly.invitee', 'salesforce.lead'] }] }; + expect(d.diff(a, b).level).toBe('minor'); + const c = example('calendly-salesforce'); c.version = '3.0.0'; c.links[0].identity.match[0].key = 'mail'; + expect(d.checkBump(a, { ...c, version: '2.1.0' }).ok).toBe(false); + expect(d.planUpgrade({}, a, c).actions.map((x) => x.kind)).toContain('reindex'); + }); + + it('changing only an option list\'s inputs is minor, not "setting type changed"', () => { + const a = example('asana-slack'); + const b = example('asana-slack'); b.version = '0.3.0'; + b.settings.project.options = { ...b.settings.project.options, with: { workspaceGid: '$.settings.workspace', archived: false } }; + expect(d.diff(a, b).changes.map((x) => `${x.level} ${x.message}`)).toEqual(['minor Option list inputs changed']); + }); +}); diff --git a/packages/core/integrations/data-defined/tests/wiring.test.js b/packages/core/integrations/data-defined/tests/wiring.test.js new file mode 100644 index 000000000..77d317639 --- /dev/null +++ b/packages/core/integrations/data-defined/tests/wiring.test.js @@ -0,0 +1,291 @@ +/** + * The data-defined class inside Frigg's real entry points: initialize() hydration, webhook receipt (handshakes), + * the worker turning a raw delivery into workflow messages (module, topic, batch, account, signature), app-level + * fan-out, and the existing authenticated config-options and user-action routes. Only IntegrationBase is faked; the + * class, compiler, runtime and webhook resolution are the real ones. + */ +const d = require('..'); +const { eventsIn, handshakeFor } = require('../webhooks'); + +const manifests = d.loadManifests(); +const example = (id) => JSON.parse(JSON.stringify(d.loadExamples().find((x) => x.id === id))); +const phoneCall = () => JSON.parse(JSON.stringify(manifests.phone.events['call.completed'].payload)); + +class FakeBase { + constructor(p = {}) { + this.events = {}; this.config = p.config; this.id = p.id; this.rows = new Map(); this.queued = []; this.patched = []; + } + async initialize() { this.initialized = true; this.registerEventHandlers(); } + registerEventHandlers() { this.on = { ...this.events }; } + async send(event, payload) { return this.on[event].handler.call(this, payload); } + async queueWebhook(data) { this.queued.push(data); } + async onWebhookReceived({ req, res }) { await this.queueWebhook({ integrationId: req.params?.integrationId || null, body: req.body, headers: req.headers, query: req.query }); res.status(200).json({ received: true }); } + async patchConfig(c) { this.patched.push(c); this.config = { ...this.config, ...c }; } + async onUpdate(p) { if (p?.config) await this.patchConfig(p.config); } + async onCreate() {} async onDelete() {} + async getMapping(k) { return this.rows.has(k) ? { mapping: this.rows.get(k) } : null; } + async upsertMapping(k, v) { this.rows.set(k, v); } + async createMapping(k, v) { if (this.rows.has(k)) { const e = new Error('exists'); e.name = 'MappingAlreadyExistsError'; throw e; } this.rows.set(k, v); } + async deleteMapping(k) { this.rows.delete(k); } +} +const res = () => { const r = { statusCode: null, headers: {}, body: undefined }; r.status = (c) => { r.statusCode = c; return r; }; r.set = (k, v) => { r.headers[k] = v; return r; }; r.json = (b) => { r.body = b; return r; }; r.send = (b) => { r.body = b; return r; }; return r; }; + +function shared(defs, { lookup, apis = {} } = {}) { + const Cls = d.defineSharedIntegrationClass({ definitions: d.memoryDefinitions(defs), manifests, modules: { phone: {}, hubspot: {}, slack: {}, asana: {}, quickbooks: {}, pipedrive: {} }, IntegrationBase: FakeBase, lookup, runs: false }); + return (config, id = 'int_1') => { const inst = new Cls({ id, config }); Object.assign(inst, apis); return inst; }; +} +const pin = (def) => ({ definition: { id: def.id, version: def.version } }); +const verified = (extra = {}) => ({ api: { verifyWebhookSignature: async () => true, ...extra }, entity: { externalId: '12345678' } }); + +describe('initialize() is where a real Frigg entry point hydrates the definition', () => { + it('loads the pinned version and registers its events; a dry instance (no pin) loads nothing', async () => { + const make = shared([example('phone-hubspot')]); + const inst = make(pin(example('phone-hubspot'))); + await inst.initialize(); + expect(inst.def.version).toBe('1.4.0'); + expect(Object.keys(inst.on)).toContain('WF_LOG_CALL'); + const dry = make(undefined, undefined); + await dry.initialize(); + expect(dry.def).toBeUndefined(); + }); + + it('switching a workflow off removes its event on recompile (not just adds new ones)', async () => { + const def = example('phone-hubspot'); + const inst = shared([def])(pin(def)); + await inst.initialize(); + expect(inst.events.WF_LOG_CALL).toBeDefined(); + await inst.onUpdate({ config: { workflows: { 'log-call': { enabled: false } } } }); + expect(inst.events.WF_LOG_CALL).toBeUndefined(); + expect(inst.on.WF_LOG_CALL).toBeUndefined(); + }); + + it('calls a module by its module name when the definition names it by another key', async () => { + const def = { + schemaVersion: '0.1', id: 'renamed', version: '1.0.0', display: { label: 'Renamed keys' }, + modules: { phone: { module: 'phone' }, crm: { module: 'hubspot' } }, + workflows: [{ id: 'look-up', label: 'Look up', trigger: { type: 'userAction', label: 'Look up' }, steps: [{ id: 'c', use: 'crm.contact.get', with: { id: '$.event.id' } }] }], + }; + expect(d.validate(def, manifests).errors).toEqual([]); + const getContactById = jest.fn(async () => ({ id: '51' })); + const inst = shared([def], { apis: { hubspot: { api: { getContactById } }, phone: { api: {} } } })(pin(def)); + await inst.initialize(); + expect((await inst.send('WF_LOOK_UP', { id: '51' })).status).toBe('completed'); + expect(getContactById).toHaveBeenCalledWith({ id: '51' }); + }); +}); + +describe('webhook resolution (webhooks.js)', () => { + it('every example delivery in every manifest routes to its own event', () => { + for (const [name, m] of Object.entries(manifests)) { + for (const [key, ev] of Object.entries(m.events || {})) { + if ((ev.delivery || 'webhook') !== 'webhook') continue; + expect([name, key, eventsIn(m, { body: ev.payload, headers: ev.headers }).some((e) => e.topic === key)]).toEqual([name, key, true]); + } + } + }); + + it('a missed Phone call is call.completed with status missed: it is both events; a completed call is only one', () => { + const missed = eventsIn(manifests.phone, { body: manifests.phone.events['call.missed'].payload }).map((e) => e.topic); + expect(missed.sort()).toEqual(['call.completed', 'call.missed']); + expect(eventsIn(manifests.phone, { body: phoneCall() }).map((e) => e.topic)).toEqual(['call.completed']); + }); + + it('splits a HubSpot batch into events, each with its own account and event id', () => { + const body = [ + { eventId: 1, subscriptionType: 'contact.creation', portalId: 111, objectId: '1' }, + { eventId: 2, subscriptionType: 'deal.propertyChange', portalId: 222, objectId: '9' }, + { eventId: 3, subscriptionType: 'company.creation', portalId: 111, objectId: '5' }, // no such event: ignored + ]; + expect(eventsIn(manifests.hubspot, { body }).map((e) => [e.topic, e.account, e.eventId])).toEqual([['contact.created', '111', '1'], ['deal.propertyChange', '222', '2']]); + }); + + it('a two-level QuickBooks batch takes each entity\'s account from its notification', () => { + const body = { eventNotifications: [ + { realmId: 'R1', dataChangeEvent: { entities: [{ name: 'Payment', id: '7', operation: 'Create', lastUpdated: 't1' }] } }, + { realmId: 'R2', dataChangeEvent: { entities: [{ name: 'Customer', id: '8', operation: 'Update', lastUpdated: 't2' }, { name: 'Invoice', id: '9', operation: 'Create', lastUpdated: 't3' }] } }, + ] }; + expect(eventsIn(manifests.quickbooks, { body }).map((e) => [e.topic, e.account, e.record.id])).toEqual([['payment.created', 'R1', '7'], ['customer.updated', 'R2', '8']]); + }); + + it('reads a topic from a header, from the first non-empty template, and from a form-encoded body', () => { + const order = manifests.shopify.events['order.created']; + expect(eventsIn(manifests.shopify, { body: order.payload, headers: { 'X-Shopify-Topic': 'orders/paid', 'X-Shopify-Webhook-Id': 'w1' } }).map((e) => [e.topic, e.eventId])).toEqual([['order.paid', 'w1']]); + const interaction = manifests.slack.events['interaction.block_actions'].payload; + expect(eventsIn(manifests.slack, { body: { payload: JSON.stringify(interaction) } }).map((e) => e.topic)).toEqual(['interaction.block_actions']); + expect(eventsIn(manifests.slack, { body: manifests.slack.events.app_mention.payload }).map((e) => e.topic)).toEqual(['app_mention']); + }); + + it('handshakes: Slack echoes the challenge, Asana echoes the secret header, anything else is not a handshake', () => { + expect(handshakeFor(manifests.slack, { body: { type: 'url_verification', challenge: 'c-123' } })).toEqual({ status: 200, headers: {}, body: { challenge: 'c-123' } }); + expect(handshakeFor(manifests.asana, { body: {}, headers: { 'X-Hook-Secret': 's-9' } })).toMatchObject({ status: 200, headers: { 'X-Hook-Secret': 's-9' } }); + expect(handshakeFor(manifests.slack, { body: manifests.slack.events.app_mention.payload })).toBeNull(); + }); +}); + +describe('receipt and worker', () => { + it('answers a handshake at receipt without queueing anything', async () => { + const inst = shared([example('hubspot-slack')])(); + const r = res(); + await inst.onWebhookReceived({ req: { body: { type: 'url_verification', challenge: 'abc' }, headers: {}, query: { module: 'slack' }, params: {} }, res: r }); + expect(r.body).toEqual({ challenge: 'abc' }); + expect(inst.queued).toEqual([]); + const normal = res(); + await inst.onWebhookReceived({ req: { body: { type: 'event_callback' }, headers: {}, query: { module: 'slack' }, params: {} }, res: normal }); + expect(inst.queued).toHaveLength(1); + }); + + it('turns a raw Phone delivery into one message per listening workflow, with the provider event id; a redelivery writes once', async () => { + const def = example('phone-hubspot'); + const createCallEngagement = jest.fn(async () => ({ id: 'e1' })); + const inst = shared([def], { apis: { phone: verified(), hubspot: { api: { createCallEngagement } } } })({ ...pin(def), workflows: { 'call-summary': { enabled: false } } }); + await inst.initialize(); + const out = await inst.onWebhook({ data: { integrationId: 'int_1', body: phoneCall(), headers: {}, query: { module: 'phone' } } }); + expect(out.routed).toBe(1); + expect(inst.queued).toEqual([expect.objectContaining({ event: 'WF_LOG_CALL', module: 'phone', topic: 'call.completed', eventId: 'EVc0a1', definitionVersion: '1.4.0' })]); + inst.rows.set('idx:contacts:phone:+15125550142', { hubspot: '51', phone: 'CT7' }); + const msg = { data: inst.queued[0] }; + expect((await inst.on.WF_LOG_CALL.handler(msg)).status).toBe('completed'); + expect((await inst.on.WF_LOG_CALL.handler(msg)).status).toBe('duplicate'); + expect(createCallEngagement).toHaveBeenCalledTimes(1); + }); + + it('drops a delivery whose signature does not verify, and one whose module has no verify method (fail closed)', async () => { + const def = example('phone-hubspot'); + const bad = shared([def], { apis: { phone: { api: { verifyWebhookSignature: async () => false } }, hubspot: { api: {} } } })(pin(def)); + await bad.initialize(); + expect(await bad.onWebhook({ data: { body: phoneCall(), query: { module: 'phone' } } })).toEqual({ routed: 0, rejected: true }); + const missing = shared([def], { apis: { phone: { api: {} }, hubspot: { api: {} } } })(pin(def)); + await missing.initialize(); + expect((await missing.onWebhook({ data: { body: phoneCall(), query: { module: 'phone' } } })).rejected).toBe(true); + const forgotReturn = shared([def], { apis: { phone: { api: { verifyWebhookSignature: async () => { /* no return */ } } }, hubspot: { api: {} } } })(pin(def)); + await forgotReturn.initialize(); + expect((await forgotReturn.onWebhook({ data: { body: phoneCall(), query: { module: 'phone' } } })).rejected).toBe(true); + expect([...bad.queued, ...missing.queued, ...forgotReturn.queued]).toEqual([]); + }); + + it('dedupes on the provider event id, not on the body\'s id', async () => { + const def = example('phone-hubspot'); + const createCallEngagement = jest.fn(async () => ({ id: 'e1' })); + const inst = shared([def], { apis: { phone: verified(), hubspot: { api: { createCallEngagement } } } })(pin(def)); + await inst.initialize(); + inst.rows.set('idx:contacts:phone:+15125550142', { hubspot: '51', phone: 'CT7' }); + const a = { ...phoneCall(), id: 'body-1' }, b = { ...phoneCall(), id: 'body-2' }; + expect((await inst.on.WF_LOG_CALL.handler({ data: { body: a, eventId: 'delivery-7' } })).status).toBe('completed'); + expect((await inst.on.WF_LOG_CALL.handler({ data: { body: b, eventId: 'delivery-7' } })).status).toBe('duplicate'); + expect(inst.rows.has('evt:phone-hubspot:log-call:delivery-7')).toBe(true); + }); + + it('finds the module without ?module when exactly one manifest recognises the delivery', async () => { + const def = example('phone-hubspot'); + const inst = shared([def], { apis: { phone: verified(), hubspot: verified() } })(pin(def)); + await inst.initialize(); + expect((await inst.onWebhook({ data: { body: phoneCall(), query: {} } })).routed).toBeGreaterThan(0); + }); + + it('an app-level batch is filtered to this installation\'s own account', async () => { + const def = example('hubspot-slack'); + const inst = shared([def], { apis: { hubspot: verified(), slack: { api: {} } } })(pin(def)); + await inst.initialize(); + const body = [ + { eventId: 10, subscriptionType: 'deal.propertyChange', portalId: 12345678, objectId: '880', propertyName: 'dealstage', propertyValue: 'closedwon' }, + { eventId: 11, subscriptionType: 'deal.propertyChange', portalId: 5555, objectId: '881', propertyName: 'dealstage', propertyValue: 'closedwon' }, + ]; + await inst.onWebhook({ data: { body, query: { module: 'hubspot' } } }); + expect(inst.queued.length).toBeGreaterThan(0); + expect(inst.queued.every((m) => m.body.portalId === 12345678)).toBe(true); + }); + + it('with no installation, an app-level delivery is queued once per installation connected to each account', async () => { + const lookup = jest.fn(async (account) => (account === '12345678' ? ['int_a', 'int_b'] : [])); + const dry = shared([example('hubspot-slack')], { lookup })(); + await dry.initialize(); + const body = [{ eventId: 1, subscriptionType: 'deal.creation', portalId: 12345678, objectId: '1' }, { eventId: 2, subscriptionType: 'deal.creation', portalId: 7777, objectId: '2' }]; + expect(await dry.onWebhook({ data: { body, headers: {}, query: { module: 'hubspot' } } })).toEqual({ queued: 2 }); + expect(lookup.mock.calls.map((c) => c.slice(0, 2))).toEqual([['12345678', 'hubspot'], ['7777', 'hubspot']]); + expect(dry.queued.map((m) => m.integrationId)).toEqual(['int_a', 'int_b']); + expect(dry.queued[0].body).toBe(body); + expect(await shared([example('phone-hubspot')], { lookup })().onWebhook({ data: { body: phoneCall(), query: { module: 'phone' } } })).toEqual({ queued: 0 }); + }); +}); + +describe('the existing authenticated routes: config options, user actions, UI surfaces', () => { + const def = example('asana-slack'); + const make = () => shared([def], { apis: { asana: { api: { getWorkspaces: async () => [{ gid: 'w1', name: 'Acme' }], getProjects: async ({ workspaceGid }) => (workspaceGid === 'w1' ? [{ gid: 'p1', name: 'Launch' }] : []) } }, slack: { api: { listChannels: async () => [{ id: 'C1', name: 'general' }], listUsers: async () => [] } } } }); + + it('GET config/options: a form whose data is the config shape; a dependent list waits for its input', async () => { + const inst = make()(pin(def)); + await inst.initialize(); + const form = await inst.getConfigOptions(); + expect(Object.keys(form.data)).toEqual(['settings', 'syncs', 'workflows', 'ui']); + expect(form.jsonSchema.properties.settings.properties.workspace.oneOf).toEqual([{ const: 'w1', title: 'Acme' }]); + expect(form.jsonSchema.properties.settings.properties.project.enum).toEqual([]); + expect(form.uiSchema.settings.project['ui:help']).toMatch(/Choose workspace first/); + for (const w of def.workflows) expect(form.data.workflows[w.id].enabled).toBe((w.default || 'on') === 'on'); + const requiredInDef = Object.entries(def.settings).filter(([, s]) => s.required && s.default === undefined).map(([k]) => k); + expect(requiredInDef.length).toBeGreaterThan(0); + expect(form.jsonSchema.properties.settings.required).toEqual(requiredInDef); + }); + + it('POST config/options/refresh: the values chosen so far fill the dependent list', async () => { + const inst = make()(pin(def)); + await inst.initialize(); + const form = await inst.refreshConfigOptions({ settings: { workspace: 'w1' } }); + expect(form.jsonSchema.properties.settings.properties.project.oneOf).toEqual([{ const: 'p1', title: 'Launch' }]); + }); + + it('NEEDS_CONFIG comes from the definition\'s required settings, without calling any API', async () => { + const getWorkspaces = jest.fn(); + const inst = shared([def], { apis: { asana: { api: { getWorkspaces } }, slack: { api: {} } } })(pin(def)); + await inst.initialize(); + expect(await inst.validateConfig()).toBe(true); + inst.config = { ...inst.config, settings: Object.fromEntries(Object.entries(def.settings).filter(([, s]) => s.required).map(([k]) => [k, 'x'])) }; + for (const w of def.workflows) for (const [k, s] of Object.entries(w.settings || {})) if (s.required && s.default === undefined) ((inst.config.workflows ||= {})[w.id] ||= { settings: {} }).settings[k] = 'x'; + expect(await inst.validateConfig()).toBe(false); + expect(getWorkspaces).not.toHaveBeenCalled(); + }); + + it('POST actions/:id runs a button workflow, renders a UI surface, and the button\'s inputs are its options form', async () => { + const qh = example('phone-hubspot'); + const sendMessage = jest.fn(async () => ({ id: 'm1' })); + const inst = shared([qh], { apis: { phone: { api: { sendMessage, listCalls: async () => [], listMessages: async () => [] } }, hubspot: { api: {} } } })({ ...pin(qh), settings: { fromNumber: '+15125550100' } }); + await inst.initialize(); + const opts = await inst.getActionOptions('WF_TEXT_FROM_CARD'); + expect(opts.jsonSchema.properties.input.properties.message).toEqual({ type: 'string', title: 'Message' }); + const run = await inst.send('WF_TEXT_FROM_CARD', { record: { properties: { phone: '+15125550142' } }, input: { message: 'On my way' } }); + expect(run.status).toBe('completed'); + expect(sendMessage).toHaveBeenCalledWith(expect.objectContaining({ content: 'On my way' })); + const card = await inst.send('UI_PHONE_CARD', { record: { properties: { phone: '+15125550142' } } }); + expect(card.view.component).toBe('Stack'); + }); + + it('per-module webhook URLs: an installation URL for installation-scope modules, the app URL for app-scope ones', async () => { + process.env.FRIGG_PUBLIC_URL = 'https://api.example.com'; + const inst = shared([example('phone-hubspot')])(pin(example('phone-hubspot')), 'int_9'); + await inst.initialize(); + expect(inst.installation().webhookUrls).toEqual({ + phone: 'https://api.example.com/api/data-defined-integration/webhooks/int_9?module=phone', + hubspot: 'https://api.example.com/api/data-defined-integration/webhooks?module=hubspot', + }); + delete process.env.FRIGG_PUBLIC_URL; + }); +}); + +describe('the default app-level lookup', () => { + it('keeps only installations of this class: one account can be used by integrations of other types too', async () => { + jest.resetModules(); + jest.doMock('../../../application/commands/integration-commands', () => ({ + createIntegrationCommands: () => ({ + listIntegrationsByEntityExternalId: async (externalId, moduleName) => (externalId === '12345678' && moduleName === 'hubspot' ? ['int_dd', 'int_other', 'int_gone'] : []), + loadIntegrationContextById: async (id) => ({ int_dd: { context: { record: { config: { type: 'data-defined' } } } }, int_other: { context: { record: { config: { type: 'hubspot-native' } } } } }[id] || { error: 404 }), + }), + })); + const dd = require('..'); + const Cls = dd.defineSharedIntegrationClass({ definitions: dd.memoryDefinitions([example('hubspot-slack')]), manifests, modules: { hubspot: {}, slack: {} }, IntegrationBase: FakeBase, runs: false }); + const dry = new Cls(); + await dry.initialize(); + const out = await dry.onWebhook({ data: { body: [{ eventId: 1, subscriptionType: 'deal.creation', portalId: 12345678, objectId: '1' }], headers: {}, query: { module: 'hubspot' } } }); + expect(out).toEqual({ queued: 1 }); + expect(dry.queued.map((m) => m.integrationId)).toEqual(['int_dd']); + jest.dontMock('../../../application/commands/integration-commands'); + }); +}); diff --git a/packages/core/integrations/data-defined/transforms.js b/packages/core/integrations/data-defined/transforms.js new file mode 100644 index 000000000..1213ca45e --- /dev/null +++ b/packages/core/integrations/data-defined/transforms.js @@ -0,0 +1,91 @@ +/** Field transforms used by sync mappings and the `map` step (v0.1). Pure functions: (value, options, record) -> value. */ +const TRANSFORMS = { + none: (v) => v, + trim: (v) => (typeof v === 'string' ? v.trim() : v), + lowercase: (v) => (typeof v === 'string' ? v.toLowerCase() : v), + uppercase: (v) => (typeof v === 'string' ? v.toUpperCase() : v), + number: (v) => (v == null || v === '' ? undefined : Number(v)), + boolean: (v) => (v == null ? undefined : v === true || v === 'true' || v === 1 || v === '1'), + to_iso_date: (v) => { if (!v) return undefined; const d = new Date(v); return isNaN(d) ? undefined : d.toISOString(); }, + /** Digits with a leading +. A 10-digit number is assumed to be NANP (options.defaultCountry overrides). */ + to_e164_phone: (v, o = {}) => { + if (v == null || v === '') return undefined; + const s = String(v).trim(); + const digits = s.replace(/\D/g, ''); + if (!digits) return undefined; + if (s.startsWith('+')) return '+' + digits; + if (digits.length === 10) return (o.defaultCountry === 'GB' ? '+44' : '+1') + digits; + if (digits.length === 11 && digits.startsWith('1')) return '+' + digits; + return '+' + digits; + }, + split_full_name: (v, o = {}) => { + if (!v) return undefined; + const parts = String(v).trim().split(/\s+/); + return o.part === 'last' ? parts.slice(1).join(' ') || undefined : parts[0]; + }, + join: (v, o = {}, record = {}) => (o.fields || []).map((f) => readField(record, f)).filter((x) => x != null && x !== '').join(o.separator ?? ' ') || undefined, + map_values: (v, o = {}) => (v == null ? undefined : (o.values || {})[v] ?? o.fallback), + default: (v, o = {}) => (v == null || v === '' ? o.value : v), + // money and time, rounded and total (bad input -> undefined, which the mapping leaves out) + units_to_cents: (v) => { const n = v == null || v === '' ? NaN : Number(v); return Number.isNaN(n) ? undefined : Math.round(Number(`${n}e2`)); }, + cents_to_units: (v) => { const n = v == null || v === '' ? NaN : Number(v); return Number.isNaN(n) ? undefined : Math.round(n) / 100; }, + from_unix_seconds: (v) => { const n = v == null || v === '' ? NaN : Number(v); return Number.isNaN(n) ? undefined : new Date(n * 1000).toISOString(); }, + to_unix_seconds: (v) => { const t = v ? Date.parse(v) : NaN; return Number.isNaN(t) ? undefined : Math.floor(t / 1000); }, +}; + +function readField(record, path) { + let cur = record; + for (const seg of String(path).split(/\.|(?=\[)/)) { + if (cur == null) return undefined; + const m = /^\[(\d+)\]$/.exec(seg); + cur = m ? cur[Number(m[1])] : cur[seg]; + } + return cur; +} + +function writeField(target, path, value) { + const segs = String(path).split(/\.|(?=\[)/); + let cur = target; + segs.forEach((seg, i) => { + const m = /^\[(\d+)\]$/.exec(seg); + const key = m ? Number(m[1]) : seg; + if (i === segs.length - 1) { cur[key] = value; return; } + const nextIsIndex = /^\[\d+\]$/.test(segs[i + 1]); + if (cur[key] == null) cur[key] = nextIsIndex ? [] : {}; + cur = cur[key]; + }); + return target; +} + +/** + * Apply a field mapping to a record. A blank result is left out, never written (a blank must not overwrite a value). + * A row either reads `source` from the record or writes a fixed `value` (QuickBooks' Line[0].DetailType, a lifecycle + * stage): { target, value }. A fixed value still goes through the row's transform. + * @param {Array<{source?:string,value?:any,target:string,transform?:string,options?:object}>} rows + */ +function applyMapping(rows, record) { + const out = {}; + for (const r of rows || []) { + const fn = TRANSFORMS[r.transform || 'none']; + if (!fn) throw new Error(`Unknown transform "${r.transform}"`); + const input = 'value' in r ? r.value : readField(record, r.source); + const v = fn(input, r.options || {}, record); + if (v === undefined || v === null || v === '') continue; + writeField(out, r.target, v); + } + return out; +} + +/** Deep merge, later objects win; arrays and scalars are replaced, not merged. Inputs are not modified. */ +function deepMerge(...objects) { + const out = {}; + for (const o of objects) { + if (!o || typeof o !== 'object' || Array.isArray(o)) continue; + for (const [k, v] of Object.entries(o)) { + out[k] = v && typeof v === 'object' && !Array.isArray(v) && out[k] && typeof out[k] === 'object' && !Array.isArray(out[k]) ? deepMerge(out[k], v) : v; + } + } + return out; +} + +module.exports = { TRANSFORMS, applyMapping, readField, writeField, deepMerge }; diff --git a/packages/core/integrations/data-defined/use-cases/definition-use-cases.js b/packages/core/integrations/data-defined/use-cases/definition-use-cases.js new file mode 100644 index 000000000..beb35d3d1 --- /dev/null +++ b/packages/core/integrations/data-defined/use-cases/definition-use-cases.js @@ -0,0 +1,187 @@ +/** + * Application layer for data-defined integrations (ADR-063): publishing definition versions, the catalogue the + * runtime reads from, and per-installation upgrades. Business rules live here; the repositories only store rows. + * + * createDefinitionCatalog() what defineSharedIntegrationClass({ definitions }) reads: get(id, version), latest(id) + * PublishDefinitionVersion validate (schema + references) → checkBump against the latest published → store + * DeprecateDefinitionVersion PUBLISHED → DEPRECATED (still runs for its pins, cannot be newly installed) + * ListPublishedDefinitions the install catalogue: latest published version of each definition + * GetDefinitionHistory every version of one definition, newest first + * PlanInstallationUpgrade planUpgrade(installation, pinned, target), without changing anything + */ +const { checksum } = require('../canonical'); +const { checkBump, planUpgrade } = require('../versioning'); + +class DefinitionError extends Error { + constructor(message, { status = 400, code, details } = {}) { super(message); this.name = 'DefinitionError'; this.status = status; this.code = code; this.details = details; } +} + +const SEMVER = /^(\d+)\.(\d+)\.(\d+)(?:-([\w.]+))?$/; +function compareVersions(a, b) { + const x = SEMVER.exec(a), y = SEMVER.exec(b); + if (!x || !y) return String(a).localeCompare(String(b)); + for (let i = 1; i <= 3; i += 1) if (Number(x[i]) !== Number(y[i])) return Number(x[i]) - Number(y[i]); + if (x[4] && !y[4]) return -1; // 1.0.0-beta < 1.0.0 + if (!x[4] && y[4]) return 1; + return String(x[4] || '').localeCompare(String(y[4] || '')); +} +const newestFirst = (rows) => [...rows].sort((a, b) => compareVersions(b.version, a.version)); +const SERVABLE = new Set(['PUBLISHED', 'DEPRECATED', 'RETIRED']); // a DRAFT never reaches the runtime + +/** + * The definition store the runtime reads from. Published rows are immutable, so they are cached by id@version for the + * life of the process; a pinned checksum is verified, so an installation provably runs the document it was pinned to. + */ +function createDefinitionCatalog({ repository } = {}) { + const repo = () => repository || (repository = require('../repositories/definition-version-repository').createDefinitionVersionRepository()); + const cache = new Map(); + return { + async get(definitionId, version, { checksum: expected } = {}) { + const key = `${definitionId}@${version}`; + let row = cache.get(key); + if (!row) { + row = await repo().findVersion(definitionId, version); + if (!row || !SERVABLE.has(row.status)) return null; + cache.set(key, row); + } + if (expected && expected !== row.checksum) throw new DefinitionError(`${key} does not match the pinned checksum`, { status: 409, code: 'CHECKSUM_MISMATCH' }); + return row.document; + }, + /** The latest PUBLISHED version (not deprecated, not draft): what a new installation pins. */ + async latest(definitionId) { + const rows = (await repo().listVersions(definitionId)).filter((r) => r.status === 'PUBLISHED'); + return rows.length ? newestFirst(rows)[0].document : null; + }, + async checksumOf(definitionId, version) { + return (await repo().findVersion(definitionId, version))?.checksum || null; + }, + /** The install catalogue: latest published version of each definition. */ + list: () => new ListPublishedDefinitions({ repository: repo() }).execute(), + /** The repository behind this catalogue, for publishing (bundled stores have none). */ + get repository() { return repo(); }, + }; +} + +class PublishDefinitionVersion { + /** + * @param {{ repository, manifests: object, check?: (def, manifests) => { valid, errors, warnings } }} deps + * check defaults to schema-check's checkDefinition (JSON Schema + references) + */ + constructor({ repository, manifests, check }) { + this.repository = repository; + this.manifests = manifests; + this.check = check || require('../schema-check').checkDefinition; + } + + /** @returns {Promise<{ version: object, created: boolean, changeLevel: string|null, warnings: array }>} */ + async execute({ document, notes, createdBy, publish = true, strict = false } = {}) { + if (!document || typeof document !== 'object') throw new DefinitionError('Send the definition as `document`'); + const result = this.check(document, this.manifests); + if (!result.valid || (strict && result.warnings.length)) throw new DefinitionError(`${document.id}@${document.version} is not valid`, { code: 'INVALID', details: { errors: result.errors, warnings: result.warnings } }); + + const sum = checksum(document); + const existing = await this.repository.findVersion(document.id, document.version); + if (existing && existing.status !== 'DRAFT') { + if (existing.checksum === sum) return { version: existing, created: false, changeLevel: existing.changeLevel, warnings: result.warnings }; + throw new DefinitionError(`${document.id}@${document.version} is already ${existing.status.toLowerCase()} and cannot change. Publish a new version.`, { status: 409, code: 'IMMUTABLE' }); + } + + const published = newestFirst((await this.repository.listVersions(document.id)).filter((r) => r.status === 'PUBLISHED' || r.status === 'DEPRECATED')); + const previous = published.find((r) => r.version !== document.version) || null; + let changeLevel = null; + if (previous) { + if (compareVersions(document.version, previous.version) <= 0) throw new DefinitionError(`${document.version} is not newer than the latest published ${previous.version}`, { status: 409, code: 'NOT_NEWER' }); + const bump = checkBump(previous.document, document); + if (!bump.ok) throw new DefinitionError(bump.message, { status: 409, code: 'BUMP_TOO_SMALL', details: { required: bump.required, declared: bump.declared, changes: bump.changes } }); + changeLevel = bump.required; + } + + const row = { + definitionId: document.id, version: document.version, schemaVersion: document.schemaVersion, checksum: sum, document, + changeLevel, previousVersion: previous?.version || null, notes: notes ?? null, createdBy: createdBy ?? null, + status: publish ? 'PUBLISHED' : 'DRAFT', publishedAt: publish ? new Date() : null, + }; + if (existing) { + // A draft may be rewritten, and published, as long as it is still a draft when we write (checked by the + // database, so two concurrent publishes cannot both win). + const updated = await this.repository.updateIfStatus(document.id, document.version, 'DRAFT', row); + if (!updated) throw new DefinitionError(`${document.id}@${document.version} changed while publishing; try again`, { status: 409, code: 'CONFLICT' }); + return { version: updated, created: false, changeLevel, warnings: result.warnings }; + } + try { + return { version: await this.repository.createVersion(row), created: true, changeLevel, warnings: result.warnings }; + } catch (e) { + if (e?.code === 'P2002') { + // Lost a race to the same version: the same document is the same publish; anything else is a conflict. + const winner = await this.repository.findVersion(document.id, document.version); + if (winner?.checksum === sum) return { version: winner, created: false, changeLevel: winner.changeLevel, warnings: result.warnings }; + throw new DefinitionError(`${document.id}@${document.version} was published with different content just now`, { status: 409, code: 'CONFLICT' }); + } + throw e; + } + } +} + +class DeprecateDefinitionVersion { + constructor({ repository }) { this.repository = repository; } + async execute({ definitionId, version }) { + const row = await this.repository.updateIfStatus(definitionId, version, 'PUBLISHED', { status: 'DEPRECATED' }); + if (row) return row; + const existing = await this.repository.findVersion(definitionId, version); + if (!existing) throw new DefinitionError(`${definitionId}@${version} does not exist`, { status: 404, code: 'NOT_FOUND' }); + throw new DefinitionError(`${definitionId}@${version} is ${existing.status.toLowerCase()}; only a published version can be deprecated`, { status: 409, code: 'WRONG_STATUS' }); + } +} + +class ListPublishedDefinitions { + constructor({ repository }) { this.repository = repository; } + /** Latest published version of each definition, with what an install screen shows. */ + async execute() { + const byId = new Map(); + for (const r of await this.repository.listByStatus(['PUBLISHED'])) { + const cur = byId.get(r.definitionId); + if (!cur || compareVersions(r.version, cur.version) > 0) byId.set(r.definitionId, r); + } + const out = []; + for (const r of byId.values()) { + const doc = (await this.repository.findVersion(r.definitionId, r.version)).document; + out.push({ + id: r.definitionId, version: r.version, checksum: r.checksum, publishedAt: r.publishedAt, + display: doc.display, modules: Object.fromEntries(Object.entries(doc.modules || {}).map(([k, m]) => [k, m.module])), + counts: { syncs: (doc.syncs || []).length, links: (doc.links || []).length, workflows: (doc.workflows || []).length, ui: (doc.ui || []).length }, + }); + } + return out.sort((a, b) => String(a.display?.label || a.id).localeCompare(String(b.display?.label || b.id))); + } +} + +class GetDefinitionHistory { + constructor({ repository }) { this.repository = repository; } + async execute({ definitionId }) { + const rows = newestFirst(await this.repository.listVersions(definitionId)); + if (!rows.length) throw new DefinitionError(`No definition "${definitionId}"`, { status: 404, code: 'NOT_FOUND' }); + return rows.map(({ document, ...meta }) => meta); + } +} + +class PlanInstallationUpgrade { + constructor({ catalog }) { this.catalog = catalog; } + /** @returns {Promise<{ from, to, allowed, needs, actions, config, changes }>} */ + async execute({ config, toVersion }) { + const pin = config?.definition; + if (!pin?.id || !pin?.version) throw new DefinitionError('This installation does not name a definition', { code: 'NO_PIN' }); + const from = await this.catalog.get(pin.id, pin.version); + const target = toVersion || (await this.catalog.latest(pin.id))?.version; + if (!target) throw new DefinitionError(`No published version of "${pin.id}"`, { status: 404, code: 'NOT_FOUND' }); + const to = await this.catalog.get(pin.id, target); + if (!from || !to) throw new DefinitionError(`${pin.id}@${!from ? pin.version : target} is not published`, { status: 404, code: 'NOT_FOUND' }); + const plan = planUpgrade(config, from, to); + plan.config.definition = { ...plan.config.definition, checksum: await this.catalog.checksumOf(pin.id, target) }; + return { from: pin.version, to: target, ...plan, changes: require('../versioning').diff(from, to).changes }; + } +} + +module.exports = { + DefinitionError, compareVersions, createDefinitionCatalog, + PublishDefinitionVersion, DeprecateDefinitionVersion, ListPublishedDefinitions, GetDefinitionHistory, PlanInstallationUpgrade, +}; diff --git a/packages/core/integrations/data-defined/validate.js b/packages/core/integrations/data-defined/validate.js new file mode 100644 index 000000000..191ac1885 --- /dev/null +++ b/packages/core/integrations/data-defined/validate.js @@ -0,0 +1,370 @@ +/** + * Validate a definition (v0.1) against the capability manifests of the modules it uses. + * + * Structural rules mirror schema/integration-definition.v0.1.schema.json (kept dependency-free so core needs no + * JSON-schema library). Semantic rules are the ones a schema cannot express: every step names a capability the + * module declares and passes its required arguments; $.steps.x refers to an earlier step; $.settings.y is declared; + * trigger topics are events the module emits; UI placements are ones the host supports. + * + * @param {object} def + * @param {Record} manifests manifest name -> manifest (capabilities/*.capabilities.json) + * @returns {{ valid: boolean, errors: Array<{path:string,message:string}>, warnings: Array<{path:string,message:string}> }} + */ +const { BUILTINS, isBuiltin, splitCapability } = require('./steps'); +const { parseCondition, referencedPaths, templateTags, FILTERS } = require('./expressions'); +const { TRANSFORMS: TRANSFORM_FNS } = require('./transforms'); + +const ID = /^[a-z][a-z0-9-]*$/; +const SEMVER = /^\d+\.\d+\.\d+(-[\w.]+)?$/; +const ROOTS = new Set(['event', 'record', 'steps', 'settings', 'install', 'vars', 'item', 'index', 'data', 'row', 'now']); +const COMPONENTS = new Set(['Stack', 'Heading', 'Text', 'KeyValue', 'List', 'Table', 'Badge', 'Button', 'Empty', 'Divider']); +const TRIGGERS = new Set(['webhook', 'schedule', 'userAction', 'syncRecord']); +const TRANSFORMS = new Set(Object.keys(TRANSFORM_FNS)); +const INSTALL_KEYS = new Set(['webhookUrl', 'webhookUrls', 'portalId', 'runtime', 'integrationId', 'accounts']); + +function validate(def, manifests = {}) { + const errors = [], warnings = []; + const err = (path, message) => errors.push({ path, message }); + const warn = (path, message) => warnings.push({ path, message }); + + if (!def || typeof def !== 'object') return { valid: false, errors: [{ path: '', message: 'Definition must be an object' }], warnings }; + if (def.schemaVersion !== '0.1') err('schemaVersion', 'Must be "0.1"'); + if (!ID.test(def.id || '')) err('id', 'Lowercase letters, digits and dashes, starting with a letter'); + if (!SEMVER.test(def.version || '')) err('version', 'Must be a semantic version (1.4.0)'); + if (!def.display?.label) err('display.label', 'Required'); + + // ---------- modules ---------- + const mods = def.modules || {}; + if (!Object.keys(mods).length) err('modules', 'At least one module is required'); + const manifestOf = {}; + for (const [key, m] of Object.entries(mods)) { + if (!m?.module) { err(`modules.${key}`, 'Needs a module name'); continue; } + if (!manifests[m.module]) err(`modules.${key}`, `No capability manifest for module "${m.module}"`); + else manifestOf[key] = manifests[m.module]; + } + + // ---------- ids are unique across kinds (they become event names) ---------- + const seen = new Map(); + for (const kind of ['syncs', 'links', 'workflows', 'queries', 'ui']) { + (def[kind] || []).forEach((x, i) => { + if (!ID.test(x?.id || '')) err(`${kind}[${i}].id`, 'Lowercase letters, digits and dashes, starting with a letter'); + else if (seen.has(x.id)) err(`${kind}[${i}].id`, `Duplicate id "${x.id}" (also ${seen.get(x.id)})`); + else seen.set(x.id, `${kind}[${i}]`); + }); + } + const installKeys = new Set([...Object.keys(def.settings || {}), ...INSTALL_KEYS]); + let warnedWebhookUrl = false; + const installStepIds = new Set((def.lifecycle?.onInstall || []).map((st) => st?.id).filter(Boolean)); + const syncs = Object.fromEntries((def.syncs || []).map((s) => [s.id, s])); + const linkSets = Object.fromEntries((def.links || []).map((l) => [l.id, l])); + const workflows = Object.fromEntries((def.workflows || []).map((w) => [w.id, w])); + const queries = Object.fromEntries((def.queries || []).map((q) => [q.id, q])); + const uis = Object.fromEntries((def.ui || []).map((u) => [u.id, u])); + + // ---------- settings ---------- + const checkSettings = (settings, path) => { + for (const [k, s] of Object.entries(settings || {})) { + const p = `${path}.${k}`; + if (!s?.label) err(p, 'Setting needs a label'); + if (!['string', 'text', 'number', 'boolean', 'enum', 'option', 'duration'].includes(s?.type)) err(p, `Unknown setting type "${s?.type}"`); + if (s?.type === 'enum' && !Array.isArray(s.enum)) err(p, 'enum settings need an enum list'); + if (s?.type === 'option') checkOptionSource(s.options, `${p}.options`, k, settings, path !== 'settings'); + } + // An option list may read earlier choices; a loop (a reads b, b reads a) could never be filled in. + const reads = (k) => Object.values(optionInputs(settings?.[k])).flatMap((v) => referencedPaths(v)).map((r) => r.split('.')).filter((x) => x[1] === 'settings').map((x) => x[2]); + for (const k of Object.keys(settings || {})) { + const seenK = new Set([k]); const stack = [...reads(k)]; + while (stack.length) { const n = stack.pop(); if (n === k) { err(`${path}.${k}.options`, `Option inputs form a loop through "${k}"`); break; } if (!seenK.has(n)) { seenK.add(n); stack.push(...reads(n)); } } + } + }; + const optionInputs = (s) => (s && typeof s.options === 'object' ? s.options.with || {} : {}); + function checkOptionSource(opts, p, self, siblings, isWorkflow) { + const ref = typeof opts === 'string' ? { source: opts } : opts; + if (!ref || typeof ref.source !== 'string') { err(p, 'options is "module.list" or { source: "module.list", with: { ... } }'); return; } + const [mk, ok] = ref.source.split('.'); + if (!manifestOf[mk]) { err(p, `options "${ref.source}" names an unknown module key`); return; } + const list = manifestOf[mk].options?.[ok]; + if (!list) { err(p, `Module "${mk}" has no option list "${ok}"`); return; } + const inputs = list.inputs || {}; + for (const [name, spec] of Object.entries(inputs)) if (spec.required && !(name in (ref.with || {}))) err(`${p}.with.${name}`, `Option list ${ref.source} needs input "${name}"`); + for (const [name, v] of Object.entries(ref.with || {})) { + if (!(name in inputs)) { err(`${p}.with.${name}`, `Option list ${ref.source} takes no input "${name}"`); continue; } + for (const r of referencedPaths(v)) { + const [, root, key] = r.split(/\.|\[/); + if (root === 'settings' && key === self) err(`${p}.with.${name}`, 'An option list cannot read its own setting'); + else if (root === 'settings' && !(key in (siblings || {}))) err(`${p}.with.${name}`, `$.settings.${key} is not a setting at this level`); + else if (root === 'install' && !(isWorkflow && key in (def.settings || {}))) err(`${p}.with.${name}`, `$.install.${key} is not an integration setting`); + else if (root !== 'settings' && root !== 'install') err(`${p}.with.${name}`, 'Option inputs can only read $.settings (or $.install from a workflow setting)'); + } + } + } + checkSettings(def.settings, 'settings'); + + const objectFields = (key, object) => (manifestOf[key]?.objects?.[object]?.fields || []).map((f) => f.name); + const splitRef = (ref) => { const [k, o] = String(ref || '').split('.'); return { k, o }; }; + + // ---------- link sets: links between records with no sync behind them ---------- + (def.links || []).forEach((l, i) => { + const p = `links[${i}]`; + if (!Array.isArray(l.between) || l.between.length !== 2) { err(`${p}.between`, 'Two "module.object" references'); return; } + l.between.forEach((ref, j) => { + const r = splitRef(ref); + if (!manifestOf[r.k]) err(`${p}.between[${j}]`, `Unknown module key "${r.k}"`); + else if (!manifestOf[r.k].objects?.[r.o]) err(`${p}.between[${j}]`, `Module "${r.k}" has no object "${r.o}"`); + }); + if (splitRef(l.between[0]).k === splitRef(l.between[1]).k) err(`${p}.between`, 'A link set joins two different modules'); + (l.identity?.match || []).forEach((m, j) => { + if (!m.key) err(`${p}.identity.match[${j}].key`, 'Name the identity key (email, phone)'); + if (m.transform && !TRANSFORMS.has(m.transform)) err(`${p}.identity.match[${j}].transform`, `Unknown transform "${m.transform}"`); + }); + }); + const checkRows = (rows, p, sf, tf, srcName, tgtName, tgtFields) => (rows || []).forEach((r, j) => { + const q = `${p}[${j}]`; + const fixed = r && 'value' in r; + if (fixed && r.source != null) err(q, 'A row has a source or a fixed value, not both'); + else if (!fixed && !r?.source) err(q, 'A row needs a source or a fixed value'); + if (!fixed && sf?.length && !sf.includes(r.source)) err(`${q}.source`, `"${r.source}" is not a field of ${srcName}`); + if (tf?.length && !tf.includes(r.target)) err(`${q}.target`, `"${r.target}" is not a field of ${tgtName}`); + const tfield = tgtFields?.find((f) => f.name === r.target); + if (tfield?.readOnly && r.direction !== 'to_source') err(`${q}.target`, `"${r.target}" is read-only in ${tgtName}`); + if (fixed && r.direction === 'to_source') err(`${q}.direction`, 'A fixed value only writes to the target'); + if (r.transform && !TRANSFORMS.has(r.transform)) err(`${q}.transform`, `Unknown transform "${r.transform}"`); + }); + + // ---------- syncs ---------- + (def.syncs || []).forEach((s, i) => { + const p = `syncs[${i}]`; + if (!s.label) err(`${p}.label`, 'Required'); + if (!['one_way', 'two_way'].includes(s.direction)) err(`${p}.direction`, 'one_way or two_way'); + const src = splitRef(s.source), tgt = splitRef(s.target); + for (const [side, r] of [['source', src], ['target', tgt]]) { + if (!manifestOf[r.k]) err(`${p}.${side}`, `Unknown module key "${r.k}"`); + else if (!manifestOf[r.k].objects?.[r.o]) err(`${p}.${side}`, `Module "${r.k}" has no object "${r.o}"`); + } + const sf = objectFields(src.k, src.o), tf = objectFields(tgt.k, tgt.o); + const match = s.identity?.match || []; + if (!match.length) err(`${p}.identity.match`, 'At least one identity key is required'); + match.forEach((m, j) => { + if (!m.key) err(`${p}.identity.match[${j}].key`, 'Name the identity key (email, phone)'); + if (sf.length && !sf.includes(m.source)) err(`${p}.identity.match[${j}].source`, `"${m.source}" is not a field of ${s.source}`); + if (tf.length && !tf.includes(m.target)) err(`${p}.identity.match[${j}].target`, `"${m.target}" is not a field of ${s.target}`); + }); + checkRows(s.mapping, `${p}.mapping`, sf, tf, s.source, s.target, manifestOf[tgt.k]?.objects?.[tgt.o]?.fields); + if (s.ongoing?.mode === 'webhook') { + const hasEvents = (k, o) => Object.values(manifestOf[k]?.events || {}).some((e) => e.object === o); + if (manifestOf[src.k] && !hasEvents(src.k, src.o)) warn(`${p}.ongoing`, `${s.source} declares no webhook events; ongoing sync will need polling`); + } + if (s.filter) checkCondition(s.filter, `${p}.filter`); + }); + + // ---------- steps ---------- + function checkCondition(src, path) { + try { parseCondition(src); } catch (e) { err(path, e.message); return; } + checkRefs(src, path, null); + } + function checkFilters(value, path) { + for (const t of templateTags(value)) for (const f of t.filters) if (!FILTERS[f.name]) err(path, `Unknown template filter "${f.name}" (${Object.keys(FILTERS).join(', ')})`); + } + function checkRefs(value, path, scope) { + checkFilters(value, path); + for (const ref of referencedPaths(value)) { + const segs = ref.replace(/^\$\./, '').split(/\.|\[/); + const root = segs[0]; + if (!ROOTS.has(root)) { err(path, `Unknown root "$.${root}" (use event, record, steps, settings, install, vars, item, data)`); continue; } + if (!scope) { if (root === 'install') checkInstallRef(segs, path, null); continue; } + if (root === 'steps' && segs[1] && !scope.steps.has(segs[1])) err(path, `$.steps.${segs[1]} is not an earlier step in this list`); + if (root === 'settings' && segs[1] && !scope.settings.has(segs[1])) err(path, `$.settings.${segs[1]} is not a setting of this workflow`); + if (root === 'install') checkInstallRef(segs, path, scope); + if ((root === 'item' || root === 'index') && !scope.inLoop) err(path, `$.${root} is only available inside foreach`); + } + } + // $.install. · $.install.accounts.. + // $.install.runtime.installed. (not readable while install itself is running) + function checkInstallRef(segs, path, scope) { + const [, key, a, b] = segs; + if (!key) return; + if (!installKeys.has(key)) { err(path, `$.install.${key} is not an integration setting`); return; } + if (key === 'accounts' && a) { + if (!manifestOf[a]) err(path, `$.install.accounts.${a}: "${a}" is not a module key`); + else if (b && !(manifestOf[a].account?.fields || []).some((f) => f.name === b)) err(path, `$.install.accounts.${a}.${b}: ${manifestOf[a].module} declares no account field "${b}" (${(manifestOf[a].account?.fields || []).map((f) => f.name).join(', ') || 'none'})`); + } + if (key === 'webhookUrls' && a && !manifestOf[a]) err(path, `$.install.webhookUrls.${a}: "${a}" is not a module key`); + if (key === 'webhookUrl' && Object.keys(mods).length > 1 && !warnedWebhookUrl) { + warnedWebhookUrl = true; + warn(path, 'Use $.install.webhookUrls.: with more than one module, the URL should say which module a delivery comes from'); + } + if (key === 'runtime' && a) { + if (a !== 'installed') err(path, `$.install.runtime.${a}: runtime only holds "installed" (the results of onInstall steps)`); + else if (scope?.installing) err(path, 'Install steps cannot read $.install.runtime.installed; refer to an earlier step with $.steps'); + else if (b && !installStepIds.has(b)) err(path, `$.install.runtime.installed.${b}: no onInstall step has id "${b}"`); + } + } + function checkSteps(steps, path, scope) { + if (!Array.isArray(steps)) { err(path, 'Steps must be a list'); return; } + steps.forEach((st, i) => { + const p = `${path}[${i}]`; + if (!st?.use) { err(p, 'Step needs "use"'); return; } + if (st.when) checkCondition(st.when, `${p}.when`), checkRefs(st.when, `${p}.when`, scope); + if (st.onError && !['fail', 'continue', 'stop'].includes(st.onError)) err(`${p}.onError`, 'fail, continue or stop'); + const args = st.with || {}; + if (st.use === 'foreach') { + checkRefs(args.over, `${p}.with.over`, scope); + const inner = { ...scope, steps: new Set(scope.steps), declared: new Set(), inLoop: true }; + if (args.where) { checkCondition(args.where, `${p}.with.where`); checkRefs(args.where, `${p}.with.where`, inner); } + if (!st.do && !('collect' in args)) err(p, 'foreach needs do steps, collect, or both'); + if (st.do) checkSteps(st.do, `${p}.do`, inner); + if ('collect' in args) checkRefs(args.collect, `${p}.with.collect`, inner); + if (args.max != null && !(Number.isInteger(args.max) && args.max > 0 && args.max <= 1000)) err(`${p}.with.max`, 'A whole number from 1 to 1000'); + } else if (st.use === 'branch') { + (st.cases || []).forEach((c, j) => { checkCondition(c.when, `${p}.cases[${j}].when`); checkRefs(c.when, `${p}.cases[${j}].when`, scope); checkSteps(c.do, `${p}.cases[${j}].do`, { ...scope, steps: new Set(scope.steps) }); }); + if (st.else) checkSteps(st.else, `${p}.else`, { ...scope, steps: new Set(scope.steps) }); + } else if (isBuiltin(st.use)) { + checkRefs(args, `${p}.with`, scope); + if (st.use.startsWith('links.')) { + const isLink = 'link' in args; + if (isLink === ('sync' in args)) err(`${p}.with`, `${st.use} names a link set (link) or a sync (sync), exactly one`); + const set = isLink ? linkSets[args.link] : syncs[args.sync]; + if (!set) err(`${p}.with.${isLink ? 'link' : 'sync'}`, `Unknown ${isLink ? 'link set' : 'sync'} "${isLink ? args.link : args.sync}"`); + else { + const keys = new Set((set.identity?.match || []).map((m) => m.key)); + const mods = new Set((isLink ? set.between : [set.source, set.target]).map((r) => splitRef(r).k)); + const label = `${isLink ? 'link set' : 'sync'} "${set.id}"`; + for (const k of Object.keys((st.use === 'links.find' ? args.key : args.keys) || {})) if (!keys.has(k)) err(`${p}.with.${st.use === 'links.find' ? 'key' : 'keys'}`, `"${k}" is not an identity key of ${label}${keys.size ? ` (${[...keys].join(', ')})` : ''}`); + if (st.use === 'links.find' && !args.key && !(args.from && args.id)) err(`${p}.with`, 'links.find needs key, or from + id'); + if (args.from && !mods.has(args.from)) err(`${p}.with.from`, `"${args.from}" is not one of the modules ${label} links`); + for (const m of Object.keys(args.ids || {})) if (!mods.has(m)) err(`${p}.with.ids.${m}`, `"${m}" is not one of the modules ${label} links`); + } + } + if (st.use === 'merge' && !(Array.isArray(args.objects) || (typeof args.objects === 'string' && args.objects.startsWith('$.')))) err(`${p}.with.objects`, 'A list of objects (or a reference to one)'); + if (st.use === 'map' && Array.isArray(args.mapping)) checkRows(args.mapping, `${p}.with.mapping`); + if (st.use === 'template' && args.id && !def.templates?.[args.id]) err(`${p}.with.id`, `Unknown template "${args.id}"`); + if (st.use === 'map' && args.sync && !syncs[args.sync]) err(`${p}.with.sync`, `Unknown sync "${args.sync}"`); + if (st.use === 'code' && !/^[@\w/.-]+#\w+@\d+\.\d+\.\d+$/.test(args.ref || '')) err(`${p}.with.ref`, 'Code steps are "package#function@x.y.z"'); + } else { + const cap = splitCapability(st.use); + const man = cap && manifestOf[cap.module]; + if (!man) err(`${p}.use`, `"${st.use}" is not a built-in, and "${cap?.module}" is not a module key`); + else { + const c = man.capabilities?.[cap.capability]; + if (!c) err(`${p}.use`, `Module "${cap.module}" (${man.module}) has no capability "${cap.capability}"`); + else { + for (const [name, spec] of Object.entries(c.input || {})) if (spec.required && !(name in args)) err(`${p}.with.${name}`, `Required by ${st.use}`); + for (const name of Object.keys(args)) if (c.input && !(name in c.input)) err(`${p}.with.${name}`, `${st.use} takes no argument "${name}"`); + } + } + checkRefs(args, `${p}.with`, scope); + } + if (st.id) { + if (scope.steps.has(st.id) && scope.declared.has(st.id)) err(`${p}.id`, `Step id "${st.id}" is used twice`); + scope.steps.add(st.id); scope.declared.add(st.id); + } + }); + } + const newScope = (settings) => ({ steps: new Set(), declared: new Set(), settings: new Set(Object.keys(settings || {})), install: installKeys, inLoop: false }); + + // ---------- workflows ---------- + (def.workflows || []).forEach((w, i) => { + const p = `workflows[${i}]`; + if (!w.label) err(`${p}.label`, 'Required'); + checkSettings(w.settings, `${p}.settings`); + const t = w.trigger || {}; + if (!TRIGGERS.has(t.type)) err(`${p}.trigger.type`, `One of ${[...TRIGGERS].join(', ')}`); + if (t.type === 'webhook') { + if (!manifestOf[t.module]) err(`${p}.trigger.module`, `Unknown module key "${t.module}"`); + else if (!manifestOf[t.module].events?.[t.topic]) err(`${p}.trigger.topic`, `Module "${t.module}" emits no event "${t.topic}"`); + } + if (t.type === 'schedule' && String(t.cron || '').trim().split(/\s+/).length !== 5) err(`${p}.trigger.cron`, 'Five-field cron expression'); + if (t.type === 'userAction' && t.from && !uis[t.from]) err(`${p}.trigger.from`, `Unknown UI surface "${t.from}"`); + if (t.type === 'syncRecord' && !syncs[t.sync]) err(`${p}.trigger.sync`, `Unknown sync "${t.sync}"`); + if (t.filter) checkCondition(t.filter, `${p}.trigger.filter`); + if (!w.steps?.length) err(`${p}.steps`, 'At least one step'); + checkSteps(w.steps, `${p}.steps`, newScope(w.settings)); + }); + + // ---------- queries ---------- + (def.queries || []).forEach((q, i) => { + const scope = newScope({}); + checkSteps(q.steps || [], `queries[${i}].steps`, scope); + checkRefs(q.returns, `queries[${i}].returns`, scope); + }); + + // ---------- ui ---------- + // A view sees $.data (the query's returns), $.record and $.install; $.item inside a List's title/meta, $.row inside + // a Table column. Anything else is a typo that would render blank, so it is an error here, not at runtime. + function checkViewRefs(value, path, ctx) { + checkFilters(value, path); + for (const ref of referencedPaths(value)) { + const [root, key] = ref.replace(/^\$\./, '').split(/\.|\[/); + const allowed = ['data', 'record', 'install', ...(ctx.item ? ['item'] : []), ...(ctx.row ? ['row'] : [])]; + if (!allowed.includes(root)) err(path, `$.${root} is not available here (a view can read ${allowed.map((r) => '$.' + r).join(', ')})`); + else if (root === 'data' && key && ctx.dataKeys && !ctx.dataKeys.has(key)) err(path, `$.data.${key} is not returned by query "${ctx.query}" (${[...ctx.dataKeys].join(', ')})`); + else if (root === 'install') checkInstallRef(ref.replace(/^\$\./, '').split(/\.|\[/), path, null); + } + } + function checkView(node, path, ctx) { + if (!node || !COMPONENTS.has(node.component)) { err(path, `Unknown component "${node?.component}" (v0.1: ${[...COMPONENTS].join(', ')})`); return; } + if (node.when) { checkCondition(node.when, `${path}.when`); checkViewRefs(node.when, `${path}.when`, ctx); } + const props = { ...(node.props || {}) }; + if (node.component === 'List') { + checkViewRefs(props.items, `${path}.props.items`, ctx); + for (const k of ['title', 'meta']) checkViewRefs(props[k], `${path}.props.${k}`, { ...ctx, item: true }); + delete props.items; delete props.title; delete props.meta; + } else if (node.component === 'Table') { + checkViewRefs(props.rows, `${path}.props.rows`, ctx); + (props.columns || []).forEach((c, j) => checkViewRefs(c.value, `${path}.props.columns[${j}].value`, { ...ctx, row: true })); + delete props.rows; delete props.columns; + } + const { action, ...rest } = props; + checkViewRefs(rest, `${path}.props`, ctx); + if (action?.url) checkViewRefs(action.url, `${path}.props.action.url`, ctx); + const a = node.props?.action; + if (a?.type === 'userAction') { + const w = workflows[a.workflow]; + if (!w) err(`${path}.props.action.workflow`, `Unknown workflow "${a.workflow}"`); + else if (w.trigger?.type !== 'userAction') err(`${path}.props.action.workflow`, `Workflow "${a.workflow}" is not triggered by a user action`); + } else if (a && a.type !== 'open_url') err(`${path}.props.action.type`, 'userAction or open_url'); + (node.children || []).forEach((c, j) => checkView(c, `${path}.children[${j}]`, ctx)); + } + (def.ui || []).forEach((u, i) => { + const p = `ui[${i}]`; + const host = manifestOf[u.host]; + if (!host) err(`${p}.host`, `Unknown module key "${u.host}"`); + else if (!(host.ui?.placements || []).includes(u.placement)) err(`${p}.placement`, `"${u.placement}" is not a placement ${host.module} supports (${(host.ui?.placements || []).join(', ') || 'none'})`); + const q = queries[u.data?.query]; + if (!q) err(`${p}.data.query`, `Unknown query "${u.data?.query}"`); + const returns = q?.returns; + const dataKeys = returns && typeof returns === 'object' && !Array.isArray(returns) ? new Set(Object.keys(returns)) : null; + checkView(u.view, `${p}.view`, { dataKeys, query: u.data?.query }); + }); + + // ---------- named templates ---------- + // A template renders in the context of the step that uses it, so its paths are checked for root and filter here + // and its $.steps references against each workflow that renders it. + const usedTemplates = new Set(); + const collectTemplateUse = (steps, w) => (steps || []).forEach((st) => { + if (st?.use === 'template' && st.with?.id) usedTemplates.add(st.with.id); + collectTemplateUse(st?.do, w); (st?.cases || []).forEach((c) => collectTemplateUse(c.do, w)); collectTemplateUse(st?.else, w); + }); + (def.workflows || []).forEach((w) => collectTemplateUse(w.steps, w)); + (def.queries || []).forEach((q) => collectTemplateUse(q.steps)); + for (const hook of ['onInstall', 'onUninstall']) collectTemplateUse(def.lifecycle?.[hook]); + for (const [id, text] of Object.entries(def.templates || {})) { + if (typeof text !== 'string') { err(`templates.${id}`, 'A template is a string'); continue; } + checkRefs(text, `templates.${id}`, null); + if (!usedTemplates.has(id)) warn(`templates.${id}`, 'No step renders this template'); + } + + // ---------- lifecycle ---------- + // What install creates, uninstall should undo: an install step whose result onUninstall never reads is probably a + // webhook nobody deletes. A warning, because some install steps create nothing that needs undoing. + const uninstallReads = JSON.stringify(def.lifecycle?.onUninstall || []); + for (const [i, st] of (def.lifecycle?.onInstall || []).entries()) { + if (!st?.id) warn(`lifecycle.onInstall[${i}]`, 'This install step has no id, so its result is not kept and uninstall cannot undo it'); + else if (!uninstallReads.includes(`install.runtime.installed.${st.id}`)) warn(`lifecycle.onInstall[${i}]`, `onUninstall never reads $.install.runtime.installed.${st.id}, so what this step creates is never removed`); + } + for (const hook of ['onInstall', 'onUninstall']) if (def.lifecycle?.[hook]) checkSteps(def.lifecycle[hook], `lifecycle.${hook}`, { ...newScope({}), installing: hook === 'onInstall' }); + + return { valid: errors.length === 0, errors, warnings }; +} + +module.exports = { validate }; diff --git a/packages/core/integrations/data-defined/versioning.js b/packages/core/integrations/data-defined/versioning.js new file mode 100644 index 000000000..5d895c6e9 --- /dev/null +++ b/packages/core/integrations/data-defined/versioning.js @@ -0,0 +1,161 @@ +/** + * Versioning for data-defined integrations. + * + * A published version is immutable and every installation pins one exact version, so a release never changes a + * running customer until that installation is upgraded. Three questions are answered here: + * + * diff(from, to) what changed, and how big each change is (patch / minor / major) + * checkBump(from, to) is the declared version bump at least as big as the changes require? + * planUpgrade(install, a, b) can this installation move from a to b, and what has to happen when it does? + * + * The rule of thumb: patch = behaviour inside the same contract (steps, templates, view, mapping rows); + * minor = additive contract (new workflow, optional setting, new UI); major = anything an installation could be + * relying on goes away or changes meaning (removed workflow, required setting, identity change, new module). + */ +const RANK = { none: 0, patch: 1, minor: 2, major: 3 }; +const max = (a, b) => (RANK[a] >= RANK[b] ? a : b); +const byId = (xs) => Object.fromEntries((xs || []).map((x) => [x.id, x])); +const same = (a, b) => JSON.stringify(a) === JSON.stringify(b); +const optionSource = (o) => (o && typeof o === 'object' ? o.source : o); + +function diffSettings(a = {}, b = {}, path, push) { + for (const k of Object.keys(a)) if (!(k in b)) push('minor', `${path}.${k}`, 'Setting removed (saved values are dropped)'); + for (const [k, s] of Object.entries(b)) { + if (!(k in a)) push(s.required && s.default === undefined ? 'major' : 'minor', `${path}.${k}`, s.required && s.default === undefined ? 'New required setting with no default' : 'New optional setting'); + else if (a[k].type !== s.type || optionSource(a[k].options) !== optionSource(s.options)) push('major', `${path}.${k}`, 'Setting type or option list changed'); + else if (!same(a[k].options, s.options)) push('minor', `${path}.${k}`, 'Option list inputs changed'); + else if (!a[k].required && s.required && s.default === undefined) push('major', `${path}.${k}`, 'Setting became required'); + else if (!same(a[k], s)) push('patch', `${path}.${k}`, 'Setting label, help or default changed'); + } +} + +function diff(from, to) { + const changes = []; + const push = (level, path, message) => changes.push({ level, path, message }); + if (from.id !== to.id) push('major', 'id', 'Different integration'); + + for (const k of Object.keys(from.modules || {})) if (!to.modules?.[k]) push('major', `modules.${k}`, 'Module removed'); + for (const [k, m] of Object.entries(to.modules || {})) { + if (!from.modules?.[k]) push('major', `modules.${k}`, 'New module: installations must connect it'); + else if (from.modules[k].module !== m.module) push('major', `modules.${k}`, 'Module replaced'); + else if (from.modules[k].version !== m.version) push('patch', `modules.${k}`, 'Module version range changed'); + } + diffSettings(from.settings, to.settings, 'settings', push); + + const sa = byId(from.syncs), sb = byId(to.syncs); + for (const id of Object.keys(sa)) if (!sb[id]) push('major', `syncs.${id}`, 'Sync removed'); + for (const [id, s] of Object.entries(sb)) { + const o = sa[id]; + if (!o) { push('minor', `syncs.${id}`, 'New sync'); continue; } + if (o.source !== s.source || o.target !== s.target) push('major', `syncs.${id}`, 'Sync objects changed'); + if (o.direction !== s.direction) push('major', `syncs.${id}.direction`, 'Sync direction changed'); + if (!same(o.identity, s.identity)) push('major', `syncs.${id}.identity`, 'Identity keys changed: existing links must be rebuilt'); + if (!same(o.mapping, s.mapping)) push('patch', `syncs.${id}.mapping`, 'Default mapping changed (installations with their own mapping keep it)'); + if (!same(o.ongoing, s.ongoing) || !same(o.backfill, s.backfill) || o.filter !== s.filter) push('minor', `syncs.${id}`, 'Sync schedule or filter changed'); + } + + const la = byId(from.links), lb = byId(to.links); + for (const id of Object.keys(la)) if (!lb[id]) push('major', `links.${id}`, 'Link set removed: workflows that read it lose their links'); + for (const [id, l] of Object.entries(lb)) { + const o = la[id]; + if (!o) { push('minor', `links.${id}`, 'New link set'); continue; } + if (!same(o.between, l.between)) push('major', `links.${id}.between`, 'Link set joins different objects'); + if (!same(o.identity, l.identity)) push('major', `links.${id}.identity`, 'Identity keys changed: existing links must be rebuilt'); + if (o.label !== l.label || o.description !== l.description) push('patch', `links.${id}`, 'Label or description changed'); + } + + const wa = byId(from.workflows), wb = byId(to.workflows); + for (const id of Object.keys(wa)) if (!wb[id]) push('major', `workflows.${id}`, 'Workflow removed'); + for (const [id, w] of Object.entries(wb)) { + const o = wa[id]; + if (!o) { push('minor', `workflows.${id}`, `New workflow (default ${w.default || 'on'})`); continue; } + if (o.trigger?.type !== w.trigger?.type || o.trigger?.module !== w.trigger?.module) push('major', `workflows.${id}.trigger`, 'Trigger source changed'); + else if (!same(o.trigger, w.trigger)) push('minor', `workflows.${id}.trigger`, 'Trigger topic, schedule or filter changed'); + diffSettings(o.settings, w.settings, `workflows.${id}.settings`, push); + if (!same(o.steps, w.steps)) push('patch', `workflows.${id}.steps`, 'Steps changed'); + if (o.label !== w.label || o.description !== w.description) push('patch', `workflows.${id}`, 'Label or description changed'); + } + + const qa = byId(from.queries), qb = byId(to.queries); + for (const id of new Set([...Object.keys(qa), ...Object.keys(qb)])) if (!same(qa[id], qb[id])) push('patch', `queries.${id}`, qa[id] ? (qb[id] ? 'Query changed' : 'Query removed') : 'New query'); + const ua = byId(from.ui), ub = byId(to.ui); + for (const id of Object.keys(ua)) if (!ub[id]) push('minor', `ui.${id}`, 'UI surface removed'); + for (const [id, u] of Object.entries(ub)) { + if (!ua[id]) push('minor', `ui.${id}`, 'New UI surface'); + else if (ua[id].host !== u.host || ua[id].placement !== u.placement) push('minor', `ui.${id}`, 'UI surface moved'); + else if (!same(ua[id], u)) push('patch', `ui.${id}`, 'UI view or data changed'); + } + if (!same(from.templates, to.templates)) push('patch', 'templates', 'Templates changed'); + if (!same(from.lifecycle, to.lifecycle)) push('minor', 'lifecycle', 'Install/uninstall steps changed: upgrades re-run onInstall'); + if (!same(from.display, to.display)) push('patch', 'display', 'Display text changed'); + + const level = changes.reduce((l, c) => max(l, c.level), 'none'); + return { level, changes }; +} + +const parse = (v) => String(v).split('-')[0].split('.').map(Number); +function bumpOf(fromV, toV) { + const [a, b, c] = parse(fromV), [x, y, z] = parse(toV); + if (x > a) return 'major'; + if (x === a && y > b) return 'minor'; + if (x === a && y === b && z > c) return 'patch'; + return 'none'; +} + +/** The declared bump must cover the computed one (a major change cannot ship as 1.4.1). */ +function checkBump(from, to) { + const d = diff(from, to); + const declared = bumpOf(from.version, to.version); + const ok = declared !== 'none' && RANK[declared] >= RANK[d.level]; + return { ok, declared, required: d.level, changes: d.changes, message: ok ? null : declared === 'none' ? `${to.version} is not newer than ${from.version}` : `Changes need a ${d.level} bump; ${from.version} -> ${to.version} is ${declared}` }; +} + +/** + * What it takes to move one installation between versions. + * @returns {{ allowed:boolean, needs:Array<{kind:string,path:string,message:string}>, actions:Array<{kind:string,id?:string,message:string}>, config:object }} + * needs things a person must supply first (a required setting, a module connection) + * actions what the runtime does on upgrade (enable/disable workflows, rebuild links, re-run onInstall) + * config the installation config after the upgrade (pinned to the new version) + */ +function planUpgrade(installation, from, to) { + const needs = [], actions = []; + const config = JSON.parse(JSON.stringify(installation || {})); + config.definition = { id: to.id, version: to.version }; + + for (const k of Object.keys(to.modules || {})) if (!from.modules?.[k]) needs.push({ kind: 'connect', path: `modules.${k}`, message: `Connect ${to.modules[k].module}` }); + for (const [k, s] of Object.entries(to.settings || {})) { + if (s.required && s.default === undefined && config.settings?.[k] === undefined) needs.push({ kind: 'setting', path: `settings.${k}`, message: `Choose "${s.label}"` }); + } + const wa = byId(from.workflows), wb = byId(to.workflows); + config.workflows ||= {}; + for (const id of Object.keys(config.workflows)) if (!wb[id]) { delete config.workflows[id]; actions.push({ kind: 'disable', id, message: `Workflow "${wa[id]?.label || id}" no longer exists` }); } + for (const [id, w] of Object.entries(wb)) { + const enabled = config.workflows[id]?.enabled ?? ((w.default || 'on') === 'on'); + if (!wa[id]) { + config.workflows[id] = { enabled: (w.default || 'on') === 'on', settings: {} }; + actions.push({ kind: config.workflows[id].enabled ? 'enable' : 'offer', id, message: `${config.workflows[id].enabled ? 'Turns on' : 'Available, off'}: ${w.label}` }); + } + if (!enabled) continue; + for (const [k, s] of Object.entries(w.settings || {})) { + if (s.required && s.default === undefined && config.workflows[id]?.settings?.[k] === undefined) needs.push({ kind: 'setting', path: `workflows.${id}.settings.${k}`, message: `"${w.label}": choose "${s.label}"` }); + } + const keep = new Set(Object.keys(w.settings || {})); + for (const k of Object.keys(config.workflows[id]?.settings || {})) if (!keep.has(k)) delete config.workflows[id].settings[k]; + } + const sa = byId(from.syncs); + for (const s of to.syncs || []) { + if (sa[s.id] && !same(sa[s.id].identity, s.identity)) actions.push({ kind: 'relink', id: s.id, message: `Rebuild ${s.label} links with the new identity keys (runs a backfill)` }); + if (!sa[s.id]) actions.push({ kind: 'backfill', id: s.id, message: `New sync: ${s.label}` }); + } + // A link set has no backfill to rebuild from: its links by id still resolve, but its identity index was written + // with the old keys, so lookups by key miss until each pair is linked again. Said plainly rather than hidden. + const la = byId(from.links), lb = byId(to.links); + for (const l of to.links || []) if (la[l.id] && !same(la[l.id].identity, l.identity)) actions.push({ kind: 'reindex', id: l.id, message: `${l.label || l.id}: links by id still resolve; lookups by the new identity keys miss until each pair is linked again` }); + for (const id of Object.keys(la)) if (!lb[id]) actions.push({ kind: 'drop-links', id, message: `Link set "${la[id].label || id}" is gone; its stored links are no longer read` }); + if (!same(from.lifecycle?.onInstall, to.lifecycle?.onInstall)) actions.push({ kind: 'reinstall', message: 'Re-run install steps (webhook subscriptions)' }); + actions.push({ kind: 'drain', message: `Messages already queued under ${from.version} finish on ${from.version}; new events run on ${to.version}` }); + + return { allowed: needs.length === 0, needs, actions, config }; +} + +module.exports = { diff, checkBump, planUpgrade, bumpOf }; diff --git a/packages/core/integrations/data-defined/webhooks.js b/packages/core/integrations/data-defined/webhooks.js new file mode 100644 index 000000000..fa9ab4906 --- /dev/null +++ b/packages/core/integrations/data-defined/webhooks.js @@ -0,0 +1,119 @@ +/** + * Turning a provider's webhook delivery into the module events a definition listens for. Pure; no I/O. + * + * Providers disagree about everything: where the topic is (a body field, a header, two fields combined), whether one + * POST carries one event or a batch (HubSpot sends an array, QuickBooks nests entities inside notifications), which + * account it is about, and whether the first request is a handshake to answer synchronously (Slack url_verification, + * Asana X-Hook-Secret). A manifest says so in its `webhook` block, with the same template expressions definitions + * use, so none of it is code: + * + * "webhook": { + * "scope": "app", // one URL for every account ("installation": one per install) + * "batch": "body", // path to the array of events; [outer, inner] for two levels + * "topic": "{{item.subscriptionType}}", // the provider's topic; a list tries each until one is non-empty + * "eventId": "{{item.eventId}}", // dedupe id + * "account": "{{item.portalId}}", // the connected account (entity externalId) the event is about + * "verify": "verifyWebhookSignature", // optional Api method: ({ headers, body, query }) -> boolean + * "handshake": { "when": "...", "status": 200, "headers": {...}, "body": {...} }, + * "unwrap": "payload" // a form-encoded field holding the JSON body (Slack interactivity) + * } + * + * and each event says which provider topic it is (`match`, default: the event key) and optionally a condition on the + * record (`when`): a phone module reports a missed call as call.completed with status missed. + * + * Template context: { body, headers, query, item, parent }. `item` is the event (the body itself when there is no + * batch); `parent` is the outer item of a two-level batch (a QuickBooks notification, which carries the realmId). + */ +const { resolve, evaluate, getPath } = require('./expressions'); + +const list = (v) => (v == null ? [] : Array.isArray(v) ? v : [v]); +const str = (v) => (v == null ? '' : typeof v === 'object' ? JSON.stringify(v) : String(v)); + +/** The first template in `tpls` that renders to a non-empty string. */ +function renderFirst(tpls, ctx) { + for (const t of list(tpls)) { + const v = str(resolve(t, ctx)).trim(); + if (v) return v; + } + return ''; +} + +/** Lower-case header names, as Express and Lambda deliver them, so manifests can always write headers.x-shopify-topic. */ +function normaliseHeaders(headers = {}) { + return Object.fromEntries(Object.entries(headers || {}).map(([k, v]) => [k.toLowerCase(), Array.isArray(v) ? v[0] : v])); +} + +/** The delivery as the manifest sees it: lower-case headers, and the JSON body unwrapped from a form field if declared. */ +function prepare(webhook = {}, delivery) { + let body = delivery.body; + if (webhook.unwrap && body && typeof body[webhook.unwrap] === 'string') { + try { body = JSON.parse(body[webhook.unwrap]); } catch { /* not JSON: leave the body as delivered */ } + } + return { body, headers: normaliseHeaders(delivery.headers), query: delivery.query || {} }; +} + +/** Split one delivery into events: [{ item, parent }]. */ +function splitBatch(webhook = {}, input) { + const paths = list(webhook.batch); + const at = (obj, p) => (p === 'body' ? obj.body : p === '$' ? obj : getPath(obj, '$.' + p)); + if (!paths.length) return [{ item: input.body, parent: null }]; + const outer = at(input, paths[0]); + const outerItems = Array.isArray(outer) ? outer : outer == null ? [] : [outer]; + if (paths.length === 1) return outerItems.map((item) => ({ item, parent: null })); + return outerItems.flatMap((parent) => { + const inner = getPath(parent, '$.' + paths[1]); + return (Array.isArray(inner) ? inner : inner == null ? [] : [inner]).map((item) => ({ item, parent })); + }); +} + +/** + * Every module event in one delivery. + * @param {object} manifest the module's capability manifest + * @param {{ body, headers?, query? }} delivery + * @returns {Array<{ topic: string, providerTopic: string, eventId: string|null, account: string|null, event: object, record: any }>} + * topic is the manifest's event key (what definitions name); event is the item the workflow sees as $.event. + */ +function eventsIn(manifest, delivery) { + const webhook = manifest?.webhook; + if (!webhook) return []; + const input = prepare(webhook, delivery); + const out = []; + for (const { item, parent } of splitBatch(webhook, input)) { + const ctx = { ...input, item, parent }; + const providerTopic = renderFirst(webhook.topic, ctx); + if (!providerTopic) continue; + const eventId = renderFirst(webhook.eventId, ctx) || null; + const account = webhook.account ? renderFirst(webhook.account, ctx) || null : null; + for (const [key, ev] of Object.entries(manifest.events || {})) { + if (ev.delivery && ev.delivery !== 'webhook') continue; + if ((ev.match ?? key) !== providerTopic) continue; + const record = ev.record ? getPath(item, '$.' + ev.record) : item; + if (ev.when && !evaluate(ev.when, { event: item, record, headers: input.headers, parent })) continue; + out.push({ topic: key, providerTopic, eventId, account, event: item, record }); + } + } + return out; +} + +/** A handshake to answer at receipt, before anything is queued: { status, headers, body } or null. */ +function handshakeFor(manifest, delivery) { + const h = manifest?.webhook?.handshake; + if (!h) return null; + const ctx = prepare(manifest.webhook, delivery); + if (!evaluate(h.when, ctx)) return null; + return { status: h.status || 200, headers: resolve(h.headers || {}, ctx), body: h.body === undefined ? '' : resolve(h.body, ctx) }; +} + +/** + * Which module of a definition a delivery came from. The URL names it (?module=); without that, the one + * module whose manifest recognises an event in the body. Returns module keys of the definition (usually one). + */ +function modulesFor(def, manifests, delivery) { + const named = delivery.query?.module; + const keys = Object.entries(def.modules || {}).filter(([, m]) => (named ? m.module === named : true)).map(([k]) => k); + if (named) return keys; + const recognised = keys.filter((k) => eventsIn(manifests[def.modules[k].module], delivery).length > 0); + return recognised.length === 1 ? recognised : []; +} + +module.exports = { eventsIn, handshakeFor, modulesFor, splitBatch, renderFirst, normaliseHeaders }; diff --git a/packages/core/integrations/integration-router.js b/packages/core/integrations/integration-router.js index 2a19342c4..11576672b 100644 --- a/packages/core/integrations/integration-router.js +++ b/packages/core/integrations/integration-router.js @@ -209,6 +209,11 @@ function createIntegrationRouter() { updateIntegration, getPossibleIntegrations, }); + // Data-defined integrations (ADR-063): only mounted when the app registers a data-defined class. + require('./data-defined/router').setDataDefinedRoutes(router, authenticateUser, { + getIntegrationInstance, + integrationClasses, + }); setEntityRoutes(router, authenticateUser, { getCredentialForUser, getModuleInstanceFromType, diff --git a/packages/core/package.json b/packages/core/package.json index dbbed0af0..74c2a25f7 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -67,6 +67,7 @@ "scripts": { "lint:fix": "prettier --write --loglevel error . && eslint . --fix", "test": "jest --passWithNoTests # TODO", + "validate:definitions": "node integrations/data-defined/scripts/validate.js", "prisma:generate:mongo": "npx prisma generate --schema ./prisma-mongodb/schema.prisma", "prisma:generate:postgres": "npx prisma generate --schema ./prisma-postgresql/schema.prisma", "prisma:generate": "npm run prisma:generate:mongo && npm run prisma:generate:postgres", diff --git a/packages/core/prisma-mongodb/schema.prisma b/packages/core/prisma-mongodb/schema.prisma index f142c0078..84b604bb0 100644 --- a/packages/core/prisma-mongodb/schema.prisma +++ b/packages/core/prisma-mongodb/schema.prisma @@ -181,6 +181,7 @@ model Integration { syncs Sync[] mappings IntegrationMapping[] processes Process[] + workflowRuns WorkflowRun[] @@index([userId]) @@index([status]) @@ -453,3 +454,58 @@ model UsageCounter { @@index([metric, integrationType, window]) @@map("UsageCounter") } + +// Data-defined integrations (ADR-063): one immutable row per published definition version. +enum DefinitionStatus { + DRAFT + PUBLISHED + DEPRECATED + RETIRED +} + +model IntegrationDefinitionVersion { + id String @id @default(auto()) @map("_id") @db.ObjectId + definitionId String + version String + schemaVersion String + status DefinitionStatus @default(DRAFT) + checksum String + document Json + changeLevel String? + previousVersion String? + notes String? + createdBy String? + publishedAt DateTime? + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@unique([definitionId, version]) + @@index([definitionId, status]) + @@map("IntegrationDefinitionVersion") +} + +// One row per data-defined workflow run. trace holds step ids, statuses, timings and error messages, never request or +// response bodies. +model WorkflowRun { + id String @id @default(auto()) @map("_id") @db.ObjectId + integrationId String @db.ObjectId + integration Integration @relation(fields: [integrationId], references: [id], onDelete: Cascade) + definitionId String + definitionVersion String + workflowId String + triggerType String + eventId String? + status String + reason String? + error String? + retryable Boolean @default(false) + trace Json @default("[]") + startedAt DateTime + endedAt DateTime? + createdAt DateTime @default(now()) + + @@index([integrationId, startedAt]) + @@index([integrationId, workflowId, startedAt]) + @@index([integrationId, status]) + @@map("WorkflowRun") +} diff --git a/packages/core/prisma-postgresql/migrations/20261018000000_add_data_defined_integrations/migration.sql b/packages/core/prisma-postgresql/migrations/20261018000000_add_data_defined_integrations/migration.sql new file mode 100644 index 000000000..9a1cf5193 --- /dev/null +++ b/packages/core/prisma-postgresql/migrations/20261018000000_add_data_defined_integrations/migration.sql @@ -0,0 +1,66 @@ +-- Data-defined integrations (ADR-063): published definition versions and workflow runs. +-- +-- Additive and non-destructive: one enum type and two tables. No existing table or row is touched; WorkflowRun +-- references Integration with ON DELETE CASCADE, so deleting an installation removes its run history. + +-- CreateEnum +CREATE TYPE "DefinitionStatus" AS ENUM ('DRAFT', 'PUBLISHED', 'DEPRECATED', 'RETIRED'); + +-- CreateTable +CREATE TABLE "IntegrationDefinitionVersion" ( + "id" SERIAL NOT NULL, + "definitionId" TEXT NOT NULL, + "version" TEXT NOT NULL, + "schemaVersion" TEXT NOT NULL, + "status" "DefinitionStatus" NOT NULL DEFAULT 'DRAFT', + "checksum" TEXT NOT NULL, + "document" JSONB NOT NULL, + "changeLevel" TEXT, + "previousVersion" TEXT, + "notes" TEXT, + "createdBy" TEXT, + "publishedAt" TIMESTAMP(3), + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + + CONSTRAINT "IntegrationDefinitionVersion_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "WorkflowRun" ( + "id" SERIAL NOT NULL, + "integrationId" INTEGER NOT NULL, + "definitionId" TEXT NOT NULL, + "definitionVersion" TEXT NOT NULL, + "workflowId" TEXT NOT NULL, + "triggerType" TEXT NOT NULL, + "eventId" TEXT, + "status" TEXT NOT NULL, + "reason" TEXT, + "error" TEXT, + "retryable" BOOLEAN NOT NULL DEFAULT false, + "trace" JSONB NOT NULL DEFAULT '[]', + "startedAt" TIMESTAMP(3) NOT NULL, + "endedAt" TIMESTAMP(3), + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + + CONSTRAINT "WorkflowRun_pkey" PRIMARY KEY ("id") +); + +-- CreateIndex +CREATE UNIQUE INDEX "IntegrationDefinitionVersion_definitionId_version_key" ON "IntegrationDefinitionVersion"("definitionId", "version"); + +-- CreateIndex +CREATE INDEX "IntegrationDefinitionVersion_definitionId_status_idx" ON "IntegrationDefinitionVersion"("definitionId", "status"); + +-- CreateIndex +CREATE INDEX "WorkflowRun_integrationId_startedAt_idx" ON "WorkflowRun"("integrationId", "startedAt"); + +-- CreateIndex +CREATE INDEX "WorkflowRun_integrationId_workflowId_startedAt_idx" ON "WorkflowRun"("integrationId", "workflowId", "startedAt"); + +-- CreateIndex +CREATE INDEX "WorkflowRun_integrationId_status_idx" ON "WorkflowRun"("integrationId", "status"); + +-- AddForeignKey +ALTER TABLE "WorkflowRun" ADD CONSTRAINT "WorkflowRun_integrationId_fkey" FOREIGN KEY ("integrationId") REFERENCES "Integration"("id") ON DELETE CASCADE ON UPDATE CASCADE; diff --git a/packages/core/prisma-postgresql/schema.prisma b/packages/core/prisma-postgresql/schema.prisma index 159faf4bd..23c8bac45 100644 --- a/packages/core/prisma-postgresql/schema.prisma +++ b/packages/core/prisma-postgresql/schema.prisma @@ -173,6 +173,7 @@ model Integration { syncs Sync[] mappings IntegrationMapping[] processes Process[] + workflowRuns WorkflowRun[] @@index([userId]) @@index([status]) @@ -434,3 +435,56 @@ model UsageCounter { // Covers series() (metric + integrationType + window range). @@index([metric, integrationType, window]) } + +// Data-defined integrations (ADR-063): one immutable row per published definition version. +enum DefinitionStatus { + DRAFT + PUBLISHED + DEPRECATED + RETIRED +} + +model IntegrationDefinitionVersion { + id Int @id @default(autoincrement()) + definitionId String + version String + schemaVersion String + status DefinitionStatus @default(DRAFT) + checksum String + document Json + changeLevel String? + previousVersion String? + notes String? + createdBy String? + publishedAt DateTime? + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@unique([definitionId, version]) + @@index([definitionId, status]) +} + +// One row per data-defined workflow run. trace holds step ids, statuses, timings and error messages, never request or +// response bodies. +model WorkflowRun { + id Int @id @default(autoincrement()) + integrationId Int + integration Integration @relation(fields: [integrationId], references: [id], onDelete: Cascade) + definitionId String + definitionVersion String + workflowId String + triggerType String + eventId String? + status String + reason String? + error String? + retryable Boolean @default(false) + trace Json @default("[]") + startedAt DateTime + endedAt DateTime? + createdAt DateTime @default(now()) + + @@index([integrationId, startedAt]) + @@index([integrationId, workflowId, startedAt]) + @@index([integrationId, status]) +}