feat(core): in-app previews (Slack, HubSpot, HTML) and workflow dry runs

docs/architecture-decisions/063-integration-definitions-as-data.md

@@ -741,7 +741,7 @@ A UI extension is a `query` plus a `view`. The view is a tree of a **fixed compo
 - A `Button` can only point at a `userAction` workflow (checked by `validate`). A surface can therefore read data
   and trigger declared workflows, and nothing else.
 
-The renderers themselves are not in v0.1.
+**Preview renderers are not host renderers.** v0.1 has *preview* renderers (`renderers/`): they turn a resolved tree into Slack blocks, a HubSpot tree or HTML, and dry-run a workflow, so an author can see in the editor what an integration will show in the apps it connects. They run inside this app, over sample or real records, and nothing deploys them to a host. The *host* renderer described above, the one deployed card or panel inside each host that calls `POST /api/integrations/:id/actions/UI_<SURFACE>` and draws the answer, is not in v0.1.
 
 ### 17. Versioning (answers ADR-030 for data-defined integrations)
 
@@ -810,7 +810,7 @@ work kept apart from bulk work.
 | Missing | Why it matters | Plan |
 |---|---|---|
 | Per-module rate limits | Bursts (a backfill, a webhook storm) hit provider limits; today a 429 fails the step and the queue retries | A token bucket per module and installation, in the ports |
-| Host renderers (HubSpot card, Phone panel) | `renderSurface` returns trees that nothing draws yet | Per host |
+| Deployed host renderers (HubSpot card, phone panel) | `renderSurface` returns trees that no host draws yet. The preview renderers draw them, but only inside this app, for authors | Per host. Not the same work as the previews |
 | Real manifests | `capabilities/*.json` are mocks written to match the Api classes | Generated from, or checked against, each module's `Api` |
 
 Their order is decided by a month of iteration on v0.1, not fixed here.

packages/core/integrations/data-defined/README.md

@@ -144,6 +144,29 @@ different document at the same version is refused. So is a version that is not n
 semver bump smaller than the changes. Publishing goes through a conditional write, so two concurrent publishes of one
 version cannot both win. With `memoryDefinitions()` (definitions bundled in the app) the admin routes answer 501.
 
+## Previews: in the app
+
+An integration shows up inside the apps it connects: a HubSpot card, a Slack message, digest or approval request. A
+preview renders exactly that from the current settings and sample or real records, with the same query, view and step
+code production runs (`renderers/`):
+
+| Function | Does |
+|---|---|
+| `previewSurface({ def, installation, surfaceId, record, ports, host? })` | `{ surface, view, html, slack?, hubspot? }`. `slack` is Block Kit (`{ blocks, text, truncated }`), `hubspot` a data description of `@hubspot/ui-extensions` components, `html` the same card framed as its host looks. `host` defaults to the surface's own |
+| `previewWorkflow({ def, installation, workflowId, event?, record?, manifests, fixtures? })` | Dry run: `{ status, sends, calls, trace, waiting? }`. Each call is `{ stepId, module, capability, kind, args, output, rendered }`; `rendered` is `{ html, slack? }` for writes (a Slack post renders as the message, anything else as a "would call" card) |
+| `recordingPorts({ def, manifests, fixtures })` | Ports that record and answer instead of calling an API; give them to `previewSurface` for sample data |
+| `viewToSlack`, `viewToHubSpot`, `viewToHtml`, `slackMessageToHtml` | The renderers on their own. `decodeAction(value)` reads a Slack button click back to `{ integrationId?, workflow, surface }` |
+
+Dry-run rules: nothing is performed (the base ports' API function throws, and the recorder sits in front of it).
+`fixtures` answers calls, keyed `"<module key>.<capability>"` (a value, or `(args) => value`; `code:<ref>` for code
+steps, which are never run), otherwise the manifest's own record shape with ids filled. A webhook workflow with no
+`event` runs on the manifest's sample payload. A pausing step ends the preview with `waiting`.
+
+Everything that is record data is escaped for its target (mrkdwn `& < >` plus fenced `* _ ~`, HTML, text children for
+HubSpot), links are http(s) only, Block Kit limits are met by truncating with a marker (50 blocks, 3000/2000/150
+characters, 25 buttons), and nothing throws on an odd tree. Text a *definition* writes into a Slack message (templates)
+is sent as authored: a preview shows it as sent, including any raw record data a template interpolates.
+
 ## Syncs
 
 `sync-engine.js` runs `syncs`: backfill (a button or `onInstall`, one queued page per message, progress at

packages/core/integrations/data-defined/examples/asana-slack.definition.json

@@ -178,14 +178,14 @@
           "use": "slack.message.post",
           "with": {
             "channel": "$.settings.channel",
-            "text": ":white_check_mark: {{steps.task.completed_by.name | default: 'Someone'}} completed <{{steps.task.permalink_url}}|{{steps.task.name}}>"
+            "text": ":white_check_mark: {{steps.task.completed_by.name | default: 'Someone' | mrkdwn}} completed <{{steps.task.permalink_url}}|{{steps.task.name | mrkdwn}}>"
           }
         }
       ]
     }
   ],
   "templates": {
-    "assignee-digest": ":spiral_calendar_pad: Hi {{item.firstName | default: item.name}}, you have {{item.count}} Asana task(s) due tomorrow:\n{{item.lines}}",
+    "assignee-digest": ":spiral_calendar_pad: Hi {{item.firstName | default: item.name | mrkdwn}}, you have {{item.count}} Asana task(s) due tomorrow:\n{{item.lines}}",
     "unassigned-digest": ":warning: {{steps.groups.unassignedCount}} task(s) due tomorrow have no assignee:\n{{steps.groups.unassignedLines}}"
   },
   "lifecycle": {

packages/core/integrations/data-defined/examples/hubspot-slack.definition.json

@@ -258,7 +258,7 @@
               "when": "$.steps.slackUser.id exists",
               "with": {
                 "channel": "$.steps.slackUser.id",
-                "text": "Due today: {{item.properties.hs_task_subject}}",
+                "text": "Due today: {{item.properties.hs_task_subject | mrkdwn}}",
                 "metadata": {
                   "event_type": "hubspot_task",
                   "event_payload": {
@@ -401,9 +401,9 @@
     }
   ],
   "templates": {
-    "deal-won": ":tada: *{{steps.deal.properties.dealname}}* closed won for ${{steps.deal.properties.amount | default: '?'}}. Nice work {{steps.owner.firstName | default: 'team'}}! <https://app.hubspot.com/contacts/{{install.portalId}}/record/0-3/{{steps.deal.id}}|Open deal>",
-    "high-value": ":moneybag: New deal over ${{settings.threshold}}: *{{steps.deal.properties.dealname}}* (${{steps.deal.properties.amount}}), closing {{steps.deal.properties.closedate | date}}. <https://app.hubspot.com/contacts/{{install.portalId}}/record/0-3/{{steps.deal.id}}|Open deal>",
-    "digest-line": "\u2022 <https://app.hubspot.com/contacts/{{install.portalId}}/record/0-3/{{item.id}}|{{item.properties.dealname}}>: ${{item.properties.amount | default: 0}}, closes {{item.properties.closedate | date}}",
+    "deal-won": ":tada: *{{steps.deal.properties.dealname | mrkdwn}}* closed won for ${{steps.deal.properties.amount | default: '?'}}. Nice work {{steps.owner.firstName | default: 'team' | mrkdwn}}! <https://app.hubspot.com/contacts/{{install.portalId}}/record/0-3/{{steps.deal.id}}|Open deal>",
+    "high-value": ":moneybag: New deal over ${{settings.threshold}}: *{{steps.deal.properties.dealname | mrkdwn}}* (${{steps.deal.properties.amount}}), closing {{steps.deal.properties.closedate | date}}. <https://app.hubspot.com/contacts/{{install.portalId}}/record/0-3/{{steps.deal.id}}|Open deal>",
+    "digest-line": "\u2022 <https://app.hubspot.com/contacts/{{install.portalId}}/record/0-3/{{item.id}}|{{item.properties.dealname | mrkdwn}}>: ${{item.properties.amount | default: 0}}, closes {{item.properties.closedate | date}}",
     "digest": ":calendar: *Closing in the next 7 days* ({{steps.deals | size}} deals)\n{{steps.build | join: '\\n'}}"
   }
 }

packages/core/integrations/data-defined/expressions.js

@@ -46,6 +46,8 @@ const FILTERS = {
     seconds: (v) => (v == null ? '' : String(Math.round(Number(v) / 1000))),
     date: (v) => (v ? new Date(v).toISOString().slice(0, 10) : ''),
     json: (v) => JSON.stringify(v),
+    // Record data inside a Slack message: escaped so a name like "*x* <!channel>" neither formats nor pings.
+    mrkdwn: (v) => require('./renderers/slack').escapeMrkdwn(v),
     default: (v, arg) => (v == null || v === '' ? arg : v),
     truncate: (v, arg) => { const s = String(v ?? ''); const n = Number(arg) || 80; return s.length > n ? s.slice(0, n - 1) + '…' : s; },
     // lists

packages/core/integrations/data-defined/index.js

@@ -15,6 +15,11 @@ const { TRANSFORMS, applyMapping } = require('./transforms');
 const { FILTERS } = require('./expressions');
 const { validateManifest } = require('./manifests');
 const { checkDefinition, checkManifest, checkInstallation } = require('./schema-check');
+const { viewToSlack, escapeMrkdwn } = require('./renderers/slack');
+const { viewToHubSpot } = require('./renderers/hubspot');
+const { viewToHtml, slackMessageToHtml } = require('./renderers/html');
+const { previewSurface, previewWorkflow, recordingPorts } = require('./renderers/preview');
+const { decodeAction } = require('./renderers/common');
 const useCases = require('./use-cases/definition-use-cases');
 
 /** Load every capabilities/*.capabilities.json (bundled mocks; real modules ship their own). */
@@ -35,5 +40,6 @@ module.exports = {
     defineIntegrationClass, defineSharedIntegrationClass, extendIntegrationClass, memoryDefinitions,
     BUILTINS, TRANSFORMS, FILTERS, applyMapping, resolveOptions,
     loadManifests, loadExamples,
+    viewToSlack, escapeMrkdwn, viewToHubSpot, viewToHtml, slackMessageToHtml, previewSurface, previewWorkflow, recordingPorts, decodeAction,
     ...useCases,
 };

packages/core/integrations/data-defined/renderers/common.js

@@ -0,0 +1,67 @@
+/**
+ * What every renderer shares: coercion of view props (they come from record data, so any type can arrive), the one
+ * URL rule, truncation that never leaves half an entity behind, and the button <-> Slack round trip.
+ */
+
+/** A view prop as text. Objects are JSON, null/undefined are empty: a renderer never prints "[object Object]" or "undefined". */
+function str(v) {
+    if (v == null) return '';
+    if (typeof v === 'string') return v;
+    if (typeof v === 'object') { try { return JSON.stringify(v); } catch (_) { return ''; } }
+    return String(v);
+}
+
+/**
+ * Only http(s) links leave a preview or a card. Record data decides these URLs ("{{data.proposal.recording}}"), and a
+ * `javascript:` or `data:` URL in a vendor field must not become a clickable thing in someone's HubSpot or Slack.
+ * @returns {string|null}
+ */
+function safeUrl(u) {
+    const s = str(u).trim();
+    if (!/^https?:\/\/[^\s<>"']+$/i.test(s)) return null;
+    try { new URL(s); } catch (_) { return null; }
+    return s;
+}
+
+/**
+ * Cut to at most `max` characters, ending in the marker. Never leaves a dangling half of a Slack entity ("&am"), which
+ * would render as literal junk instead of the character it stood for.
+ */
+function truncate(s, max, marker = '…') {
+    s = String(s);
+    if (s.length <= max) return s;
+    let cut = s.slice(0, Math.max(0, max - marker.length));
+    cut = cut.replace(/&(?:[a-z]{1,4}|#\d{0,6}|#x[0-9a-f]{0,5})?$/i, '');
+    return cut + marker;
+}
+
+/** Badge tone (`neutral`, or a Jira statusCategory key like `done`) -> one of four. Unknown is neutral, never an error. */
+function toneOf(tone) {
+    const t = str(tone).toLowerCase();
+    if (['positive', 'success', 'good', 'done', 'green', 'complete', 'completed'].includes(t)) return 'positive';
+    if (['warning', 'caution', 'indeterminate', 'yellow', 'in_progress', 'pending'].includes(t)) return 'warning';
+    if (['critical', 'danger', 'error', 'bad', 'red', 'failed'].includes(t)) return 'critical';
+    return 'neutral';
+}
+
+const ACTION_VALUE_MAX = 2000; // Slack's cap on a button's value
+
+/**
+ * A userAction button's identity as a Slack button value: which installation, which workflow, which surface it was on.
+ * `needsInput` marks a button whose workflow asks for values first (a Slack button cannot, so the handler opens a form).
+ */
+function encodeAction({ integrationId, workflow, surface, needsInput }) {
+    const value = JSON.stringify({ ...(integrationId ? { integrationId: String(integrationId) } : {}), workflow, surface, ...(needsInput ? { needsInput: true } : {}) });
+    return value.length <= ACTION_VALUE_MAX ? value : null;
+}
+
+/** The inverse, for the handler that receives a click. null for anything that is not one of ours. */
+function decodeAction(value) {
+    let v;
+    try { v = JSON.parse(value); } catch (_) { return null; }
+    if (!v || typeof v !== 'object' || typeof v.workflow !== 'string' || !v.workflow || typeof v.surface !== 'string') return null;
+    if (v.integrationId !== undefined && typeof v.integrationId !== 'string') return null;
+    return { ...(v.integrationId ? { integrationId: v.integrationId } : {}), workflow: v.workflow, surface: v.surface, ...(v.needsInput ? { needsInput: true } : {}) };
+}
+
+module.exports = { str, safeUrl, truncate, toneOf, encodeAction, decodeAction, ACTION_VALUE_MAX };