Repository navigation
CC-5667 - [Sentinel] Add the AI Interaction Logs connector and SIEM token setup - #61
ilia-sretenskii wants to merge 4 commits into
Conversation
System (L1): the Miro solution for Microsoft Sentinel ingests Miro audit, content and AI interaction logs from the Miro REST API (https://api.miro.com) through the Codeless Connector Framework (CCF). Unchanged: the publisher, the offer, the CCF RestApiPoller pattern (APIKey bearer auth, 5-minute windows, from/to or createdAfter/createdBefore, NextPageToken paging on $.cursor, records in $.data). Changing: a third data source, AI interaction logs, and the setup text of every connector. Container (L2): the solution package (Solutions/Miro) deployed from the Content Hub into a Log Analytics workspace. Unchanged: the Audit Logs and Content Logs connectors, their pollers and the MiroAuditLogs_CL table. Changing: version 3.0.0 -> 3.1.0, a new connector with its own DCR and the MiroAILogs_CL table, new columns in MiroContentLogs_CL. Component (L3): - Miro AI Interaction Logs connector (Enterprise Plan + Enterprise Guard): polls GET /v2/orgs/{organizationId}/ai-interaction-logs with from/to and sorting asc; needs the aiinteractionlogs:read scope and the organization ID. Raw nested objects (actor, object, session, model, tool, details) are dynamic columns, so keys added to the API later reach the table without an update; actor_id, actor_email, actor_name, object_id and session_id are also flattened as static columns because the API documents them as filter parameters. The description carries the API reference's note that details may contain unstructured data from LLM interactions. The requirements and Step 1 name the organization setting a Miro admin turns on first (Collect AI interaction logs in Admin console > Security > Audit logs > Settings; only later interactions are recorded) and point to the Miro account manager when the setting is not visible. The connector is not marked as preview, so the listing needs no update when the API leaves its beta stage. - Content Logs connector: the DCR keeps the raw ai and actor objects as dynamic columns and derives ai_assisted (bool) from ai.assisted, so AI-assisted content changes can be queried directly. - Setup text of every connector and the README: the SIEM toggle of the Enterprise integrations page creates one access token with every log scope the organization is entitled to (auditlogs:read with the Enterprise Plan, contentlogs:export and aiinteractionlogs:read with the Enterprise Guard add-on); a toggle enabled before these scopes were offered must be turned off and on to get them. The eDiscovery toggle stays an alternative for the content logs connector. Code (L4): - Data Connectors/MiroAILogs_CCF/: MiroAILogs_DataConnectorDefinition.json, MiroAILogs_PollingConfig.json, MiroAILogs_DCR.json, MiroAILogs_Table.json (new). - Data Connectors/MiroContentLogs_CCF/: MiroContentLogs_DCR.json, MiroContentLogs_Table.json (ai, ai_assisted, actor), MiroContentLogs_DataConnectorDefinition.json (SIEM setup note). - Data Connectors/MiroAuditLogs_CCF/MiroAuditLogs_DataConnectorDefinition.json (SIEM setup note). - Data/Solution_Miro.json: the third connector, description and version 3.1.0. - README.md: the third connector in every section, the SIEM token setup, the logo path ../../Logos/Miro.svg (the Solutions/Miro/Logo directory no longer exists). - ReleaseNotes.md: 3.1.0 entry. - Package/: mainTemplate.json, createUiDefinition.json and 3.1.0.zip regenerated with Tools/Create-Azure-Sentinel-Solution/V3/createSolutionV3.ps1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
d28c2bc to
a7e197f
Compare
annaavanesyan
left a comment
There was a problem hiding this comment.
The package matches the sources, the schema changes are additive and the SIEM-token story checks out against server, but I want the pagination rate limiting and the unexplained double ingestion after reconnect understood before this goes upstream.
| }, | ||
| "request": { | ||
| "apiEndpoint": "https://api.miro.com/v2/orgs/{{organizationId}}/ai-interaction-logs", | ||
| "rateLimitQPS": 10, |
There was a problem hiding this comment.
This endpoint is rate-limit level 4 (2000 credits a call, so 50 calls a minute per user and app), and in CCF rateLimitQPS only throttles the first request of a window; paginated calls run unthrottled unless PaginatedCallsPerSecond is set. AI logs are up to six rows per interaction at 100 rows a page, so a busy window or a catch-up after an outage goes past 50 pages a minute, gets 429s, and after three retries the window is gone. I'd set PaginatedCallsPerSecond to about 0.8 and add a rateLimitConfig on X-RateLimit-Remaining/X-RateLimit-Reset here, and the content poller has the same problem.
There was a problem hiding this comment.
@annaavanesyan Added paginatedCallsPerSecond 0.8 and rateLimitConfig to both pollers. Audit calls are 20x cheaper, so that poller is unchanged.
9f07678
| "description": "Tool information (tool invocation events only), as returned by the API, in JSON format" | ||
| }, | ||
| { | ||
| "name": "details", |
There was a problem hiding this comment.
The Logs Ingestion API truncates any field value above 64 KB, and details for model_invocation_request carries the full prompt with context, so a long one arrives as a cut JSON string with no error anywhere. For an eDiscovery table that is a silent gap; have we measured the size distribution, and do we at least want to document it next to the sensitivity note?
There was a problem hiding this comment.
@annaavanesyan Confirmed on Microsoft's limits page and documented next to the sensitivity note. Not measured yet.
9f07678
| } | ||
| ], | ||
| "availability": { | ||
| "isPreview": false |
There was a problem hiding this comment.
The upstream submission is planned for when AI interaction logs reach Public Beta, so the API behind this connector will still be beta when it shows up in the gallery. Should this go out as isPreview: true and flip at GA?
There was a problem hiding this comment.
@annaavanesyan Agreed after all: the connector is now marked as preview and named (Beta) until GA.
930cb8d
| }, | ||
| { | ||
| "description": "**Option 1: Enterprise integrations (recommended)**\n\n1. Open [Miro Company Settings](https://miro.com/app/settings/).\n2. Expand the **Apps and integrations** section.\n3. Click **Enterprise integrations**.\n4. Enable the **SIEM** toggle.\n5. Copy the **Access Token** value that appears.\n6. **Important:** Store the token securely—it provides full access to audit logs.\n7. The token will work until you disable the toggle.\n8. Proceed to Step 3." | ||
| "description": "**Option 1: Enterprise integrations (recommended)**\n\n1. Open [Miro Company Settings](https://miro.com/app/settings/).\n2. Expand the **Apps and integrations** section.\n3. Click **Enterprise integrations**.\n4. Enable the **SIEM** toggle.\n5. Copy the **Access Token** value that appears.\n6. **Important:** Store the token securely—it provides full access to audit logs.\n7. The token will work until you disable the toggle.\n8. Proceed to Step 3.\n\n**One token for all Miro connectors:** the SIEM toggle creates one access token with every log scope your organization is entitled to: `auditlogs:read` (Enterprise Plan), `contentlogs:export` (Enterprise Guard add-on) and `aiinteractionlogs:read` (Enterprise Guard add-on with AI interaction logs). If the SIEM toggle was enabled before it offered these scopes, turn it off and on again to get a token that includes them, then update the token in every connector that uses it." |
There was a problem hiding this comment.
Turning the SIEM toggle off deletes the app installation, so the current token dies at that moment and every connector using it fails until the new one is pasted in. I'd say that plainly here and in the content and AI steps, and I'd like to know whether CCF re-polls the windows that failed on 401 once the token is updated or whether that gap is simply lost.
There was a problem hiding this comment.
@annaavanesyan Documented in all three connectors. Microsoft doesn't say whether failed windows are polled again.
9f07678
| "Data Connectors/MiroAILogs_CCF/MiroAILogs_DataConnectorDefinition.json" | ||
| ], | ||
| "Version": "3.0.0", | ||
| "Version": "3.1.0", |
There was a problem hiding this comment.
SolutionMetadata.json still has no firstPublishDate/lastPublishDate, so the generated mainTemplate.json metadata carries neither; the V3 packaging guide documents both and Microsoft reviewers expect lastPublishDate to move with each version. Since this branch goes upstream as is, I'd add 2025-12-05 and 2026-10-19 now and regenerate the package.
There was a problem hiding this comment.
@annaavanesyan Added both dates and regenerated the package.
9f07678
…dates System (L1): the Miro solution for Microsoft Sentinel ingests Miro audit, content and AI interaction logs from the Miro REST API (https://api.miro.com) through the Codeless Connector Framework (CCF). Unchanged: the three connectors, their tables, DCRs and query windows. Changing: how fast the content and AI pollers page, the setup text about the SIEM token, and the solution metadata. Container (L2): the solution package (Solutions/Miro) deployed from the Content Hub. Unchanged: version 3.1.0 and the package contents other than the edits below, regenerated with the V3 tool. Component (L3): - Content Logs and AI Interaction Logs pollers: Miro charges both endpoints 2000 credits per call against a 100,000-credit budget per minute, evaluated per method, so each endpoint allows 50 calls a minute. CCF's rateLimitQPS throttles only the first request of a window (https://learn.microsoft.com/azure/sentinel/data-connector-connection-rules-reference), so a busy window paged unthrottled and could hit 429. paginatedCallsPerSecond 0.8 keeps paging under the budget, rateLimitQPS drops to 1 to match, and rateLimitConfig reads Miro's X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset (epoch seconds) headers on a 429, in the shape of Microsoft's documented example. The Audit Logs poller is unchanged: that endpoint costs 100 credits per call. - Setup text of every connector and the README: turning the SIEM toggle off uninstalls the SIEM app and invalidates its token, so every connector using that token fails to poll until the new token is entered. - AI Interaction Logs connector and the README: Azure Monitor truncates field values longer than 64 KB at ingestion (https://learn.microsoft.com/azure/azure-monitor/fundamentals/service-limits), so a long details value can arrive cut off; the note sits next to the existing sensitivity note. - Solution metadata: firstPublishDate (the 3.0.0 release) and lastPublishDate (this release), as documented in Tools/Create-Azure-Sentinel-Solution/V3/README.md. Code (L4): - Data Connectors/MiroContentLogs_CCF/MiroContentLogs_PollingConfig.json and Data Connectors/MiroAILogs_CCF/MiroAILogs_PollingConfig.json: rateLimitQPS, paginatedCallsPerSecond, rateLimitConfig. - Data Connectors/*/*_DataConnectorDefinition.json and README.md: token invalidation sentence; the AI definition and README also carry the 64 KB note. - SolutionMetadata.json: firstPublishDate 2025-12-05, lastPublishDate 2026-10-19. - ReleaseNotes.md: the 3.1.0 entry names the paging throttle. - Package/: mainTemplate.json and 3.1.0.zip regenerated with Tools/Create-Azure-Sentinel-Solution/V3/createSolutionV3.ps1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uidance System (L1): the Miro solution for Microsoft Sentinel ingests Miro audit, content and AI interaction logs through the Codeless Connector Framework. Unchanged: every poller, DCR and table. Changing: the AI connector's availability label and the setup guidance. Container (L2): the solution package (Solutions/Miro), version 3.1.0, regenerated with the V3 tool. Component (L3): - AI Interaction Logs connector: the Miro AI interaction logs API is in Beta (Help Center article "AI interaction logs (Beta)") and may change before general availability, so the connector sets availability.isPreview to true, its title carries "(Beta)", and its description says the API may change. The audit and content connectors stay generally available. - README, token setup: how to rotate a token (turn the SIEM toggle off and on, or reinstall the custom app) and revoke one whose scopes are no longer needed (turn the toggle off, or uninstall the app), then update every connector that uses it. - README and AI Interaction Logs connector: MiroAILogs_CL holds personal data and AI prompts, so the note recommends restricting who can read the table with granular RBAC, which Microsoft's "Manage access to Log Analytics workspaces" page names the recommended method for table-level access control (https://learn.microsoft.com/azure/azure-monitor/logs/granular-rbac-log-analytics). Code (L4): - Data Connectors/MiroAILogs_CCF/MiroAILogs_DataConnectorDefinition.json: title, isPreview, the Beta sentence and the table access sentence. - README.md: the connector name, the Beta sentence, a token rotation paragraph and the table access sentence. - ReleaseNotes.md: the 3.1.0 entry marks the connector as Beta. - Package/: mainTemplate.json, createUiDefinition.json and 3.1.0.zip regenerated with Tools/Create-Azure-Sentinel-Solution/V3/createSolutionV3.ps1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
|
@annaavanesyan Retested the double ingestion after reconnecting all three connectors: the first poll and a later one ingested every AI event once. The cause of the earlier duplicates is unknown. |
System (L1): the Miro solution for Microsoft Sentinel ingests Miro audit, content and AI interaction logs through the Codeless Connector Framework. Unchanged: every connector, poller, DCR and table. Changing: one sentence of setup guidance. Container (L2): the solution package (Solutions/Miro), version 3.1.0, regenerated with the V3 tool. Component (L3): - AI Interaction Logs connector and README: the note recommended restricting who can read MiroAILogs_CL with granular RBAC. How a workspace assigns roles is the Sentinel administrators' own decision and applies to any table, so the connector documentation leaves it out. The note keeps the sensitivity warning for the details column and the 64 KB truncation limit. Code (L4): - Data Connectors/MiroAILogs_CCF/MiroAILogs_DataConnectorDefinition.json and README.md: the granular RBAC sentence removed from the AI logs note. - Package/: mainTemplate.json and 3.1.0.zip regenerated with Tools/Create-Azure-Sentinel-Solution/V3/createSolutionV3.ps1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
Oplane Security ReviewOplane identified security requirements relevant to this PR and checked the implementation against them. All requirements are satisfied. Review the generated threat model.
Change comment settings · Set up Oplane in your agent · Workflow guidance |
|
@annaavanesyan this work merged upstream as Azure#15357, so this internal pull request is closed without merging. |
Internal review before the upstream submission
Team review of the Miro solution 3.1.0 before the branch goes to Azure/Azure-Sentinel without internal references.
See https://miro.atlassian.net/browse/CC-5667 for details.
Change(s)
MiroAILogs_CLtable inSolutions/Miro; its note covers 64 KB truncation.ai,ai_assistedandactorcolumns.Testing Completed
ARM-TTK passes except its known resource ID check. In a test workspace with one SIEM-toggle token, the AI connector shows "(Beta)" as preview, the new rate limits are active, and AI translations reached all three tables once each, on the first poll after reconnecting and later.
Notes
An earlier run ingested AI events twice after reconnecting; the retest did not, and that cause is unknown.
Pipeline: Dependency Reviewcannot run on this public fork because its workflow lives in an internal repository.AI reviewers: read the C4 model context in all commit messages and the open review threads before reviewing the file changes.
🤖 Generated with Claude Code