diff --git a/docs/README.md b/docs/README.md index 7b8d0d9..f7d35f0 100644 --- a/docs/README.md +++ b/docs/README.md @@ -14,7 +14,7 @@ The Docker Compose configuration is modular, allowing you to choose which data s - [Container Image Access](./container-images.md) - [Configuration Reference](./configuration.md) - [Enable SAML SSO](./sso-saml.md) -- [Use AWS S3 for Artifact Storage](./object-storage.md) +- [Use AWS S3 for Object Storage](./object-storage.md) - [Logging Configuration](./logging.md) - [Backup and Restore](./backup-restore.md) - [Upgrading Currents On-Prem](./upgrading.md) diff --git a/docs/configuration.md b/docs/configuration.md index 2b87fa8..a6821ad 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -41,6 +41,11 @@ These have defaults but you'll likely want to customize them. | `FILE_STORAGE_ACCESS_KEY_ID` | string | `${RUSTFS_ACCESS_KEY}` | Object storage access key ID | | `FILE_STORAGE_SECRET_ACCESS_KEY` | string | `${RUSTFS_SECRET_KEY}` | Object storage secret access key | | `FILE_STORAGE_REGION` | string | _(commented)_ | S3 region (required for AWS S3) | +| `APP_CONTENT_STORAGE_ENDPOINT` | string | `${FILE_STORAGE_ENDPOINT}` | App content bucket endpoint (external access): context shares, organization logos and session files. Unset, context sharing and logo editing are off and session files cannot be uploaded ([details](./object-storage.md#app-content-bucket)) | +| `APP_CONTENT_STORAGE_BUCKET` | string | `currents-app-content` | App content bucket name | +| `APP_CONTENT_STORAGE_ACCESS_KEY_ID` | string | `${FILE_STORAGE_ACCESS_KEY_ID}` | App content bucket access key ID | +| `APP_CONTENT_STORAGE_SECRET_ACCESS_KEY` | string | `${FILE_STORAGE_SECRET_ACCESS_KEY}` | App content bucket secret access key | +| `APP_CONTENT_STORAGE_REGION` | string | _(commented)_ | S3 region (required for AWS S3) | | `CLICKHOUSE_URL` | string | `http://clickhouse:8123` | ClickHouse HTTP endpoint | | `CLICKHOUSE_USERNAME` | string | `currents` | ClickHouse username | | `MONGODB_USERNAME` | string | `currents-user` | MongoDB username | @@ -64,11 +69,28 @@ Less commonly changed settings with sensible defaults. | `RUSTFS_ACCESS_KEY` | string | `rustfs-access-key` | RustFS/S3 access key | | `FILE_STORAGE_INTERNAL_ENDPOINT` | string | `http://host.docker.internal:9000` | Object storage endpoint the services use from inside Docker. For AWS S3, set it to `FILE_STORAGE_ENDPOINT` ([details](./object-storage.md)) | | `FILE_STORAGE_FORCE_PATH_STYLE` | bool | _(commented)_ | Use path-style S3 URLs (auto-set to `true` when using RustFS profile) | +| `APP_CONTENT_STORAGE_INTERNAL_ENDPOINT` | string | `${FILE_STORAGE_INTERNAL_ENDPOINT}` | App content bucket endpoint the services use from inside Docker | +| `APP_CONTENT_STORAGE_FORCE_PATH_STYLE` | bool | _(commented)_ | Path-style URLs for the app content bucket (auto-set to `true` when using RustFS profile) | +| `API_BASE_URL` | string | `${APP_BASE_URL}` | Public API origin, used in share links and as the OAuth token audience | | `AUTOMATED_REPORTS_EMAIL_BCC` | string | _(empty)_ | BCC address for automated reports | | `INVITE_EMAIL_BCC` | string | _(empty)_ | BCC address for invitation emails | | `INVITE_EXPIRATION_DAYS` | int | `14` | Number of days before invitations expire | | `EMAIL_LINKS_BASE_URL` | string | `${APP_BASE_URL}` | Base URL for links in emails (derived from APP_BASE_URL) | +### Remote MCP and OAuth + +Remote MCP lets coding agents use Currents tools over HTTP at `/mcp`. Clients authenticate with a Currents API key or personal access token, or, with the OAuth provider on, by signing in with a Currents account. Both endpoints are reachable by anyone who can reach the install; they refuse requests without a valid credential. + +| Variable | Type | Default | Description | +|----------|------|---------|-------------| +| `MCP_ENABLED` | bool | `true` | Serve remote MCP at `/mcp`. Set `false` to turn it off; `/mcp` then answers 503 | +| `MCP_ORIGINS` | string | _(empty)_ | Other origins clients reach `/mcp` on, comma separated | +| `OAUTH_PROVIDER_ENABLED` | bool | `true` in `.env.example` | Serve OAuth authorization (`/api/auth/oauth2/*`, `/.well-known/oauth-*`). Issuer, token audience and JWKS URL are derived from `APP_BASE_URL`. Off when absent, so an upgraded `.env` keeps it off until added. Use an `https` `APP_BASE_URL` for any real setup. Over `http` it works only on `localhost`, or, for an evaluation, on a `.localhost` name in `OAUTH_HTTP_ALLOWED_HOSTS`. On any other `http` host the API refuses to start with it on, so set it to `false` there | +| `OAUTH_DYNAMIC_CLIENT_REGISTRATION_ENABLED` | bool | `true` in `.env.example` | Let clients register themselves, the way to add a client without internet access | +| `OAUTH_CIMD_ENABLED` | bool | `true` | Accept clients identified by a client ID metadata document: an https URL the server fetches. Hosts resolving to private, loopback or link-local addresses are refused | +| `OAUTH_HTTP_ALLOWED_HOSTS` | string | _(empty)_ | For an evaluation without TLS: a `.localhost` name (e.g. `currents.localhost`) the OAuth endpoints may serve over `http`. Browsers resolve `.localhost` names to this machine. Another `http` host starts, but OAuth clients are sent to an `https` issuer the install does not serve | +| `OAUTH_JWKS_URL` | string | `/api/auth/jwks` | Where the API fetches its own signing keys to check access tokens. With a `.localhost` `APP_BASE_URL`, set `http://localhost:4000/api/auth/jwks`: the container cannot resolve `.localhost` names, so every token is rejected. Set the same when the container cannot reach or trust `APP_BASE_URL` (split DNS, an internal CA) | + ### SAML SSO (Optional) Enable SAML single sign-on by setting the two required variables and mounting your IdP metadata file into `./data/sso`. See [Enable SAML SSO](sso-saml.md) for the full walkthrough. SSO turns on automatically once both required variables are set — there is no separate enable flag. @@ -80,8 +102,7 @@ Enable SAML single sign-on by setting the two required variables and mounting yo | `SSO_SAML_PROVIDER_ID` | string | `onprem-saml` | Provider id; recommend your IdP name (e.g. `okta`). Becomes the last path segment of the ACS callback URL. | | `SSO_SAML_DEFAULT_ROLE` | string | `member` | Role granted to auto-provisioned SSO users | | `SSO_ALLOWED_DOMAINS` | string | _(empty)_ | Comma-separated email domains allowed to sign in via SSO | -| `SSO_SAML_AUTHN_REQUESTS_SIGNED` | bool | `false` | Sign outbound AuthnRequests (requires the SP key/cert below) | -| `SSO_SAML_SP_CERT_FILE` | string | _(empty)_ | Path to the mounted SP certificate PEM (only when signing requests) | +| `SSO_SAML_AUTHN_REQUESTS_SIGNED` | bool | `false` | Sign outbound AuthnRequests (requires the SP key below) | | `SSO_SAML_SP_KEY_FILE` | string | _(empty)_ | Path to the mounted SP private key PEM (only when signing requests) | ### Docker Compose Configuration @@ -100,6 +121,12 @@ These variables configure Docker Compose behavior only (not passed to containers | `DC_RUSTFS_IMAGE` | string | `rustfs/rustfs:1.0.1` | RustFS image | | `DC_AWS_CLI_IMAGE` | string | `amazon/aws-cli:latest` | AWS CLI image (for bucket init) | +#### Included RustFS + +| Variable | Type | Default | Description | +|----------|------|---------|-------------| +| `DC_RUSTFS_DELETE_ARTIFACTS_AFTER_DAYS` | int | _(unset)_ | Delete artifacts (screenshots, videos, traces, stdout) older than this many days, through a lifecycle rule `rustfs-init` applies. Unset keeps them until deleted. Other storage: [Artifact expiry](./object-storage.md#artifact-expiry) | + #### Port Configuration | Variable | Type | Default | Description | diff --git a/docs/index.md b/docs/index.md index 4ad3333..dee5f2d 100644 --- a/docs/index.md +++ b/docs/index.md @@ -13,7 +13,7 @@ Docker Compose configuration for self-hosted Currents deployment. - [Quickstart Guide](quickstart.md) — Get up and running with Docker Compose - [Configuration Reference](configuration.md) — All environment variables and settings - [Enable SAML SSO](sso-saml.md) — Delegate sign-in to your SAML identity provider -- [Use AWS S3 for Artifact Storage](object-storage.md) — IAM policy and CORS rules for your bucket +- [Use AWS S3 for Object Storage](object-storage.md) — IAM policy, CORS and lifecycle rules for your buckets - [Logging Configuration](logging.md) — Configure container logging for production - [Backup and Restore](backup-restore.md) — Backup and restore procedures - [Upgrading Currents On-Prem](upgrading.md) — Upgrade workflows and version management diff --git a/docs/object-storage.md b/docs/object-storage.md index 7b866a2..3298522 100644 --- a/docs/object-storage.md +++ b/docs/object-storage.md @@ -1,6 +1,6 @@ -# Use AWS S3 for Artifact Storage +# Use AWS S3 for Object Storage -Currents stores test artifacts (screenshots, videos, Playwright traces, stdout and attachments) in an S3-compatible bucket. This page covers the bucket permissions and CORS rules Currents needs when you use AWS S3 instead of the included RustFS service. +Currents stores test artifacts (screenshots, videos, Playwright traces, stdout and attachments) in an S3-compatible bucket, and context shares, organization logos and session files in a second one, the [app content bucket](#app-content-bucket). This page covers the bucket permissions, CORS and lifecycle rules Currents needs when you use AWS S3 instead of the included RustFS service. ## How Artifacts Reach the Bucket @@ -92,6 +92,52 @@ aws s3api put-bucket-cors --bucket currents-artifacts --cors-configuration '{ Uploads come from the CI reporter and from Currents services rather than a browser, so they don't need a CORS rule. +## App Content Bucket + +Context shares, organization logos and session files go to a separate bucket, configured with the `APP_CONTENT_STORAGE_*` variables. Without it, context sharing and logo editing are off and uploading session files answers `503`. + +```bash +APP_CONTENT_STORAGE_ENDPOINT=https://s3.us-east-1.amazonaws.com +APP_CONTENT_STORAGE_INTERNAL_ENDPOINT=https://s3.us-east-1.amazonaws.com +APP_CONTENT_STORAGE_REGION=us-east-1 +APP_CONTENT_STORAGE_BUCKET=currents-app-content +APP_CONTENT_STORAGE_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE +APP_CONTENT_STORAGE_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY +``` + +The same rules as the artifact bucket apply, with these differences: + +- **IAM policy:** the [policy above](#iam-policy) on the app content bucket, plus `s3:DeleteObject` on its objects. Deleting a session or organization removes its files. +- **CORS:** add `PUT` to `AllowedMethods`. Admins upload the organization logo from the browser. trace.playwright.dev is not needed. +- **Lifecycle rules:** shares expire, but nothing deletes their objects. Add these rules, and scope them to these prefixes only: the bucket also holds logos (`/logo/`) and session files (`/sessions/`), and a bucket-wide rule deletes them. + +```bash +aws s3api put-bucket-lifecycle-configuration --bucket currents-app-content --lifecycle-configuration '{ + "Rules": [ + { "ID": "ai-context-share", "Status": "Enabled", "Filter": { "Prefix": "ai-context-share/" }, "Expiration": { "Days": 2 } }, + { "ID": "share-3d", "Status": "Enabled", "Filter": { "Prefix": "share-3d/" }, "Expiration": { "Days": 4 } }, + { "ID": "share-7d", "Status": "Enabled", "Filter": { "Prefix": "share-7d/" }, "Expiration": { "Days": 8 } }, + { "ID": "trace-links", "Status": "Enabled", "Filter": { "Prefix": "trace-links/" }, "Expiration": { "Days": 8 } } + ] +}' +``` + +Each rule outlives the longest share written under its prefix by a day. The included RustFS service gets these rules from `rustfs-init`. + +## Artifact Expiry + +Currents does not delete artifacts. To limit how long they are kept, add a lifecycle rule to the artifact bucket. Replace `30` with the number of days and `currents-artifacts` with your bucket name: + +```bash +aws s3api put-bucket-lifecycle-configuration --bucket currents-artifacts --lifecycle-configuration '{ + "Rules": [ + { "ID": "delete-artifacts-after-days", "Status": "Enabled", "Filter": { "Prefix": "" }, "Expiration": { "Days": 30 } } + ] +}' +``` + +Runs older than that still show in the dashboard, without their screenshots, videos, traces and stdout. With the included RustFS service, set `DC_RUSTFS_DELETE_ARTIFACTS_AFTER_DAYS` instead. + ## Troubleshooting 403 Errors Open the failing request in the browser's developer tools (**Network** tab): diff --git a/docs/quickstart.md b/docs/quickstart.md index 97f30a9..4619fe1 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -78,6 +78,8 @@ APP_BASE_URL=http://localhost:4000 CURRENTS_RECORD_API_URL=http://localhost:1234 ``` +The OAuth provider for remote MCP, on in `.env.example`, needs `https` or `localhost`. With an `http` `APP_BASE_URL` on any other host, set `OAUTH_PROVIDER_ENABLED=false`, or the API refuses to start. To evaluate OAuth without TLS on a name other than `localhost`, use a `.localhost` name such as `http://currents.localhost:4000`, and set `OAUTH_HTTP_ALLOWED_HOSTS=currents.localhost` and `OAUTH_JWKS_URL=http://localhost:4000/api/auth/jwks`. + ### Root User Account The `ON_PREM_EMAIL` is the email address used to create the initial root admin user: @@ -108,9 +110,11 @@ FILE_STORAGE_REGION=us-east-1 # FILE_STORAGE_FORCE_PATH_STYLE=true ``` -For AWS S3, also set `FILE_STORAGE_INTERNAL_ENDPOINT` to the same endpoint, and give the bucket the IAM policy and CORS rules in [Use AWS S3 for Artifact Storage](./object-storage.md). +For AWS S3, also set `FILE_STORAGE_INTERNAL_ENDPOINT` to the same endpoint, and give the bucket the IAM policy and CORS rules in [Use AWS S3 for Object Storage](./object-storage.md). + +Context shares, organization logos and session files need a second bucket, set with the `APP_CONTENT_STORAGE_*` variables. `.env.example` reuses the artifact storage endpoint and credentials for it. Create the bucket and add the lifecycle rules in [App Content Bucket](./object-storage.md#app-content-bucket). -If using the included RustFS for testing, configure the `RUSTFS_*` variables instead. The RustFS profile automatically sets `FILE_STORAGE_FORCE_PATH_STYLE=true` for all services. +If using the included RustFS for testing, configure the `RUSTFS_*` variables instead. The RustFS profile automatically sets `FILE_STORAGE_FORCE_PATH_STYLE=true` and `APP_CONTENT_STORAGE_FORCE_PATH_STYLE=true` for all services, and creates both buckets. > ⚠️ **Production Note:** RustFS is intended for local development and testing only—it is **not recommended for production deployments**. The included Docker Compose configuration is designed for local development; production environments should use external, production-grade object storage backends such as AWS S3, Google Cloud Storage, or a managed MinIO cluster. diff --git a/docs/sso-saml.md b/docs/sso-saml.md index eaf15e7..fcd4467 100644 --- a/docs/sso-saml.md +++ b/docs/sso-saml.md @@ -74,12 +74,12 @@ Go to your dashboard and enter your email on the sign-in screen. If your email i |----------|---------| | `SSO_SAML_DEFAULT_ROLE` | Role for auto-provisioned SSO users. Default `member`. | | `SSO_ALLOWED_DOMAINS` | Comma-separated allow-list of email domains, e.g. `acme.com,acme.io`. Defense-in-depth on top of your IdP. | -| `SSO_SAML_AUTHN_REQUESTS_SIGNED` | Set `true` to sign outbound AuthnRequests. Requires an SP key/cert (below). | -| `SSO_SAML_SP_CERT_FILE` / `SSO_SAML_SP_KEY_FILE` | Paths to your SP certificate/private key PEMs (place them in `./data/sso` too). Only needed for signed requests. | +| `SSO_SAML_AUTHN_REQUESTS_SIGNED` | Set `true` to sign outbound AuthnRequests. Requires an SP key (below). | +| `SSO_SAML_SP_KEY_FILE` | Path to your SP private key PEM (place it in `./data/sso` too). Only needed for signed requests. | ### Signing AuthnRequests (advanced) -If your IdP requires signed requests, generate a self-signed SP keypair, drop it in `./data/sso`, and point the vars at it: +If your IdP requires signed requests, generate a self-signed SP keypair in `./data/sso`, upload `sp-cert.pem` to your IdP so it can verify the signatures, and point Currents at the key: ```bash openssl req -x509 -newkey rsa:2048 -nodes \ @@ -89,7 +89,6 @@ openssl req -x509 -newkey rsa:2048 -nodes \ ```bash SSO_SAML_AUTHN_REQUESTS_SIGNED=true -SSO_SAML_SP_CERT_FILE=/etc/currents/sso/sp-cert.pem SSO_SAML_SP_KEY_FILE=/etc/currents/sso/sp-key.pem ``` diff --git a/on-prem/.env.example b/on-prem/.env.example index 88b9178..159cbbb 100644 --- a/on-prem/.env.example +++ b/on-prem/.env.example @@ -76,6 +76,16 @@ FILE_STORAGE_ACCESS_KEY_ID=${RUSTFS_ACCESS_KEY} FILE_STORAGE_SECRET_ACCESS_KEY=${RUSTFS_SECRET_KEY} # FILE_STORAGE_REGION=us-east-1 +# App content storage: context shares, organization logos and session files. +# A second bucket (the included RustFS creates it). Unset, context sharing and +# logo editing are off and session files cannot be uploaded. On your own +# storage, add the lifecycle rules in docs/object-storage.md. +APP_CONTENT_STORAGE_ENDPOINT=${FILE_STORAGE_ENDPOINT} +APP_CONTENT_STORAGE_BUCKET="currents-app-content" +APP_CONTENT_STORAGE_ACCESS_KEY_ID=${FILE_STORAGE_ACCESS_KEY_ID} +APP_CONTENT_STORAGE_SECRET_ACCESS_KEY=${FILE_STORAGE_SECRET_ACCESS_KEY} +# APP_CONTENT_STORAGE_REGION=us-east-1 + # ClickHouse CLICKHOUSE_URL=http://clickhouse:8123 CLICKHOUSE_USERNAME=currents @@ -115,6 +125,8 @@ REDIS_URI_SLAVE=redis://redis:6379 # Object Storage (addition configuration) FILE_STORAGE_INTERNAL_ENDPOINT="http://host.docker.internal:9000" # FILE_STORAGE_FORCE_PATH_STYLE=true # Set automatically when using RustFS profile +APP_CONTENT_STORAGE_INTERNAL_ENDPOINT=${FILE_STORAGE_INTERNAL_ENDPOINT} +# APP_CONTENT_STORAGE_FORCE_PATH_STYLE=true # Set automatically when using RustFS profile # Email (additional settings) AUTOMATED_REPORTS_EMAIL_BCC= @@ -126,6 +138,40 @@ INVITE_EXPIRATION_DAYS= # CORALOGIX_API_KEY= # CORALOGIX_APP_NAME="customer-name" +# ============================================================================= +# Remote MCP and OAuth +# ============================================================================= +# Remote MCP at /mcp is on unless set to false. It takes a +# Currents API key or personal access token as a bearer token, or an OAuth +# token when the provider below is on. Anyone who can reach the install can +# reach the endpoint; it answers 401 without a valid credential. +# MCP_ENABLED=true +# MCP_ORIGINS= # other origins clients reach /mcp on, comma separated + +# OAuth provider: MCP clients sign in with a Currents account instead of a key. +# Issuer, token audience and JWKS URL are derived from APP_BASE_URL. Adds the +# /api/auth/oauth2/* and /.well-known/oauth-* endpoints. Use an https +# APP_BASE_URL for any real setup. Over http it works only on localhost, or, for +# an evaluation, on a .localhost name listed in OAUTH_HTTP_ALLOWED_HOSTS +# (APP_BASE_URL=http://currents.localhost:4000). On any other http host the API +# refuses to start with it on, so set it to false there; MCP keeps working +# with keys. +OAUTH_PROVIDER_ENABLED=true +# Lets a client register itself: the way to add one without internet access. +OAUTH_DYNAMIC_CLIENT_REGISTRATION_ENABLED=true +# Client ID metadata documents: a client names an https URL the server fetches +# its metadata from. Hosts resolving to private, loopback or link-local +# addresses are refused. On unless set to false. +# OAUTH_CIMD_ENABLED=true +# For a .localhost APP_BASE_URL, both of these. The API checks each access token +# against its own signing keys, fetched from APP_BASE_URL unless +# OAUTH_JWKS_URL says otherwise, and the container cannot resolve .localhost. +# OAUTH_HTTP_ALLOWED_HOSTS=currents.localhost +# OAUTH_JWKS_URL=http://localhost:4000/api/auth/jwks + +# Public API origin in share links and the OAuth token audience (default: APP_BASE_URL) +# API_BASE_URL= + # ============================================================================= # SAML SSO (Optional) # ============================================================================= @@ -152,8 +198,7 @@ INVITE_EXPIRATION_DAYS= # Optional: # SSO_SAML_DEFAULT_ROLE=member # role for auto-provisioned SSO users (default: member) # SSO_ALLOWED_DOMAINS=acme.com,acme.io # restrict SSO to these email domains -# SSO_SAML_AUTHN_REQUESTS_SIGNED=false # sign AuthnRequests (requires the SP key/cert below) -# SSO_SAML_SP_CERT_FILE=/etc/currents/sso/sp-cert.pem +# SSO_SAML_AUTHN_REQUESTS_SIGNED=false # sign AuthnRequests (requires the SP key below) # SSO_SAML_SP_KEY_FILE=/etc/currents/sso/sp-key.pem # ============================================================================= @@ -168,6 +213,11 @@ INVITE_EXPIRATION_DAYS= # DC_RUSTFS_IMAGE=rustfs/rustfs:1.0.1 # DC_AWS_CLI_IMAGE=amazon/aws-cli:latest +# Included RustFS only: delete artifacts (screenshots, videos, traces, stdout) +# older than this many days. Unset keeps them until deleted. Other storage: +# see docs/object-storage.md. +# DC_RUSTFS_DELETE_ARTIFACTS_AFTER_DAYS=30 + # Port Configuration # Currents services default to all interfaces, databases default to localhost only # DC_DIRECTOR_PORT=1234 # Director API (default: all interfaces) diff --git a/on-prem/CHANGELOG.md b/on-prem/CHANGELOG.md index cef5b61..adcda64 100644 --- a/on-prem/CHANGELOG.md +++ b/on-prem/CHANGELOG.md @@ -9,10 +9,22 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Compose File Changes - The bundled RustFS image is `rustfs/rustfs:1.0.1` (requires `./scripts/generate-compose.sh` if using custom templates) - The `rustfs` service sets `RUSTFS_CORS_ALLOWED_ORIGINS: "*"`. `1.0.1` sends no CORS headers without it, which breaks trace.playwright.dev and the dashboard's stdout and attachment previews. If you run `1.0.1` from your own compose file, add it there too. +- The `api`, `changestreams-worker` and `write-worker` services read the app content bucket from `APP_CONTENT_STORAGE_*`, and `api` sets `MCP_ENABLED` (requires `./scripts/generate-compose.sh` if using custom templates) +- `rustfs-init` creates the app content bucket beside the artifact bucket and adds its lifecycle rules. Each run rewrites only the rules it owns, matched by ID (`ai-context-share`, `share-3d`, `share-7d`, `trace-links`, `delete-artifacts-after-days`), and keeps any others + +### New Environment Variables +- `APP_CONTENT_STORAGE_ENDPOINT`, `APP_CONTENT_STORAGE_BUCKET`, `APP_CONTENT_STORAGE_ACCESS_KEY_ID`, `APP_CONTENT_STORAGE_SECRET_ACCESS_KEY`, `APP_CONTENT_STORAGE_REGION`, `APP_CONTENT_STORAGE_INTERNAL_ENDPOINT`, `APP_CONTENT_STORAGE_FORCE_PATH_STYLE` — the app content bucket for context shares, organization logos and session files. Without it, context sharing and logo editing are off and uploading session files answers `503`. Copy the block from `.env.example`; on your own storage, create the bucket and add the lifecycle rules in [docs/object-storage.md](../docs/object-storage.md#app-content-bucket) +- `MCP_ENABLED` (default `true`), `MCP_ORIGINS` — remote MCP at `/mcp` +- `OAUTH_PROVIDER_ENABLED`, `OAUTH_DYNAMIC_CLIENT_REGISTRATION_ENABLED`, `OAUTH_CIMD_ENABLED`, `OAUTH_HTTP_ALLOWED_HOSTS`, `OAUTH_JWKS_URL` — sign-in to remote MCP with a Currents account. `.env.example` turns the provider on; an upgraded `.env` without `OAUTH_PROVIDER_ENABLED=true` keeps it off. It needs an `https` `APP_BASE_URL`, or `http` on `localhost` (or, for an evaluation, a `.localhost` name in `OAUTH_HTTP_ALLOWED_HOSTS`, with `OAUTH_JWKS_URL` on `localhost`): with `http` on any other host the API refuses to start, so set `OAUTH_PROVIDER_ENABLED=false` +- `API_BASE_URL` — public API origin, default `APP_BASE_URL` +- `DC_RUSTFS_DELETE_ARTIFACTS_AFTER_DAYS` — delete artifacts in the bundled RustFS after this many days. Unset keeps them ### Changed Environment Variables - `DC_RUSTFS_IMAGE` defaults to `rustfs/rustfs:1.0.1`. If your `.env` sets it, update the value to pick up the fixes below. +### Removed Environment Variables +- `SSO_SAML_SP_CERT_FILE` — the API never read it. Signed AuthnRequests need only `SSO_SAML_SP_KEY_FILE`; upload the certificate to your IdP. + ### Fixed - The bundled RustFS enforces bucket lifecycle rules. `1.0.0-alpha.79` accepted a lifecycle configuration and returned it from `GetBucketLifecycleConfiguration`, but never expired the matching objects. - The bundled RustFS includes the fix for CVE-2026-73288, where lifecycle and scanner sweeps could expire objects under COMPLIANCE retention when lock metadata was missing or unreadable. diff --git a/on-prem/docker-compose.cache.yml b/on-prem/docker-compose.cache.yml index c8104fd..e7b1fc5 100644 --- a/on-prem/docker-compose.cache.yml +++ b/on-prem/docker-compose.cache.yml @@ -47,6 +47,18 @@ services: BETTER_AUTH_ENABLED: "true" CLICKHOUSE_PASSWORD: ${CLICKHOUSE_CURRENTS_PASSWORD} EMAIL_TRANSPORTER: smtp + # Remote MCP at /mcp, which takes an API key or a personal access token. + MCP_ENABLED: ${MCP_ENABLED:-true} + # The app-content bucket (shares, logos, session files). APP_CONTENT_STORAGE_* + # in .env is the operator-facing name; the services read PERMANENT_STORAGE_*, + # which .env may also set directly. + PERMANENT_STORAGE_ENDPOINT: ${APP_CONTENT_STORAGE_ENDPOINT:-${PERMANENT_STORAGE_ENDPOINT:-}} + PERMANENT_STORAGE_INTERNAL_ENDPOINT: ${APP_CONTENT_STORAGE_INTERNAL_ENDPOINT:-${PERMANENT_STORAGE_INTERNAL_ENDPOINT:-}} + PERMANENT_STORAGE_BUCKET: ${APP_CONTENT_STORAGE_BUCKET:-${PERMANENT_STORAGE_BUCKET:-}} + PERMANENT_STORAGE_REGION: ${APP_CONTENT_STORAGE_REGION:-${PERMANENT_STORAGE_REGION:-}} + PERMANENT_STORAGE_ACCESS_KEY_ID: ${APP_CONTENT_STORAGE_ACCESS_KEY_ID:-${PERMANENT_STORAGE_ACCESS_KEY_ID:-}} + PERMANENT_STORAGE_SECRET_ACCESS_KEY: ${APP_CONTENT_STORAGE_SECRET_ACCESS_KEY:-${PERMANENT_STORAGE_SECRET_ACCESS_KEY:-}} + PERMANENT_STORAGE_FORCE_PATH_STYLE: ${APP_CONTENT_STORAGE_FORCE_PATH_STYLE:-${PERMANENT_STORAGE_FORCE_PATH_STYLE:-}} volumes: - ${DC_SSO_VOLUME:-./data/sso}:/etc/currents/sso:ro depends_on: @@ -64,6 +76,16 @@ services: CURRENTS_ENV: onprem CLICKHOUSE_PASSWORD: ${CLICKHOUSE_CURRENTS_PASSWORD} EMAIL_TRANSPORTER: smtp + # The app-content bucket (shares, logos, session files). APP_CONTENT_STORAGE_* + # in .env is the operator-facing name; the services read PERMANENT_STORAGE_*, + # which .env may also set directly. + PERMANENT_STORAGE_ENDPOINT: ${APP_CONTENT_STORAGE_ENDPOINT:-${PERMANENT_STORAGE_ENDPOINT:-}} + PERMANENT_STORAGE_INTERNAL_ENDPOINT: ${APP_CONTENT_STORAGE_INTERNAL_ENDPOINT:-${PERMANENT_STORAGE_INTERNAL_ENDPOINT:-}} + PERMANENT_STORAGE_BUCKET: ${APP_CONTENT_STORAGE_BUCKET:-${PERMANENT_STORAGE_BUCKET:-}} + PERMANENT_STORAGE_REGION: ${APP_CONTENT_STORAGE_REGION:-${PERMANENT_STORAGE_REGION:-}} + PERMANENT_STORAGE_ACCESS_KEY_ID: ${APP_CONTENT_STORAGE_ACCESS_KEY_ID:-${PERMANENT_STORAGE_ACCESS_KEY_ID:-}} + PERMANENT_STORAGE_SECRET_ACCESS_KEY: ${APP_CONTENT_STORAGE_SECRET_ACCESS_KEY:-${PERMANENT_STORAGE_SECRET_ACCESS_KEY:-}} + PERMANENT_STORAGE_FORCE_PATH_STYLE: ${APP_CONTENT_STORAGE_FORCE_PATH_STYLE:-${PERMANENT_STORAGE_FORCE_PATH_STYLE:-}} depends_on: redis: condition: service_started @@ -79,6 +101,16 @@ services: CURRENTS_ENV: onprem CLICKHOUSE_PASSWORD: ${CLICKHOUSE_CURRENTS_PASSWORD} EMAIL_TRANSPORTER: smtp + # The app-content bucket (shares, logos, session files). APP_CONTENT_STORAGE_* + # in .env is the operator-facing name; the services read PERMANENT_STORAGE_*, + # which .env may also set directly. + PERMANENT_STORAGE_ENDPOINT: ${APP_CONTENT_STORAGE_ENDPOINT:-${PERMANENT_STORAGE_ENDPOINT:-}} + PERMANENT_STORAGE_INTERNAL_ENDPOINT: ${APP_CONTENT_STORAGE_INTERNAL_ENDPOINT:-${PERMANENT_STORAGE_INTERNAL_ENDPOINT:-}} + PERMANENT_STORAGE_BUCKET: ${APP_CONTENT_STORAGE_BUCKET:-${PERMANENT_STORAGE_BUCKET:-}} + PERMANENT_STORAGE_REGION: ${APP_CONTENT_STORAGE_REGION:-${PERMANENT_STORAGE_REGION:-}} + PERMANENT_STORAGE_ACCESS_KEY_ID: ${APP_CONTENT_STORAGE_ACCESS_KEY_ID:-${PERMANENT_STORAGE_ACCESS_KEY_ID:-}} + PERMANENT_STORAGE_SECRET_ACCESS_KEY: ${APP_CONTENT_STORAGE_SECRET_ACCESS_KEY:-${PERMANENT_STORAGE_SECRET_ACCESS_KEY:-}} + PERMANENT_STORAGE_FORCE_PATH_STYLE: ${APP_CONTENT_STORAGE_FORCE_PATH_STYLE:-${PERMANENT_STORAGE_FORCE_PATH_STYLE:-}} depends_on: redis: condition: service_started diff --git a/on-prem/docker-compose.database.yml b/on-prem/docker-compose.database.yml index 0037a2b..c86966c 100644 --- a/on-prem/docker-compose.database.yml +++ b/on-prem/docker-compose.database.yml @@ -50,6 +50,18 @@ services: BETTER_AUTH_ENABLED: "true" CLICKHOUSE_PASSWORD: ${CLICKHOUSE_CURRENTS_PASSWORD} EMAIL_TRANSPORTER: smtp + # Remote MCP at /mcp, which takes an API key or a personal access token. + MCP_ENABLED: ${MCP_ENABLED:-true} + # The app-content bucket (shares, logos, session files). APP_CONTENT_STORAGE_* + # in .env is the operator-facing name; the services read PERMANENT_STORAGE_*, + # which .env may also set directly. + PERMANENT_STORAGE_ENDPOINT: ${APP_CONTENT_STORAGE_ENDPOINT:-${PERMANENT_STORAGE_ENDPOINT:-}} + PERMANENT_STORAGE_INTERNAL_ENDPOINT: ${APP_CONTENT_STORAGE_INTERNAL_ENDPOINT:-${PERMANENT_STORAGE_INTERNAL_ENDPOINT:-}} + PERMANENT_STORAGE_BUCKET: ${APP_CONTENT_STORAGE_BUCKET:-${PERMANENT_STORAGE_BUCKET:-}} + PERMANENT_STORAGE_REGION: ${APP_CONTENT_STORAGE_REGION:-${PERMANENT_STORAGE_REGION:-}} + PERMANENT_STORAGE_ACCESS_KEY_ID: ${APP_CONTENT_STORAGE_ACCESS_KEY_ID:-${PERMANENT_STORAGE_ACCESS_KEY_ID:-}} + PERMANENT_STORAGE_SECRET_ACCESS_KEY: ${APP_CONTENT_STORAGE_SECRET_ACCESS_KEY:-${PERMANENT_STORAGE_SECRET_ACCESS_KEY:-}} + PERMANENT_STORAGE_FORCE_PATH_STYLE: ${APP_CONTENT_STORAGE_FORCE_PATH_STYLE:-${PERMANENT_STORAGE_FORCE_PATH_STYLE:-}} volumes: - ${DC_SSO_VOLUME:-./data/sso}:/etc/currents/sso:ro depends_on: @@ -70,6 +82,16 @@ services: CURRENTS_ENV: onprem CLICKHOUSE_PASSWORD: ${CLICKHOUSE_CURRENTS_PASSWORD} EMAIL_TRANSPORTER: smtp + # The app-content bucket (shares, logos, session files). APP_CONTENT_STORAGE_* + # in .env is the operator-facing name; the services read PERMANENT_STORAGE_*, + # which .env may also set directly. + PERMANENT_STORAGE_ENDPOINT: ${APP_CONTENT_STORAGE_ENDPOINT:-${PERMANENT_STORAGE_ENDPOINT:-}} + PERMANENT_STORAGE_INTERNAL_ENDPOINT: ${APP_CONTENT_STORAGE_INTERNAL_ENDPOINT:-${PERMANENT_STORAGE_INTERNAL_ENDPOINT:-}} + PERMANENT_STORAGE_BUCKET: ${APP_CONTENT_STORAGE_BUCKET:-${PERMANENT_STORAGE_BUCKET:-}} + PERMANENT_STORAGE_REGION: ${APP_CONTENT_STORAGE_REGION:-${PERMANENT_STORAGE_REGION:-}} + PERMANENT_STORAGE_ACCESS_KEY_ID: ${APP_CONTENT_STORAGE_ACCESS_KEY_ID:-${PERMANENT_STORAGE_ACCESS_KEY_ID:-}} + PERMANENT_STORAGE_SECRET_ACCESS_KEY: ${APP_CONTENT_STORAGE_SECRET_ACCESS_KEY:-${PERMANENT_STORAGE_SECRET_ACCESS_KEY:-}} + PERMANENT_STORAGE_FORCE_PATH_STYLE: ${APP_CONTENT_STORAGE_FORCE_PATH_STYLE:-${PERMANENT_STORAGE_FORCE_PATH_STYLE:-}} depends_on: mongodb: condition: service_healthy @@ -89,6 +111,16 @@ services: CURRENTS_ENV: onprem CLICKHOUSE_PASSWORD: ${CLICKHOUSE_CURRENTS_PASSWORD} EMAIL_TRANSPORTER: smtp + # The app-content bucket (shares, logos, session files). APP_CONTENT_STORAGE_* + # in .env is the operator-facing name; the services read PERMANENT_STORAGE_*, + # which .env may also set directly. + PERMANENT_STORAGE_ENDPOINT: ${APP_CONTENT_STORAGE_ENDPOINT:-${PERMANENT_STORAGE_ENDPOINT:-}} + PERMANENT_STORAGE_INTERNAL_ENDPOINT: ${APP_CONTENT_STORAGE_INTERNAL_ENDPOINT:-${PERMANENT_STORAGE_INTERNAL_ENDPOINT:-}} + PERMANENT_STORAGE_BUCKET: ${APP_CONTENT_STORAGE_BUCKET:-${PERMANENT_STORAGE_BUCKET:-}} + PERMANENT_STORAGE_REGION: ${APP_CONTENT_STORAGE_REGION:-${PERMANENT_STORAGE_REGION:-}} + PERMANENT_STORAGE_ACCESS_KEY_ID: ${APP_CONTENT_STORAGE_ACCESS_KEY_ID:-${PERMANENT_STORAGE_ACCESS_KEY_ID:-}} + PERMANENT_STORAGE_SECRET_ACCESS_KEY: ${APP_CONTENT_STORAGE_SECRET_ACCESS_KEY:-${PERMANENT_STORAGE_SECRET_ACCESS_KEY:-}} + PERMANENT_STORAGE_FORCE_PATH_STYLE: ${APP_CONTENT_STORAGE_FORCE_PATH_STYLE:-${PERMANENT_STORAGE_FORCE_PATH_STYLE:-}} depends_on: clickhouse: condition: service_healthy diff --git a/on-prem/docker-compose.full.yml b/on-prem/docker-compose.full.yml index add57cd..771cff8 100644 --- a/on-prem/docker-compose.full.yml +++ b/on-prem/docker-compose.full.yml @@ -51,6 +51,18 @@ services: BETTER_AUTH_ENABLED: "true" CLICKHOUSE_PASSWORD: ${CLICKHOUSE_CURRENTS_PASSWORD} EMAIL_TRANSPORTER: smtp + # Remote MCP at /mcp, which takes an API key or a personal access token. + MCP_ENABLED: ${MCP_ENABLED:-true} + # The app-content bucket (shares, logos, session files). APP_CONTENT_STORAGE_* + # in .env is the operator-facing name; the services read PERMANENT_STORAGE_*, + # which .env may also set directly. + PERMANENT_STORAGE_ENDPOINT: ${APP_CONTENT_STORAGE_ENDPOINT:-${PERMANENT_STORAGE_ENDPOINT:-}} + PERMANENT_STORAGE_INTERNAL_ENDPOINT: ${APP_CONTENT_STORAGE_INTERNAL_ENDPOINT:-${PERMANENT_STORAGE_INTERNAL_ENDPOINT:-}} + PERMANENT_STORAGE_BUCKET: ${APP_CONTENT_STORAGE_BUCKET:-${PERMANENT_STORAGE_BUCKET:-}} + PERMANENT_STORAGE_REGION: ${APP_CONTENT_STORAGE_REGION:-${PERMANENT_STORAGE_REGION:-}} + PERMANENT_STORAGE_ACCESS_KEY_ID: ${APP_CONTENT_STORAGE_ACCESS_KEY_ID:-${PERMANENT_STORAGE_ACCESS_KEY_ID:-}} + PERMANENT_STORAGE_SECRET_ACCESS_KEY: ${APP_CONTENT_STORAGE_SECRET_ACCESS_KEY:-${PERMANENT_STORAGE_SECRET_ACCESS_KEY:-}} + PERMANENT_STORAGE_FORCE_PATH_STYLE: "true" FILE_STORAGE_FORCE_PATH_STYLE: "true" volumes: - ${DC_SSO_VOLUME:-./data/sso}:/etc/currents/sso:ro @@ -72,6 +84,16 @@ services: CURRENTS_ENV: onprem CLICKHOUSE_PASSWORD: ${CLICKHOUSE_CURRENTS_PASSWORD} EMAIL_TRANSPORTER: smtp + # The app-content bucket (shares, logos, session files). APP_CONTENT_STORAGE_* + # in .env is the operator-facing name; the services read PERMANENT_STORAGE_*, + # which .env may also set directly. + PERMANENT_STORAGE_ENDPOINT: ${APP_CONTENT_STORAGE_ENDPOINT:-${PERMANENT_STORAGE_ENDPOINT:-}} + PERMANENT_STORAGE_INTERNAL_ENDPOINT: ${APP_CONTENT_STORAGE_INTERNAL_ENDPOINT:-${PERMANENT_STORAGE_INTERNAL_ENDPOINT:-}} + PERMANENT_STORAGE_BUCKET: ${APP_CONTENT_STORAGE_BUCKET:-${PERMANENT_STORAGE_BUCKET:-}} + PERMANENT_STORAGE_REGION: ${APP_CONTENT_STORAGE_REGION:-${PERMANENT_STORAGE_REGION:-}} + PERMANENT_STORAGE_ACCESS_KEY_ID: ${APP_CONTENT_STORAGE_ACCESS_KEY_ID:-${PERMANENT_STORAGE_ACCESS_KEY_ID:-}} + PERMANENT_STORAGE_SECRET_ACCESS_KEY: ${APP_CONTENT_STORAGE_SECRET_ACCESS_KEY:-${PERMANENT_STORAGE_SECRET_ACCESS_KEY:-}} + PERMANENT_STORAGE_FORCE_PATH_STYLE: "true" FILE_STORAGE_FORCE_PATH_STYLE: "true" depends_on: mongodb: @@ -92,6 +114,16 @@ services: CURRENTS_ENV: onprem CLICKHOUSE_PASSWORD: ${CLICKHOUSE_CURRENTS_PASSWORD} EMAIL_TRANSPORTER: smtp + # The app-content bucket (shares, logos, session files). APP_CONTENT_STORAGE_* + # in .env is the operator-facing name; the services read PERMANENT_STORAGE_*, + # which .env may also set directly. + PERMANENT_STORAGE_ENDPOINT: ${APP_CONTENT_STORAGE_ENDPOINT:-${PERMANENT_STORAGE_ENDPOINT:-}} + PERMANENT_STORAGE_INTERNAL_ENDPOINT: ${APP_CONTENT_STORAGE_INTERNAL_ENDPOINT:-${PERMANENT_STORAGE_INTERNAL_ENDPOINT:-}} + PERMANENT_STORAGE_BUCKET: ${APP_CONTENT_STORAGE_BUCKET:-${PERMANENT_STORAGE_BUCKET:-}} + PERMANENT_STORAGE_REGION: ${APP_CONTENT_STORAGE_REGION:-${PERMANENT_STORAGE_REGION:-}} + PERMANENT_STORAGE_ACCESS_KEY_ID: ${APP_CONTENT_STORAGE_ACCESS_KEY_ID:-${PERMANENT_STORAGE_ACCESS_KEY_ID:-}} + PERMANENT_STORAGE_SECRET_ACCESS_KEY: ${APP_CONTENT_STORAGE_SECRET_ACCESS_KEY:-${PERMANENT_STORAGE_SECRET_ACCESS_KEY:-}} + PERMANENT_STORAGE_FORCE_PATH_STYLE: "true" FILE_STORAGE_FORCE_PATH_STYLE: "true" depends_on: clickhouse: @@ -302,9 +334,64 @@ services: AWS_ACCESS_KEY_ID: ${RUSTFS_ACCESS_KEY} AWS_SECRET_ACCESS_KEY: ${RUSTFS_SECRET_KEY} AWS_DEFAULT_REGION: us-east-1 - entrypoint: > - sh -c "aws --endpoint-url=http://rustfs:9000 s3 mb s3://currents || true" - + AWS_ENDPOINT_URL: http://rustfs:9000 + ARTIFACT_BUCKET: ${FILE_STORAGE_BUCKET:-currents} + APP_CONTENT_BUCKET: ${APP_CONTENT_STORAGE_BUCKET:-currents-app-content} + DELETE_ARTIFACTS_AFTER_DAYS: ${DC_RUSTFS_DELETE_ARTIFACTS_AFTER_DAYS:-} + # The app-content rules expire share markdown a day after its link and trace + # links after their longest lifetime. Never bucket-wide: the same bucket keeps + # organization logos and session files for good. Each run rewrites the rules + # this init owns, matched by ID, and keeps any other rules on the buckets. + entrypoint: + - sh + - -c + - | + set -e + for bucket in "$$ARTIFACT_BUCKET" "$$APP_CONTENT_BUCKET"; do + # A create that loses a race to another init finds the bucket there. + aws s3api head-bucket --bucket "$$bucket" 2>/dev/null || + aws s3 mb "s3://$$bucket" || + aws s3api head-bucket --bucket "$$bucket" + done + # The bucket's rules whose ID is not in $$2, as JSON. Any error other than + # an absent configuration stops the init rather than dropping those rules. + other_rules() { + filter="" + for id in $$2; do filter="$${filter:+$$filter && }ID!='$$id'"; done + if out=$$(aws s3api get-bucket-lifecycle-configuration --bucket "$$1" --query "Rules[?$$filter]" --output json 2>&1); then + echo "$$out" + else + case "$$out" in + *NoSuchLifecycleConfiguration*) echo "[]" ;; + *) echo "$$out" >&2; return 1 ;; + esac + fi + } + # Sets bucket $$1's rules to its other rules plus $$3 (comma-separated rule + # objects, or empty), deleting the configuration when none are left. + put_rules() { + others=$$(other_rules "$$1" "$$2") + rules=$$(printf '%s' "$$others" | tr -d '\n' | sed 's/^ *\[ *//; s/ *\] *$$//') + if [ -n "$$rules" ] && [ -n "$$3" ]; then rules="$$rules, $$3"; else rules="$$rules$$3"; fi + if [ -n "$$rules" ]; then + aws s3api put-bucket-lifecycle-configuration --bucket "$$1" --lifecycle-configuration "{ \"Rules\": [$$rules] }" + else + aws s3api delete-bucket-lifecycle --bucket "$$1" + fi + } + put_rules "$$APP_CONTENT_BUCKET" "ai-context-share share-3d share-7d trace-links" ' + { "ID": "ai-context-share", "Status": "Enabled", "Filter": { "Prefix": "ai-context-share/" }, "Expiration": { "Days": 2 } }, + { "ID": "share-3d", "Status": "Enabled", "Filter": { "Prefix": "share-3d/" }, "Expiration": { "Days": 4 } }, + { "ID": "share-7d", "Status": "Enabled", "Filter": { "Prefix": "share-7d/" }, "Expiration": { "Days": 8 } }, + { "ID": "trace-links", "Status": "Enabled", "Filter": { "Prefix": "trace-links/" }, "Expiration": { "Days": 8 } }' + artifact_rule="" + if [ -n "$$DELETE_ARTIFACTS_AFTER_DAYS" ]; then + case "$$DELETE_ARTIFACTS_AFTER_DAYS" in + *[!0-9]* | 0*) echo "DC_RUSTFS_DELETE_ARTIFACTS_AFTER_DAYS must be a whole number of days above 0" >&2; exit 1 ;; + esac + artifact_rule="{ \"ID\": \"delete-artifacts-after-days\", \"Status\": \"Enabled\", \"Filter\": { \"Prefix\": \"\" }, \"Expiration\": { \"Days\": $$DELETE_ARTIFACTS_AFTER_DAYS } }" + fi + put_rules "$$ARTIFACT_BUCKET" "delete-artifacts-after-days" "$$artifact_rule" networks: - default networks: diff --git a/on-prem/templates/compose.currents.yml b/on-prem/templates/compose.currents.yml index 4fae7a7..1e8cc2e 100644 --- a/on-prem/templates/compose.currents.yml +++ b/on-prem/templates/compose.currents.yml @@ -31,6 +31,18 @@ services: BETTER_AUTH_ENABLED: "true" CLICKHOUSE_PASSWORD: ${CLICKHOUSE_CURRENTS_PASSWORD} EMAIL_TRANSPORTER: smtp + # Remote MCP at /mcp, which takes an API key or a personal access token. + MCP_ENABLED: ${MCP_ENABLED:-true} + # The app-content bucket (shares, logos, session files). APP_CONTENT_STORAGE_* + # in .env is the operator-facing name; the services read PERMANENT_STORAGE_*, + # which .env may also set directly. + PERMANENT_STORAGE_ENDPOINT: ${APP_CONTENT_STORAGE_ENDPOINT:-${PERMANENT_STORAGE_ENDPOINT:-}} + PERMANENT_STORAGE_INTERNAL_ENDPOINT: ${APP_CONTENT_STORAGE_INTERNAL_ENDPOINT:-${PERMANENT_STORAGE_INTERNAL_ENDPOINT:-}} + PERMANENT_STORAGE_BUCKET: ${APP_CONTENT_STORAGE_BUCKET:-${PERMANENT_STORAGE_BUCKET:-}} + PERMANENT_STORAGE_REGION: ${APP_CONTENT_STORAGE_REGION:-${PERMANENT_STORAGE_REGION:-}} + PERMANENT_STORAGE_ACCESS_KEY_ID: ${APP_CONTENT_STORAGE_ACCESS_KEY_ID:-${PERMANENT_STORAGE_ACCESS_KEY_ID:-}} + PERMANENT_STORAGE_SECRET_ACCESS_KEY: ${APP_CONTENT_STORAGE_SECRET_ACCESS_KEY:-${PERMANENT_STORAGE_SECRET_ACCESS_KEY:-}} + PERMANENT_STORAGE_FORCE_PATH_STYLE: ${APP_CONTENT_STORAGE_FORCE_PATH_STYLE:-${PERMANENT_STORAGE_FORCE_PATH_STYLE:-}} volumes: - ${DC_SSO_VOLUME:-./data/sso}:/etc/currents/sso:ro @@ -45,6 +57,16 @@ services: CURRENTS_ENV: onprem CLICKHOUSE_PASSWORD: ${CLICKHOUSE_CURRENTS_PASSWORD} EMAIL_TRANSPORTER: smtp + # The app-content bucket (shares, logos, session files). APP_CONTENT_STORAGE_* + # in .env is the operator-facing name; the services read PERMANENT_STORAGE_*, + # which .env may also set directly. + PERMANENT_STORAGE_ENDPOINT: ${APP_CONTENT_STORAGE_ENDPOINT:-${PERMANENT_STORAGE_ENDPOINT:-}} + PERMANENT_STORAGE_INTERNAL_ENDPOINT: ${APP_CONTENT_STORAGE_INTERNAL_ENDPOINT:-${PERMANENT_STORAGE_INTERNAL_ENDPOINT:-}} + PERMANENT_STORAGE_BUCKET: ${APP_CONTENT_STORAGE_BUCKET:-${PERMANENT_STORAGE_BUCKET:-}} + PERMANENT_STORAGE_REGION: ${APP_CONTENT_STORAGE_REGION:-${PERMANENT_STORAGE_REGION:-}} + PERMANENT_STORAGE_ACCESS_KEY_ID: ${APP_CONTENT_STORAGE_ACCESS_KEY_ID:-${PERMANENT_STORAGE_ACCESS_KEY_ID:-}} + PERMANENT_STORAGE_SECRET_ACCESS_KEY: ${APP_CONTENT_STORAGE_SECRET_ACCESS_KEY:-${PERMANENT_STORAGE_SECRET_ACCESS_KEY:-}} + PERMANENT_STORAGE_FORCE_PATH_STYLE: ${APP_CONTENT_STORAGE_FORCE_PATH_STYLE:-${PERMANENT_STORAGE_FORCE_PATH_STYLE:-}} write-worker: image: ${DC_CURRENTS_IMAGE_REPOSITORY:-currents-}writer:${DC_CURRENTS_IMAGE_TAG:-staging} @@ -57,6 +79,16 @@ services: CURRENTS_ENV: onprem CLICKHOUSE_PASSWORD: ${CLICKHOUSE_CURRENTS_PASSWORD} EMAIL_TRANSPORTER: smtp + # The app-content bucket (shares, logos, session files). APP_CONTENT_STORAGE_* + # in .env is the operator-facing name; the services read PERMANENT_STORAGE_*, + # which .env may also set directly. + PERMANENT_STORAGE_ENDPOINT: ${APP_CONTENT_STORAGE_ENDPOINT:-${PERMANENT_STORAGE_ENDPOINT:-}} + PERMANENT_STORAGE_INTERNAL_ENDPOINT: ${APP_CONTENT_STORAGE_INTERNAL_ENDPOINT:-${PERMANENT_STORAGE_INTERNAL_ENDPOINT:-}} + PERMANENT_STORAGE_BUCKET: ${APP_CONTENT_STORAGE_BUCKET:-${PERMANENT_STORAGE_BUCKET:-}} + PERMANENT_STORAGE_REGION: ${APP_CONTENT_STORAGE_REGION:-${PERMANENT_STORAGE_REGION:-}} + PERMANENT_STORAGE_ACCESS_KEY_ID: ${APP_CONTENT_STORAGE_ACCESS_KEY_ID:-${PERMANENT_STORAGE_ACCESS_KEY_ID:-}} + PERMANENT_STORAGE_SECRET_ACCESS_KEY: ${APP_CONTENT_STORAGE_SECRET_ACCESS_KEY:-${PERMANENT_STORAGE_SECRET_ACCESS_KEY:-}} + PERMANENT_STORAGE_FORCE_PATH_STYLE: ${APP_CONTENT_STORAGE_FORCE_PATH_STYLE:-${PERMANENT_STORAGE_FORCE_PATH_STYLE:-}} scheduler: image: ${DC_CURRENTS_IMAGE_REPOSITORY:-currents-}scheduler:${DC_CURRENTS_IMAGE_TAG:-staging} diff --git a/on-prem/templates/compose.rustfs.yml b/on-prem/templates/compose.rustfs.yml index 0b641bf..01bc205 100644 --- a/on-prem/templates/compose.rustfs.yml +++ b/on-prem/templates/compose.rustfs.yml @@ -37,8 +37,64 @@ services: AWS_ACCESS_KEY_ID: ${RUSTFS_ACCESS_KEY} AWS_SECRET_ACCESS_KEY: ${RUSTFS_SECRET_KEY} AWS_DEFAULT_REGION: us-east-1 - entrypoint: > - sh -c "aws --endpoint-url=http://rustfs:9000 s3 mb s3://currents || true" + AWS_ENDPOINT_URL: http://rustfs:9000 + ARTIFACT_BUCKET: ${FILE_STORAGE_BUCKET:-currents} + APP_CONTENT_BUCKET: ${APP_CONTENT_STORAGE_BUCKET:-currents-app-content} + DELETE_ARTIFACTS_AFTER_DAYS: ${DC_RUSTFS_DELETE_ARTIFACTS_AFTER_DAYS:-} + # The app-content rules expire share markdown a day after its link and trace + # links after their longest lifetime. Never bucket-wide: the same bucket keeps + # organization logos and session files for good. Each run rewrites the rules + # this init owns, matched by ID, and keeps any other rules on the buckets. + entrypoint: + - sh + - -c + - | + set -e + for bucket in "$$ARTIFACT_BUCKET" "$$APP_CONTENT_BUCKET"; do + # A create that loses a race to another init finds the bucket there. + aws s3api head-bucket --bucket "$$bucket" 2>/dev/null || + aws s3 mb "s3://$$bucket" || + aws s3api head-bucket --bucket "$$bucket" + done + # The bucket's rules whose ID is not in $$2, as JSON. Any error other than + # an absent configuration stops the init rather than dropping those rules. + other_rules() { + filter="" + for id in $$2; do filter="$${filter:+$$filter && }ID!='$$id'"; done + if out=$$(aws s3api get-bucket-lifecycle-configuration --bucket "$$1" --query "Rules[?$$filter]" --output json 2>&1); then + echo "$$out" + else + case "$$out" in + *NoSuchLifecycleConfiguration*) echo "[]" ;; + *) echo "$$out" >&2; return 1 ;; + esac + fi + } + # Sets bucket $$1's rules to its other rules plus $$3 (comma-separated rule + # objects, or empty), deleting the configuration when none are left. + put_rules() { + others=$$(other_rules "$$1" "$$2") + rules=$$(printf '%s' "$$others" | tr -d '\n' | sed 's/^ *\[ *//; s/ *\] *$$//') + if [ -n "$$rules" ] && [ -n "$$3" ]; then rules="$$rules, $$3"; else rules="$$rules$$3"; fi + if [ -n "$$rules" ]; then + aws s3api put-bucket-lifecycle-configuration --bucket "$$1" --lifecycle-configuration "{ \"Rules\": [$$rules] }" + else + aws s3api delete-bucket-lifecycle --bucket "$$1" + fi + } + put_rules "$$APP_CONTENT_BUCKET" "ai-context-share share-3d share-7d trace-links" ' + { "ID": "ai-context-share", "Status": "Enabled", "Filter": { "Prefix": "ai-context-share/" }, "Expiration": { "Days": 2 } }, + { "ID": "share-3d", "Status": "Enabled", "Filter": { "Prefix": "share-3d/" }, "Expiration": { "Days": 4 } }, + { "ID": "share-7d", "Status": "Enabled", "Filter": { "Prefix": "share-7d/" }, "Expiration": { "Days": 8 } }, + { "ID": "trace-links", "Status": "Enabled", "Filter": { "Prefix": "trace-links/" }, "Expiration": { "Days": 8 } }' + artifact_rule="" + if [ -n "$$DELETE_ARTIFACTS_AFTER_DAYS" ]; then + case "$$DELETE_ARTIFACTS_AFTER_DAYS" in + *[!0-9]* | 0*) echo "DC_RUSTFS_DELETE_ARTIFACTS_AFTER_DAYS must be a whole number of days above 0" >&2; exit 1 ;; + esac + artifact_rule="{ \"ID\": \"delete-artifacts-after-days\", \"Status\": \"Enabled\", \"Filter\": { \"Prefix\": \"\" }, \"Expiration\": { \"Days\": $$DELETE_ARTIFACTS_AFTER_DAYS } }" + fi + put_rules "$$ARTIFACT_BUCKET" "delete-artifacts-after-days" "$$artifact_rule" networks: - default @@ -55,14 +111,17 @@ services: api: environment: FILE_STORAGE_FORCE_PATH_STYLE: "true" + PERMANENT_STORAGE_FORCE_PATH_STYLE: "true" changestreams-worker: environment: FILE_STORAGE_FORCE_PATH_STYLE: "true" + PERMANENT_STORAGE_FORCE_PATH_STYLE: "true" write-worker: environment: FILE_STORAGE_FORCE_PATH_STYLE: "true" + PERMANENT_STORAGE_FORCE_PATH_STYLE: "true" scheduler: environment: