From e38081145c45a3a96db46da43f5fb4f5cebc24a5 Mon Sep 17 00:00:00 2001 From: Cheese Date: Tue, 15 Sep 2026 16:59:24 -0400 Subject: [PATCH 01/14] docs: add TiDB Cloud Filesystem product documentation --- TOC-tidb-cloud-filesystem.md | 26 +++ ai/_index.md | 2 +- tidb-cloud-filesystem/_index.md | 74 ++++++++ .../filesystem-authorization.md | 104 +++++++++++ .../filesystem-branches-checkpoints.md | 164 ++++++++++++++++++ .../filesystem-mount-docker.md | 129 ++++++++++++++ .../filesystem-mount-linux.md | 82 +++++++++ .../filesystem-mount-macos.md | 82 +++++++++ tidb-cloud-filesystem/filesystem-mount.md | 98 +++++++++++ .../filesystem-quick-start.md | 127 ++++++++++++++ tidb-cloud-filesystem/filesystem-sharing.md | 113 ++++++++++++ 11 files changed, 1000 insertions(+), 1 deletion(-) create mode 100644 TOC-tidb-cloud-filesystem.md create mode 100644 tidb-cloud-filesystem/_index.md create mode 100644 tidb-cloud-filesystem/filesystem-authorization.md create mode 100644 tidb-cloud-filesystem/filesystem-branches-checkpoints.md create mode 100644 tidb-cloud-filesystem/filesystem-mount-docker.md create mode 100644 tidb-cloud-filesystem/filesystem-mount-linux.md create mode 100644 tidb-cloud-filesystem/filesystem-mount-macos.md create mode 100644 tidb-cloud-filesystem/filesystem-mount.md create mode 100644 tidb-cloud-filesystem/filesystem-quick-start.md create mode 100644 tidb-cloud-filesystem/filesystem-sharing.md diff --git a/TOC-tidb-cloud-filesystem.md b/TOC-tidb-cloud-filesystem.md new file mode 100644 index 0000000000000..2b6014848677d --- /dev/null +++ b/TOC-tidb-cloud-filesystem.md @@ -0,0 +1,26 @@ + + + +# Table of Contents + +## Getting Started + +- [Introduction](/tidb-cloud-filesystem/_index.md) +- [Quick Start](/tidb-cloud-filesystem/filesystem-quick-start.md) +- Mounting Locally + - [Overview](/tidb-cloud-filesystem/filesystem-mount.md) + - [Linux](/tidb-cloud-filesystem/filesystem-mount-linux.md) + - [macOS](/tidb-cloud-filesystem/filesystem-mount-macos.md) + - [Docker and Docker Compose](/tidb-cloud-filesystem/filesystem-mount-docker.md) + +## Concepts + +- [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md) +- [Sharing Filesystems](/tidb-cloud-filesystem/filesystem-sharing.md) +- [Branches and Checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md) + +## Reference + +- [Filesystem CLI Commands](/ai/ti/reference/ti-filesystem.md) +- [Regions and Limitations](/ai/ti/reference/ti-regions-security-and-limitations.md) +- [Troubleshooting](/ai/ti/reference/ti-troubleshooting.md) diff --git a/ai/_index.md b/ai/_index.md index 78b4767293763..ebbd5e69eee72 100644 --- a/ai/_index.md +++ b/ai/_index.md @@ -8,7 +8,7 @@ summary: Build AI applications and agent workflows with TiDB using SQL, integrat TiDB provides data and workspace capabilities for building AI applications and running AI agent workflows. - For application development, you can use SQL or [Python SDK for TiDB AI (`pytidb`)](https://github.com/pingcap/pytidb) with structured data, vector search, full-text search, hybrid search, and AI-powered retrieval. -- For AI agents and automation, you can use [TiDB Cloud CLI (`ti`)](https://github.com/tidbcloud/ti-cli) to manage TiDB Cloud Starter instances and SQL workflows, and use TiDB Cloud Filesystems as persistent, shared storage across local machines, CI jobs, and ephemeral agent sandboxes. Filesystems also support mounted workspaces, Git workflows, journals, and delegated secrets. +- For AI agents and automation, you can use [TiDB Cloud CLI (`ti`)](https://github.com/tidbcloud/ti-cli) to manage TiDB Cloud Starter instances and SQL workflows, and use [TiDB Cloud Filesystems](/tidb-cloud-filesystem/_index.md) as persistent, shared storage across local machines, CI jobs, and ephemeral agent sandboxes. Filesystems also support mounted workspaces, Git workflows, journals, and delegated secrets. ## Get started diff --git a/tidb-cloud-filesystem/_index.md b/tidb-cloud-filesystem/_index.md new file mode 100644 index 0000000000000..a053fbea6f69c --- /dev/null +++ b/tidb-cloud-filesystem/_index.md @@ -0,0 +1,74 @@ +--- +title: TiDB Cloud Filesystem +summary: Learn how TiDB Cloud Filesystem keeps agent files available across sessions, shares workspaces, and isolates changes with layers. +--- + +# TiDB Cloud Filesystem + +TiDB Cloud Filesystem is persistent, shared file storage for applications and AI agents. Keep a workspace in the cloud, access it from different machines, and retain its files after a sandbox or process ends. + +Use the TiDB Cloud Command Line Interface (`ti`) to create a Filesystem, upload and download files, or mount a remote directory for tools that work with local paths. You do not need to provision or manage a separate database to get started. + +> **Note:** +> +> TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. + +## Keep work beyond one session + +An agent might collect source documents in one sandbox, generate a report in another, and hand the result to a person on a laptop. Files stored only on the sandbox's local disk disappear when that environment is removed. Repeatedly exporting and importing archives also creates separate copies that can drift apart. + +With a Filesystem, these participants use the same remote files. A new environment needs an access token and the Filesystem's region code, not the creator's TiDB Cloud API keys or local configuration directory. + +- **Continue a task in another environment.** Mount the existing workspace instead of recreating its input files. +- **Share results with a reviewer.** Give the reviewer a read-only scoped token for a report directory. +- **Compare parallel approaches.** Fork layers for independent drafts, create checkpoints, and commit a selected result to the shared base. + +## Work with files through the CLI or a mount + +The CLI provides two ways to use the same remote Filesystem: + +- **Direct file commands:** upload, download, list, read, and search files with `ti fs`. These commands do not require FUSE or a mount. +- **Local mounts:** expose remote files at a local directory so existing tools can use commands such as `cat`, `cp`, and `ls`. Linux uses FUSE. macOS uses WebDAV by default and can use macFUSE for layer and checkpoint mounts. + +For example, after selecting a Filesystem and supplying its token: + +```bash +# Read a remote file without mounting it. +ti fs read-file --path /workspace/report.md +``` + +Or mount the workspace on macOS or Linux: + +```bash +# Give local tools a directory backed by the remote workspace. +mkdir -p "$HOME/workspace" +ti fs mount-file-system --remote-path /workspace --mount-path "$HOME/workspace" +cat "$HOME/workspace/report.md" +``` + +## Control sharing and changes + +- [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md) distinguishes resource-management credentials from owner and scoped Filesystem tokens. +- [Sharing Filesystems](/tidb-cloud-filesystem/filesystem-sharing.md) explains how participants access one remote workspace without sharing account credentials. +- [Branches and Checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md) explains layers, saved history points, and publishing changes to the base Filesystem. + +## Before you begin + +Choose how to access the Filesystem: + +- **Create and manage a Filesystem:** obtain a TiDB Cloud API public key and private key from the [API Keys page](https://tidbcloud.com/org-settings/api-keys), then run `ti configure`. Follow the [Quick Start](/tidb-cloud-filesystem/filesystem-quick-start.md) to create a Filesystem and write your first file. +- **Use an existing Filesystem:** if someone has supplied an FS token, set `TI_FS_TOKEN` and `TI_REGION_CODE`. You do not need TiDB Cloud API keys or `ti configure` to access files within the token's permissions. Follow [Mounting Locally](/tidb-cloud-filesystem/filesystem-mount.md#use-a-token-without-configuring-a-profile). + +See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md) for the differences between API keys, owner tokens, and scoped tokens. An FS token does not grant permission to create or delete a Filesystem through the CLI. + +Choose a supported Filesystem region: `aws-us-east-1`, `aws-ap-southeast-1`, `aws-us-west-2`, or `alicloud-ap-southeast-1`. Keep compute close to that region when possible to reduce network latency. + +Mount support depends on the operating system and its permissions. Windows supports direct file commands but not native mounting through `ti`. WebDAV does not support layer or checkpoint mounts. Buffered writes must reach the service before you remove an environment; a successful local write alone is not a durability guarantee. + +These guides focus on file storage and access. TiDB Cloud Filesystem does not provision an agent's compute environment. A layer is not a security sandbox, and checkpoints do not snapshot the entire live base Filesystem. + +## What's next + +- [Create a Filesystem and write your first file](/tidb-cloud-filesystem/filesystem-quick-start.md). +- [Mount an existing Filesystem locally](/tidb-cloud-filesystem/filesystem-mount.md). +- [Try the interactive agent sandbox lab](https://labs.tidb.io/labs/demo_901). diff --git a/tidb-cloud-filesystem/filesystem-authorization.md b/tidb-cloud-filesystem/filesystem-authorization.md new file mode 100644 index 0000000000000..a1ae7f4152ab6 --- /dev/null +++ b/tidb-cloud-filesystem/filesystem-authorization.md @@ -0,0 +1,104 @@ +--- +title: TiDB Cloud Filesystem Authorization +summary: Choose TiDB Cloud API keys, owner tokens, or scoped tokens to separate Filesystem administration from application data access. +--- + +# TiDB Cloud Filesystem Authorization + +The person who creates a Filesystem and the agent that uses its files do not need the same credentials. Keep account-level management on a trusted machine and give each application only the Filesystem access it needs. + +> **Note:** +> +> TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. + +## Choose the credential type + +### TiDB Cloud API keys + +A TiDB Cloud public/private API key pair authorizes resource-management operations according to the account's permissions. Use it to create, list, describe, and delete Filesystems, generate owner tokens, and configure Filesystem AI providers. + +Configure the keys with `ti configure`, or provide `TIDB_CLOUD_PUBLIC_KEY` and `TIDB_CLOUD_PRIVATE_KEY` together. They are not FS tokens and do not directly replace the token used by a mount. + +### Owner FS tokens + +An owner token grants broad access within one Filesystem, including reading, writing, and deleting files. It can issue scoped tokens, list token metadata, and revoke tokens in that Filesystem. It can enable or disable scoped tokens, but enabling or disabling owner tokens requires TiDB Cloud API keys. It is a high-privilege secret, not a read-only mount credential. + +An owner FS token is not interchangeable with TiDB Cloud API keys: it cannot create or delete the Filesystem resource or generate another owner token through `ti`. In particular, deleting a file and deleting its Filesystem are different permissions. + +Filesystem creation returns an owner token and stores it locally for the creating profile. To generate another owner token, use `ti fs generate-file-system-token` with TiDB Cloud API keys. + +### Scoped FS tokens + +A scoped token limits access to specified path prefixes and operations. The supported operations are `read`, `list`, `search`, `write`, and `delete`. The service enforces these permissions, including when requests arrive through a mount. + +- A reporting agent might need `read,list` for `/inputs` and `read,list,write` for `/reports`. +- A reviewer might need only `read,list` for `/reports`. +- `search` also requires `read` on the scope. + +Scoped tokens cannot generate child tokens or manage token inventory. A scoped token can refresh itself while valid; refresh does not turn it into an owner token or broaden its permissions. + +## Delegate access to an agent + +On a trusted machine with a locally stored owner token, select the Filesystem and generate a limited token: + +```bash +# Capture the one-time token response without printing the secret. +SCOPED_TOKEN="$(ti fs generate-file-system-scoped-token \ + --file-system-id "" \ + --subject report-agent \ + --ttl 24h \ + --allow /workspace:read,list,write \ + --query fs_token --output text)" +``` + +Transfer `SCOPED_TOKEN` through a secret manager. In the agent's environment, inject it as `TI_FS_TOKEN` and provide the Filesystem's region: + +```bash +# In the agent environment, these values normally come from secret injection. +export TI_FS_TOKEN="" +export TI_REGION_CODE="aws-us-east-1" +ti fs list-files --path /workspace +``` + +The remote `/workspace` directory must already exist. For a mount, select the allowed subtree with `--remote-path /workspace`. Mounting the root `/` is not appropriate for a token that has access only to `/workspace`. + +## Understand local selection + +One Filesystem can have multiple remote tokens, but one CLI profile stores at most one selected local token for that Filesystem. Local state is a credential selection, not the authoritative remote token inventory. + +- `--fs-token` takes precedence over `TI_FS_TOKEN` for token-based operations. +- Without an explicit token, data-access commands use the locally stored credential for `--file-system-id` or `TI_FS_FILE_SYSTEM_ID`. +- An explicit token contains the Filesystem ID. A clean environment therefore needs only `TI_FS_TOKEN` and `TI_REGION_CODE` for data access. +- Generating a token does not select it locally unless you pass `--store-locally`. Replacing the local selection does not revoke the previous remote token. + +For token list, enable, disable, and delete commands, an explicitly supplied owner token selects bearer authentication; otherwise the CLI uses TiDB Cloud API keys and requires a Filesystem ID. A scoped token does not gain administrative capability merely because account keys also exist in the profile. + +## Rotate and revoke credentials + +List token metadata to identify the token you want to manage: + +```bash +# Use TiDB Cloud API keys when no explicit FS token is present. +ti fs list-file-system-tokens --file-system-id "" --output text +``` + +The list does not return token plaintext. Preserve newly generated or refreshed tokens in a secret manager. If you lose an owner token, generate a replacement using TiDB Cloud API keys; do not expect to recover the original secret by listing tokens. + +To retire a token, stop its consumers and use its token ID: + +```bash +# Revoke only the retired token, not the Filesystem itself. +ti fs delete-file-system-token --file-system-id "" --token-id "" +``` + +Token changes can take time to propagate through authorization caches. Rotate by distributing and validating a replacement before retiring the old token. Do not treat disabling an owner token as an implicit replacement for reviewing and revoking previously issued scoped tokens. + +> **Warning:** +> +> Stop writes and unmount consumers before refreshing, disabling, or deleting their token. The CLI checks known local mounts but cannot discover every remote machine using the secret. Token refresh is non-idempotent: if the request might have succeeded but the response was lost, do not blindly retry it with the old token. + +## What's next + +- [Share one Filesystem with multiple machines](/tidb-cloud-filesystem/filesystem-sharing.md). +- [Look up token management commands](/ai/ti/guides/manage-filesystem-tokens.md). +- [Mount a token-scoped directory](/tidb-cloud-filesystem/filesystem-mount.md). diff --git a/tidb-cloud-filesystem/filesystem-branches-checkpoints.md b/tidb-cloud-filesystem/filesystem-branches-checkpoints.md new file mode 100644 index 0000000000000..29bf09021cd79 --- /dev/null +++ b/tidb-cloud-filesystem/filesystem-branches-checkpoints.md @@ -0,0 +1,164 @@ +--- +title: TiDB Cloud Filesystem Branches and Checkpoints +summary: Use Filesystem layers to compare parallel agent drafts, checkpoint progress, inspect history, and publish a selected result. +--- + +# TiDB Cloud Filesystem Branches and Checkpoints + +When two agents explore different answers, they should not overwrite each other's drafts. A Filesystem layer gives each agent a writable view over a base directory. Fork layers to work in parallel, record checkpoints for review, and commit a selected result to the shared base. + +The CLI calls these objects **layers**, not Git branches. All commands in this guide use `ti fs`; no Git repository is required. + +> **Note:** +> +> TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. + +## Understand the model + +- **Base Filesystem:** the live shared files read by ordinary file commands and mounts without a layer selector. +- **Layer:** a set of changes over a base path. Its writes do not change the base until committed. +- **Fork:** a child layer that pins the parent's layer history at its current tip or a specified checkpoint. The child's later writes are independent of the parent's later layer changes. +- **Checkpoint:** a named point in one layer's durable history. It does not flush another client's pending writes or publish changes to the base. +- **Commit:** apply the layer's effective changes to the base Filesystem. This is not the same as a Git commit. + +> **Warning:** +> +> A fork or checkpoint pins layer history, not the entire base Filesystem. Paths resolved from the live base can still reflect later base changes. Layers are neither full Filesystem snapshots nor authorization boundaries. Do not use them as a substitute for backups or scoped tokens. + +## Prepare a parallel review + +This example has two agents write alternative reports, then publishes one result. It uses new filenames for successive drafts and avoids modifying the same file across checkpoints. + +Before running it: + +- Install `ti` v0.2.4 or later and select a Filesystem with an owner token as described in [Mounting Locally](/tidb-cloud-filesystem/filesystem-mount.md#select-a-filesystem). +- Use Linux with FUSE3 or macOS with macFUSE. WebDAV cannot mount layers or checkpoints. +- Use the same terminal for the shell variables below. Do not run other writers against this example's base path. + +```bash +# Isolate this run from existing work and earlier examples. +RUN_ID="$(date +%s)" +REMOTE_ROOT="/layer-demo-$RUN_ID" +LOCAL_ROOT="$HOME/layer-demo-$RUN_ID" +ti fs create-directory --path "$REMOTE_ROOT" +ti fs create-directory --path "$REMOTE_ROOT/reports" +mkdir -p "$LOCAL_ROOT/brief" "$LOCAL_ROOT/analyst" "$LOCAL_ROOT/review" +``` + +## Create a seed and fork two drafts + +```bash +# Create the seed layer and retain its immutable ID. +BASE_LAYER_ID="$(ti fs create-layer \ + --base-root-path "$REMOTE_ROOT" \ + --layer-name "seed-$RUN_ID" \ + --query layer_id --output text)" + +# Direct layer upload avoids a pending local mount write. +printf 'Compare two approaches to the same research question.\n' | ti fs copy-file \ + --from-stdin --to-remote "$REMOTE_ROOT/input.txt" \ + --layer-id "$BASE_LAYER_ID" + +SEED_ID="$(ti fs create-layer-checkpoint \ + --layer-id "$BASE_LAYER_ID" \ + --checkpoint-id "seed-$RUN_ID" \ + --label workspace-seed --query checkpoint_id --output text)" +``` + +Fork both drafts before committing the seed: + +```bash +# Both children start from the same parent checkpoint. +BRIEF_ID="$(ti fs fork-layer \ + --parent-layer-ref "$BASE_LAYER_ID" \ + --layer-name "brief-$RUN_ID" --checkpoint-id "$SEED_ID" \ + --query layer_id --output text)" +ANALYST_ID="$(ti fs fork-layer \ + --parent-layer-ref "$BASE_LAYER_ID" \ + --layer-name "analyst-$RUN_ID" --checkpoint-id "$SEED_ID" \ + --query layer_id --output text)" +ti fs list-layer-chain --layer-ref "$ANALYST_ID" + +# Publish the shared input before the children start writing new reports. +ti fs commit-layer --layer-id "$BASE_LAYER_ID" +``` + +The children retain their pinned parent history. Creating the shared directories in the base and committing the seed first also avoids having each child publish inherited directory creation as part of its own report. + +## Write the alternatives + +```bash +# Give each agent its own writable FUSE mount. +ti fs mount-file-system --driver fuse --remote-path "$REMOTE_ROOT" \ + --mount-path "$LOCAL_ROOT/brief" --layer-ref "$BRIEF_ID" +ti fs mount-file-system --driver fuse --remote-path "$REMOTE_ROOT" \ + --mount-path "$LOCAL_ROOT/analyst" --layer-ref "$ANALYST_ID" + +printf 'A concise recommendation.\n' > "$LOCAL_ROOT/brief/reports/brief.txt" +printf 'An evidence-led first draft.\n' > "$LOCAL_ROOT/analyst/reports/analyst-v1.txt" + +# Make the first analyst draft durable before checkpointing it. +ti fs drain-file-system --mount-path "$LOCAL_ROOT/analyst" --timeout 30s +V1_ID="$(ti fs create-layer-checkpoint \ + --layer-id "$ANALYST_ID" --checkpoint-id "v1-$RUN_ID" \ + --label first-draft --query checkpoint_id --output text)" +``` + +The two mounts contain different reports. Neither report is published in the base yet. Do not mount the same writable layer at multiple paths concurrently. + +## Review a historical checkpoint + +```bash +# Keep revisions in different files for this preview workflow. +printf 'A second draft with additional evidence.\n' > "$LOCAL_ROOT/analyst/reports/analyst-v2.txt" +ti fs drain-file-system --mount-path "$LOCAL_ROOT/analyst" --timeout 30s +ti fs create-layer-checkpoint --layer-id "$ANALYST_ID" \ + --checkpoint-id "v2-$RUN_ID" --label second-draft + +# Mount the earlier checkpoint read-only alongside the current writable tip. +ti fs mount-file-system --driver fuse --remote-path "$REMOTE_ROOT" \ + --mount-path "$LOCAL_ROOT/review" \ + --layer-ref "$ANALYST_ID" --checkpoint-id "$V1_ID" +ls "$LOCAL_ROOT/review/reports" +ls "$LOCAL_ROOT/analyst/reports" +ti fs diff-layer --layer-id "$ANALYST_ID" +``` + +The historical view includes `analyst-v1.txt` but not the later `analyst-v2.txt`. The writable tip includes both. `diff-layer` reports layer changes; it is not a Git-style line-by-line patch. + +To continue writing from a checkpoint, create a new child with `ti fs fork-layer --parent-layer-ref --checkpoint-id --layer-name ` while the parent remains forkable. A checkpoint mount itself is read-only. `rollback-layer` discards a layer; it does not reset that layer to a selected checkpoint. + +## Publish the selected result + +Stop all writers and unmount the example mounts before changing layer lifecycle state: + +```bash +# Flush writes and release both writable and historical mounts. +ti fs unmount-file-system --mount-path "$LOCAL_ROOT/review" +ti fs drain-file-system --mount-path "$LOCAL_ROOT/brief" --timeout 30s +ti fs unmount-file-system --mount-path "$LOCAL_ROOT/brief" +ti fs drain-file-system --mount-path "$LOCAL_ROOT/analyst" --timeout 30s +ti fs unmount-file-system --mount-path "$LOCAL_ROOT/analyst" + +# Abandon the rejected draft and publish the selected layer to the base. +ti fs delete-layer --layer-ref "$BRIEF_ID" +ti fs commit-layer --layer-id "$ANALYST_ID" +ti fs read-file --path "$REMOTE_ROOT/reports/analyst-v2.txt" +``` + +Committing publishes all effective changes in the selected layer, including both analyst draft files in this example. It does not select a single file automatically. Deleting a layer logically abandons it; it does not guarantee immediate physical removal of history still pinned by descendants. + +## Preview boundaries + +- Use immutable layer IDs in automation. Names are not guaranteed to be unique. +- Run a successful FUSE drain before a checkpoint of mounted writes. A checkpoint cannot capture data still buffered only on another machine. +- Recursive `ti fs copy-file --recursive` cannot be combined with `--layer-id`. To populate a layer with a directory tree, copy through its writable FUSE mount. +- Plain `ti fs read-file` and `ti fs list-files` read the base, not an uncommitted layer. Use a layer mount to inspect its files. +- Commit can fail on conflicting base changes. Retain the layer and inspect the conflict; do not automatically delete and recreate it or assume a transactional multi-file merge. +- Repeated changes to layer-created files and inherited metadata can have preview limitations. This example deliberately uses separate revision files and publishes the seed before child work. Test your own workflow before relying on more complex edit histories. +- Layer commit is not a merge into its parent, and the CLI does not provide Git-style merge, rebase, or an in-place reset to a checkpoint. + +## What's next + +- [Control which participants can read and write](/tidb-cloud-filesystem/filesystem-authorization.md). +- [Review the layer command reference](/ai/ti/reference/ti-filesystem.md). diff --git a/tidb-cloud-filesystem/filesystem-mount-docker.md b/tidb-cloud-filesystem/filesystem-mount-docker.md new file mode 100644 index 0000000000000..01b9f322bf33e --- /dev/null +++ b/tidb-cloud-filesystem/filesystem-mount-docker.md @@ -0,0 +1,129 @@ +--- +title: Mount TiDB Cloud Filesystem in Docker +summary: Configure Docker or Docker Compose with a FUSE device, mount permissions, and a Filesystem token to use persistent remote files. +--- + +# Mount TiDB Cloud Filesystem in Docker + +A container needs both FUSE userspace tools and permission to use the Linux host's FUSE device. Installing `fuse3` inside an otherwise restricted container is not sufficient. + +> **Note:** +> +> TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. + +## Prerequisites + +- A Linux Docker host with an accessible `/dev/fuse` device. +- An FS token for the existing Filesystem and its region code. +- Permission to start a container with the device and capabilities below. + +These examples target a Linux Docker host. Do not assume Docker Desktop's VM or a managed sandbox exposes the same device and security controls. If mounting is unavailable, use direct `ti fs` file commands instead. + +> **Warning:** +> +> `SYS_ADMIN` grants broad privileges, and `apparmor=unconfined` disables the container's AppArmor profile. Use these settings only in an environment whose security policy permits them. A scoped FS token limits remote data access but does not restore container isolation. Do not grant these privileges to untrusted agent code without an appropriate isolation boundary. + +## Start with Docker + +On the host, provide the token through your secret manager and set the matching region: + +```bash +# Do not put a real token in a Dockerfile or commit it to source control. +export TI_FS_TOKEN="" +export TI_REGION_CODE="aws-us-east-1" +``` + +Start an interactive container: + +```bash +# Expose FUSE and the permissions needed to create the mount. +docker run --rm -it \ + --device /dev/fuse \ + --cap-add SYS_ADMIN \ + --security-opt apparmor=unconfined \ + --env TI_FS_TOKEN \ + --env TI_REGION_CODE \ + ubuntu:24.04 bash +``` + +The device option exposes `/dev/fuse`; `SYS_ADMIN` permits the mount operation; the security option removes AppArmor restrictions that could otherwise reject it. A host's other security controls can still prohibit mounting. + +Continue with [Install and mount inside the container](/tidb-cloud-filesystem/filesystem-mount-docker.md#install-and-mount-inside-the-container). + +## Start with Docker Compose + +As an alternative to `docker run`, use the following `compose.yaml`. It reads the same two environment variables from the host: + +```yaml +services: + agent: + image: ubuntu:24.04 + command: ["sleep", "infinity"] + devices: + - /dev/fuse:/dev/fuse + cap_add: + - SYS_ADMIN + security_opt: + - apparmor=unconfined + environment: + TI_FS_TOKEN: ${TI_FS_TOKEN:?Set TI_FS_TOKEN} + TI_REGION_CODE: ${TI_REGION_CODE:?Set TI_REGION_CODE} +``` + +```bash +# Start the container, then open its shell. +docker compose up -d +docker compose exec agent bash +``` + +## Install and mount inside the container + +Run these commands inside the container. This Ubuntu image starts as root, so no `sudo` is needed for package installation: + +```bash +# Install HTTPS download support and the FUSE3 mount helper. +apt-get update +apt-get install -y --no-install-recommends ca-certificates curl fuse3 +``` + +Install the CLI: + +```bash +# The installer includes the Filesystem mount runtime. +curl -fsSL https://github.com/tidbcloud/ti-cli/releases/latest/download/install.sh | sh -s -- --yes +``` + +After installation, prepare the current shell and mount: + +```bash +# The token identifies the Filesystem; no profile configuration is needed. +export PATH="$HOME/.ti/bin:$PATH" +mkdir -p "$HOME/workspace" +ti fs mount-file-system --mount-path "$HOME/workspace" --driver fuse +ls "$HOME/workspace" +``` + +For a token restricted to a subtree, add the matching `--remote-path`, for example `/workspace`. Add `--read-only` for a read-only token. Keep the application and mount under the same OS user. If your application image uses a non-root user, install dependencies while building the image and mount as the application user at runtime. + +## Stop the container safely + +Stop application writes, close files, and unmount inside the container: + +```bash +# Wait for a successful unmount before destroying the container. +ti fs unmount-file-system --mount-path "$HOME/workspace" && exit +``` + +The `docker run --rm` container is removed after its shell exits. For Compose, run the following on the host only after the unmount succeeds: + +```bash +# Remove the container after the mount has stopped cleanly. +docker compose down +``` + +The remote Filesystem remains available. Removing the container before pending writes reach the service can lose those writes; an automatic container timeout is not a graceful unmount. + +## What's next + +- [Share data with the next sandbox](/tidb-cloud-filesystem/filesystem-sharing.md). +- [Restrict tokens to the required paths and operations](/tidb-cloud-filesystem/filesystem-authorization.md). diff --git a/tidb-cloud-filesystem/filesystem-mount-linux.md b/tidb-cloud-filesystem/filesystem-mount-linux.md new file mode 100644 index 0000000000000..74bfaa0bbe48a --- /dev/null +++ b/tidb-cloud-filesystem/filesystem-mount-linux.md @@ -0,0 +1,82 @@ +--- +title: Mount TiDB Cloud Filesystem on Linux +summary: Set up FUSE3 on Linux, mount a TiDB Cloud Filesystem as the application user, and troubleshoot mount permission errors. +--- + +# Mount TiDB Cloud Filesystem on Linux + +Linux mounts use FUSE to route local filesystem operations to TiDB Cloud Filesystem. Applications can use the mounted directory without adopting a storage SDK. + +> **Note:** +> +> TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. + +## Prerequisites + +- Install `ti` and select a Filesystem using [a local credential or an FS token](/tidb-cloud-filesystem/filesystem-mount.md#select-a-filesystem). +- Use a Linux host with FUSE support. For a container, follow [Docker and Docker Compose](/tidb-cloud-filesystem/filesystem-mount-docker.md) instead. +- Run the mount and application as the same OS user. + +## Install FUSE3 + +On Ubuntu or Debian: + +```bash +# Install the mount helper; this administrative step might require sudo. +sudo apt-get update +sudo apt-get install -y --no-install-recommends fuse3 +``` + +Check the helper and device: + +```bash +# Both the helper and the kernel device must be available. +command -v fusermount3 +ls -l /dev/fuse +``` + +On another distribution, install its FUSE3 package. If `/dev/fuse` is missing or inaccessible, have the host administrator enable FUSE and grant the mounting user access. Installing a library alone does not provide a usable mount environment. + +## Mount and verify + +Use an empty directory under your home directory: + +```bash +# Keep the mount owned and used by the current OS user. +mkdir -p "$HOME/workspace" +ti fs mount-file-system --mount-path "$HOME/workspace" --driver fuse +``` + +With a writable token, write a test file and confirm it has reached the remote Filesystem: + +```bash +# Use a unique filename so the test does not overwrite an existing file. +TEST_FILE="mount-check-$(date +%s).txt" +printf 'Hello from Linux\n' > "$HOME/workspace/$TEST_FILE" +ti fs drain-file-system --mount-path "$HOME/workspace" --timeout 30s +ti fs read-file --path "/$TEST_FILE" +``` + +This example mounts the remote root `/`. If you mounted a subtree, include that remote prefix in the `read-file` path. + +When finished, stop writers and unmount: + +```bash +# Stop the mount without deleting the remote Filesystem. +ti fs unmount-file-system --mount-path "$HOME/workspace" +``` + +## Diagnose permission errors + +If `fusermount3` reports `Permission denied`, check the local mount path, `/dev/fuse` access, and the host's security policy. This error is not necessarily a TiDB Cloud credential failure. + +Ubuntu 26.04 can apply an AppArmor profile to `fusermount3` that allows paths under home directories but rejects a top-level path such as `/workspace`. Becoming root or running `chown` on `/workspace` does not bypass that policy. Prefer `$HOME/workspace`; if the application requires another path, ask the administrator to update the narrow AppArmor rule described in [Ubuntu mount-path restrictions](/ai/ti/guides/mount-filesystem.md#ubuntu-2604-mount-paths). + +A root-created FUSE mount is also not automatically usable by an application running as a different user. Mount as the user that will run the application rather than trying to repair access with `chown` afterward. + +For startup failures, inspect the diagnostic log path shown by the CLI. Do not repeatedly start mounts at the same path without checking whether a previous mount is still present. + +## What's next + +- [Use layers and checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md). +- [Share files with another machine](/tidb-cloud-filesystem/filesystem-sharing.md). diff --git a/tidb-cloud-filesystem/filesystem-mount-macos.md b/tidb-cloud-filesystem/filesystem-mount-macos.md new file mode 100644 index 0000000000000..1d7aad6b81317 --- /dev/null +++ b/tidb-cloud-filesystem/filesystem-mount-macos.md @@ -0,0 +1,82 @@ +--- +title: Mount TiDB Cloud Filesystem on macOS +summary: Mount a TiDB Cloud Filesystem with macOS WebDAV, or choose macFUSE when your workflow needs layers and historical checkpoints. +--- + +# Mount TiDB Cloud Filesystem on macOS + +On macOS, `ti` uses WebDAV by default, so you can start with a local directory without installing FUSE. Install macFUSE and explicitly select the FUSE driver when you need layer mounts, checkpoint mounts, or the FUSE drain operation. + +> **Note:** +> +> TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. + +## Prerequisites + +Install `ti` and select a Filesystem using [a local credential or an FS token](/tidb-cloud-filesystem/filesystem-mount.md#select-a-filesystem). Use a writable owner or scoped token for the write examples below. + +## Mount with the default WebDAV driver + +```bash +# Mount under your home directory, not at a root-level path such as /workspace. +mkdir -p "$HOME/workspace" +ti fs mount-file-system --mount-path "$HOME/workspace" +``` + +The successful result identifies the `webdav` driver. You can explicitly request the same mode with `--driver webdav`. + +```bash +# Verify a write through the local mount. +TEST_FILE="mount-check-$(date +%s).txt" +printf 'Hello from macOS\n' > "$HOME/workspace/$TEST_FILE" +cat "$HOME/workspace/$TEST_FILE" +``` + +Before a handoff or shutdown, close application files and unmount: + +```bash +# WebDAV has no drain command; finish file operations and unmount normally. +ti fs unmount-file-system --mount-path "$HOME/workspace" +ti fs read-file --path "/$TEST_FILE" +``` + +The final command reads from the remote service, independently of the mount. WebDAV and FUSE are different filesystem interfaces; do not assume WebDAV has every FUSE or POSIX capability. + +## Use macFUSE for layers and checkpoints + +Install [macFUSE](https://macfuse.github.io/) and complete the installation and security approvals required for your macOS version. Installing `ti` does not install macFUSE. Follow the macFUSE installation guidance for your system; do not assume every macFUSE backend is supported by the bundled Filesystem runtime. + +Unmount an existing WebDAV mount before reusing its directory with FUSE: + +```bash +# Select FUSE explicitly after installing macFUSE. +ti fs mount-file-system --mount-path "$HOME/workspace" --driver fuse +``` + +To keep a FUSE mount online while waiting for pending writes to reach the service: + +```bash +# Use this durability barrier before a layer checkpoint or cross-machine handoff. +ti fs drain-file-system --mount-path "$HOME/workspace" --timeout 30s +``` + +For writable layer and read-only checkpoint examples, see [Branches and Checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md). They cannot run through WebDAV. + +When finished with the FUSE mount, stop writers and unmount it: + +```bash +# Release the mount after pending writes have been flushed. +ti fs unmount-file-system --mount-path "$HOME/workspace" +``` + +## Troubleshoot startup + +- Use an empty directory you can write to, such as `$HOME/workspace`. A root-level path might not be writable on macOS. +- For FUSE, verify that macFUSE installation and required approvals are complete. +- If startup fails, inspect the diagnostic log path in the CLI error. The top-level background-process error does not by itself identify the cause. +- If unmount fails, keep the process, local cache, and machine available until you resolve the failure and verify remote data. + +## What's next + +- [Understand mount lifecycle and durability](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). +- [Share a read-only workspace](/tidb-cloud-filesystem/filesystem-sharing.md). diff --git a/tidb-cloud-filesystem/filesystem-mount.md b/tidb-cloud-filesystem/filesystem-mount.md new file mode 100644 index 0000000000000..4a0eedacbc212 --- /dev/null +++ b/tidb-cloud-filesystem/filesystem-mount.md @@ -0,0 +1,98 @@ +--- +title: Mount TiDB Cloud Filesystem Locally +summary: Select a Filesystem and mount driver, access remote files from a local directory, and stop a mount without losing pending writes. +--- + +# Mount TiDB Cloud Filesystem Locally + +A mount makes remote files available at a local directory. Use it when your editor, application, or agent expects filesystem paths instead of file-transfer commands. The remote Filesystem persists independently of the mount process. + +> **Note:** +> +> TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. + +## Choose your environment + +- [Linux](/tidb-cloud-filesystem/filesystem-mount-linux.md): use FUSE3 and an accessible `/dev/fuse` device. +- [macOS](/tidb-cloud-filesystem/filesystem-mount-macos.md): use the default WebDAV driver, or install macFUSE and explicitly select FUSE for layers and checkpoints. +- [Docker and Docker Compose](/tidb-cloud-filesystem/filesystem-mount-docker.md): expose the Linux host's FUSE device and allow mounting inside the container. + +Native Windows mounting is not supported by `ti`. Use direct commands such as `ti fs copy-file`, `ti fs read-file`, and `ti fs list-files` instead. + +## Select a Filesystem + +### Use a locally stored token + +After creating a Filesystem with the CLI, select its ID: + +```bash +# Select the Filesystem whose token is already stored locally. +export TI_FS_FILE_SYSTEM_ID="" +``` + +The CLI uses the selected resource's stored token and region information. You can pass `--file-system-id` on each command instead. There is no default Filesystem selected merely because only one resource exists. + +### Use a token without configuring a profile + +On a machine with `ti` installed, provide the token through a secret manager or the environment: + +```bash +# Inject these values from your secret manager in automation. +export TI_FS_TOKEN="" +export TI_REGION_CODE="aws-us-east-1" +``` + +Set the region to where this Filesystem was created. `ti` derives the Filesystem ID from the token; neither `ti configure` nor `TI_FS_FILE_SYSTEM_ID` is required. A supplied ID must match the token. If your shell already has an ID for another Filesystem, clear it before using the token-only workflow. + +Use an owner token for full Filesystem access, or a scoped token with the operations required by the application. For a token restricted to `/workspace`, mount that subtree rather than `/`. See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md). + +## Mount and use the files + +After installing the platform dependencies, mount the selected Filesystem: + +```bash +# Use an empty directory owned by the user who will access the mount. +mkdir -p "$HOME/workspace" +ti fs mount-file-system --mount-path "$HOME/workspace" +``` + +The command waits for readiness and returns a structured result with `status: mounted`. A background companion process keeps the mount alive. Closing the terminal does not unmount it, but terminating that process or the machine interrupts access. + +To expose only one remote directory, pass `--remote-path /workspace`. To make a mount read-only, add `--read-only`. These are client-side mount settings, not substitutes for a scoped token's server-enforced permissions. + +```bash +# These are ordinary local filesystem commands, not CLI subcommands. +ls "$HOME/workspace" +printf 'Written through the mount\n' > "$HOME/workspace/mounted.txt" +cat "$HOME/workspace/mounted.txt" +``` + +The write example requires a writable mount and token. Use the same OS user for mounting and file access; changing a directory's ownership does not grant another user access through an existing FUSE mount. + +## Finish safely + +Stop applications writing to the mount and close their files. If you need pending writes to reach the service while keeping a FUSE mount online, such as before creating a checkpoint or handing work to another machine, drain it: + +```bash +# FUSE only: wait for pending remote writes without unmounting. +ti fs drain-file-system --mount-path "$HOME/workspace" --timeout 30s +``` + +When finished, unmount: + +```bash +# Graceful unmount flushes pending FUSE work before stopping the mount. +ti fs unmount-file-system --mount-path "$HOME/workspace" +``` + +An explicit drain is not required before every normal unmount. WebDAV does not support drain: close application files and complete a normal unmount before a handoff. Drain and unmount use the local mount locator and do not require you to provide the region or token again. The running mount still needs valid credentials to finish remote work. + +> **Warning:** +> +> Do not destroy a sandbox, stop its mount process, or delete its local cache after a drain or unmount error. Pending writes might exist only on that machine. Resolve the error and verify remote data before removing the environment. Unmounting does not delete the remote Filesystem. + +## What's next + +- [Share a Filesystem across environments](/tidb-cloud-filesystem/filesystem-sharing.md). +- [Mount a layer or historical checkpoint](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md). +- [Look up mount options](/ai/ti/reference/ti-fs-mount-file-system.md). diff --git a/tidb-cloud-filesystem/filesystem-quick-start.md b/tidb-cloud-filesystem/filesystem-quick-start.md new file mode 100644 index 0000000000000..6bb8ec93a308f --- /dev/null +++ b/tidb-cloud-filesystem/filesystem-quick-start.md @@ -0,0 +1,127 @@ +--- +title: Get Started with TiDB Cloud Filesystem +summary: Create a TiDB Cloud Filesystem with the CLI, write and read a file, and keep the workspace available for another session. +--- + +# Get Started with TiDB Cloud Filesystem + +Create a workspace when your application's or agent's files need to outlive the machine that produces them. This quick start writes and reads a remote file without requiring a mount. You can mount the same Filesystem afterward. + +> **Note:** +> +> TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. + +## Prerequisites + +Obtain a TiDB Cloud API public key and private key from the [API Keys page](https://tidbcloud.com/org-settings/api-keys). The keys must have permission to create a Filesystem in your organization. + +If someone has already supplied you with an FS token, skip resource creation and follow [Mounting Locally](/tidb-cloud-filesystem/filesystem-mount.md#use-a-token-without-configuring-a-profile). + +## Step 1. Install the CLI + + + +
+ +Run the installer: + +```bash +# Install the CLI and its bundled Filesystem runtime. +curl -fsSL https://github.com/tidbcloud/ti-cli/releases/latest/download/install.sh | sh -s -- --yes +``` + +After installation, add the binary directory to your current shell: + +```bash +# Make the installed CLI available in this terminal. +export PATH="$HOME/.ti/bin:$PATH" +ti --version +``` + +Add the same `export PATH` line to your shell profile to use `ti` in future terminals. + +
+ +
+ +Run the installer: + +```powershell +# Download and run the PowerShell installer. +$script = "$env:TEMP\install-ti.ps1" +Invoke-WebRequest https://github.com/tidbcloud/ti-cli/releases/latest/download/install.ps1 -OutFile $script +powershell -ExecutionPolicy Bypass -File $script -Yes +``` + +After installation, add the binary directory to your current session: + +```powershell +# Make the installed CLI available in this PowerShell session. +$env:Path = "$HOME\.ti\bin;$env:Path" +ti --version +``` + +Add `$HOME\.ti\bin` to your user `PATH` for future sessions. Windows supports the direct file commands in this quick start, but not native Filesystem mounts through `ti`. + +
+ + +For other installation and upgrade details, see [Install, Configure, and Update TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md). + +## Step 2. Configure access + +```shell +# Follow the prompts to save your API keys and default region. +ti configure +``` + +Choose one of these Filesystem regions: + +- `aws-us-east-1` +- `aws-ap-southeast-1` +- `aws-us-west-2` +- `alicloud-ap-southeast-1` + +Configuration saves the inputs locally. Your first remote command verifies the credentials with the service. + +## Step 3. Create the Filesystem + +```shell +# Wait until the new Filesystem's root is readable. +ti fs create-file-system --display-name my-workspace --wait +``` + +Copy the returned `file_system_id` for the next step. The CLI stores this Filesystem's token locally, so you do not need to export a token. Treat the returned `fs_token` as a secret; do not paste the output into a public issue or log. + +Display names help identify resources but are not unique selectors. Subsequent commands use the ID. A failed wait does not automatically delete the created Filesystem; inspect the reported resource before creating another one. If creation reports a free-plan quota error, follow its billing link to add a payment method. + +## Step 4. Write and read a file + +Replace `` with the ID returned by creation: + +```shell +# Store a file in the remote workspace. +echo "Hello from my workspace" | ti fs copy-file --file-system-id "" --from-stdin --to-remote /hello.txt +``` + +```shell +# Read the same file from the service. +ti fs read-file --file-system-id "" --path /hello.txt +``` + +The read returns `Hello from my workspace`. The file remains available after you close the terminal. There is no local mount to keep running for this workflow. + +## What's next + +- [Mount the Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) to use it through a local directory. +- [Share the workspace](/tidb-cloud-filesystem/filesystem-sharing.md) with another machine or agent. +- [Use layers and checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md) to review changes before publishing them. + +When you no longer need this tutorial Filesystem, delete it using your TiDB Cloud API credentials: + +```shell +# Permanently request deletion of only the tutorial resource. +ti fs delete-file-system --file-system-id "" +``` + +Deletion removes the remote resource and its data, not just a local registration. The command reports `deleting` when the asynchronous request is accepted. diff --git a/tidb-cloud-filesystem/filesystem-sharing.md b/tidb-cloud-filesystem/filesystem-sharing.md new file mode 100644 index 0000000000000..712ea6d485dbe --- /dev/null +++ b/tidb-cloud-filesystem/filesystem-sharing.md @@ -0,0 +1,113 @@ +--- +title: Share TiDB Cloud Filesystems +summary: Share a remote workspace across machines and sandboxes with separate access tokens, and hand off files after writes reach the service. +--- + +# Share TiDB Cloud Filesystems + +Sharing a Filesystem gives participants access to the same remote namespace, not independent copies. For example, an agent can produce a report in a sandbox and a reviewer can open that report from a laptop without downloading and redistributing an archive for every revision. + +> **Note:** +> +> TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. + +## What each participant needs + +The trusted machine uses TiDB Cloud API keys to create the Filesystem. Other participants need `ti`, an FS token, and the Filesystem region. They do not need a copy of `~/.ti/`, account API keys, or their own Filesystem resource. + +Use separate tokens so you can retire one participant's access without changing every participant's credentials. Choose a scoped token for a restricted task; an owner token grants broad Filesystem access. See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md). + +## Prepare the workspace on machine A + +With `ti` installed and configured, create a Filesystem or use an existing one. This example creates a new resource: + +```bash +# Retain the ID; the creator's token is stored locally by the CLI. +FILE_SYSTEM_ID="$(ti fs create-file-system \ + --display-name shared-reports --wait \ + --query file_system_id --output text)" +``` + +Create a directory and publish the first report: + +```bash +# Write through the direct file interface, without a local mount. +ti fs create-directory --file-system-id "$FILE_SYSTEM_ID" --path /reports +printf 'The first report is ready for review.\n' | ti fs copy-file \ + --file-system-id "$FILE_SYSTEM_ID" \ + --from-stdin --to-remote /reports/summary.txt +``` + +Issue a read-only token for the reviewer: + +```bash +# Keep this value in a secret manager, not in a shared log. +REVIEW_TOKEN="$(ti fs generate-file-system-scoped-token \ + --file-system-id "$FILE_SYSTEM_ID" \ + --subject reviewer --ttl 24h \ + --allow /reports:read,list \ + --query fs_token --output text)" +``` + +Deliver `REVIEW_TOKEN` and the Filesystem's region code securely to machine B. Retain the Filesystem ID on machine A for administration. The token expires after the requested lifetime; a saved environment variable does not extend it. + +## Open the report on machine B + +Inject the reviewer's token and matching region into the environment: + +```bash +# No ti configure is needed on the receiving machine. +export TI_FS_TOKEN="" +export TI_REGION_CODE="" +ti fs read-file --path /reports/summary.txt +``` + +On macOS or Linux with the [mount dependencies](/tidb-cloud-filesystem/filesystem-mount.md#choose-your-environment), expose the allowed directory locally: + +```bash +# Mount only the scope allowed by the reviewer token. +mkdir -p "$HOME/reports" +ti fs mount-file-system \ + --remote-path /reports \ + --mount-path "$HOME/reports" \ + --read-only +cat "$HOME/reports/summary.txt" +``` + +The remote `/reports` prefix becomes the local mount root, so the local file is `$HOME/reports/summary.txt`, not `$HOME/reports/reports/summary.txt`. + +The token enforces read-only access at the service. `--read-only` also tells the local mount to reject writes; using that flag with an owner token alone would not restrict the owner's other API or CLI access. + +## Hand off new data safely + +A successful write to a FUSE-mounted file might still be buffered on the producing machine. Before telling the reviewer that a revision is ready, stop the application's writes and drain its FUSE mount, or unmount it successfully. For WebDAV, close application files and finish a normal unmount. See [Finish safely](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). + +Use a direct remote read to verify a handoff independently of another mount's cache. Existing open handles and client caches can retain older content; do not assume every reader instantly sees each local write. + +Coordinate writers to the same path. Shared storage is not a distributed lock or an automatic merge system. Use separate paths or [layers](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md) for independent drafts, and publish only after review. + +## End access without deleting the workspace + +On machine B, stop readers and unmount: + +```bash +# Remove the local mount, not the shared remote data. +ti fs unmount-file-system --mount-path "$HOME/reports" +unset TI_FS_TOKEN TI_REGION_CODE +``` + +On machine A, identify and revoke the reviewer's token when no longer needed: + +```bash +# Find the reviewer token ID in the metadata, then revoke that token only. +ti fs list-file-system-tokens --file-system-id "$FILE_SYSTEM_ID" --output text +ti fs delete-file-system-token \ + --file-system-id "$FILE_SYSTEM_ID" --token-id "" +``` + +Do not delete the Filesystem to disconnect one participant: resource deletion affects everyone and removes the shared data. + +## What's next + +- [Use branches to compare independent drafts](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md). +- [Run the agent sandbox example](/ai/ti/guides/ti-agent-sandbox-example.md). From 43eee5c07693e411e9e36b4a89242e8dd1aa30e9 Mon Sep 17 00:00:00 2001 From: qiancai Date: Thu, 17 Sep 2026 12:38:12 +0800 Subject: [PATCH 02/14] move filesystem guides from TOC-ai.md to TOC-tidb-cloud-filesystem.md --- TOC-ai.md | 11 +--- TOC-tidb-cloud-filesystem.md | 29 +++++++--- ai/_index.md | 4 +- ai/ti/guides/manage-filesystems-via-cli.md | 56 +++++++++++++++++++ ai/ti/guides/ti-agent-sandbox-example.md | 4 +- .../guides/ti-ci-artifact-handoff-example.md | 2 +- ai/ti/reference/ti-filesystem-git.md | 2 +- ai/ti/reference/ti-filesystem-journal.md | 2 +- ai/ti/reference/ti-filesystem-vault.md | 2 +- ai/ti/reference/ti-filesystem.md | 18 +++--- .../reference/ti-install-configure-update.md | 4 +- ai/ti/reference/ti-migrate-from-tdc.md | 2 +- ai/ti/reference/ti-troubleshooting.md | 2 +- ai/ti/ti-overview.md | 2 +- ai/ti/ti-quick-start.md | 2 +- .../configure-filesystem-ai-providers.md | 6 +- .../filesystem-authorization.md | 2 +- .../filesystem-branches-checkpoints.md | 2 + .../filesystem-mount-linux.md | 2 +- tidb-cloud-filesystem/filesystem-mount.md | 2 + .../manage-filesystem-layers.md | 12 ++-- .../manage-filesystem-resources.md | 8 +-- .../manage-filesystem-tokens.md | 9 +-- .../manage-filesystem-vault-secrets.md | 6 +- .../manage-git-workspaces.md | 7 ++- .../mount-filesystem.md | 12 ++-- ...filesystem-for-automation-and-ai-agents.md | 28 ++++++++++ .../use-filesystem-journals.md | 6 +- .../work-with-filesystem-data.md | 11 ++-- 29 files changed, 177 insertions(+), 78 deletions(-) create mode 100644 ai/ti/guides/manage-filesystems-via-cli.md rename {ai/ti/guides => tidb-cloud-filesystem}/configure-filesystem-ai-providers.md (93%) rename {ai/ti/guides => tidb-cloud-filesystem}/manage-filesystem-layers.md (71%) rename {ai/ti/guides => tidb-cloud-filesystem}/manage-filesystem-resources.md (87%) rename {ai/ti/guides => tidb-cloud-filesystem}/manage-filesystem-tokens.md (80%) rename {ai/ti/guides => tidb-cloud-filesystem}/manage-filesystem-vault-secrets.md (85%) rename {ai/ti/guides => tidb-cloud-filesystem}/manage-git-workspaces.md (75%) rename {ai/ti/guides => tidb-cloud-filesystem}/mount-filesystem.md (80%) create mode 100644 tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md rename {ai/ti/guides => tidb-cloud-filesystem}/use-filesystem-journals.md (72%) rename {ai/ti/guides => tidb-cloud-filesystem}/work-with-filesystem-data.md (66%) diff --git a/TOC-ai.md b/TOC-ai.md index e4a251b30ec68..574518319f0a0 100644 --- a/TOC-ai.md +++ b/TOC-ai.md @@ -45,16 +45,7 @@ - [Overview](/ai/ti/ti-overview.md) - [Install, Configure, and Update TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md) - [Manage TiDB Cloud Starter Instances](/ai/ti/guides/manage-starter-instances.md) - - Manage TiDB Cloud Filesystem - - [Manage Filesystem Resources](/ai/ti/guides/manage-filesystem-resources.md) - - [Configure Filesystem AI Providers](/ai/ti/guides/configure-filesystem-ai-providers.md) - - [Manage Filesystem Tokens](/ai/ti/guides/manage-filesystem-tokens.md) - - [Work with Filesystem Data](/ai/ti/guides/work-with-filesystem-data.md) - - [Manage Filesystem Layers and Checkpoints](/ai/ti/guides/manage-filesystem-layers.md) - - [Mount a TiDB Cloud Filesystem](/ai/ti/guides/mount-filesystem.md) - - [Manage Git Workspaces](/ai/ti/guides/manage-git-workspaces.md) - - [Use Filesystem Journals](/ai/ti/guides/use-filesystem-journals.md) - - [Manage Filesystem Vault Secrets](/ai/ti/guides/manage-filesystem-vault-secrets.md) + - [Manage TiDB Cloud Filesystems via CLI](/ai/ti/guides/manage-filesystems-via-cli.md) - Scenarios for Users and Automation - [Run a Daily TiDB Cloud CLI Workflow](/ai/ti/guides/ti-daily-workflow-example.md) - [Query TiDB Cloud Starter with Explicit SQL Roles](/ai/ti/guides/ti-query-sql-with-roles-example.md) diff --git a/TOC-tidb-cloud-filesystem.md b/TOC-tidb-cloud-filesystem.md index 2b6014848677d..bc35a1f80447c 100644 --- a/TOC-tidb-cloud-filesystem.md +++ b/TOC-tidb-cloud-filesystem.md @@ -3,23 +3,36 @@ # Table of Contents -## Getting Started +## GET STARTED - [Introduction](/tidb-cloud-filesystem/_index.md) - [Quick Start](/tidb-cloud-filesystem/filesystem-quick-start.md) + +## CONCEPTS + +- [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md) +- [Sharing Filesystems](/tidb-cloud-filesystem/filesystem-sharing.md) +- [Layers, Checkpoints, and Parallel Review](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md) + +## GUIDES + +- [Manage Filesystem Resources](/tidb-cloud-filesystem/manage-filesystem-resources.md) +- [Manage Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) +- [Work with Filesystem Data](/tidb-cloud-filesystem/work-with-filesystem-data.md) - Mounting Locally - [Overview](/tidb-cloud-filesystem/filesystem-mount.md) - [Linux](/tidb-cloud-filesystem/filesystem-mount-linux.md) - [macOS](/tidb-cloud-filesystem/filesystem-mount-macos.md) - [Docker and Docker Compose](/tidb-cloud-filesystem/filesystem-mount-docker.md) +- [Manage Filesystem Mounts with the CLI](/tidb-cloud-filesystem/mount-filesystem.md) +- [Manage Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) +- [Manage Git Workspaces](/tidb-cloud-filesystem/manage-git-workspaces.md) +- [Use Filesystem Journals](/tidb-cloud-filesystem/use-filesystem-journals.md) +- [Manage Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md) +- [Configure Filesystem AI Providers](/tidb-cloud-filesystem/configure-filesystem-ai-providers.md) +- [Use TiDB Cloud Filesystem for Automation and AI Agents](/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md) -## Concepts - -- [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md) -- [Sharing Filesystems](/tidb-cloud-filesystem/filesystem-sharing.md) -- [Branches and Checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md) - -## Reference +## REFERENCES - [Filesystem CLI Commands](/ai/ti/reference/ti-filesystem.md) - [Regions and Limitations](/ai/ti/reference/ti-regions-security-and-limitations.md) diff --git a/ai/_index.md b/ai/_index.md index ebbd5e69eee72..30dfee54a5351 100644 --- a/ai/_index.md +++ b/ai/_index.md @@ -101,8 +101,8 @@ The TiDB Cloud CLI (`ti`) lets users, scripts, CI jobs, and AI agents manage TiD | Understand what `ti` manages and when to use it | [TiDB Cloud CLI Overview](/ai/ti/ti-overview.md) | | Install and configure `ti`, then complete a first workflow | [Get Started with TiDB Cloud CLI](/ai/ti/ti-quick-start.md) | | Automate TiDB Cloud Starter instance, branch, and SQL operations | [Manage TiDB Cloud Starter Instances](/ai/ti/guides/manage-starter-instances.md) | -| Persist and share files across machines, CI jobs, and sandboxes | [Manage Filesystem Resources](/ai/ti/guides/manage-filesystem-resources.md) and [Work with Filesystem Data](/ai/ti/guides/work-with-filesystem-data.md) | -| Use mounted workspaces, Git workspaces, journals, or delegated secrets | [Mount a Filesystem](/ai/ti/guides/mount-filesystem.md), [Manage Git Workspaces](/ai/ti/guides/manage-git-workspaces.md), [Use Filesystem Journals](/ai/ti/guides/use-filesystem-journals.md), and [Manage Filesystem Vault Secrets](/ai/ti/guides/manage-filesystem-vault-secrets.md) | +| Persist and share files across machines, CI jobs, and sandboxes | [Manage TiDB Cloud Filesystems via CLI](/ai/ti/guides/manage-filesystems-via-cli.md) | +| Use mounted workspaces, Git workspaces, journals, or delegated secrets | [Mount a Filesystem](/tidb-cloud-filesystem/mount-filesystem.md), [Manage Git Workspaces](/tidb-cloud-filesystem/manage-git-workspaces.md), [Use Filesystem Journals](/tidb-cloud-filesystem/use-filesystem-journals.md), and [Manage Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md) | | Follow an end-to-end automation or agent example | [Run a Daily TiDB Cloud CLI Workflow](/ai/ti/guides/ti-daily-workflow-example.md) or [Use TiDB Cloud Filesystem in an Agent Sandbox](/ai/ti/guides/ti-agent-sandbox-example.md) | | Look up commands, global options, output behavior, and errors | [TiDB Cloud CLI Command Reference](/ai/ti/reference/ti-cli-reference.md) | diff --git a/ai/ti/guides/manage-filesystems-via-cli.md b/ai/ti/guides/manage-filesystems-via-cli.md new file mode 100644 index 0000000000000..4004bfcc3bc8e --- /dev/null +++ b/ai/ti/guides/manage-filesystems-via-cli.md @@ -0,0 +1,56 @@ +--- +title: Manage TiDB Cloud Filesystems via CLI +summary: Learn how to create a TiDB Cloud Filesystem with the CLI, write and read a file, and find detailed guides for more Filesystem tasks. +--- + +# Manage TiDB Cloud Filesystems via CLI + +Use the TiDB Cloud CLI (`ti`) to create a Filesystem and work with its files from a terminal or automation workflow. + +## Prerequisites + +- [Install and configure TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md) with API keys that can create a Filesystem in your organization. +- Choose a supported Filesystem region when you run `ti configure`. For region choices and installation details, see the [Filesystem Quick Start](/tidb-cloud-filesystem/filesystem-quick-start.md). + +## Create and use a Filesystem + +Create a Filesystem and wait until it is ready: + +```shell +ti fs create-file-system --display-name my-workspace --wait +``` + +Save the returned `file_system_id` for the next commands. The CLI stores this Filesystem's token locally. Treat the returned `fs_token` as a secret; do not put it in a public issue or log. + +Replace `` with the returned ID, write a file, and read it back: + +```shell +echo "Hello from my workspace" | ti fs copy-file --file-system-id "" --from-stdin --to-remote /hello.txt +ti fs read-file --file-system-id "" --path /hello.txt +``` + +The read returns `Hello from my workspace`. The file remains available after you close the terminal. For a guided first run and cleanup instructions, see the [Filesystem Quick Start](/tidb-cloud-filesystem/filesystem-quick-start.md). + +## More Filesystem operations + +For more Filesystem operations, see the following guides in the TiDB Cloud Filesystem documentation: + +- Manage a Filesystem + + - [Create, inspect, check, and delete Filesystem resources](/tidb-cloud-filesystem/manage-filesystem-resources.md) + - [Manage Filesystem tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) + - [Configure Filesystem AI providers](/tidb-cloud-filesystem/configure-filesystem-ai-providers.md) + +- Work with Filesystem data + + - [Copy, read, organize, and search files](/tidb-cloud-filesystem/work-with-filesystem-data.md) + - [Manage layers and checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) + - [Mount a Filesystem with the CLI](/tidb-cloud-filesystem/mount-filesystem.md) + +- Use advanced Filesystem features + + - [Manage Git workspaces](/tidb-cloud-filesystem/manage-git-workspaces.md) + - [Use Filesystem journals](/tidb-cloud-filesystem/use-filesystem-journals.md) + - [Manage Filesystem Vault secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md) + +For details about commands, command flags, and output fields, see [`ti fs` command reference](/ai/ti/reference/ti-filesystem.md). \ No newline at end of file diff --git a/ai/ti/guides/ti-agent-sandbox-example.md b/ai/ti/guides/ti-agent-sandbox-example.md index 98e01ecf5d5cb..602d0abc8cc50 100644 --- a/ai/ti/guides/ti-agent-sandbox-example.md +++ b/ai/ti/guides/ti-agent-sandbox-example.md @@ -79,7 +79,7 @@ ti fs mount-file-system \ cat "$HOME/workspace/sandbox/status.txt" ``` -On macOS, omit `--driver fuse` to use WebDAV, which requires no FUSE installation. Install macFUSE and select FUSE when you need FUSE-specific capabilities such as Git workspaces, layers, or online drain. For platform requirements and mount-path restrictions, see [Mount a TiDB Cloud Filesystem](/ai/ti/guides/mount-filesystem.md). +On macOS, omit `--driver fuse` to use WebDAV, which requires no FUSE installation. Install macFUSE and select FUSE when you need FUSE-specific capabilities such as Git workspaces, layers, or online drain. For platform requirements and mount-path restrictions, see [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md). After mounting, you can use `ti fs-git`, `ti fs-journal`, and owner-authorized `ti fs-vault` commands with the same FS environment. Give agents a delegated `TI_VAULT_TOKEN` instead of the owner token when they need only selected secret fields. @@ -91,7 +91,7 @@ Stop writers and unmount. A graceful FUSE unmount automatically flushes and drai ti fs unmount-file-system --mount-path "$HOME/workspace" ``` -For a FUSE mount, use `ti fs drain-file-system --mount-path "$HOME/workspace"` separately when you need to verify remote durability while keeping the mount online. `drain-file-system` is not supported for WebDAV. For more information, see [Drain or unmount](/ai/ti/guides/mount-filesystem.md#drain-or-unmount). Back on the trusted machine: +For a FUSE mount, use `ti fs drain-file-system --mount-path "$HOME/workspace"` separately when you need to verify remote durability while keeping the mount online. `drain-file-system` is not supported for WebDAV. For more information, see [Drain or unmount](/tidb-cloud-filesystem/mount-filesystem.md#drain-or-unmount). Back on the trusted machine: ```bash ti fs delete-file-system \ diff --git a/ai/ti/guides/ti-ci-artifact-handoff-example.md b/ai/ti/guides/ti-ci-artifact-handoff-example.md index a0a39282375a3..d0ff031a19596 100644 --- a/ai/ti/guides/ti-ci-artifact-handoff-example.md +++ b/ai/ti/guides/ti-ci-artifact-handoff-example.md @@ -17,7 +17,7 @@ The pipeline injects one Filesystem token and region into both jobs. The token i ## Prerequisites -[Create a Filesystem](/ai/ti/guides/manage-filesystem-resources.md#create-a-filesystem) on a trusted machine, and store these values as protected CI secrets or variables: +[Create a Filesystem](/tidb-cloud-filesystem/manage-filesystem-resources.md#create-a-filesystem) on a trusted machine, and store these values as protected CI secrets or variables: ```text TI_FS_TOKEN diff --git a/ai/ti/reference/ti-filesystem-git.md b/ai/ti/reference/ti-filesystem-git.md index d5567ac92c7cb..52de33fe513e0 100644 --- a/ai/ti/reference/ti-filesystem-git.md +++ b/ai/ti/reference/ti-filesystem-git.md @@ -18,5 +18,5 @@ summary: Reference every `ti fs-git` command for cloning, hydrating, and managin ## See also -- [Manage Git Workspaces on TiDB Cloud Filesystem](/ai/ti/guides/manage-git-workspaces.md) +- [Manage Git Workspaces on TiDB Cloud Filesystem](/tidb-cloud-filesystem/manage-git-workspaces.md) - [Prepare a Git Workspace for Agents on TiDB Cloud Filesystem](/ai/ti/guides/ti-git-workspace-for-agents-example.md) diff --git a/ai/ti/reference/ti-filesystem-journal.md b/ai/ti/reference/ti-filesystem-journal.md index 9e721d7129301..8dd50b49c05ea 100644 --- a/ai/ti/reference/ti-filesystem-journal.md +++ b/ai/ti/reference/ti-filesystem-journal.md @@ -19,5 +19,5 @@ summary: Reference every `ti fs-journal` command for creating, appending, readin ## See also -- [Use TiDB Cloud Filesystem Journals](/ai/ti/guides/use-filesystem-journals.md) +- [Use TiDB Cloud Filesystem Journals](/tidb-cloud-filesystem/use-filesystem-journals.md) - [Record an Agent Workflow in a TiDB Cloud Filesystem Journal](/ai/ti/guides/ti-journal-agent-workflow-example.md) diff --git a/ai/ti/reference/ti-filesystem-vault.md b/ai/ti/reference/ti-filesystem-vault.md index cece96304ca50..7dc479df1d4f2 100644 --- a/ai/ti/reference/ti-filesystem-vault.md +++ b/ai/ti/reference/ti-filesystem-vault.md @@ -27,5 +27,5 @@ Most secret-management commands identify a secret by its name, such as `db-prod` ## See also -- [Manage TiDB Cloud Filesystem Vault Secrets](/ai/ti/guides/manage-filesystem-vault-secrets.md) +- [Manage TiDB Cloud Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md) - [Delegate TiDB Cloud Filesystem Vault Secrets to an Agent](/ai/ti/guides/ti-vault-agent-secrets-example.md) diff --git a/ai/ti/reference/ti-filesystem.md b/ai/ti/reference/ti-filesystem.md index efa09ce87b292..3705eef04255d 100644 --- a/ai/ti/reference/ti-filesystem.md +++ b/ai/ti/reference/ti-filesystem.md @@ -124,12 +124,12 @@ Aliases use the same options, authentication, output, query, and error behavior ## See also -- [Manage TiDB Cloud Filesystem Resources](/ai/ti/guides/manage-filesystem-resources.md) -- [Configure TiDB Cloud Filesystem AI Providers](/ai/ti/guides/configure-filesystem-ai-providers.md) -- [Manage TiDB Cloud Filesystem Tokens](/ai/ti/guides/manage-filesystem-tokens.md) -- [Work with TiDB Cloud Filesystem Data](/ai/ti/guides/work-with-filesystem-data.md) -- [Manage Filesystem Layers and Checkpoints](/ai/ti/guides/manage-filesystem-layers.md) -- [Mount a TiDB Cloud Filesystem](/ai/ti/guides/mount-filesystem.md) -- [Manage Git Workspaces on TiDB Cloud Filesystem](/ai/ti/guides/manage-git-workspaces.md) -- [Use TiDB Cloud Filesystem Journals](/ai/ti/guides/use-filesystem-journals.md) -- [Manage TiDB Cloud Filesystem Vault Secrets](/ai/ti/guides/manage-filesystem-vault-secrets.md) +- [Manage TiDB Cloud Filesystem Resources](/tidb-cloud-filesystem/manage-filesystem-resources.md) +- [Configure TiDB Cloud Filesystem AI Providers](/tidb-cloud-filesystem/configure-filesystem-ai-providers.md) +- [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) +- [Work with TiDB Cloud Filesystem Data](/tidb-cloud-filesystem/work-with-filesystem-data.md) +- [Manage Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) +- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md) +- [Manage Git Workspaces on TiDB Cloud Filesystem](/tidb-cloud-filesystem/manage-git-workspaces.md) +- [Use TiDB Cloud Filesystem Journals](/tidb-cloud-filesystem/use-filesystem-journals.md) +- [Manage TiDB Cloud Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md) diff --git a/ai/ti/reference/ti-install-configure-update.md b/ai/ti/reference/ti-install-configure-update.md index 1c5335c6e482a..37fabb47229aa 100644 --- a/ai/ti/reference/ti-install-configure-update.md +++ b/ai/ti/reference/ti-install-configure-update.md @@ -171,7 +171,7 @@ ti update --dry-run > ti fs unmount-file-system --mount-path > ``` > -> For a Vault mount, use `ti fs-vault unmount-vault --mount-path `. For details, see [Mount a TiDB Cloud Filesystem](/ai/ti/guides/mount-filesystem.md) and [Manage Filesystem Vault Secrets](/ai/ti/guides/manage-filesystem-vault-secrets.md). +> For a Vault mount, use `ti fs-vault unmount-vault --mount-path `. For details, see [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md) and [Manage Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md). Apply the latest update: @@ -209,7 +209,7 @@ ti fs unmount-file-system --mount-path ti fs-vault unmount-vault --mount-path ``` -For details, see [Mount a TiDB Cloud Filesystem](/ai/ti/guides/mount-filesystem.md) and [Manage Filesystem Vault Secrets](/ai/ti/guides/manage-filesystem-vault-secrets.md). +For details, see [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md) and [Manage Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md). diff --git a/ai/ti/reference/ti-migrate-from-tdc.md b/ai/ti/reference/ti-migrate-from-tdc.md index 7a87e1e49a59b..042b60d7e37f6 100644 --- a/ai/ti/reference/ti-migrate-from-tdc.md +++ b/ai/ti/reference/ti-migrate-from-tdc.md @@ -92,5 +92,5 @@ During the v0.2.x transition, `ti` accepts a legacy `TDC_*` environment variable - [Install, Configure, and Update TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md) - [TiDB Cloud CLI Configuration and Credentials](/ai/ti/reference/ti-configuration-and-credentials.md) - [Manage TiDB Cloud Starter Instances](/ai/ti/guides/manage-starter-instances.md) -- [Manage Filesystem Resources](/ai/ti/guides/manage-filesystem-resources.md) +- [Manage Filesystem Resources](/tidb-cloud-filesystem/manage-filesystem-resources.md) - [Troubleshoot TiDB Cloud CLI](/ai/ti/reference/ti-troubleshooting.md) diff --git a/ai/ti/reference/ti-troubleshooting.md b/ai/ti/reference/ti-troubleshooting.md index b8f35127d3be8..7c09a44a4a4b9 100644 --- a/ai/ti/reference/ti-troubleshooting.md +++ b/ai/ti/reference/ti-troubleshooting.md @@ -197,7 +197,7 @@ mkdir -p "$HOME/workspace" ti fs mount-file-system --mount-path "$HOME/workspace" ``` -Changing the owner or mode of `/workspace` does not bypass AppArmor. If the path cannot change, add explicit `/workspace` mount and unmount rules to `/etc/apparmor.d/local/fusermount3` as described in [Mount a TiDB Cloud Filesystem](/ai/ti/guides/mount-filesystem.md#ubuntu-2604-mount-paths). +Changing the owner or mode of `/workspace` does not bypass AppArmor. If the path cannot change, add explicit `/workspace` mount and unmount rules to `/etc/apparmor.d/local/fusermount3` as described in [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md#ubuntu-2604-mount-paths). ## Mount becomes stale after a process crash diff --git a/ai/ti/ti-overview.md b/ai/ti/ti-overview.md index ef567bfc2b802..954b2d21d5ec2 100644 --- a/ai/ti/ti-overview.md +++ b/ai/ti/ti-overview.md @@ -84,7 +84,7 @@ If you are new to the TiDB Cloud CLI, start with the [Quick Start](/ai/ti/ti-qui Then continue based on what you want to do: - [Manage TiDB Cloud Starter Instances](/ai/ti/guides/manage-starter-instances.md) -- [Manage TiDB Cloud Filesystems](/ai/ti/guides/manage-filesystem-resources.md) +- [Manage TiDB Cloud Filesystems via CLI](/ai/ti/guides/manage-filesystems-via-cli.md) - **Follow end-to-end workflows**: Start with [Run a Daily TiDB Cloud CLI Workflow](/ai/ti/guides/ti-daily-workflow-example.md) or [Use TiDB Cloud Filesystem in an Agent Sandbox](/ai/ti/guides/ti-agent-sandbox-example.md) - **Look up a specific command**: Check the [TiDB Cloud CLI Command Reference](/ai/ti/reference/ti-cli-reference.md) - **See what’s new in TiDB Cloud CLI**: Check the [TiDB Cloud CLI (`ti`) Release Notes](https://github.com/tidbcloud/ti-cli/releases) diff --git a/ai/ti/ti-quick-start.md b/ai/ti/ti-quick-start.md index 37fd7198be9ca..c39578746759b 100644 --- a/ai/ti/ti-quick-start.md +++ b/ai/ti/ti-quick-start.md @@ -192,6 +192,6 @@ A TiDB Cloud Filesystem is a persistent, shareable cloud file system that you ca ## What's next - Read the [TiDB Cloud Command Line Interface Overview](/ai/ti/ti-overview.md) to understand what `ti` manages and when to use it. -- Follow the task guides to manage [TiDB Cloud Starter](https://docs.pingcap.com/tidbcloud/select-cluster-tier/?plan=starter#starter) or [Filesystem resources](/ai/ti/guides/manage-filesystem-resources.md). +- Follow the task guides to manage [TiDB Cloud Starter](https://docs.pingcap.com/tidbcloud/select-cluster-tier/?plan=starter#starter) or [Filesystem resources](/tidb-cloud-filesystem/manage-filesystem-resources.md). - Explore the [TiDB Cloud CLI Command Reference](/ai/ti/reference/ti-cli-reference.md) for command groups, global options, and shared CLI behavior. - Learn about [TiDB Cloud CLI Configuration and Credentials](/ai/ti/reference/ti-configuration-and-credentials.md) to set up multiple profiles or non-interactive authentication. diff --git a/ai/ti/guides/configure-filesystem-ai-providers.md b/tidb-cloud-filesystem/configure-filesystem-ai-providers.md similarity index 93% rename from ai/ti/guides/configure-filesystem-ai-providers.md rename to tidb-cloud-filesystem/configure-filesystem-ai-providers.md index 138ee6f720ee4..b4219bb949b9f 100644 --- a/ai/ti/guides/configure-filesystem-ai-providers.md +++ b/tidb-cloud-filesystem/configure-filesystem-ai-providers.md @@ -1,6 +1,7 @@ --- title: Configure TiDB Cloud Filesystem AI Providers summary: Learn how to inspect and configure media extraction and embedding providers for a TiDB Cloud Filesystem by using the CLI. +aliases: ['/ai/configure-filesystem-ai-providers'] --- # Configure TiDB Cloud Filesystem AI Providers @@ -9,8 +10,7 @@ A TiDB Cloud Filesystem can optionally extract text from media files and generat ## Prerequisites -- [Install and configure TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md). -- Obtain the target Filesystem ID. +- Follow [Get Started with TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-quick-start.md) to install the TiDB Cloud CLI, create a Filesystem, and obtain the Filesystem ID. - Obtain the API key required by your selected AI provider. The configuration commands require TiDB Cloud API credentials and an explicit Filesystem ID. Set the provider key through `TI_FS_AI_PROVIDER_API_KEY`. The CLI does not persist the key locally. The Filesystem service stores it encrypted and returns only a masked value in subsequent configuration output. @@ -110,5 +110,5 @@ If an update fails because of a timeout, lost response, or another ambiguous net ## What's next -- [Work with TiDB Cloud Filesystem Data](/ai/ti/guides/work-with-filesystem-data.md) +- [Work with TiDB Cloud Filesystem Data](/tidb-cloud-filesystem/work-with-filesystem-data.md) - [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) diff --git a/tidb-cloud-filesystem/filesystem-authorization.md b/tidb-cloud-filesystem/filesystem-authorization.md index a1ae7f4152ab6..494d4369dafdd 100644 --- a/tidb-cloud-filesystem/filesystem-authorization.md +++ b/tidb-cloud-filesystem/filesystem-authorization.md @@ -100,5 +100,5 @@ Token changes can take time to propagate through authorization caches. Rotate by ## What's next - [Share one Filesystem with multiple machines](/tidb-cloud-filesystem/filesystem-sharing.md). -- [Look up token management commands](/ai/ti/guides/manage-filesystem-tokens.md). +- [Look up token management commands](/tidb-cloud-filesystem/manage-filesystem-tokens.md). - [Mount a token-scoped directory](/tidb-cloud-filesystem/filesystem-mount.md). diff --git a/tidb-cloud-filesystem/filesystem-branches-checkpoints.md b/tidb-cloud-filesystem/filesystem-branches-checkpoints.md index 29bf09021cd79..8cf1d90e55f9b 100644 --- a/tidb-cloud-filesystem/filesystem-branches-checkpoints.md +++ b/tidb-cloud-filesystem/filesystem-branches-checkpoints.md @@ -9,6 +9,8 @@ When two agents explore different answers, they should not overwrite each other' The CLI calls these objects **layers**, not Git branches. All commands in this guide use `ti fs`; no Git repository is required. +This page explains the model through a parallel review scenario. For individual commands to create, inspect, commit, or roll back layers, see [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md). + > **Note:** > > TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. diff --git a/tidb-cloud-filesystem/filesystem-mount-linux.md b/tidb-cloud-filesystem/filesystem-mount-linux.md index 74bfaa0bbe48a..459304405885f 100644 --- a/tidb-cloud-filesystem/filesystem-mount-linux.md +++ b/tidb-cloud-filesystem/filesystem-mount-linux.md @@ -70,7 +70,7 @@ ti fs unmount-file-system --mount-path "$HOME/workspace" If `fusermount3` reports `Permission denied`, check the local mount path, `/dev/fuse` access, and the host's security policy. This error is not necessarily a TiDB Cloud credential failure. -Ubuntu 26.04 can apply an AppArmor profile to `fusermount3` that allows paths under home directories but rejects a top-level path such as `/workspace`. Becoming root or running `chown` on `/workspace` does not bypass that policy. Prefer `$HOME/workspace`; if the application requires another path, ask the administrator to update the narrow AppArmor rule described in [Ubuntu mount-path restrictions](/ai/ti/guides/mount-filesystem.md#ubuntu-2604-mount-paths). +Ubuntu 26.04 can apply an AppArmor profile to `fusermount3` that allows paths under home directories but rejects a top-level path such as `/workspace`. Becoming root or running `chown` on `/workspace` does not bypass that policy. Prefer `$HOME/workspace`; if the application requires another path, ask the administrator to update the narrow AppArmor rule described in [Ubuntu mount-path restrictions](/tidb-cloud-filesystem/mount-filesystem.md#ubuntu-2604-mount-paths). A root-created FUSE mount is also not automatically usable by an application running as a different user. Mount as the user that will run the application rather than trying to repair access with `chown` afterward. diff --git a/tidb-cloud-filesystem/filesystem-mount.md b/tidb-cloud-filesystem/filesystem-mount.md index 4a0eedacbc212..a37f0d05f1d05 100644 --- a/tidb-cloud-filesystem/filesystem-mount.md +++ b/tidb-cloud-filesystem/filesystem-mount.md @@ -7,6 +7,8 @@ summary: Select a Filesystem and mount driver, access remote files from a local A mount makes remote files available at a local directory. Use it when your editor, application, or agent expects filesystem paths instead of file-transfer commands. The remote Filesystem persists independently of the mount process. +Start here to choose a platform, select credentials, and complete a first mount. For CLI driver options, container configuration, and mount lifecycle commands, see [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md). + > **Note:** > > TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. diff --git a/ai/ti/guides/manage-filesystem-layers.md b/tidb-cloud-filesystem/manage-filesystem-layers.md similarity index 71% rename from ai/ti/guides/manage-filesystem-layers.md rename to tidb-cloud-filesystem/manage-filesystem-layers.md index 1ece63ff88283..24d1829b1caa7 100644 --- a/ai/ti/guides/manage-filesystem-layers.md +++ b/tidb-cloud-filesystem/manage-filesystem-layers.md @@ -1,17 +1,19 @@ --- title: Manage TiDB Cloud Filesystem Layers and Checkpoints summary: Learn how to safely create, inspect, fork, checkpoint, roll back, commit, pack, and restore TiDB Cloud Filesystem layers. +aliases: ['/ai/manage-filesystem-layers'] --- # Manage TiDB Cloud Filesystem Layers and Checkpoints Use layers to record isolated changes over a Filesystem base path before you commit or discard them. +This guide covers individual CLI operations. For the layer model and an end-to-end parallel review example, see [Branches and Checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md). + ## Prerequisites -- [Install and configure TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md). -- Select a Filesystem by passing `--file-system-id`, setting `TI_FS_FILE_SYSTEM_ID`, or supplying an FS token that identifies the Filesystem. -- Provide an FS token with the required read or write permission by using `--fs-token`, `TI_FS_TOKEN`, or the local credential stored for the selected Filesystem. +- [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). +- For the commands below, set `TI_FS_FILE_SYSTEM_ID` to the Filesystem ID and use its locally stored FS token. Alternatively, set `TI_FS_TOKEN` and `TI_REGION_CODE` for token-only access; the token identifies the Filesystem. To select a Filesystem per command instead, add `--file-system-id ""` to each command. Use a token with the required read or write permission. See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md#understand-local-selection) for selection details. - Choose the base path whose data the layer overlays. ## Create and inspect a layer @@ -68,7 +70,7 @@ A checkpoint mount is read-only. To continue working from a checkpoint, fork a n > **Warning:** > -> Before you create a checkpoint for a layer with a writable FUSE mount, run [`drain-file-system`](/ai/ti/guides/mount-filesystem.md#drain-or-unmount). A checkpoint includes only changes that have reached the service. Before you roll back or commit the layer, drain and then [`unmount-file-system`](/ai/ti/guides/mount-filesystem.md#drain-or-unmount). The CLI does not perform these steps automatically. +> Before you create a checkpoint for a layer with a writable FUSE mount, run [`drain-file-system`](/tidb-cloud-filesystem/mount-filesystem.md#drain-or-unmount). A checkpoint includes only changes that have reached the service. Before you roll back or commit the layer, drain and then [`unmount-file-system`](/tidb-cloud-filesystem/mount-filesystem.md#drain-or-unmount). The CLI does not perform these steps automatically. Choose one outcome for a layer: @@ -112,5 +114,5 @@ Use the same local overlay root when you mount the Filesystem on the destination ## What's next -- [Mount a TiDB Cloud Filesystem](/ai/ti/guides/mount-filesystem.md) +- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md) - [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) diff --git a/ai/ti/guides/manage-filesystem-resources.md b/tidb-cloud-filesystem/manage-filesystem-resources.md similarity index 87% rename from ai/ti/guides/manage-filesystem-resources.md rename to tidb-cloud-filesystem/manage-filesystem-resources.md index 0ba5d578fd957..e9c3339e85d0b 100644 --- a/ai/ti/guides/manage-filesystem-resources.md +++ b/tidb-cloud-filesystem/manage-filesystem-resources.md @@ -1,6 +1,7 @@ --- title: Manage TiDB Cloud Filesystem Resources summary: Learn how to safely create, inspect, check, select, and delete TiDB Cloud Filesystem resources by using TiDB Cloud CLI. +aliases: ['/ai/manage-filesystem-resources'] --- # Manage TiDB Cloud Filesystem Resources @@ -13,8 +14,7 @@ This document describes how to use [`ti fs` commands](/ai/ti/reference/ti-filesy ## Prerequisites -- [Install and configure TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md). -- Configure a profile with TiDB Cloud API credentials. +- [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli) and [configure access](/tidb-cloud-filesystem/filesystem-quick-start.md#step-2-configure-access) with TiDB Cloud API credentials. - Install `jq`, or use another JSON processor to capture command output safely. ## Create a Filesystem @@ -80,6 +80,6 @@ Filesystem deletion is asynchronous. After the service accepts the request, the ## What's next -- [Manage TiDB Cloud Filesystem Tokens](/ai/ti/guides/manage-filesystem-tokens.md) -- [Work with TiDB Cloud Filesystem Data](/ai/ti/guides/work-with-filesystem-data.md) +- [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) +- [Work with TiDB Cloud Filesystem Data](/tidb-cloud-filesystem/work-with-filesystem-data.md) - [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) diff --git a/ai/ti/guides/manage-filesystem-tokens.md b/tidb-cloud-filesystem/manage-filesystem-tokens.md similarity index 80% rename from ai/ti/guides/manage-filesystem-tokens.md rename to tidb-cloud-filesystem/manage-filesystem-tokens.md index b7a861d775410..cf00412618de4 100644 --- a/ai/ti/guides/manage-filesystem-tokens.md +++ b/tidb-cloud-filesystem/manage-filesystem-tokens.md @@ -1,6 +1,7 @@ --- title: Manage TiDB Cloud Filesystem Tokens summary: Learn how to import, generate, scope, inspect, disable, refresh, and revoke access tokens for a TiDB Cloud Filesystem. +aliases: ['/ai/manage-filesystem-tokens'] --- # Manage TiDB Cloud Filesystem Tokens @@ -9,9 +10,9 @@ You can use Filesystem tokens to give users or automation access to a TiDB Cloud ## Prerequisites -- [Install and configure TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md). -- For owner-token generation and TiDB Cloud-authenticated token management, configure TiDB Cloud API credentials and obtain the Filesystem ID. -- For scoped-token generation or bearer-authenticated token management, obtain an owner FS token. You can pass it through `--fs-token`, set `TI_FS_TOKEN`, or use the local token stored for an explicitly selected Filesystem. +- [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). +- For owner-token generation and TiDB Cloud-authenticated token management, [configure TiDB Cloud API credentials](/tidb-cloud-filesystem/filesystem-quick-start.md#step-2-configure-access) and obtain the Filesystem ID. +- For scoped-token generation or bearer-authenticated token management, obtain an [owner FS token](/tidb-cloud-filesystem/filesystem-authorization.md#owner-fs-tokens). You can pass it through `--fs-token`, set `TI_FS_TOKEN`, or use the local token stored for an explicitly selected Filesystem. > **Note:** > @@ -70,7 +71,7 @@ Use [`delete-file-system-token`](/ai/ti/reference/ti-fs-delete-file-system-token > **Note:** > -> Before you rotate, disable, or delete a token used by an active local mount, run [`drain-file-system`](/ai/ti/guides/mount-filesystem.md#drain-or-unmount) and then [`unmount-file-system`](/ai/ti/guides/mount-filesystem.md#drain-or-unmount). The CLI checks for known active mounts and refuses the operation if the token is still in use. +> Before you rotate, disable, or delete a token used by an active local mount, run [`drain-file-system`](/tidb-cloud-filesystem/mount-filesystem.md#drain-or-unmount) and then [`unmount-file-system`](/tidb-cloud-filesystem/mount-filesystem.md#drain-or-unmount). The CLI checks for known active mounts and refuses the operation if the token is still in use. ## What's next diff --git a/ai/ti/guides/manage-filesystem-vault-secrets.md b/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md similarity index 85% rename from ai/ti/guides/manage-filesystem-vault-secrets.md rename to tidb-cloud-filesystem/manage-filesystem-vault-secrets.md index 4b222e12983b4..a433182c01ed4 100644 --- a/ai/ti/guides/manage-filesystem-vault-secrets.md +++ b/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md @@ -1,6 +1,7 @@ --- title: Manage TiDB Cloud Filesystem Vault Secrets summary: Learn how to store, read, delegate, inject, audit, revoke, and mount secrets securely with TiDB Cloud Filesystem Vault. +aliases: ['/ai/manage-filesystem-vault-secrets'] --- # Manage TiDB Cloud Filesystem Vault Secrets @@ -9,9 +10,8 @@ TiDB Cloud Filesystem Vault lets you store secrets, delegate narrowly scoped and ## Prerequisites -- [Install and configure TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md). -- Select a Filesystem by passing `--file-system-id`, setting `TI_FS_FILE_SYSTEM_ID`, or supplying an FS token that identifies the Filesystem. -- For owner operations, provide an owner FS token through `--fs-token`, `TI_FS_TOKEN`, or the local credential stored for the selected Filesystem. +- [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). +- For the secret commands below, set `TI_FS_FILE_SYSTEM_ID` to the Filesystem ID and use its locally stored FS token. Alternatively, set `TI_FS_TOKEN` and `TI_REGION_CODE` for token-only access; the token identifies the Filesystem. Owner operations require an [owner FS token](/tidb-cloud-filesystem/filesystem-authorization.md#owner-fs-tokens). See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md#understand-local-selection) for selection details. > **Note:** > diff --git a/ai/ti/guides/manage-git-workspaces.md b/tidb-cloud-filesystem/manage-git-workspaces.md similarity index 75% rename from ai/ti/guides/manage-git-workspaces.md rename to tidb-cloud-filesystem/manage-git-workspaces.md index ad2ba0ec29752..0d86ef0501a87 100644 --- a/ai/ti/guides/manage-git-workspaces.md +++ b/tidb-cloud-filesystem/manage-git-workspaces.md @@ -1,6 +1,7 @@ --- title: Manage Git Workspaces on TiDB Cloud Filesystem summary: Learn how to clone, hydrate, create linked worktrees, and remove Git workspaces on a mounted TiDB Cloud Filesystem. +aliases: ['/ai/manage-git-workspaces'] --- # Manage Git Workspaces on TiDB Cloud Filesystem @@ -9,9 +10,9 @@ Use `ti fs-git` to accelerate Git workspace setup on a mounted TiDB Cloud Filesy ## Prerequisites -- [Install and configure TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md). -- [Mount a TiDB Cloud Filesystem](/ai/ti/guides/mount-filesystem.md) through FUSE. -- Select the mounted Filesystem by passing `--file-system-id`, setting `TI_FS_FILE_SYSTEM_ID`, or supplying an FS token that identifies it. Provide an FS token with Git workspace permissions. +- [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). +- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md) through FUSE. +- For the commands below, set `TI_FS_FILE_SYSTEM_ID` to the mounted Filesystem ID and use its locally stored FS token. Alternatively, set `TI_FS_TOKEN` and `TI_REGION_CODE` for token-only access; the token identifies the Filesystem. To select a Filesystem per command instead, add `--file-system-id ""` to each command. Use a token with Git workspace permissions. See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md#understand-local-selection) for selection details. - Install Git and configure repository credentials independently. ## Clone a workspace diff --git a/ai/ti/guides/mount-filesystem.md b/tidb-cloud-filesystem/mount-filesystem.md similarity index 80% rename from ai/ti/guides/mount-filesystem.md rename to tidb-cloud-filesystem/mount-filesystem.md index 52a73e39ba801..784f45238b86d 100644 --- a/ai/ti/guides/mount-filesystem.md +++ b/tidb-cloud-filesystem/mount-filesystem.md @@ -1,18 +1,20 @@ --- title: Mount a TiDB Cloud Filesystem summary: Learn how to safely mount, use, drain, and unmount a TiDB Cloud Filesystem on macOS, Linux, or in a container. +aliases: ['/ai/mount-filesystem'] --- # Mount a TiDB Cloud Filesystem In TiDB Cloud CLI, you can mount a TiDB Cloud Filesystem when an application needs to access remote data through a local filesystem path. +This guide covers driver selection, container setup, and mount lifecycle commands. If this is your first mount, start with [Mount TiDB Cloud Filesystem Locally](/tidb-cloud-filesystem/filesystem-mount.md) for platform-specific setup and credential selection. + ## Prerequisites -- [Install and configure TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md). -- Select a Filesystem by passing `--file-system-id`, setting `TI_FS_FILE_SYSTEM_ID`, or supplying an FS token that identifies the Filesystem. -- Provide an FS token through `--fs-token`, `TI_FS_TOKEN`, or the local credential stored for the selected Filesystem. -- On Linux, install FUSE3 and provide access to `/dev/fuse`. +- [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). +- Replace `` in the mount examples with the Filesystem ID. Use its locally stored FS token, or provide a token through `--fs-token` or `TI_FS_TOKEN`. For token-only access without a configured profile, also set `TI_REGION_CODE`; the supplied ID must match the token. See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md#understand-local-selection) for selection details. +- On Linux, [install FUSE3](/tidb-cloud-filesystem/filesystem-mount-linux.md#install-fuse3) and provide access to `/dev/fuse`. ## Choose a mount driver @@ -126,5 +128,5 @@ ti fs drain-file-system --mount-path /path/to/workspace --timeout 30s ## What's next -- [Manage TiDB Cloud Filesystem Layers and Checkpoints](/ai/ti/guides/manage-filesystem-layers.md) +- [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) - [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) diff --git a/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md b/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md new file mode 100644 index 0000000000000..449f5f607e0d1 --- /dev/null +++ b/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md @@ -0,0 +1,28 @@ +--- +title: Use TiDB Cloud Filesystem for Automation and AI Agents +summary: Learn how to use TiDB Cloud Filesystem to share workspaces, hand off CI artifacts, and support agent work with data, Git, journals, and secrets. +--- + +# Use TiDB Cloud Filesystem for Automation and AI Agents + +TiDB Cloud Filesystem keeps files available when a machine, CI job, or agent sandbox is temporary. Choose a workflow below based on what you need to share or preserve. These examples use the TiDB Cloud CLI and open in the TiDB for AI documentation section. + +If you are new to Filesystem, start with the [Quick Start](/tidb-cloud-filesystem/filesystem-quick-start.md) to create a Filesystem and work with your first file. + +To share work across machines and CI jobs, see the following guides in the TiDB for AI documentation. + +| Guide | What you can do | +| --- | --- | +| [Share a TiDB Cloud Filesystem Across Machines](/ai/ti/guides/ti-share-filesystem-across-machines-example.md) | Give users or automation on separate machines access to the same workspace without exchanging file copies. | +| [Hand Off CI Artifacts Between Isolated Jobs with TiDB Cloud Filesystem](/ai/ti/guides/ti-ci-artifact-handoff-example.md) | Keep build output in a Filesystem so a later CI job can retrieve it without copying a CLI profile. | + +To support AI agent workflows, see the following guides in the TiDB for AI documentation. + +| Guide | What you can do | +| --- | --- | +| [Use TiDB Cloud Filesystem in an Agent Sandbox](/ai/ti/guides/ti-agent-sandbox-example.md) | Give an ephemeral agent a shared workspace without exposing TiDB Cloud API keys to its sandbox. | +| [Persist Agent State Across Disposable Sandboxes with TiDB Cloud Filesystem](/ai/ti/guides/ti-persistent-agent-state-example.md) | Keep plans, results, and workflow history available after replacing a sandbox. | +| [Share a Read-Only Dataset Across Parallel Agents with TiDB Cloud Filesystem](/ai/ti/guides/ti-parallel-agent-dataset-example.md) | Let multiple agents use the same dataset without downloading a separate copy for each worker. | +| [Prepare a Git Workspace for Agents on TiDB Cloud Filesystem](/ai/ti/guides/ti-git-workspace-for-agents-example.md) | Make a large repository available to an agent while its clean Git data hydrates in the background. | +| [Record an Agent Workflow in a TiDB Cloud Filesystem Journal](/ai/ti/guides/ti-journal-agent-workflow-example.md) | Record and verify an ordered history of agent actions and handoffs. | +| [Delegate TiDB Cloud Filesystem Vault Secrets to an Agent](/ai/ti/guides/ti-vault-agent-secrets-example.md) | Give an agent limited access to a secret field without sharing the Filesystem owner token. | diff --git a/ai/ti/guides/use-filesystem-journals.md b/tidb-cloud-filesystem/use-filesystem-journals.md similarity index 72% rename from ai/ti/guides/use-filesystem-journals.md rename to tidb-cloud-filesystem/use-filesystem-journals.md index e3ea5e7154873..95bb43f730d95 100644 --- a/ai/ti/guides/use-filesystem-journals.md +++ b/tidb-cloud-filesystem/use-filesystem-journals.md @@ -1,6 +1,7 @@ --- title: Use TiDB Cloud Filesystem Journals summary: Learn how to create, append, read, search, and verify append-only journals for agent and automation events in a Filesystem. +aliases: ['/ai/use-filesystem-journals'] --- # Use TiDB Cloud Filesystem Journals @@ -9,9 +10,8 @@ Journals provide append-only, hash-chained event logs for agent workflows and au ## Prerequisites -- [Install and configure TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md). -- Select a Filesystem by passing `--file-system-id`, setting `TI_FS_FILE_SYSTEM_ID`, or supplying an FS token that identifies the Filesystem. -- Provide an FS token with journal permissions through `--fs-token`, `TI_FS_TOKEN`, or the local credential stored for the selected Filesystem. +- [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). +- For the commands below, set `TI_FS_FILE_SYSTEM_ID` to the Filesystem ID and use its locally stored FS token. Alternatively, set `TI_FS_TOKEN` and `TI_REGION_CODE` for token-only access; the token identifies the Filesystem. To select a Filesystem per command instead, add `--file-system-id ""` to each command. Use a token with journal permissions. See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md#understand-local-selection) for selection details. ## Create a journal diff --git a/ai/ti/guides/work-with-filesystem-data.md b/tidb-cloud-filesystem/work-with-filesystem-data.md similarity index 66% rename from ai/ti/guides/work-with-filesystem-data.md rename to tidb-cloud-filesystem/work-with-filesystem-data.md index 8b75a0708ced6..8749a2e17229b 100644 --- a/ai/ti/guides/work-with-filesystem-data.md +++ b/tidb-cloud-filesystem/work-with-filesystem-data.md @@ -1,6 +1,7 @@ --- title: Work with TiDB Cloud Filesystem Data summary: Learn how to copy, read, organize, search, and inspect files and directories in TiDB Cloud Filesystem by using the CLI. +aliases: ['/ai/work-with-filesystem-data'] --- # Work with TiDB Cloud Filesystem Data @@ -9,9 +10,9 @@ In TiDB Cloud CLI, you can use [`ti fs` commands](/ai/ti/reference/ti-filesystem ## Prerequisites -- [Install and configure TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md). -- [Create a Filesystem](/ai/ti/guides/manage-filesystem-resources.md) or obtain access to an existing one. -- Select the Filesystem by passing `--file-system-id`, setting `TI_FS_FILE_SYSTEM_ID`, or supplying an FS token that identifies it. Provide an FS token with the permissions required for each operation. +- [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). +- [Create a Filesystem](/tidb-cloud-filesystem/manage-filesystem-resources.md) or obtain access to an existing one. +- For the commands below, set `TI_FS_FILE_SYSTEM_ID` to the Filesystem ID and use its locally stored FS token. Alternatively, set `TI_FS_TOKEN` and `TI_REGION_CODE` for token-only access; the token identifies the Filesystem. To select a Filesystem per command instead, add `--file-system-id ""` to each command. Use a token with the permissions required for each operation. See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md#understand-local-selection) for selection details. ## Copy data @@ -70,6 +71,6 @@ ti fs find-files --path /reports --file-name-pattern "*.md" --tag stage=review ## What's next -- [Manage TiDB Cloud Filesystem Layers and Checkpoints](/ai/ti/guides/manage-filesystem-layers.md) -- [Mount a TiDB Cloud Filesystem](/ai/ti/guides/mount-filesystem.md) +- [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) +- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md) - [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) From 4981df70ac6ae44247487a24d93b83935b3738c1 Mon Sep 17 00:00:00 2001 From: qiancai Date: Thu, 17 Sep 2026 15:26:06 +0800 Subject: [PATCH 03/14] add categories in guides --- TOC-ai.md | 2 +- TOC-tidb-cloud-filesystem.md | 27 +-- ai/_index.md | 4 +- ai/ti/guides/manage-filesystems-via-cli.md | 51 ++---- ai/ti/guides/ti-agent-sandbox-example.md | 4 +- ai/ti/reference/ti-filesystem.md | 6 +- .../reference/ti-install-configure-update.md | 4 +- ai/ti/reference/ti-troubleshooting.md | 2 +- ai/ti/ti-overview.md | 2 +- tidb-cloud-filesystem/_index.md | 48 ++--- .../configure-filesystem-ai-providers.md | 2 +- .../filesystem-authorization.md | 53 +----- .../filesystem-branches-checkpoints.md | 166 +++--------------- .../filesystem-mount-linux.md | 23 ++- .../filesystem-mount-macos.md | 2 +- tidb-cloud-filesystem/filesystem-mount.md | 9 +- .../filesystem-quick-start.md | 2 +- tidb-cloud-filesystem/filesystem-sharing.md | 6 +- .../manage-filesystem-layers.md | 6 +- .../manage-filesystem-resources.md | 6 +- .../manage-filesystem-tokens.md | 40 ++++- .../manage-git-workspaces.md | 2 +- tidb-cloud-filesystem/mount-filesystem.md | 132 -------------- ...filesystem-for-automation-and-ai-agents.md | 32 ++-- .../work-with-filesystem-data.md | 6 +- 25 files changed, 171 insertions(+), 466 deletions(-) delete mode 100644 tidb-cloud-filesystem/mount-filesystem.md diff --git a/TOC-ai.md b/TOC-ai.md index 574518319f0a0..b492604ab13f4 100644 --- a/TOC-ai.md +++ b/TOC-ai.md @@ -45,7 +45,7 @@ - [Overview](/ai/ti/ti-overview.md) - [Install, Configure, and Update TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md) - [Manage TiDB Cloud Starter Instances](/ai/ti/guides/manage-starter-instances.md) - - [Manage TiDB Cloud Filesystems via CLI](/ai/ti/guides/manage-filesystems-via-cli.md) + - [Use TiDB Cloud Filesystem with TiDB Cloud CLI](/ai/ti/guides/manage-filesystems-via-cli.md) - Scenarios for Users and Automation - [Run a Daily TiDB Cloud CLI Workflow](/ai/ti/guides/ti-daily-workflow-example.md) - [Query TiDB Cloud Starter with Explicit SQL Roles](/ai/ti/guides/ti-query-sql-with-roles-example.md) diff --git a/TOC-tidb-cloud-filesystem.md b/TOC-tidb-cloud-filesystem.md index bc35a1f80447c..25297bddc8015 100644 --- a/TOC-tidb-cloud-filesystem.md +++ b/TOC-tidb-cloud-filesystem.md @@ -11,26 +11,27 @@ ## CONCEPTS - [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md) -- [Sharing Filesystems](/tidb-cloud-filesystem/filesystem-sharing.md) -- [Layers, Checkpoints, and Parallel Review](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md) +- [Layers and Checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md) ## GUIDES -- [Manage Filesystem Resources](/tidb-cloud-filesystem/manage-filesystem-resources.md) -- [Manage Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) -- [Work with Filesystem Data](/tidb-cloud-filesystem/work-with-filesystem-data.md) -- Mounting Locally +- Basic Operations + - [Manage TiDB Cloud Filesystems](/tidb-cloud-filesystem/manage-filesystem-resources.md) + - [Manage Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) + - [Work with Files and Directories](/tidb-cloud-filesystem/work-with-filesystem-data.md) + - [Share a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-sharing.md) +- Mount a Filesystem - [Overview](/tidb-cloud-filesystem/filesystem-mount.md) - [Linux](/tidb-cloud-filesystem/filesystem-mount-linux.md) - [macOS](/tidb-cloud-filesystem/filesystem-mount-macos.md) - [Docker and Docker Compose](/tidb-cloud-filesystem/filesystem-mount-docker.md) -- [Manage Filesystem Mounts with the CLI](/tidb-cloud-filesystem/mount-filesystem.md) -- [Manage Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) -- [Manage Git Workspaces](/tidb-cloud-filesystem/manage-git-workspaces.md) -- [Use Filesystem Journals](/tidb-cloud-filesystem/use-filesystem-journals.md) -- [Manage Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md) -- [Configure Filesystem AI Providers](/tidb-cloud-filesystem/configure-filesystem-ai-providers.md) -- [Use TiDB Cloud Filesystem for Automation and AI Agents](/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md) +- Advanced Capabilities + - [Manage Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) + - [Manage Git Workspaces](/tidb-cloud-filesystem/manage-git-workspaces.md) + - [Use Filesystem Journals](/tidb-cloud-filesystem/use-filesystem-journals.md) + - [Manage Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md) + - [Configure Filesystem AI Providers](/tidb-cloud-filesystem/configure-filesystem-ai-providers.md) +- [Automation and AI Agent Workflows](/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md) ## REFERENCES diff --git a/ai/_index.md b/ai/_index.md index 30dfee54a5351..039edf068fd3f 100644 --- a/ai/_index.md +++ b/ai/_index.md @@ -101,8 +101,8 @@ The TiDB Cloud CLI (`ti`) lets users, scripts, CI jobs, and AI agents manage TiD | Understand what `ti` manages and when to use it | [TiDB Cloud CLI Overview](/ai/ti/ti-overview.md) | | Install and configure `ti`, then complete a first workflow | [Get Started with TiDB Cloud CLI](/ai/ti/ti-quick-start.md) | | Automate TiDB Cloud Starter instance, branch, and SQL operations | [Manage TiDB Cloud Starter Instances](/ai/ti/guides/manage-starter-instances.md) | -| Persist and share files across machines, CI jobs, and sandboxes | [Manage TiDB Cloud Filesystems via CLI](/ai/ti/guides/manage-filesystems-via-cli.md) | -| Use mounted workspaces, Git workspaces, journals, or delegated secrets | [Mount a Filesystem](/tidb-cloud-filesystem/mount-filesystem.md), [Manage Git Workspaces](/tidb-cloud-filesystem/manage-git-workspaces.md), [Use Filesystem Journals](/tidb-cloud-filesystem/use-filesystem-journals.md), and [Manage Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md) | +| Persist and share files across machines, CI jobs, and sandboxes | [Use TiDB Cloud Filesystem with TiDB Cloud CLI](/ai/ti/guides/manage-filesystems-via-cli.md) | +| Use mounted workspaces, Git workspaces, journals, or delegated secrets | [Mount a Filesystem](/tidb-cloud-filesystem/filesystem-mount.md), [Manage Git Workspaces](/tidb-cloud-filesystem/manage-git-workspaces.md), [Use Filesystem Journals](/tidb-cloud-filesystem/use-filesystem-journals.md), and [Manage Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md) | | Follow an end-to-end automation or agent example | [Run a Daily TiDB Cloud CLI Workflow](/ai/ti/guides/ti-daily-workflow-example.md) or [Use TiDB Cloud Filesystem in an Agent Sandbox](/ai/ti/guides/ti-agent-sandbox-example.md) | | Look up commands, global options, output behavior, and errors | [TiDB Cloud CLI Command Reference](/ai/ti/reference/ti-cli-reference.md) | diff --git a/ai/ti/guides/manage-filesystems-via-cli.md b/ai/ti/guides/manage-filesystems-via-cli.md index 4004bfcc3bc8e..696fbda84a297 100644 --- a/ai/ti/guides/manage-filesystems-via-cli.md +++ b/ai/ti/guides/manage-filesystems-via-cli.md @@ -1,56 +1,41 @@ --- -title: Manage TiDB Cloud Filesystems via CLI -summary: Learn how to create a TiDB Cloud Filesystem with the CLI, write and read a file, and find detailed guides for more Filesystem tasks. +title: Manage TiDB Cloud Filesystem +summary: Learn how to create a TiDB Cloud Filesystem and write and read a file with `ti fs`, then find guides for other Filesystem tasks. --- -# Manage TiDB Cloud Filesystems via CLI +# Manage TiDB Cloud Filesystem -Use the TiDB Cloud CLI (`ti`) to create a Filesystem and work with its files from a terminal or automation workflow. - -## Prerequisites - -- [Install and configure TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md) with API keys that can create a Filesystem in your organization. -- Choose a supported Filesystem region when you run `ti configure`. For region choices and installation details, see the [Filesystem Quick Start](/tidb-cloud-filesystem/filesystem-quick-start.md). +With TiDB Cloud CLI (`ti`), you can create and manage TiDB Cloud Filesystems and work with their files from a terminal or automation workflow. ## Create and use a Filesystem +Before you begin, [install and configure TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md) with API keys that can create a Filesystem in your organization. Choose a supported Filesystem region when you run `ti configure`. + Create a Filesystem and wait until it is ready: ```shell ti fs create-file-system --display-name my-workspace --wait ``` -Save the returned `file_system_id` for the next commands. The CLI stores this Filesystem's token locally. Treat the returned `fs_token` as a secret; do not put it in a public issue or log. - -Replace `` with the returned ID, write a file, and read it back: +Copy the returned `file_system_id` and replace `` in the following commands. The CLI stores the Filesystem token locally. Treat the returned `fs_token` as a secret; do not share the command output publicly. ```shell echo "Hello from my workspace" | ti fs copy-file --file-system-id "" --from-stdin --to-remote /hello.txt ti fs read-file --file-system-id "" --path /hello.txt ``` -The read returns `Hello from my workspace`. The file remains available after you close the terminal. For a guided first run and cleanup instructions, see the [Filesystem Quick Start](/tidb-cloud-filesystem/filesystem-quick-start.md). - -## More Filesystem operations - -For more Filesystem operations, see the following guides in the TiDB Cloud Filesystem documentation: - -- Manage a Filesystem - - - [Create, inspect, check, and delete Filesystem resources](/tidb-cloud-filesystem/manage-filesystem-resources.md) - - [Manage Filesystem tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) - - [Configure Filesystem AI providers](/tidb-cloud-filesystem/configure-filesystem-ai-providers.md) - -- Work with Filesystem data +The read returns `Hello from my workspace`. For supported regions, detailed setup, and cleanup instructions, follow [Get Started with TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-quick-start.md). - - [Copy, read, organize, and search files](/tidb-cloud-filesystem/work-with-filesystem-data.md) - - [Manage layers and checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) - - [Mount a Filesystem with the CLI](/tidb-cloud-filesystem/mount-filesystem.md) +## More Filesystem tasks -- Use advanced Filesystem features +For common tasks, see the following guides in the **TiDB Cloud Filesystem** documentation: - - [Manage Git workspaces](/tidb-cloud-filesystem/manage-git-workspaces.md) - - [Use Filesystem journals](/tidb-cloud-filesystem/use-filesystem-journals.md) - - [Manage Filesystem Vault secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md) +| What you want to do | Guide | +| --- | --- | +| Create, inspect, or delete a Filesystem | [Manage TiDB Cloud Filesystems](/tidb-cloud-filesystem/manage-filesystem-resources.md) | +| Manage access tokens | [Manage Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) | +| Copy, read, and organize files | [Work with Files and Directories](/tidb-cloud-filesystem/work-with-filesystem-data.md) | +| Access files through a local mount | [Mount a Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) | +| Isolate changes with layers and checkpoints | [Manage Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) | -For details about commands, command flags, and output fields, see [`ti fs` command reference](/ai/ti/reference/ti-filesystem.md). \ No newline at end of file +For syntax, flags, and output fields, see the [`ti fs` command reference](/ai/ti/reference/ti-filesystem.md). diff --git a/ai/ti/guides/ti-agent-sandbox-example.md b/ai/ti/guides/ti-agent-sandbox-example.md index 602d0abc8cc50..d1366bb154e4b 100644 --- a/ai/ti/guides/ti-agent-sandbox-example.md +++ b/ai/ti/guides/ti-agent-sandbox-example.md @@ -79,7 +79,7 @@ ti fs mount-file-system \ cat "$HOME/workspace/sandbox/status.txt" ``` -On macOS, omit `--driver fuse` to use WebDAV, which requires no FUSE installation. Install macFUSE and select FUSE when you need FUSE-specific capabilities such as Git workspaces, layers, or online drain. For platform requirements and mount-path restrictions, see [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md). +On macOS, omit `--driver fuse` to use WebDAV, which requires no FUSE installation. Install macFUSE and select FUSE when you need FUSE-specific capabilities such as Git workspaces, layers, or online drain. For platform requirements and mount-path restrictions, see [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-mount.md). After mounting, you can use `ti fs-git`, `ti fs-journal`, and owner-authorized `ti fs-vault` commands with the same FS environment. Give agents a delegated `TI_VAULT_TOKEN` instead of the owner token when they need only selected secret fields. @@ -91,7 +91,7 @@ Stop writers and unmount. A graceful FUSE unmount automatically flushes and drai ti fs unmount-file-system --mount-path "$HOME/workspace" ``` -For a FUSE mount, use `ti fs drain-file-system --mount-path "$HOME/workspace"` separately when you need to verify remote durability while keeping the mount online. `drain-file-system` is not supported for WebDAV. For more information, see [Drain or unmount](/tidb-cloud-filesystem/mount-filesystem.md#drain-or-unmount). Back on the trusted machine: +For a FUSE mount, use `ti fs drain-file-system --mount-path "$HOME/workspace"` separately when you need to verify remote durability while keeping the mount online. `drain-file-system` is not supported for WebDAV. For more information, see [Finish safely](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). Back on the trusted machine: ```bash ti fs delete-file-system \ diff --git a/ai/ti/reference/ti-filesystem.md b/ai/ti/reference/ti-filesystem.md index 3705eef04255d..9408dce5b9c6d 100644 --- a/ai/ti/reference/ti-filesystem.md +++ b/ai/ti/reference/ti-filesystem.md @@ -124,12 +124,12 @@ Aliases use the same options, authentication, output, query, and error behavior ## See also -- [Manage TiDB Cloud Filesystem Resources](/tidb-cloud-filesystem/manage-filesystem-resources.md) +- [Manage TiDB Cloud Filesystems](/tidb-cloud-filesystem/manage-filesystem-resources.md) - [Configure TiDB Cloud Filesystem AI Providers](/tidb-cloud-filesystem/configure-filesystem-ai-providers.md) - [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) -- [Work with TiDB Cloud Filesystem Data](/tidb-cloud-filesystem/work-with-filesystem-data.md) +- [Work with Files and Directories](/tidb-cloud-filesystem/work-with-filesystem-data.md) - [Manage Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) -- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md) +- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) - [Manage Git Workspaces on TiDB Cloud Filesystem](/tidb-cloud-filesystem/manage-git-workspaces.md) - [Use TiDB Cloud Filesystem Journals](/tidb-cloud-filesystem/use-filesystem-journals.md) - [Manage TiDB Cloud Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md) diff --git a/ai/ti/reference/ti-install-configure-update.md b/ai/ti/reference/ti-install-configure-update.md index 37fabb47229aa..2bbe56f17f2da 100644 --- a/ai/ti/reference/ti-install-configure-update.md +++ b/ai/ti/reference/ti-install-configure-update.md @@ -171,7 +171,7 @@ ti update --dry-run > ti fs unmount-file-system --mount-path > ``` > -> For a Vault mount, use `ti fs-vault unmount-vault --mount-path `. For details, see [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md) and [Manage Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md). +> For a Vault mount, use `ti fs-vault unmount-vault --mount-path `. For details, see [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) and [Manage Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md). Apply the latest update: @@ -209,7 +209,7 @@ ti fs unmount-file-system --mount-path ti fs-vault unmount-vault --mount-path ``` -For details, see [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md) and [Manage Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md). +For details, see [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) and [Manage Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md). diff --git a/ai/ti/reference/ti-troubleshooting.md b/ai/ti/reference/ti-troubleshooting.md index 7c09a44a4a4b9..f8b9256180d41 100644 --- a/ai/ti/reference/ti-troubleshooting.md +++ b/ai/ti/reference/ti-troubleshooting.md @@ -197,7 +197,7 @@ mkdir -p "$HOME/workspace" ti fs mount-file-system --mount-path "$HOME/workspace" ``` -Changing the owner or mode of `/workspace` does not bypass AppArmor. If the path cannot change, add explicit `/workspace` mount and unmount rules to `/etc/apparmor.d/local/fusermount3` as described in [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md#ubuntu-2604-mount-paths). +Changing the owner or mode of `/workspace` does not bypass AppArmor. If the path cannot change, add explicit `/workspace` mount and unmount rules to `/etc/apparmor.d/local/fusermount3` as described in [Ubuntu 26.04 mount-path restrictions](/tidb-cloud-filesystem/filesystem-mount-linux.md#ubuntu-2604-mount-path-restrictions). ## Mount becomes stale after a process crash diff --git a/ai/ti/ti-overview.md b/ai/ti/ti-overview.md index 954b2d21d5ec2..0b7a5a6d7d08d 100644 --- a/ai/ti/ti-overview.md +++ b/ai/ti/ti-overview.md @@ -84,7 +84,7 @@ If you are new to the TiDB Cloud CLI, start with the [Quick Start](/ai/ti/ti-qui Then continue based on what you want to do: - [Manage TiDB Cloud Starter Instances](/ai/ti/guides/manage-starter-instances.md) -- [Manage TiDB Cloud Filesystems via CLI](/ai/ti/guides/manage-filesystems-via-cli.md) +- [Use TiDB Cloud Filesystem with TiDB Cloud CLI](/ai/ti/guides/manage-filesystems-via-cli.md) - **Follow end-to-end workflows**: Start with [Run a Daily TiDB Cloud CLI Workflow](/ai/ti/guides/ti-daily-workflow-example.md) or [Use TiDB Cloud Filesystem in an Agent Sandbox](/ai/ti/guides/ti-agent-sandbox-example.md) - **Look up a specific command**: Check the [TiDB Cloud CLI Command Reference](/ai/ti/reference/ti-cli-reference.md) - **See what’s new in TiDB Cloud CLI**: Check the [TiDB Cloud CLI (`ti`) Release Notes](https://github.com/tidbcloud/ti-cli/releases) diff --git a/tidb-cloud-filesystem/_index.md b/tidb-cloud-filesystem/_index.md index a053fbea6f69c..133713ffbd849 100644 --- a/tidb-cloud-filesystem/_index.md +++ b/tidb-cloud-filesystem/_index.md @@ -23,52 +23,30 @@ With a Filesystem, these participants use the same remote files. A new environme - **Share results with a reviewer.** Give the reviewer a read-only scoped token for a report directory. - **Compare parallel approaches.** Fork layers for independent drafts, create checkpoints, and commit a selected result to the shared base. -## Work with files through the CLI or a mount +## Choose how to work with files The CLI provides two ways to use the same remote Filesystem: -- **Direct file commands:** upload, download, list, read, and search files with `ti fs`. These commands do not require FUSE or a mount. -- **Local mounts:** expose remote files at a local directory so existing tools can use commands such as `cat`, `cp`, and `ls`. Linux uses FUSE. macOS uses WebDAV by default and can use macFUSE for layer and checkpoint mounts. +- **Direct file commands:** [Copy, read, organize, and search files](/tidb-cloud-filesystem/work-with-filesystem-data.md) with `ti fs`, without setting up a mount. +- **Local mounts:** [Expose remote files at a local directory](/tidb-cloud-filesystem/filesystem-mount.md) so existing tools can use filesystem paths. Mount support depends on your operating system. -For example, after selecting a Filesystem and supplying its token: +## Control access and changes -```bash -# Read a remote file without mounting it. -ti fs read-file --path /workspace/report.md -``` +- [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md) explains when to use API keys, owner tokens, or scoped tokens. +- [Share a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-sharing.md) shows how to give another machine access without sharing account credentials. +- [Layers and Checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md) explains how to isolate drafts and publish selected changes. -Or mount the workspace on macOS or Linux: +## Get started -```bash -# Give local tools a directory backed by the remote workspace. -mkdir -p "$HOME/workspace" -ti fs mount-file-system --remote-path /workspace --mount-path "$HOME/workspace" -cat "$HOME/workspace/report.md" -``` +If you need to create a Filesystem, follow [Get Started with TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-quick-start.md). You will need TiDB Cloud API keys with permission to create one. -## Control sharing and changes +If someone has given you access to an existing Filesystem, [use its FS token and region without configuring a profile](/tidb-cloud-filesystem/filesystem-mount.md#use-a-token-without-configuring-a-profile). You do not need the creator's API keys to access files within your token's permissions. -- [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md) distinguishes resource-management credentials from owner and scoped Filesystem tokens. -- [Sharing Filesystems](/tidb-cloud-filesystem/filesystem-sharing.md) explains how participants access one remote workspace without sharing account credentials. -- [Branches and Checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md) explains layers, saved history points, and publishing changes to the base Filesystem. - -## Before you begin - -Choose how to access the Filesystem: - -- **Create and manage a Filesystem:** obtain a TiDB Cloud API public key and private key from the [API Keys page](https://tidbcloud.com/org-settings/api-keys), then run `ti configure`. Follow the [Quick Start](/tidb-cloud-filesystem/filesystem-quick-start.md) to create a Filesystem and write your first file. -- **Use an existing Filesystem:** if someone has supplied an FS token, set `TI_FS_TOKEN` and `TI_REGION_CODE`. You do not need TiDB Cloud API keys or `ti configure` to access files within the token's permissions. Follow [Mounting Locally](/tidb-cloud-filesystem/filesystem-mount.md#use-a-token-without-configuring-a-profile). - -See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md) for the differences between API keys, owner tokens, and scoped tokens. An FS token does not grant permission to create or delete a Filesystem through the CLI. - -Choose a supported Filesystem region: `aws-us-east-1`, `aws-ap-southeast-1`, `aws-us-west-2`, or `alicloud-ap-southeast-1`. Keep compute close to that region when possible to reduce network latency. - -Mount support depends on the operating system and its permissions. Windows supports direct file commands but not native mounting through `ti`. WebDAV does not support layer or checkpoint mounts. Buffered writes must reach the service before you remove an environment; a successful local write alone is not a durability guarantee. - -These guides focus on file storage and access. TiDB Cloud Filesystem does not provision an agent's compute environment. A layer is not a security sandbox, and checkpoints do not snapshot the entire live base Filesystem. +For supported regions, platform requirements, and current constraints, see [Regions and Limitations](/ai/ti/reference/ti-regions-security-and-limitations.md). ## What's next - [Create a Filesystem and write your first file](/tidb-cloud-filesystem/filesystem-quick-start.md). -- [Mount an existing Filesystem locally](/tidb-cloud-filesystem/filesystem-mount.md). +- [Share a Filesystem across machines](/tidb-cloud-filesystem/filesystem-sharing.md). +- [Explore automation and AI agent workflows](/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md). - [Try the interactive agent sandbox lab](https://labs.tidb.io/labs/demo_901). diff --git a/tidb-cloud-filesystem/configure-filesystem-ai-providers.md b/tidb-cloud-filesystem/configure-filesystem-ai-providers.md index b4219bb949b9f..5fbd82ae26125 100644 --- a/tidb-cloud-filesystem/configure-filesystem-ai-providers.md +++ b/tidb-cloud-filesystem/configure-filesystem-ai-providers.md @@ -110,5 +110,5 @@ If an update fails because of a timeout, lost response, or another ambiguous net ## What's next -- [Work with TiDB Cloud Filesystem Data](/tidb-cloud-filesystem/work-with-filesystem-data.md) +- [Work with Files and Directories](/tidb-cloud-filesystem/work-with-filesystem-data.md) - [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) diff --git a/tidb-cloud-filesystem/filesystem-authorization.md b/tidb-cloud-filesystem/filesystem-authorization.md index 494d4369dafdd..32626d9534df7 100644 --- a/tidb-cloud-filesystem/filesystem-authorization.md +++ b/tidb-cloud-filesystem/filesystem-authorization.md @@ -25,7 +25,7 @@ An owner token grants broad access within one Filesystem, including reading, wri An owner FS token is not interchangeable with TiDB Cloud API keys: it cannot create or delete the Filesystem resource or generate another owner token through `ti`. In particular, deleting a file and deleting its Filesystem are different permissions. -Filesystem creation returns an owner token and stores it locally for the creating profile. To generate another owner token, use `ti fs generate-file-system-token` with TiDB Cloud API keys. +Filesystem creation returns an owner token and stores it locally for the creating profile. To generate another owner token, use [`ti fs generate-file-system-token`](/tidb-cloud-filesystem/manage-filesystem-tokens.md#generate-an-owner-token) with TiDB Cloud API keys. ### Scoped FS tokens @@ -37,30 +37,7 @@ A scoped token limits access to specified path prefixes and operations. The supp Scoped tokens cannot generate child tokens or manage token inventory. A scoped token can refresh itself while valid; refresh does not turn it into an owner token or broaden its permissions. -## Delegate access to an agent - -On a trusted machine with a locally stored owner token, select the Filesystem and generate a limited token: - -```bash -# Capture the one-time token response without printing the secret. -SCOPED_TOKEN="$(ti fs generate-file-system-scoped-token \ - --file-system-id "" \ - --subject report-agent \ - --ttl 24h \ - --allow /workspace:read,list,write \ - --query fs_token --output text)" -``` - -Transfer `SCOPED_TOKEN` through a secret manager. In the agent's environment, inject it as `TI_FS_TOKEN` and provide the Filesystem's region: - -```bash -# In the agent environment, these values normally come from secret injection. -export TI_FS_TOKEN="" -export TI_REGION_CODE="aws-us-east-1" -ti fs list-files --path /workspace -``` - -The remote `/workspace` directory must already exist. For a mount, select the allowed subtree with `--remote-path /workspace`. Mounting the root `/` is not appropriate for a token that has access only to `/workspace`. +Choose a scoped token when delegating access to an agent or another machine. Keep the owner token on a trusted machine, and use a secret manager to deliver only the scoped token. To generate and use one, see [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md#generate-and-delegate-a-scoped-token). ## Understand local selection @@ -73,32 +50,8 @@ One Filesystem can have multiple remote tokens, but one CLI profile stores at mo For token list, enable, disable, and delete commands, an explicitly supplied owner token selects bearer authentication; otherwise the CLI uses TiDB Cloud API keys and requires a Filesystem ID. A scoped token does not gain administrative capability merely because account keys also exist in the profile. -## Rotate and revoke credentials - -List token metadata to identify the token you want to manage: - -```bash -# Use TiDB Cloud API keys when no explicit FS token is present. -ti fs list-file-system-tokens --file-system-id "" --output text -``` - -The list does not return token plaintext. Preserve newly generated or refreshed tokens in a secret manager. If you lose an owner token, generate a replacement using TiDB Cloud API keys; do not expect to recover the original secret by listing tokens. - -To retire a token, stop its consumers and use its token ID: - -```bash -# Revoke only the retired token, not the Filesystem itself. -ti fs delete-file-system-token --file-system-id "" --token-id "" -``` - -Token changes can take time to propagate through authorization caches. Rotate by distributing and validating a replacement before retiring the old token. Do not treat disabling an owner token as an implicit replacement for reviewing and revoking previously issued scoped tokens. - -> **Warning:** -> -> Stop writes and unmount consumers before refreshing, disabling, or deleting their token. The CLI checks known local mounts but cannot discover every remote machine using the secret. Token refresh is non-idempotent: if the request might have succeeded but the response was lost, do not blindly retry it with the old token. - ## What's next - [Share one Filesystem with multiple machines](/tidb-cloud-filesystem/filesystem-sharing.md). -- [Look up token management commands](/tidb-cloud-filesystem/manage-filesystem-tokens.md). +- [Generate, rotate, or revoke Filesystem tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md). - [Mount a token-scoped directory](/tidb-cloud-filesystem/filesystem-mount.md). diff --git a/tidb-cloud-filesystem/filesystem-branches-checkpoints.md b/tidb-cloud-filesystem/filesystem-branches-checkpoints.md index 8cf1d90e55f9b..4c5ffb7035fdf 100644 --- a/tidb-cloud-filesystem/filesystem-branches-checkpoints.md +++ b/tidb-cloud-filesystem/filesystem-branches-checkpoints.md @@ -1,15 +1,13 @@ --- -title: TiDB Cloud Filesystem Branches and Checkpoints -summary: Use Filesystem layers to compare parallel agent drafts, checkpoint progress, inspect history, and publish a selected result. +title: TiDB Cloud Filesystem Layers and Checkpoints +summary: Understand how Filesystem layers isolate changes, how forks and checkpoints preserve layer history, and when changes become visible in the base Filesystem. --- -# TiDB Cloud Filesystem Branches and Checkpoints +# TiDB Cloud Filesystem Layers and Checkpoints -When two agents explore different answers, they should not overwrite each other's drafts. A Filesystem layer gives each agent a writable view over a base directory. Fork layers to work in parallel, record checkpoints for review, and commit a selected result to the shared base. +A Filesystem layer gives you a writable view over a base path without immediately changing the shared files. Use layers to prepare independent drafts, inspect changes, and publish a selected result after review. The CLI calls these objects **layers**, not Git branches; no Git repository is required. -The CLI calls these objects **layers**, not Git branches. All commands in this guide use `ti fs`; no Git repository is required. - -This page explains the model through a parallel review scenario. For individual commands to create, inspect, commit, or roll back layers, see [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md). +For commands to create, fork, checkpoint, commit, or roll back a layer, see [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md). > **Note:** > @@ -18,149 +16,27 @@ This page explains the model through a parallel review scenario. For individual ## Understand the model - **Base Filesystem:** the live shared files read by ordinary file commands and mounts without a layer selector. -- **Layer:** a set of changes over a base path. Its writes do not change the base until committed. -- **Fork:** a child layer that pins the parent's layer history at its current tip or a specified checkpoint. The child's later writes are independent of the parent's later layer changes. -- **Checkpoint:** a named point in one layer's durable history. It does not flush another client's pending writes or publish changes to the base. -- **Commit:** apply the layer's effective changes to the base Filesystem. This is not the same as a Git commit. - -> **Warning:** -> -> A fork or checkpoint pins layer history, not the entire base Filesystem. Paths resolved from the live base can still reflect later base changes. Layers are neither full Filesystem snapshots nor authorization boundaries. Do not use them as a substitute for backups or scoped tokens. - -## Prepare a parallel review - -This example has two agents write alternative reports, then publishes one result. It uses new filenames for successive drafts and avoids modifying the same file across checkpoints. - -Before running it: - -- Install `ti` v0.2.4 or later and select a Filesystem with an owner token as described in [Mounting Locally](/tidb-cloud-filesystem/filesystem-mount.md#select-a-filesystem). -- Use Linux with FUSE3 or macOS with macFUSE. WebDAV cannot mount layers or checkpoints. -- Use the same terminal for the shell variables below. Do not run other writers against this example's base path. - -```bash -# Isolate this run from existing work and earlier examples. -RUN_ID="$(date +%s)" -REMOTE_ROOT="/layer-demo-$RUN_ID" -LOCAL_ROOT="$HOME/layer-demo-$RUN_ID" -ti fs create-directory --path "$REMOTE_ROOT" -ti fs create-directory --path "$REMOTE_ROOT/reports" -mkdir -p "$LOCAL_ROOT/brief" "$LOCAL_ROOT/analyst" "$LOCAL_ROOT/review" -``` - -## Create a seed and fork two drafts - -```bash -# Create the seed layer and retain its immutable ID. -BASE_LAYER_ID="$(ti fs create-layer \ - --base-root-path "$REMOTE_ROOT" \ - --layer-name "seed-$RUN_ID" \ - --query layer_id --output text)" - -# Direct layer upload avoids a pending local mount write. -printf 'Compare two approaches to the same research question.\n' | ti fs copy-file \ - --from-stdin --to-remote "$REMOTE_ROOT/input.txt" \ - --layer-id "$BASE_LAYER_ID" - -SEED_ID="$(ti fs create-layer-checkpoint \ - --layer-id "$BASE_LAYER_ID" \ - --checkpoint-id "seed-$RUN_ID" \ - --label workspace-seed --query checkpoint_id --output text)" -``` - -Fork both drafts before committing the seed: - -```bash -# Both children start from the same parent checkpoint. -BRIEF_ID="$(ti fs fork-layer \ - --parent-layer-ref "$BASE_LAYER_ID" \ - --layer-name "brief-$RUN_ID" --checkpoint-id "$SEED_ID" \ - --query layer_id --output text)" -ANALYST_ID="$(ti fs fork-layer \ - --parent-layer-ref "$BASE_LAYER_ID" \ - --layer-name "analyst-$RUN_ID" --checkpoint-id "$SEED_ID" \ - --query layer_id --output text)" -ti fs list-layer-chain --layer-ref "$ANALYST_ID" - -# Publish the shared input before the children start writing new reports. -ti fs commit-layer --layer-id "$BASE_LAYER_ID" -``` - -The children retain their pinned parent history. Creating the shared directories in the base and committing the seed first also avoids having each child publish inherited directory creation as part of its own report. - -## Write the alternatives - -```bash -# Give each agent its own writable FUSE mount. -ti fs mount-file-system --driver fuse --remote-path "$REMOTE_ROOT" \ - --mount-path "$LOCAL_ROOT/brief" --layer-ref "$BRIEF_ID" -ti fs mount-file-system --driver fuse --remote-path "$REMOTE_ROOT" \ - --mount-path "$LOCAL_ROOT/analyst" --layer-ref "$ANALYST_ID" - -printf 'A concise recommendation.\n' > "$LOCAL_ROOT/brief/reports/brief.txt" -printf 'An evidence-led first draft.\n' > "$LOCAL_ROOT/analyst/reports/analyst-v1.txt" - -# Make the first analyst draft durable before checkpointing it. -ti fs drain-file-system --mount-path "$LOCAL_ROOT/analyst" --timeout 30s -V1_ID="$(ti fs create-layer-checkpoint \ - --layer-id "$ANALYST_ID" --checkpoint-id "v1-$RUN_ID" \ - --label first-draft --query checkpoint_id --output text)" -``` - -The two mounts contain different reports. Neither report is published in the base yet. Do not mount the same writable layer at multiple paths concurrently. - -## Review a historical checkpoint - -```bash -# Keep revisions in different files for this preview workflow. -printf 'A second draft with additional evidence.\n' > "$LOCAL_ROOT/analyst/reports/analyst-v2.txt" -ti fs drain-file-system --mount-path "$LOCAL_ROOT/analyst" --timeout 30s -ti fs create-layer-checkpoint --layer-id "$ANALYST_ID" \ - --checkpoint-id "v2-$RUN_ID" --label second-draft - -# Mount the earlier checkpoint read-only alongside the current writable tip. -ti fs mount-file-system --driver fuse --remote-path "$REMOTE_ROOT" \ - --mount-path "$LOCAL_ROOT/review" \ - --layer-ref "$ANALYST_ID" --checkpoint-id "$V1_ID" -ls "$LOCAL_ROOT/review/reports" -ls "$LOCAL_ROOT/analyst/reports" -ti fs diff-layer --layer-id "$ANALYST_ID" -``` - -The historical view includes `analyst-v1.txt` but not the later `analyst-v2.txt`. The writable tip includes both. `diff-layer` reports layer changes; it is not a Git-style line-by-line patch. - -To continue writing from a checkpoint, create a new child with `ti fs fork-layer --parent-layer-ref --checkpoint-id --layer-name ` while the parent remains forkable. A checkpoint mount itself is read-only. `rollback-layer` discards a layer; it does not reset that layer to a selected checkpoint. - -## Publish the selected result +- **Layer:** a set of changes over a base path. Its writes remain separate from the base until committed. +- **Fork:** a child layer that pins the parent's layer history at its current tip or a specified checkpoint. Later writes to either layer are independent of the other layer's later changes. +- **Checkpoint:** a named point in one layer's durable history. It does not publish changes to the base or flush another client's pending writes. +- **Commit:** apply all of the layer's effective changes to the base Filesystem. This is not a Git commit or a selective per-file publish operation. -Stop all writers and unmount the example mounts before changing layer lifecycle state: +Use immutable layer IDs in automation because layer names are not guaranteed to be unique. A checkpoint mount is read-only; to continue writing from a checkpoint, fork a new writable layer. Rolling back a layer discards its changes rather than resetting it to a selected checkpoint. -```bash -# Flush writes and release both writable and historical mounts. -ti fs unmount-file-system --mount-path "$LOCAL_ROOT/review" -ti fs drain-file-system --mount-path "$LOCAL_ROOT/brief" --timeout 30s -ti fs unmount-file-system --mount-path "$LOCAL_ROOT/brief" -ti fs drain-file-system --mount-path "$LOCAL_ROOT/analyst" --timeout 30s -ti fs unmount-file-system --mount-path "$LOCAL_ROOT/analyst" +## Understand visibility and boundaries -# Abandon the rejected draft and publish the selected layer to the base. -ti fs delete-layer --layer-ref "$BRIEF_ID" -ti fs commit-layer --layer-id "$ANALYST_ID" -ti fs read-file --path "$REMOTE_ROOT/reports/analyst-v2.txt" -``` +Plain `ti fs read-file` and `ti fs list-files` commands read the base, not an uncommitted layer. Use a layer mount to inspect files in that layer. A fork or checkpoint pins layer history, not the entire base Filesystem: paths resolved from the live base can still reflect later base changes. -Committing publishes all effective changes in the selected layer, including both analyst draft files in this example. It does not select a single file automatically. Deleting a layer logically abandons it; it does not guarantee immediate physical removal of history still pinned by descendants. +Before checkpointing a layer with a writable FUSE mount, [drain the mount](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely) so its writes reach the service. Before committing or rolling back the layer, drain and unmount it. Do not mount the same writable layer at multiple local paths concurrently. -## Preview boundaries +Layers are not full Filesystem snapshots, backups, or authorization boundaries. Use [scoped tokens](/tidb-cloud-filesystem/filesystem-authorization.md#scoped-fs-tokens) to limit access. A layer commit can fail if the base has conflicting changes; retain the layer and inspect the conflict rather than assuming an automatic merge. -- Use immutable layer IDs in automation. Names are not guaranteed to be unique. -- Run a successful FUSE drain before a checkpoint of mounted writes. A checkpoint cannot capture data still buffered only on another machine. -- Recursive `ti fs copy-file --recursive` cannot be combined with `--layer-id`. To populate a layer with a directory tree, copy through its writable FUSE mount. -- Plain `ti fs read-file` and `ti fs list-files` read the base, not an uncommitted layer. Use a layer mount to inspect its files. -- Commit can fail on conflicting base changes. Retain the layer and inspect the conflict; do not automatically delete and recreate it or assume a transactional multi-file merge. -- Repeated changes to layer-created files and inherited metadata can have preview limitations. This example deliberately uses separate revision files and publishes the seed before child work. Test your own workflow before relying on more complex edit histories. -- Layer commit is not a merge into its parent, and the CLI does not provide Git-style merge, rebase, or an in-place reset to a checkpoint. +Other limitations to consider: -## What's next +- The CLI does not provide Git-style merge, rebase, or an in-place reset to a checkpoint. +- A layer commit does not merge changes into its parent layer. +- Deleting a layer logically abandons it but does not guarantee immediate physical removal of history pinned by descendants. +- Recursive `ti fs copy-file --recursive` cannot be combined with `--layer-id`. Use a writable FUSE mount to copy a directory tree into a layer. +- Repeated changes to layer-created files and inherited metadata can have preview limitations. Test complex edit histories before relying on them. -- [Control which participants can read and write](/tidb-cloud-filesystem/filesystem-authorization.md). -- [Review the layer command reference](/ai/ti/reference/ti-filesystem.md). +For command flags and output fields, see the [`ti fs` command reference](/ai/ti/reference/ti-filesystem.md). diff --git a/tidb-cloud-filesystem/filesystem-mount-linux.md b/tidb-cloud-filesystem/filesystem-mount-linux.md index 459304405885f..9d83b2ab1a3dd 100644 --- a/tidb-cloud-filesystem/filesystem-mount-linux.md +++ b/tidb-cloud-filesystem/filesystem-mount-linux.md @@ -70,13 +70,32 @@ ti fs unmount-file-system --mount-path "$HOME/workspace" If `fusermount3` reports `Permission denied`, check the local mount path, `/dev/fuse` access, and the host's security policy. This error is not necessarily a TiDB Cloud credential failure. -Ubuntu 26.04 can apply an AppArmor profile to `fusermount3` that allows paths under home directories but rejects a top-level path such as `/workspace`. Becoming root or running `chown` on `/workspace` does not bypass that policy. Prefer `$HOME/workspace`; if the application requires another path, ask the administrator to update the narrow AppArmor rule described in [Ubuntu mount-path restrictions](/tidb-cloud-filesystem/mount-filesystem.md#ubuntu-2604-mount-paths). +Ubuntu 26.04 can apply an AppArmor profile to `fusermount3` that allows paths under home directories but rejects a top-level path such as `/workspace`. Becoming root or running `chown` on `/workspace` does not bypass that policy. Prefer `$HOME/workspace`; if the application requires another path, see [Ubuntu 26.04 mount-path restrictions](#ubuntu-2604-mount-path-restrictions). A root-created FUSE mount is also not automatically usable by an application running as a different user. Mount as the user that will run the application rather than trying to repair access with `chown` afterward. For startup failures, inspect the diagnostic log path shown by the CLI. Do not repeatedly start mounts at the same path without checking whether a previous mount is still present. +## Ubuntu 26.04 mount-path restrictions + +Ubuntu 26.04 applies an AppArmor profile to `/usr/bin/fusermount3`. By default, mount under the current user's home directory, `/mnt`, `/media`, `/tmp`, or `/run/user/` instead of `/workspace`. + +If an application requires `/workspace`, ask the host administrator to add these rules to `/etc/apparmor.d/local/fusermount3`: + +```text +mount fstype=@{fuse_types} options=(nosuid,nodev) options in (ro,rw,noatime,dirsync,nodiratime,noexec,sync) -> /workspace/{,**/}, +umount /workspace/{,**/}, +``` + +The administrator can then reload the profile: + +```shell +sudo apparmor_parser -r /etc/apparmor.d/fusermount3 +``` + +For related errors, see [Troubleshoot TiDB Cloud CLI](/ai/ti/reference/ti-troubleshooting.md). + ## What's next -- [Use layers and checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md). +- [Manage layers and checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md). - [Share files with another machine](/tidb-cloud-filesystem/filesystem-sharing.md). diff --git a/tidb-cloud-filesystem/filesystem-mount-macos.md b/tidb-cloud-filesystem/filesystem-mount-macos.md index 1d7aad6b81317..0392974f5999a 100644 --- a/tidb-cloud-filesystem/filesystem-mount-macos.md +++ b/tidb-cloud-filesystem/filesystem-mount-macos.md @@ -60,7 +60,7 @@ To keep a FUSE mount online while waiting for pending writes to reach the servic ti fs drain-file-system --mount-path "$HOME/workspace" --timeout 30s ``` -For writable layer and read-only checkpoint examples, see [Branches and Checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md). They cannot run through WebDAV. +For layer and checkpoint mount options, see the [`mount-file-system` command reference](/ai/ti/reference/ti-fs-mount-file-system.md). Layer and checkpoint mounts cannot run through WebDAV. When finished with the FUSE mount, stop writers and unmount it: diff --git a/tidb-cloud-filesystem/filesystem-mount.md b/tidb-cloud-filesystem/filesystem-mount.md index a37f0d05f1d05..bb8aba3a6f08a 100644 --- a/tidb-cloud-filesystem/filesystem-mount.md +++ b/tidb-cloud-filesystem/filesystem-mount.md @@ -1,14 +1,13 @@ --- title: Mount TiDB Cloud Filesystem Locally summary: Select a Filesystem and mount driver, access remote files from a local directory, and stop a mount without losing pending writes. +aliases: ['/ai/mount-filesystem'] --- # Mount TiDB Cloud Filesystem Locally A mount makes remote files available at a local directory. Use it when your editor, application, or agent expects filesystem paths instead of file-transfer commands. The remote Filesystem persists independently of the mount process. -Start here to choose a platform, select credentials, and complete a first mount. For CLI driver options, container configuration, and mount lifecycle commands, see [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md). - > **Note:** > > TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. @@ -21,6 +20,8 @@ Start here to choose a platform, select credentials, and complete a first mount. Native Windows mounting is not supported by `ti`. Use direct commands such as `ti fs copy-file`, `ti fs read-file`, and `ti fs list-files` instead. +With `--driver auto`, the CLI selects WebDAV on macOS and FUSE on Linux. To mount a layer or checkpoint on macOS, install macFUSE and select `--driver fuse`. WebDAV mounting is not supported on Linux. + ## Select a Filesystem ### Use a locally stored token @@ -62,6 +63,8 @@ The command waits for readiness and returns a structured result with `status: mo To expose only one remote directory, pass `--remote-path /workspace`. To make a mount read-only, add `--read-only`. These are client-side mount settings, not substitutes for a scoped token's server-enforced permissions. +To mount a layer or a read-only checkpoint, select the FUSE driver and use the layer options in the [`mount-file-system` command reference](/ai/ti/reference/ti-fs-mount-file-system.md). + ```bash # These are ordinary local filesystem commands, not CLI subcommands. ls "$HOME/workspace" @@ -96,5 +99,5 @@ An explicit drain is not required before every normal unmount. WebDAV does not s ## What's next - [Share a Filesystem across environments](/tidb-cloud-filesystem/filesystem-sharing.md). -- [Mount a layer or historical checkpoint](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md). +- [Manage layers and checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md). - [Look up mount options](/ai/ti/reference/ti-fs-mount-file-system.md). diff --git a/tidb-cloud-filesystem/filesystem-quick-start.md b/tidb-cloud-filesystem/filesystem-quick-start.md index 6bb8ec93a308f..a79db4b252d26 100644 --- a/tidb-cloud-filesystem/filesystem-quick-start.md +++ b/tidb-cloud-filesystem/filesystem-quick-start.md @@ -115,7 +115,7 @@ The read returns `Hello from my workspace`. The file remains available after you - [Mount the Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) to use it through a local directory. - [Share the workspace](/tidb-cloud-filesystem/filesystem-sharing.md) with another machine or agent. -- [Use layers and checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md) to review changes before publishing them. +- [Manage layers and checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) to review changes before publishing them. When you no longer need this tutorial Filesystem, delete it using your TiDB Cloud API credentials: diff --git a/tidb-cloud-filesystem/filesystem-sharing.md b/tidb-cloud-filesystem/filesystem-sharing.md index 712ea6d485dbe..e19cd1c5c58bb 100644 --- a/tidb-cloud-filesystem/filesystem-sharing.md +++ b/tidb-cloud-filesystem/filesystem-sharing.md @@ -1,9 +1,9 @@ --- -title: Share TiDB Cloud Filesystems +title: Share a TiDB Cloud Filesystem summary: Share a remote workspace across machines and sandboxes with separate access tokens, and hand off files after writes reach the service. --- -# Share TiDB Cloud Filesystems +# Share a TiDB Cloud Filesystem Sharing a Filesystem gives participants access to the same remote namespace, not independent copies. For example, an agent can produce a report in a sandbox and a reviewer can open that report from a laptop without downloading and redistributing an archive for every revision. @@ -109,5 +109,5 @@ Do not delete the Filesystem to disconnect one participant: resource deletion af ## What's next -- [Use branches to compare independent drafts](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md). +- [Understand layers and checkpoints for independent drafts](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md). - [Run the agent sandbox example](/ai/ti/guides/ti-agent-sandbox-example.md). diff --git a/tidb-cloud-filesystem/manage-filesystem-layers.md b/tidb-cloud-filesystem/manage-filesystem-layers.md index 24d1829b1caa7..dc7b6d0abb25c 100644 --- a/tidb-cloud-filesystem/manage-filesystem-layers.md +++ b/tidb-cloud-filesystem/manage-filesystem-layers.md @@ -8,7 +8,7 @@ aliases: ['/ai/manage-filesystem-layers'] Use layers to record isolated changes over a Filesystem base path before you commit or discard them. -This guide covers individual CLI operations. For the layer model and an end-to-end parallel review example, see [Branches and Checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md). +This guide covers individual CLI operations. To understand layer visibility, forks, checkpoints, and their boundaries before using these commands, see [Layers and Checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md). ## Prerequisites @@ -70,7 +70,7 @@ A checkpoint mount is read-only. To continue working from a checkpoint, fork a n > **Warning:** > -> Before you create a checkpoint for a layer with a writable FUSE mount, run [`drain-file-system`](/tidb-cloud-filesystem/mount-filesystem.md#drain-or-unmount). A checkpoint includes only changes that have reached the service. Before you roll back or commit the layer, drain and then [`unmount-file-system`](/tidb-cloud-filesystem/mount-filesystem.md#drain-or-unmount). The CLI does not perform these steps automatically. +> Before you create a checkpoint for a layer with a writable FUSE mount, run [`drain-file-system`](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). A checkpoint includes only changes that have reached the service. Before you roll back or commit the layer, drain and then [`unmount-file-system`](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). The CLI does not perform these steps automatically. Choose one outcome for a layer: @@ -114,5 +114,5 @@ Use the same local overlay root when you mount the Filesystem on the destination ## What's next -- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md) +- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) - [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) diff --git a/tidb-cloud-filesystem/manage-filesystem-resources.md b/tidb-cloud-filesystem/manage-filesystem-resources.md index e9c3339e85d0b..afa036e875e65 100644 --- a/tidb-cloud-filesystem/manage-filesystem-resources.md +++ b/tidb-cloud-filesystem/manage-filesystem-resources.md @@ -1,10 +1,10 @@ --- -title: Manage TiDB Cloud Filesystem Resources +title: Manage TiDB Cloud Filesystems summary: Learn how to safely create, inspect, check, select, and delete TiDB Cloud Filesystem resources by using TiDB Cloud CLI. aliases: ['/ai/manage-filesystem-resources'] --- -# Manage TiDB Cloud Filesystem Resources +# Manage TiDB Cloud Filesystems TiDB Cloud Filesystem is a serverless distributed file system designed for AI agents and automation workloads. It provides a persistent, shareable file namespace that remains available independently of the local machine, sandbox, or CI runner that accesses it. @@ -81,5 +81,5 @@ Filesystem deletion is asynchronous. After the service accepts the request, the ## What's next - [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) -- [Work with TiDB Cloud Filesystem Data](/tidb-cloud-filesystem/work-with-filesystem-data.md) +- [Work with Files and Directories](/tidb-cloud-filesystem/work-with-filesystem-data.md) - [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) diff --git a/tidb-cloud-filesystem/manage-filesystem-tokens.md b/tidb-cloud-filesystem/manage-filesystem-tokens.md index cf00412618de4..d676521fb2678 100644 --- a/tidb-cloud-filesystem/manage-filesystem-tokens.md +++ b/tidb-cloud-filesystem/manage-filesystem-tokens.md @@ -26,7 +26,7 @@ When you run `import-file-system-token`, the CLI validates the token format, ext ti fs import-file-system-token --from-file ./fs-token --region aws-us-east-1 ``` -## Generate a token +## Generate an owner token Generate another owner token by using TiDB Cloud API credentials. The CLI does not store the generated token locally by default, so you must capture its one-time plaintext response securely: @@ -40,23 +40,39 @@ ti fs generate-file-system-token \ To have the CLI store the generated token locally, add `--store-locally`. Use `--replace` if a different token is already stored for this Filesystem. -For least-privilege access, generate a path-and-operation-limited token from an owner token: +## Generate and delegate a scoped token + +On a trusted machine with an owner token, generate a path-and-operation-limited token for an agent: ```shell -ti fs generate-file-system-scoped-token \ +SCOPED_TOKEN="$(ti fs generate-file-system-scoped-token \ --file-system-id "" \ + --subject report-agent \ --ttl 24h \ - --allow /workspace:read,list > ./scoped-token.json + --allow /workspace:read,list,write \ + --query fs_token --output text)" ``` +Transfer the token through a secret manager. In the agent's environment, inject the token and the Filesystem's region: + +```shell +export TI_FS_TOKEN="" +export TI_REGION_CODE="" +ti fs list-files --path /workspace +``` + +The remote `/workspace` directory must already exist. For a mount, select the allowed subtree with `--remote-path /workspace`; a token restricted to `/workspace` cannot mount the root `/`. For the permission model and credential precedence, see [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md). + ## Inspect and change token status List non-secret token metadata: ```shell -ti fs list-file-system-tokens --file-system-id "" +ti fs list-file-system-tokens --file-system-id "" --output text ``` +The list does not return token plaintext. Preserve newly generated or refreshed tokens in a secret manager. If you lose an owner token, generate a replacement using TiDB Cloud API credentials; you cannot recover the original secret by listing tokens. + Use [`disable-file-system-token`](/ai/ti/reference/ti-fs-disable-file-system-token.md) to suspend a token temporarily and [`enable-file-system-token`](/ai/ti/reference/ti-fs-enable-file-system-token.md) to restore it. ## Rotate or revoke a token @@ -67,11 +83,17 @@ Use [`refresh-file-system-token`](/ai/ti/reference/ti-fs-refresh-file-system-tok > > Refresh is non-idempotent. If a request might have succeeded but its response was lost, do not retry with the old token. Instead, generate a new owner token using TiDB Cloud credentials. -Use [`delete-file-system-token`](/ai/ti/reference/ti-fs-delete-file-system-token.md) to revoke a token permanently. If the deleted token matches the locally stored token, the CLI automatically removes the local credential. - -> **Note:** +> **Warning:** > -> Before you rotate, disable, or delete a token used by an active local mount, run [`drain-file-system`](/tidb-cloud-filesystem/mount-filesystem.md#drain-or-unmount) and then [`unmount-file-system`](/tidb-cloud-filesystem/mount-filesystem.md#drain-or-unmount). The CLI checks for known active mounts and refuses the operation if the token is still in use. +> Before you rotate, disable, or delete a token used by an active local mount, [stop writers and unmount it safely](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). Drain a FUSE mount before unmounting; WebDAV does not support drain. The CLI checks known local mounts but cannot discover every remote machine using the token. Stop writes and unmount consumers on those machines before changing it. + +Before retiring a token, distribute and validate a replacement. Then use [`delete-file-system-token`](/ai/ti/reference/ti-fs-delete-file-system-token.md) to revoke the retired token by its token ID: + +```shell +ti fs delete-file-system-token --file-system-id "" --token-id "" +``` + +If the deleted token matches the locally stored token, the CLI automatically removes the local credential. Token changes can take time to propagate through authorization caches. Disabling an owner token does not replace reviewing and revoking scoped tokens that it issued. ## What's next diff --git a/tidb-cloud-filesystem/manage-git-workspaces.md b/tidb-cloud-filesystem/manage-git-workspaces.md index 0d86ef0501a87..af422032d099a 100644 --- a/tidb-cloud-filesystem/manage-git-workspaces.md +++ b/tidb-cloud-filesystem/manage-git-workspaces.md @@ -11,7 +11,7 @@ Use `ti fs-git` to accelerate Git workspace setup on a mounted TiDB Cloud Filesy ## Prerequisites - [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). -- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md) through FUSE. +- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) through FUSE. - For the commands below, set `TI_FS_FILE_SYSTEM_ID` to the mounted Filesystem ID and use its locally stored FS token. Alternatively, set `TI_FS_TOKEN` and `TI_REGION_CODE` for token-only access; the token identifies the Filesystem. To select a Filesystem per command instead, add `--file-system-id ""` to each command. Use a token with Git workspace permissions. See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md#understand-local-selection) for selection details. - Install Git and configure repository credentials independently. diff --git a/tidb-cloud-filesystem/mount-filesystem.md b/tidb-cloud-filesystem/mount-filesystem.md deleted file mode 100644 index 784f45238b86d..0000000000000 --- a/tidb-cloud-filesystem/mount-filesystem.md +++ /dev/null @@ -1,132 +0,0 @@ ---- -title: Mount a TiDB Cloud Filesystem -summary: Learn how to safely mount, use, drain, and unmount a TiDB Cloud Filesystem on macOS, Linux, or in a container. -aliases: ['/ai/mount-filesystem'] ---- - -# Mount a TiDB Cloud Filesystem - -In TiDB Cloud CLI, you can mount a TiDB Cloud Filesystem when an application needs to access remote data through a local filesystem path. - -This guide covers driver selection, container setup, and mount lifecycle commands. If this is your first mount, start with [Mount TiDB Cloud Filesystem Locally](/tidb-cloud-filesystem/filesystem-mount.md) for platform-specific setup and credential selection. - -## Prerequisites - -- [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). -- Replace `` in the mount examples with the Filesystem ID. Use its locally stored FS token, or provide a token through `--fs-token` or `TI_FS_TOKEN`. For token-only access without a configured profile, also set `TI_REGION_CODE`; the supplied ID must match the token. See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md#understand-local-selection) for selection details. -- On Linux, [install FUSE3](/tidb-cloud-filesystem/filesystem-mount-linux.md#install-fuse3) and provide access to `/dev/fuse`. - -## Choose a mount driver - -| Platform | `--driver auto` | Notes | -|---|---|---| -| macOS | WebDAV | Install macFUSE and select `--driver fuse` for FUSE support. | -| Linux | FUSE | WebDAV mounting is not supported. | -| Windows | Not supported | Use `ti fs` data-plane commands without a mount. | - -## Mount the Filesystem - -On macOS or Linux, create a local path and mount the Filesystem in the background: - -```shell -mkdir -p /path/to/workspace -ti fs mount-file-system \ - --file-system-id "" \ - --mount-path /path/to/workspace -``` - -The CLI starts a background mount process and writes a local mount locator so that the drain and unmount commands can find the correct process. - -Use `--remote-path` to expose a subtree or `--read-only` to prevent writes. To mount a layer or checkpoint, select the FUSE driver and pass the appropriate layer options described in the [`mount-file-system` reference](/ai/ti/reference/ti-fs-mount-file-system.md). - -## Mount in a container - -Installing FUSE3 in an image is not sufficient. The host must expose `/dev/fuse`, and the container must be allowed to perform the mount. For Docker, provide settings equivalent to the following: - -```shell -docker run --rm -it \ - --device /dev/fuse \ - --cap-add SYS_ADMIN \ - --security-opt apparmor=unconfined \ - --env TI_FS_TOKEN \ - --env TI_REGION_CODE \ - --env TI_FS_FILE_SYSTEM_ID \ - -``` - -For Docker Compose, pass the same device, capability, security, and environment settings: - -```yaml -services: - agent: - image: - devices: - - /dev/fuse:/dev/fuse - cap_add: - - SYS_ADMIN - security_opt: - - apparmor=unconfined - environment: - TI_FS_TOKEN: ${TI_FS_TOKEN} - TI_REGION_CODE: ${TI_REGION_CODE} - TI_FS_FILE_SYSTEM_ID: ${TI_FS_FILE_SYSTEM_ID} -``` - -> **Warning:** -> -> `SYS_ADMIN` and an unconfined AppArmor profile weaken container isolation. Use them only for a dedicated, trusted container. When FUSE access is unavailable, use `ti fs` data commands without a mount. - -## Ubuntu 26.04 mount paths - -Ubuntu 26.04 applies an AppArmor profile to `/usr/bin/fusermount3`. By default, use a path under the current user's home directory, `/mnt`, `/media`, `/tmp`, or `/run/user/` instead of `/workspace`. - -For example: - -```shell -mkdir -p "$HOME/workspace" -ti fs mount-file-system \ - --file-system-id "" \ - --mount-path "$HOME/workspace" -``` - -If an application requires `/workspace`, add the following rules to `/etc/apparmor.d/local/fusermount3`: - -```text -mount fstype=@{fuse_types} options=(nosuid,nodev) options in (ro,rw,noatime,dirsync,nodiratime,noexec,sync) -> /workspace/{,**/}, -umount /workspace/{,**/}, -``` - -Then reload the profile: - -```shell -sudo apparmor_parser -r /etc/apparmor.d/fusermount3 -``` - -For related errors, see [Troubleshoot TiDB Cloud CLI](/ai/ti/reference/ti-troubleshooting.md). - -## Drain or unmount - -When you run `unmount-file-system`, the CLI automatically flushes open file handles and pending FUSE work before stopping the mount: - -```shell -ti fs unmount-file-system --mount-path /path/to/workspace -``` - -If you need a durability barrier while keeping a FUSE mount online (for example, before creating a layer checkpoint), run `drain-file-system` explicitly. This command flushes pending writes and waits for them to complete without unmounting: - -```shell -ti fs drain-file-system --mount-path /path/to/workspace --timeout 30s -``` - -> **Note:** -> -> Drain is supported only for FUSE mounts. WebDAV mounts flush writes through normal file close operations. - -> **Warning:** -> -> Do not terminate a machine while writes remain pending or after an unmount returns an error. In-memory writes and local-only overlay files can be lost. For a FUSE mount, run `drain-file-system` before shutdown to confirm that pending writes have reached the remote Filesystem. For a WebDAV mount, close files in the application and verify that `unmount-file-system` succeeds. - -## What's next - -- [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) -- [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) diff --git a/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md b/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md index 449f5f607e0d1..e02519b24f354 100644 --- a/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md +++ b/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md @@ -1,28 +1,28 @@ --- -title: Use TiDB Cloud Filesystem for Automation and AI Agents -summary: Learn how to use TiDB Cloud Filesystem to share workspaces, hand off CI artifacts, and support agent work with data, Git, journals, and secrets. +title: Explore Automation and AI Agent Workflows +summary: Find TiDB for AI examples for sharing Filesystem workspaces, handing off CI artifacts, and supporting agents with data, Git, journals, and secrets. --- -# Use TiDB Cloud Filesystem for Automation and AI Agents +# Explore Automation and AI Agent Workflows -TiDB Cloud Filesystem keeps files available when a machine, CI job, or agent sandbox is temporary. Choose a workflow below based on what you need to share or preserve. These examples use the TiDB Cloud CLI and open in the TiDB for AI documentation section. +TiDB Cloud Filesystem keeps files available when a machine, CI job, or agent sandbox is temporary. Choose a workflow below based on what you need to share or preserve. If you are new to Filesystem, start with the [Quick Start](/tidb-cloud-filesystem/filesystem-quick-start.md) to create a Filesystem and work with your first file. -To share work across machines and CI jobs, see the following guides in the TiDB for AI documentation. +To share work across machines and CI jobs, see the following guides in the **TiDB for AI** documentation. -| Guide | What you can do | +| What you want to do | Guide | | --- | --- | -| [Share a TiDB Cloud Filesystem Across Machines](/ai/ti/guides/ti-share-filesystem-across-machines-example.md) | Give users or automation on separate machines access to the same workspace without exchanging file copies. | -| [Hand Off CI Artifacts Between Isolated Jobs with TiDB Cloud Filesystem](/ai/ti/guides/ti-ci-artifact-handoff-example.md) | Keep build output in a Filesystem so a later CI job can retrieve it without copying a CLI profile. | +| Access the same workspace from multiple machines | [Share a TiDB Cloud Filesystem Across Machines](/ai/ti/guides/ti-share-filesystem-across-machines-example.md) | +| Pass build artifacts between isolated CI jobs | [Hand Off CI Artifacts Between Isolated Jobs with TiDB Cloud Filesystem](/ai/ti/guides/ti-ci-artifact-handoff-example.md) | -To support AI agent workflows, see the following guides in the TiDB for AI documentation. +To support AI agent workflows, see the following guides in the **TiDB for AI** documentation. -| Guide | What you can do | +| What you want to do | Guide | | --- | --- | -| [Use TiDB Cloud Filesystem in an Agent Sandbox](/ai/ti/guides/ti-agent-sandbox-example.md) | Give an ephemeral agent a shared workspace without exposing TiDB Cloud API keys to its sandbox. | -| [Persist Agent State Across Disposable Sandboxes with TiDB Cloud Filesystem](/ai/ti/guides/ti-persistent-agent-state-example.md) | Keep plans, results, and workflow history available after replacing a sandbox. | -| [Share a Read-Only Dataset Across Parallel Agents with TiDB Cloud Filesystem](/ai/ti/guides/ti-parallel-agent-dataset-example.md) | Let multiple agents use the same dataset without downloading a separate copy for each worker. | -| [Prepare a Git Workspace for Agents on TiDB Cloud Filesystem](/ai/ti/guides/ti-git-workspace-for-agents-example.md) | Make a large repository available to an agent while its clean Git data hydrates in the background. | -| [Record an Agent Workflow in a TiDB Cloud Filesystem Journal](/ai/ti/guides/ti-journal-agent-workflow-example.md) | Record and verify an ordered history of agent actions and handoffs. | -| [Delegate TiDB Cloud Filesystem Vault Secrets to an Agent](/ai/ti/guides/ti-vault-agent-secrets-example.md) | Give an agent limited access to a secret field without sharing the Filesystem owner token. | +| Give an agent sandbox access to a shared workspace | [Use TiDB Cloud Filesystem in an Agent Sandbox](/ai/ti/guides/ti-agent-sandbox-example.md) | +| Keep agent state across disposable sandboxes | [Persist Agent State Across Disposable Sandboxes with TiDB Cloud Filesystem](/ai/ti/guides/ti-persistent-agent-state-example.md) | +| Share a read-only dataset with parallel agents | [Share a Read-Only Dataset Across Parallel Agents with TiDB Cloud Filesystem](/ai/ti/guides/ti-parallel-agent-dataset-example.md) | +| Prepare a large Git workspace for an agent | [Prepare a Git Workspace for Agents on TiDB Cloud Filesystem](/ai/ti/guides/ti-git-workspace-for-agents-example.md) | +| Record and verify an agent workflow | [Record an Agent Workflow in a TiDB Cloud Filesystem Journal](/ai/ti/guides/ti-journal-agent-workflow-example.md) | +| Give an agent limited access to a Vault secret | [Delegate TiDB Cloud Filesystem Vault Secrets to an Agent](/ai/ti/guides/ti-vault-agent-secrets-example.md) | diff --git a/tidb-cloud-filesystem/work-with-filesystem-data.md b/tidb-cloud-filesystem/work-with-filesystem-data.md index 8749a2e17229b..405b3c47eea29 100644 --- a/tidb-cloud-filesystem/work-with-filesystem-data.md +++ b/tidb-cloud-filesystem/work-with-filesystem-data.md @@ -1,10 +1,10 @@ --- -title: Work with TiDB Cloud Filesystem Data +title: Work with Files and Directories in TiDB Cloud Filesystem summary: Learn how to copy, read, organize, search, and inspect files and directories in TiDB Cloud Filesystem by using the CLI. aliases: ['/ai/work-with-filesystem-data'] --- -# Work with TiDB Cloud Filesystem Data +# Work with Files and Directories in TiDB Cloud Filesystem In TiDB Cloud CLI, you can use [`ti fs` commands](/ai/ti/reference/ti-filesystem.md) to transfer data between local storage and TiDB Cloud Filesystem and to manage its remote namespace. @@ -72,5 +72,5 @@ ti fs find-files --path /reports --file-name-pattern "*.md" --tag stage=review ## What's next - [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) -- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/mount-filesystem.md) +- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) - [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) From 4ed5e3441e6f6db686f2f0ce1082e9d82254764f Mon Sep 17 00:00:00 2001 From: qiancai Date: Thu, 17 Sep 2026 15:49:44 +0800 Subject: [PATCH 04/14] revise descriptions --- TOC-tidb-cloud-filesystem.md | 12 ++++++++---- tidb-cloud-filesystem/_index.md | 6 +++--- .../configure-filesystem-ai-providers.md | 2 +- .../filesystem-authorization.md | 16 +++++++++++++++- ...oints.md => filesystem-layers-checkpoints.md} | 8 ++++++-- tidb-cloud-filesystem/filesystem-mount.md | 13 ++++++++++++- tidb-cloud-filesystem/filesystem-quick-start.md | 10 +++++++++- tidb-cloud-filesystem/filesystem-sharing.md | 5 +++-- .../manage-filesystem-layers.md | 12 ++++++++++-- .../manage-filesystem-resources.md | 8 +++++++- .../manage-filesystem-tokens.md | 10 ++++++---- .../manage-filesystem-vault-secrets.md | 14 ++++++++++++++ tidb-cloud-filesystem/manage-git-workspaces.md | 6 ++++-- ...se-filesystem-for-automation-and-ai-agents.md | 2 +- tidb-cloud-filesystem/use-filesystem-journals.md | 6 +++++- .../work-with-filesystem-data.md | 8 +++++++- 16 files changed, 111 insertions(+), 27 deletions(-) rename tidb-cloud-filesystem/{filesystem-branches-checkpoints.md => filesystem-layers-checkpoints.md} (95%) diff --git a/TOC-tidb-cloud-filesystem.md b/TOC-tidb-cloud-filesystem.md index 25297bddc8015..81171caebf7e1 100644 --- a/TOC-tidb-cloud-filesystem.md +++ b/TOC-tidb-cloud-filesystem.md @@ -11,7 +11,7 @@ ## CONCEPTS - [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md) -- [Layers and Checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md) +- [Layers and Checkpoints](/tidb-cloud-filesystem/filesystem-layers-checkpoints.md) ## GUIDES @@ -31,10 +31,14 @@ - [Use Filesystem Journals](/tidb-cloud-filesystem/use-filesystem-journals.md) - [Manage Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md) - [Configure Filesystem AI Providers](/tidb-cloud-filesystem/configure-filesystem-ai-providers.md) + +## USE CASES + - [Automation and AI Agent Workflows](/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md) ## REFERENCES -- [Filesystem CLI Commands](/ai/ti/reference/ti-filesystem.md) -- [Regions and Limitations](/ai/ti/reference/ti-regions-security-and-limitations.md) -- [Troubleshooting](/ai/ti/reference/ti-troubleshooting.md) +- TiDB for AI References + - [Filesystem CLI Commands](/ai/ti/reference/ti-filesystem.md) + - [Regions and Limitations](/ai/ti/reference/ti-regions-security-and-limitations.md) + - [Troubleshooting](/ai/ti/reference/ti-troubleshooting.md) diff --git a/tidb-cloud-filesystem/_index.md b/tidb-cloud-filesystem/_index.md index 133713ffbd849..cccdf2b2652ea 100644 --- a/tidb-cloud-filesystem/_index.md +++ b/tidb-cloud-filesystem/_index.md @@ -7,7 +7,7 @@ summary: Learn how TiDB Cloud Filesystem keeps agent files available across sess TiDB Cloud Filesystem is persistent, shared file storage for applications and AI agents. Keep a workspace in the cloud, access it from different machines, and retain its files after a sandbox or process ends. -Use the TiDB Cloud Command Line Interface (`ti`) to create a Filesystem, upload and download files, or mount a remote directory for tools that work with local paths. You do not need to provision or manage a separate database to get started. +Use TiDB Cloud CLI (`ti`) to create a Filesystem, upload and download files, or mount a remote directory for tools that work with local paths. You do not need to provision or manage a separate database to get started. > **Note:** > @@ -17,7 +17,7 @@ Use the TiDB Cloud Command Line Interface (`ti`) to create a Filesystem, upload An agent might collect source documents in one sandbox, generate a report in another, and hand the result to a person on a laptop. Files stored only on the sandbox's local disk disappear when that environment is removed. Repeatedly exporting and importing archives also creates separate copies that can drift apart. -With a Filesystem, these participants use the same remote files. A new environment needs an access token and the Filesystem's region code, not the creator's TiDB Cloud API keys or local configuration directory. +With a Filesystem, these participants use the same remote files. To access an existing Filesystem from a new environment, provide an access token and the Filesystem's region code. You do not need to copy the creator's TiDB Cloud API keys or local configuration directory. - **Continue a task in another environment.** Mount the existing workspace instead of recreating its input files. - **Share results with a reviewer.** Give the reviewer a read-only scoped token for a report directory. @@ -34,7 +34,7 @@ The CLI provides two ways to use the same remote Filesystem: - [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md) explains when to use API keys, owner tokens, or scoped tokens. - [Share a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-sharing.md) shows how to give another machine access without sharing account credentials. -- [Layers and Checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md) explains how to isolate drafts and publish selected changes. +- [Layers and Checkpoints](/tidb-cloud-filesystem/filesystem-layers-checkpoints.md) explains how to isolate drafts and publish selected changes. ## Get started diff --git a/tidb-cloud-filesystem/configure-filesystem-ai-providers.md b/tidb-cloud-filesystem/configure-filesystem-ai-providers.md index 5fbd82ae26125..332cccdbef7c9 100644 --- a/tidb-cloud-filesystem/configure-filesystem-ai-providers.md +++ b/tidb-cloud-filesystem/configure-filesystem-ai-providers.md @@ -86,7 +86,7 @@ ti fs update-file-system-embedding-configuration \ --provider-model text-embedding-3-small ``` -Application-managed embeddings require an OpenAI-compatible endpoint that returns 1024-dimensional vectors. They are available for Shared Filesystems and Native Filesystems whose effective embedding mode is `fts_only`. If a Native Filesystem uses database-managed automatic embeddings, the service rejects this update and reports `source=database_auto`. +Application-managed embeddings require an OpenAI-compatible endpoint that returns exactly 1024-dimensional vectors. Choose a model that supports this width; models that return a different width are not supported. Before updating the configuration, run `describe-file-system-embedding-configuration` to check the effective source. If it reports `source=database_auto`, the service manages embeddings and rejects an application-managed update. After you finish configuring providers, remove the key from the current shell: diff --git a/tidb-cloud-filesystem/filesystem-authorization.md b/tidb-cloud-filesystem/filesystem-authorization.md index 32626d9534df7..5dd92cc872741 100644 --- a/tidb-cloud-filesystem/filesystem-authorization.md +++ b/tidb-cloud-filesystem/filesystem-authorization.md @@ -13,6 +13,12 @@ The person who creates a Filesystem and the agent that uses its files do not nee ## Choose the credential type +| Credential | Use it for | Scope | Who should hold it | +| --- | --- | --- | --- | +| TiDB Cloud API keys | Create and manage Filesystem resources and generate owner tokens | The account's permissions | A trusted administrator or automation environment | +| Owner FS token | Manage files and tokens within one Filesystem | One Filesystem | A trusted machine or secret manager | +| Scoped FS token | Delegate selected file operations | Specified paths and operations in one Filesystem | The user or application that needs that access | + ### TiDB Cloud API keys A TiDB Cloud public/private API key pair authorizes resource-management operations according to the account's permissions. Use it to create, list, describe, and delete Filesystems, generate owner tokens, and configure Filesystem AI providers. @@ -43,12 +49,20 @@ Choose a scoped token when delegating access to an agent or another machine. Kee One Filesystem can have multiple remote tokens, but one CLI profile stores at most one selected local token for that Filesystem. Local state is a credential selection, not the authoritative remote token inventory. +### Select a token for data access + - `--fs-token` takes precedence over `TI_FS_TOKEN` for token-based operations. - Without an explicit token, data-access commands use the locally stored credential for `--file-system-id` or `TI_FS_FILE_SYSTEM_ID`. - An explicit token contains the Filesystem ID. A clean environment therefore needs only `TI_FS_TOKEN` and `TI_REGION_CODE` for data access. - Generating a token does not select it locally unless you pass `--store-locally`. Replacing the local selection does not revoke the previous remote token. -For token list, enable, disable, and delete commands, an explicitly supplied owner token selects bearer authentication; otherwise the CLI uses TiDB Cloud API keys and requires a Filesystem ID. A scoped token does not gain administrative capability merely because account keys also exist in the profile. +### Keep management credentials separate + +For token list, enable, disable, and delete commands, an explicitly supplied owner token selects bearer authentication; otherwise the CLI uses TiDB Cloud API keys and requires a Filesystem ID. + +> **Note:** +> +> A scoped token does not gain administrative capability merely because account keys also exist in the profile. ## What's next diff --git a/tidb-cloud-filesystem/filesystem-branches-checkpoints.md b/tidb-cloud-filesystem/filesystem-layers-checkpoints.md similarity index 95% rename from tidb-cloud-filesystem/filesystem-branches-checkpoints.md rename to tidb-cloud-filesystem/filesystem-layers-checkpoints.md index 4c5ffb7035fdf..2e14a6f15d108 100644 --- a/tidb-cloud-filesystem/filesystem-branches-checkpoints.md +++ b/tidb-cloud-filesystem/filesystem-layers-checkpoints.md @@ -31,12 +31,16 @@ Before checkpointing a layer with a writable FUSE mount, [drain the mount](/tidb Layers are not full Filesystem snapshots, backups, or authorization boundaries. Use [scoped tokens](/tidb-cloud-filesystem/filesystem-authorization.md#scoped-fs-tokens) to limit access. A layer commit can fail if the base has conflicting changes; retain the layer and inspect the conflict rather than assuming an automatic merge. -Other limitations to consider: +## Limitations + +### Unsupported operations - The CLI does not provide Git-style merge, rebase, or an in-place reset to a checkpoint. - A layer commit does not merge changes into its parent layer. + +### Other boundaries + - Deleting a layer logically abandons it but does not guarantee immediate physical removal of history pinned by descendants. -- Recursive `ti fs copy-file --recursive` cannot be combined with `--layer-id`. Use a writable FUSE mount to copy a directory tree into a layer. - Repeated changes to layer-created files and inherited metadata can have preview limitations. Test complex edit histories before relying on them. For command flags and output fields, see the [`ti fs` command reference](/ai/ti/reference/ti-filesystem.md). diff --git a/tidb-cloud-filesystem/filesystem-mount.md b/tidb-cloud-filesystem/filesystem-mount.md index bb8aba3a6f08a..d9ea320816bab 100644 --- a/tidb-cloud-filesystem/filesystem-mount.md +++ b/tidb-cloud-filesystem/filesystem-mount.md @@ -22,6 +22,15 @@ Native Windows mounting is not supported by `ti`. Use direct commands such as `t With `--driver auto`, the CLI selects WebDAV on macOS and FUSE on Linux. To mount a layer or checkpoint on macOS, install macFUSE and select `--driver fuse`. WebDAV mounting is not supported on Linux. +| Capability | FUSE | WebDAV | +| --- | --- | --- | +| Supported platforms | Linux; macOS with macFUSE | macOS | +| Layer and checkpoint mounts | Supported | Not supported | +| Drain pending writes without unmounting | Supported | Not supported; close files and unmount normally | +| Read-only mount with `--read-only` | Supported | Supported | + +The two drivers do not provide identical filesystem behavior. Use the platform guides to choose a driver rather than assuming full POSIX feature parity. + ## Select a Filesystem ### Use a locally stored token @@ -59,7 +68,7 @@ mkdir -p "$HOME/workspace" ti fs mount-file-system --mount-path "$HOME/workspace" ``` -The command waits for readiness and returns a structured result with `status: mounted`. A background companion process keeps the mount alive. Closing the terminal does not unmount it, but terminating that process or the machine interrupts access. +The command waits for readiness and returns a structured result with `status: mounted`. The bundled `ti-drive9` companion keeps the mount alive in the background. Closing the terminal does not unmount it, but terminating that process or the machine interrupts access. If startup fails, inspect the diagnostic log path reported by the CLI. To expose only one remote directory, pass `--remote-path /workspace`. To make a mount read-only, add `--read-only`. These are client-side mount settings, not substitutes for a scoped token's server-enforced permissions. @@ -83,6 +92,8 @@ Stop applications writing to the mount and close their files. If you need pendin ti fs drain-file-system --mount-path "$HOME/workspace" --timeout 30s ``` +The timeout is how long the CLI waits for dirty handles and pending writes to drain. If the command times out or returns an error, it has not confirmed that all writes reached the service. Keep the mount and machine available, resolve the error, and verify remote data before ending the session. + When finished, unmount: ```bash diff --git a/tidb-cloud-filesystem/filesystem-quick-start.md b/tidb-cloud-filesystem/filesystem-quick-start.md index a79db4b252d26..ffe4625ad17d3 100644 --- a/tidb-cloud-filesystem/filesystem-quick-start.md +++ b/tidb-cloud-filesystem/filesystem-quick-start.md @@ -61,11 +61,15 @@ $env:Path = "$HOME\.ti\bin;$env:Path" ti --version ``` -Add `$HOME\.ti\bin` to your user `PATH` for future sessions. Windows supports the direct file commands in this quick start, but not native Filesystem mounts through `ti`. +Add `$HOME\.ti\bin` to your user `PATH` for future sessions. +> **Note:** +> +> On Windows, the direct file commands in this quick start are supported, but native Filesystem mounts through `ti` are not. + For other installation and upgrade details, see [Install, Configure, and Update TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md). ## Step 2. Configure access @@ -82,6 +86,8 @@ Choose one of these Filesystem regions: - `aws-us-west-2` - `alicloud-ap-southeast-1` +Choose a region where you want to store the Filesystem's data. For the provider and location of each region, see [Supported regions](/ai/ti/reference/ti-regions-security-and-limitations.md#supported-regions). + Configuration saves the inputs locally. Your first remote command verifies the credentials with the service. ## Step 3. Create the Filesystem @@ -117,6 +123,8 @@ The read returns `Hello from my workspace`. The file remains available after you - [Share the workspace](/tidb-cloud-filesystem/filesystem-sharing.md) with another machine or agent. - [Manage layers and checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) to review changes before publishing them. +## Clean up (optional) + When you no longer need this tutorial Filesystem, delete it using your TiDB Cloud API credentials: ```shell diff --git a/tidb-cloud-filesystem/filesystem-sharing.md b/tidb-cloud-filesystem/filesystem-sharing.md index e19cd1c5c58bb..39804c55113b2 100644 --- a/tidb-cloud-filesystem/filesystem-sharing.md +++ b/tidb-cloud-filesystem/filesystem-sharing.md @@ -56,6 +56,7 @@ Deliver `REVIEW_TOKEN` and the Filesystem's region code securely to machine B. R Inject the reviewer's token and matching region into the environment: ```bash +# In production, inject the token from a secret manager instead of pasting it into a shell. # No ti configure is needed on the receiving machine. export TI_FS_TOKEN="" export TI_REGION_CODE="" @@ -84,7 +85,7 @@ A successful write to a FUSE-mounted file might still be buffered on the produci Use a direct remote read to verify a handoff independently of another mount's cache. Existing open handles and client caches can retain older content; do not assume every reader instantly sees each local write. -Coordinate writers to the same path. Shared storage is not a distributed lock or an automatic merge system. Use separate paths or [layers](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md) for independent drafts, and publish only after review. +Coordinate writers to the same path. Shared storage is not a distributed lock or an automatic merge system. Use separate paths or [layers](/tidb-cloud-filesystem/filesystem-layers-checkpoints.md) for independent drafts, and publish only after review. ## End access without deleting the workspace @@ -109,5 +110,5 @@ Do not delete the Filesystem to disconnect one participant: resource deletion af ## What's next -- [Understand layers and checkpoints for independent drafts](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md). +- [Understand layers and checkpoints for independent drafts](/tidb-cloud-filesystem/filesystem-layers-checkpoints.md). - [Run the agent sandbox example](/ai/ti/guides/ti-agent-sandbox-example.md). diff --git a/tidb-cloud-filesystem/manage-filesystem-layers.md b/tidb-cloud-filesystem/manage-filesystem-layers.md index dc7b6d0abb25c..eb7f90f919109 100644 --- a/tidb-cloud-filesystem/manage-filesystem-layers.md +++ b/tidb-cloud-filesystem/manage-filesystem-layers.md @@ -8,7 +8,7 @@ aliases: ['/ai/manage-filesystem-layers'] Use layers to record isolated changes over a Filesystem base path before you commit or discard them. -This guide covers individual CLI operations. To understand layer visibility, forks, checkpoints, and their boundaries before using these commands, see [Layers and Checkpoints](/tidb-cloud-filesystem/filesystem-branches-checkpoints.md). +This guide covers individual CLI operations. To understand layer visibility, forks, checkpoints, and their boundaries before using these commands, see [Layers and Checkpoints](/tidb-cloud-filesystem/filesystem-layers-checkpoints.md). ## Prerequisites @@ -26,6 +26,14 @@ ti fs create-layer \ --tag task=review ``` +`restore-safe` is the only `--durability-mode` value accepted by the current CLI. See the [`create-layer` command reference](/ai/ti/reference/ti-fs-create-layer.md) for the current option contract. + +List the layers in the selected Filesystem: + +```shell +ti fs list-layers --output text +``` + Use the returned layer ID to write and inspect changes: ```shell @@ -90,7 +98,7 @@ Choose one outcome for a layer: > > Do not run both `rollback-layer` and `commit-layer` in sequence for the same layer. -## Move local state to another machine +## Move local state to another machine (advanced) When a FUSE mount uses write-back cache, some data can remain in its local overlay directory. To move this local state to another machine, pack it to an explicit remote archive path: diff --git a/tidb-cloud-filesystem/manage-filesystem-resources.md b/tidb-cloud-filesystem/manage-filesystem-resources.md index afa036e875e65..83dcf3ab60608 100644 --- a/tidb-cloud-filesystem/manage-filesystem-resources.md +++ b/tidb-cloud-filesystem/manage-filesystem-resources.md @@ -15,12 +15,14 @@ This document describes how to use [`ti fs` commands](/ai/ti/reference/ti-filesy ## Prerequisites - [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli) and [configure access](/tidb-cloud-filesystem/filesystem-quick-start.md#step-2-configure-access) with TiDB Cloud API credentials. -- Install `jq`, or use another JSON processor to capture command output safely. +- Install `jq`, or use another JSON processor to extract fields from the CLI's JSON output. ## Create a Filesystem Create a Filesystem and save the returned ID and one-time owner token in a file that is not world-readable. The `--wait` flag tells the CLI to poll until data-plane access is ready before returning: +`umask 077` restricts the output file to the current user because it contains the owner token. + ```shell umask 077 ti fs create-file-system \ @@ -56,6 +58,8 @@ ti fs describe-file-system --file-system-id "" If you have access to more than one Filesystem, pass `--file-system-id` explicitly or set the `TI_FS_FILE_SYSTEM_ID` environment variable. The CLI does not automatically select a Filesystem for you. +The current CLI does not provide a command to change a Filesystem's display name or labels after creation. Choose these values when creating the resource. + ## Check access Verify resource selection, endpoint resolution, credentials, and companion access: @@ -64,6 +68,8 @@ Verify resource selection, endpoint resolution, credentials, and companion acces ti fs check-file-system --file-system-id "" ``` +The result includes an overall `status` and checks for local credentials, endpoint selection, the companion binary, and remote reachability. A `passed` status means all checks passed; `warning` or `failed` identifies a check that needs attention. + ## Delete a Filesystem > **Warning:** diff --git a/tidb-cloud-filesystem/manage-filesystem-tokens.md b/tidb-cloud-filesystem/manage-filesystem-tokens.md index d676521fb2678..c60d6e06eacad 100644 --- a/tidb-cloud-filesystem/manage-filesystem-tokens.md +++ b/tidb-cloud-filesystem/manage-filesystem-tokens.md @@ -61,6 +61,8 @@ export TI_REGION_CODE="" ti fs list-files --path /workspace ``` +The `--allow` value uses `:`. In this example, the token permits `read`, `list`, and `write` under `/workspace`. + The remote `/workspace` directory must already exist. For a mount, select the allowed subtree with `--remote-path /workspace`; a token restricted to `/workspace` cannot mount the root `/`. For the permission model and credential precedence, see [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md). ## Inspect and change token status @@ -81,7 +83,7 @@ Use [`refresh-file-system-token`](/ai/ti/reference/ti-fs-refresh-file-system-tok > **Note:** > -> Refresh is non-idempotent. If a request might have succeeded but its response was lost, do not retry with the old token. Instead, generate a new owner token using TiDB Cloud credentials. +> Refresh is non-idempotent. For example, after a network timeout, the service might have rotated the token even though you did not receive the new value. Do not retry with the old token. Instead, generate a new owner token using TiDB Cloud credentials. > **Warning:** > @@ -97,6 +99,6 @@ If the deleted token matches the locally stored token, the CLI automatically rem ## What's next -- [Share a TiDB Cloud Filesystem Across Machines](/ai/ti/guides/ti-share-filesystem-across-machines-example.md) -- [Use TiDB Cloud Filesystem in an Agent Sandbox](/ai/ti/guides/ti-agent-sandbox-example.md) -- [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) +- [Share a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-sharing.md). +- [Explore automation and AI agent workflows](/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md). +- [TiDB Cloud Filesystem CLI Command Reference (in TiDB for AI documentation)](/ai/ti/reference/ti-filesystem.md). diff --git a/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md b/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md index a433182c01ed4..9b724add7a1bf 100644 --- a/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md +++ b/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md @@ -28,10 +28,22 @@ ti fs-vault create-secret \ ti fs-vault read-secret --secret-name db-prod ``` +In `--field PASSWORD=@./password.txt`, the `@` prefix reads the field value from the local file instead of using the literal text as the value. + > **Note:** > > All `read-secret` output formats, including the default JSON format, contain plaintext secret values. Direct the output only to the intended process. +## Replace a secret + +To rotate a stored value, put each replacement field in a separate file in a local directory. For example, to change `DB_URL`, write its new value to `./secret-fields/DB_URL` and include a `./secret-fields/PASSWORD` file if you want to retain that field. Then replace the secret: + +```shell +ti fs-vault replace-secret --secret-path /n/vault/db-prod --from-directory ./secret-fields +``` + +`replace-secret` replaces **all** fields, not only the changed field. Keep the local files out of source control and remove them after use. See the [`replace-secret` reference](/ai/ti/reference/ti-fs-vault-replace-secret.md) for the full command contract. + ## Delegate limited access Create a short-lived read grant and capture its token: @@ -51,6 +63,8 @@ Prefer `TI_VAULT_TOKEN` to a command-line token because command-line values can The CLI can inject secret fields as environment variables into a child process without writing plaintext to disk. When you run the following command, the CLI reads the secret, sets each field as an environment variable (for example, `DB_URL`, `PASSWORD`), removes its own credential environment variables from the child, and then executes the specified command: +`/n/vault/db-prod` is the canonical Vault path for the secret named `db-prod`; it is not a local mount path. + ```shell ti fs-vault run-with-secret --secret-path /n/vault/db-prod -- ``` diff --git a/tidb-cloud-filesystem/manage-git-workspaces.md b/tidb-cloud-filesystem/manage-git-workspaces.md index af422032d099a..707608cd43ab8 100644 --- a/tidb-cloud-filesystem/manage-git-workspaces.md +++ b/tidb-cloud-filesystem/manage-git-workspaces.md @@ -6,7 +6,7 @@ aliases: ['/ai/manage-git-workspaces'] # Manage Git Workspaces on TiDB Cloud Filesystem -Use `ti fs-git` to accelerate Git workspace setup on a mounted TiDB Cloud Filesystem while continuing to use ordinary Git commands for daily work. +Use `ti fs-git` when you want a Git workspace on a mounted TiDB Cloud Filesystem without waiting for all file content to download before work begins. Unlike a regular `git clone` into the mount, the CLI can combine a blobless clone with background hydration. Continue to use ordinary Git commands for daily work after setup. ## Prerequisites @@ -23,7 +23,7 @@ ti fs-git clone-git-workspace \ --target-path /path/to/workspace/tidb ``` -For a large repository, add `--blobless --hydrate background` to make the directory tree available immediately. The CLI starts a background process that downloads clean file content and Git objects after the clone command returns. Use `--hydrate sync` when your workflow requires hydration to finish before the command returns. +For a large repository, add `--blobless --hydrate background` to make the directory tree available immediately. A blobless clone initially fetches Git history and tree metadata without downloading all file contents. The CLI starts a background process that downloads clean file content and Git objects after the clone command returns. Use `--hydrate sync` when your workflow requires hydration to finish before the command returns. ## Hydrate an existing workspace @@ -37,6 +37,8 @@ ti fs-git hydrate-git-workspace \ Hydration fetches missing blob data from the remote repository without discarding your working-tree changes. +If cloning or hydration fails, inspect the CLI error and diagnostic log before retrying. See [Troubleshoot TiDB Cloud CLI](/ai/ti/reference/ti-troubleshooting.md) for common Filesystem and companion issues. + ## Add and use a linked worktree ```shell diff --git a/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md b/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md index e02519b24f354..e68cd2130c0b7 100644 --- a/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md +++ b/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md @@ -16,7 +16,7 @@ To share work across machines and CI jobs, see the following guides in the **TiD | Access the same workspace from multiple machines | [Share a TiDB Cloud Filesystem Across Machines](/ai/ti/guides/ti-share-filesystem-across-machines-example.md) | | Pass build artifacts between isolated CI jobs | [Hand Off CI Artifacts Between Isolated Jobs with TiDB Cloud Filesystem](/ai/ti/guides/ti-ci-artifact-handoff-example.md) | -To support AI agent workflows, see the following guides in the **TiDB for AI** documentation. +To support AI agent workflows, see the following guides in the **TiDB for AI** documentation. If you are evaluating Filesystem for agents, start with the sandbox example before moving to longer-running or parallel workflows. | What you want to do | Guide | | --- | --- | diff --git a/tidb-cloud-filesystem/use-filesystem-journals.md b/tidb-cloud-filesystem/use-filesystem-journals.md index 95bb43f730d95..a10f60dfd057f 100644 --- a/tidb-cloud-filesystem/use-filesystem-journals.md +++ b/tidb-cloud-filesystem/use-filesystem-journals.md @@ -8,6 +8,8 @@ aliases: ['/ai/use-filesystem-journals'] Journals provide append-only, hash-chained event logs for agent workflows and automation pipelines running on a TiDB Cloud Filesystem. Use [`ti fs-journal` commands](/ai/ti/reference/ti-filesystem-journal.md) to create a journal, append ordered events, search or read them, and verify the hash chain. +Use a journal when you need to trace the order of agent actions or handoffs across sessions. The hash chain lets you check the integrity and order of recorded entries. A journal records events; it does not replay actions or replace the files produced by a workflow. + ## Prerequisites - [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). @@ -32,7 +34,7 @@ ti fs-journal append-journal-entries \ --entry-json '{"type":"review_started"}' ``` -For supported input forms and entry fields, see the [`append-journal-entries` reference](/ai/ti/reference/ti-fs-journal-append-journal-entries.md). +`--entry-json` accepts a JSON object. Each entry needs a `type`, unless you supply `--entry-type`; optional fields include `summary`, `actor`, and `occurred_at`. For the complete fields and input forms, see the [`append-journal-entries` reference](/ai/ti/reference/ti-fs-journal-append-journal-entries.md). ## Read and search entries @@ -58,6 +60,8 @@ Verify that the journal's hash chain is intact: ti fs-journal verify-journal --journal-id "" ``` +The current public CLI has no journal delete command. Keep retention needs in mind before recording sensitive or high-volume events. + ## What's next - [Record an Agent Workflow in a TiDB Cloud Filesystem Journal](/ai/ti/guides/ti-journal-agent-workflow-example.md) diff --git a/tidb-cloud-filesystem/work-with-filesystem-data.md b/tidb-cloud-filesystem/work-with-filesystem-data.md index 405b3c47eea29..6b72ebce4696a 100644 --- a/tidb-cloud-filesystem/work-with-filesystem-data.md +++ b/tidb-cloud-filesystem/work-with-filesystem-data.md @@ -49,7 +49,7 @@ ti fs move-file --from-remote /draft.md --to-remote /reports/final.md ti fs delete-file --path /scratch --recursive ``` -You can also use `chmod-file`, `create-symlink`, and `create-hardlink` to manage POSIX-style metadata and links. +You can also use [`chmod-file`](/ai/ti/reference/ti-fs-chmod-file.md), [`create-symlink`](/ai/ti/reference/ti-fs-create-symlink.md), and [`create-hardlink`](/ai/ti/reference/ti-fs-create-hardlink.md) to manage POSIX-style metadata and links. > **Warning:** > @@ -63,12 +63,18 @@ Search file content below a path: ti fs search-file-content --path /reports --pattern "TODO" ``` +`--pattern` is a text query, not a regular expression or glob. Use `--limit` to cap the number of results; `0` uses the service default. See the [`search-file-content` reference](/ai/ti/reference/ti-fs-search-file-content.md). + Find paths by name, type, tags, size, or timestamps: ```shell ti fs find-files --path /reports --file-name-pattern "*.md" --tag stage=review ``` +`find-files` searches beneath the specified path and can filter by file name, type, tags, size, or modification time. See its [options and result limit](/ai/ti/reference/ti-fs-find-files.md). + +For supported regions and current CLI and platform limitations, see [Regions and Limitations](/ai/ti/reference/ti-regions-security-and-limitations.md). + ## What's next - [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) From ccdb67588b409fde3ca3cd9674d004d5ae586904 Mon Sep 17 00:00:00 2001 From: qiancai Date: Thu, 17 Sep 2026 16:54:34 +0800 Subject: [PATCH 05/14] add dedicated docs for regions, limitations, and troubleshooting for filesystem --- TOC-tidb-cloud-filesystem.md | 7 +- .../ti-regions-security-and-limitations.md | 44 +---- ai/ti/reference/ti-troubleshooting.md | 169 +--------------- tidb-cloud-filesystem/_index.md | 2 +- .../filesystem-mount-linux.md | 2 +- .../filesystem-quick-start.md | 2 +- .../filesystem-regions-and-limitations.md | 56 ++++++ .../filesystem-troubleshooting.md | 186 ++++++++++++++++++ .../manage-git-workspaces.md | 2 +- .../work-with-filesystem-data.md | 2 +- 10 files changed, 259 insertions(+), 213 deletions(-) create mode 100644 tidb-cloud-filesystem/filesystem-regions-and-limitations.md create mode 100644 tidb-cloud-filesystem/filesystem-troubleshooting.md diff --git a/TOC-tidb-cloud-filesystem.md b/TOC-tidb-cloud-filesystem.md index 81171caebf7e1..8e4b16390764e 100644 --- a/TOC-tidb-cloud-filesystem.md +++ b/TOC-tidb-cloud-filesystem.md @@ -38,7 +38,6 @@ ## REFERENCES -- TiDB for AI References - - [Filesystem CLI Commands](/ai/ti/reference/ti-filesystem.md) - - [Regions and Limitations](/ai/ti/reference/ti-regions-security-and-limitations.md) - - [Troubleshooting](/ai/ti/reference/ti-troubleshooting.md) +- [Filesystem CLI Commands](/ai/ti/reference/ti-filesystem.md) +- [Regions and Limitations](/tidb-cloud-filesystem/filesystem-regions-and-limitations.md) +- [Troubleshooting](/tidb-cloud-filesystem/filesystem-troubleshooting.md) diff --git a/ai/ti/reference/ti-regions-security-and-limitations.md b/ai/ti/reference/ti-regions-security-and-limitations.md index 0be90892f8d60..94edb15819e11 100644 --- a/ai/ti/reference/ti-regions-security-and-limitations.md +++ b/ai/ti/reference/ti-regions-security-and-limitations.md @@ -1,11 +1,11 @@ --- title: TiDB Cloud CLI Regions, Security, and Limitations -summary: Reference supported regions, authentication boundaries, platform dependencies, preview constraints, and Filesystem companion behavior. +summary: Reference supported regions, authentication boundaries, security best practices, and current TiDB Cloud CLI limitations. --- # TiDB Cloud CLI Regions, Security, and Limitations -This reference describes current placement, authentication, platform, and preview boundaries. +This reference describes current regions, authentication, platform, and preview boundaries for TiDB Cloud CLI. For Filesystem regions and limitations, see [TiDB Cloud Filesystem Regions and Limitations](/tidb-cloud-filesystem/filesystem-regions-and-limitations.md). > **Note:** > @@ -51,56 +51,22 @@ TiDB Cloud API calls use Digest authentication. SQL HTTPS execution uses generat - Create TiDB Cloud API keys with only the access required for the workflow. Do not reuse a personal administrator key in unattended automation. - Inject automation credentials from a CI secret store or runtime secret manager. Do not place credentials in source control, container images, shell scripts, or command-line arguments that can appear in process listings and shell history. - Do not copy the complete `~/.ti/` directory into an agent sandbox. For an existing Filesystem, pass only `TI_FS_TOKEN` and `TI_REGION_CODE`; use `TI_FS_FILE_SYSTEM_ID` only as an optional assertion. -- Treat an FS owner token as full access to that Filesystem. When an agent needs only selected secrets, create a vault grant with the narrowest field scope and shortest practical TTL, and pass the delegated vault token instead. -- Use a separate Filesystem token for each machine, CI workflow, or sandbox class so that one environment can be disabled or revoked without interrupting others. Token names are operational labels, not unique identifiers; mutate tokens only by `token_id`. -- Capture generated and refreshed token plaintext immediately because it is returned only once. A token refreshed from `TI_FS_TOKEN` is not written back to an external secret manager. Refresh is non-idempotent, so do not retry after an ambiguous network failure. -- For shared-token rotation, generate and distribute a replacement, validate access, then disable and delete the old token. Allow approximately 10 seconds for authentication caches to converge after a state change. -- Pass an AI provider key only through `TI_FS_AI_PROVIDER_API_KEY`. The TiDB Cloud CLI does not persist this value locally, and the Filesystem service returns it only in masked form. Do not retry an AI configuration update after an ambiguous failure until you describe the effective configuration. -- Enabling extraction shares Filesystem media with the configured extraction provider. Enabling app-managed embedding shares text or extracted descriptions with the configured embedding provider. Review that provider's data retention and security terms before enabling either feature. - Use `--read-only` for SQL inspection by untrusted or exploratory agents. Use `--admin` only for DDL or privilege management, and use `--read-write` only when data changes are intended. - Use `--dry-run` before destructive control-plane operations. Keep `~/.ti/credentials`, resource credentials, and DB SQL credentials owner-readable only. -- Grant Docker access to `/dev/fuse`, `SYS_ADMIN`, and an unconfined AppArmor profile only to dedicated, trusted containers. These settings reduce container isolation. - Review local operation logs before sharing diagnostics. The logs exclude SQL text, paths, payloads, and credential values, but command names, flag names, profile and region metadata, status codes, and operational timing can still be sensitive. -## Mount platform limitations - -| Platform | Filesystem mount | Vault mount | Requirements and alternatives | -| --- | --- | --- | --- | -| macOS | WebDAV by default; FUSE with explicit `--driver fuse` | FUSE | The built-in WebDAV helper supports Filesystem mounts. Install macFUSE and approve its system extension for FUSE or Vault mounts. | -| Linux | FUSE | FUSE | Install FUSE3 and provide access to `/dev/fuse`. WebDAV mounting is not supported. | -| Windows | Not supported | Not supported | Use `ti fs` data-plane commands and non-mount Vault commands instead. | - -FUSE and WebDAV are implemented by the bundled [Drive9](https://github.com/mem9-ai/drive9) companion. The TiDB Cloud CLI does not fall back to a separate native mount implementation. - -Ubuntu 26.04 additionally confines `fusermount3` with AppArmor. Use a mount path under `$HOME` or `/mnt`; `/workspace` requires an explicit local AppArmor rule even when `ti` runs as root. - -## Durability limitations - -- Default FUSE behavior uses local buffering and asynchronous remote work where permitted by the companion. -- A successful `unmount-file-system` gracefully flushes and drains FUSE work; a separate drain is not required first. -- `drain-file-system` is a FUSE-only online durability barrier that leaves the mount active. -- Abruptly killing the mount process or deleting a machine can lose uncommitted memory/write-back state. -- The default coding-agent mount profile stores dependency trees, generated output, caches, and Git internals locally. Local-only data disappears when its disk disappears unless it is packed or otherwise preserved. -- A running mount remains on the companion version loaded at mount time. Unmount and remount after updating the TiDB Cloud CLI. -- Remote-committed Filesystem data survives client or sandbox deletion; deleting the machine does not delete the remote resource. +For Filesystem token, mount, Vault, and AI provider security, see [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md), [Manage Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md), and [Configure Filesystem AI Providers](/tidb-cloud-filesystem/configure-filesystem-ai-providers.md). ## Product limitations - The TiDB Cloud CLI is in preview, and command contracts can change. -- Database management targets TiDB Cloud Starter, not every TiDB Cloud cluster tier. +- Database management targets TiDB Cloud Starter instances, not other TiDB Cloud database plans. - SQL execution accepts one statement per invocation. - Read-write is the default SQL role; use explicit role flags in security-sensitive automation. -- Journals are append-only and the current public command surface has no journal delete command. -- Filesystem list and describe commands query the region-scoped remote inventory with TiDB Cloud credentials. They do not aggregate across regions. -- The local credential store keeps one selected token per profile and Filesystem. It does not mirror all remote tokens. Older create/import credentials without a known token ID remain usable, but cannot be correlated with remote token metadata. -- Filesystem extraction and embedding provider configuration is optional. Leaving it unconfigured does not block resource administration, file access, search, layers, Git, journal, vault, or mount workflows. -- OpenAI provider interfaces are supported for embedding and image, audio, and video extraction. Alibaba Cloud Model Studio Qwen ASR is supported only for audio extraction. Other vendors are conditionally compatible only through the exact OpenAI-compatible contract; native Anthropic, Gemini, Vertex AI, Bedrock, and Azure OpenAI interfaces are not supported. -- App-managed embedding requires a provider model that returns exactly 1024 dimensions. Filesystems that report `source=database_auto` use database-managed embedding and reject app-managed configuration. - Telemetry management commands are intentionally not implemented. Control telemetry through `~/.ti/.preferences` or `TI_TELEMETRY`; serverless-function deployment, Homebrew, and Scoop distribution are not implemented. -- The TiDB Cloud CLI depends on its installed `ti-drive9` companion for all public Filesystem runtime behavior, including direct file operations, layers, mounts, Git workspaces, journals, and Vault operations. ## Related documentation -- [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) +- [TiDB Cloud Filesystem Regions and Limitations](/tidb-cloud-filesystem/filesystem-regions-and-limitations.md) - [TiDB Cloud CLI Configuration and Credentials](/ai/ti/reference/ti-configuration-and-credentials.md) - [Troubleshoot TiDB Cloud CLI](/ai/ti/reference/ti-troubleshooting.md) diff --git a/ai/ti/reference/ti-troubleshooting.md b/ai/ti/reference/ti-troubleshooting.md index f8b9256180d41..9ffe01f136643 100644 --- a/ai/ti/reference/ti-troubleshooting.md +++ b/ai/ti/reference/ti-troubleshooting.md @@ -1,11 +1,11 @@ --- title: Troubleshoot TiDB Cloud CLI -summary: Diagnose TiDB Cloud CLI authentication, project, Filesystem selection, companion, quota, SQL user, mount, and interrupted-cleanup failures. +summary: Learn how to diagnose TiDB Cloud CLI API authentication, Starter quota, SQL credential, and interrupted-command failures safely. --- # Troubleshoot TiDB Cloud CLI -Use this reference to diagnose common current TiDB Cloud CLI failures. Add `--debug` only when needed; debug output is redacted but should still be reviewed before sharing. +Use this reference to diagnose CLI authentication, Starter, SQL, and interrupted-command failures. For Filesystem tokens, regions, companion processes, and mounts, see [Troubleshoot TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-troubleshooting.md). Add `--debug` only when needed; review redacted output before sharing it. > **Note:** > @@ -31,107 +31,12 @@ ti db list-db-clusters --db-cluster-type starter --profile default An API key can authenticate successfully but still lack the permission declared by a command. Use a key with the access required by that operation. `ti configure` validates and stores local values without contacting TiDB Cloud, so credential failures first appear on a remote command. -## Filesystem token is missing +## Starter creation reaches quota -For a clean sandbox, provide the token and region. `ti` derives the file system ID from the token: - -```bash -export TI_FS_TOKEN="" -export TI_REGION_CODE="" -ti fs check-file-system -``` - -The FS token is not the TiDB Cloud API private key. `TI_FS_FILE_SYSTEM_ID` is optional when a token is supplied; set it only when you want `ti` to verify that a separately distributed ID matches the token. - -If the token is known but is not stored on the current machine, import it and then select the derived ID: - -```bash -# Store a known token without requiring TiDB Cloud API keys. -chmod 600 ./fs-token -ti fs import-file-system-token --from-file ./fs-token --region -ti fs list-files --file-system-id --path / -``` - -If every known token is lost or revoked, use TiDB Cloud API keys to generate another owner token: - -```bash -ti fs generate-file-system-token \ - --file-system-id "" \ - --token-name recovery \ - --ttl 24h -``` - -The new plaintext appears once in the response. Store it securely or add `--store-locally` to select it on the current machine. - -## Filesystem token is rejected - -A data-plane HTTP 401 cannot distinguish a token that was disabled, expired, refreshed on another machine, or revoked. Inspect remote metadata with TiDB Cloud API keys: - -```bash -ti fs list-file-system-tokens \ - --file-system-id "" \ - --include-expired \ - --output text -``` - -Token names are not unique. Use the immutable `token_id` from this output for enable, disable, or delete operations. Old credentials created or imported without token lifecycle metadata can remain valid, but `ti` cannot safely identify their list row and never guesses a match. - -After enable, disable, delete, or refresh, allow approximately 10 seconds for authentication caches to converge. If refresh reports `fs.token_refresh_ambiguous`, the server might have rotated the token even though the response was lost. The outcome is unknown: the old token might still work if the refresh did not commit, or it might already be invalid. The replacement token from a committed refresh cannot be recovered because its response was lost. Do not retry the refresh with the old token. Instead, use TiDB Cloud credentials to generate an independent owner token. - -If token mutation reports `fs.token_mount_active`, use the exact mount path in the error: - -```bash -ti fs drain-file-system --mount-path /path/to/workspace -ti fs unmount-file-system --mount-path /path/to/workspace -``` - -Then retry the token operation. A mount on another machine is not visible locally; coordinate rotation with that machine separately. - -## Filesystem selection is missing - -List remote resources in the configured region with TiDB Cloud API keys and select one explicitly: - -```bash -ti fs list-file-systems --output text -ti fs list-files --file-system-id --path / -``` - -Or select the Filesystem for subsequent commands in the current shell: - -```bash -export TI_FS_FILE_SYSTEM_ID="" -``` - -The TiDB Cloud CLI intentionally does not infer a Filesystem from local credential count, including when only one credential exists. Supply its ID or an FS token whose embedded ID can be derived. - -## Filesystem region is unsupported - -The configured TiDB Cloud region might not be one of the Filesystem endpoints built into the installed TiDB Cloud CLI release. Compare it with [current Filesystem regions](/ai/ti/reference/ti-regions-security-and-limitations.md#supported-regions). Change placement with a valid profile or command-scoped `--region`; do not configure a raw server URL. - -## Companion is missing or incompatible - -The release installer places `ti-drive9`, the companion runtime for Filesystem commands, next to `ti`. You do not invoke `ti-drive9` directly. Re-run the current installer when the TiDB Cloud CLI reports a missing companion: - -```bash -curl -fsSL https://github.com/tidbcloud/ti-cli/releases/latest/download/install.sh | sh -s -- --yes -``` - -Verify that `PATH` resolves the expected `ti`: - -```bash -command -v ti -ti --version -``` - -Do not copy an arbitrary standalone Drive9 binary into place. - -## Starter or Filesystem creation reaches quota - -Quota and capacity errors can mean the organization has reached its free Starter limit. List existing resources before creating another: +Quota and capacity errors can mean the organization has reached its free Starter limit. List existing Starter resources before creating another: ```bash ti db list-db-clusters --db-cluster-type starter --output text -ti fs list-file-systems --output text ``` Never delete an unrelated resource to make automation pass. A Starter spending limit can require configured billing. @@ -155,72 +60,6 @@ ti db execute-sql-statement \ Deleting `~/.ti/db_users//credentials` removes local passwords. Run the create/repair command rather than inventing credentials. -## Mount does not become ready - -Background mount success prints the TiDB Cloud CLI result without Drive9 startup messages. If startup fails or times out, inspect the companion log path in the error. Confirm: - -- the mount path exists and is writable; -- no existing mount covers the path; -- the FS token and region are valid; -- FUSE prerequisites or the WebDAV helper are installed; -- the remote region is reachable. - -macOS defaults to WebDAV. To request FUSE after installing macFUSE: - -```bash -ti fs mount-file-system \ - --mount-path /path/to/workspace \ - --driver fuse -``` - -Linux needs FUSE3 and access to `/dev/fuse`. Filesystem and Vault mounts are not supported on Windows; use `ti fs` data-plane commands or non-mount Vault commands instead. - -## Ubuntu 26.04 rejects a FUSE mount under `/workspace` - -Ubuntu 26.04 applies an AppArmor profile to `fusermount3`. Its default mount-path allowlist does not include `/workspace`, so root and non-root users can both receive: - -```text -/usr/bin/fusermount3: mount failed: Permission denied -``` - -Confirm the denial: - -```bash -sudo journalctl -k --since "10 minutes ago" | - grep 'profile="fusermount3"' -``` - -An entry with `operation="mount"`, `name="/workspace/"`, and `info="failed mntpnt match"` identifies this restriction. Mount under `$HOME` or `/mnt` instead: - -```bash -mkdir -p "$HOME/workspace" -ti fs mount-file-system --mount-path "$HOME/workspace" -``` - -Changing the owner or mode of `/workspace` does not bypass AppArmor. If the path cannot change, add explicit `/workspace` mount and unmount rules to `/etc/apparmor.d/local/fusermount3` as described in [Ubuntu 26.04 mount-path restrictions](/tidb-cloud-filesystem/filesystem-mount-linux.md#ubuntu-2604-mount-path-restrictions). - -## Mount becomes stale after a process crash - -If the companion is killed without graceful unmount, FUSE access can return `EIO` or `Transport endpoint is not connected`. Stop processes with open files, then try: - -```bash -ti fs unmount-file-system \ - --mount-path /path/to/workspace \ - --force -``` - -Use `--ignore-absent` when cleanup should succeed if no locator remains. Abrupt cleanup cannot guarantee recovery of pending writes from a deleted local disk. - -## Unmount reports busy - -Close editors, shells whose working directory is inside the mount, and other open file handles, and then retry: - -```bash -ti fs unmount-file-system --mount-path /path/to/workspace -``` - -Unmount performs the graceful FUSE drain automatically. Running `drain-file-system` separately does not close file descriptors or resolve a busy mount; use it only when you need to flush pending work while leaving the mount online. Drain is not supported for WebDAV. - ## An interrupted command leaves resources List resources and identify only those created by your workflow. Use describe before delete: diff --git a/tidb-cloud-filesystem/_index.md b/tidb-cloud-filesystem/_index.md index cccdf2b2652ea..5c3503903e476 100644 --- a/tidb-cloud-filesystem/_index.md +++ b/tidb-cloud-filesystem/_index.md @@ -42,7 +42,7 @@ If you need to create a Filesystem, follow [Get Started with TiDB Cloud Filesyst If someone has given you access to an existing Filesystem, [use its FS token and region without configuring a profile](/tidb-cloud-filesystem/filesystem-mount.md#use-a-token-without-configuring-a-profile). You do not need the creator's API keys to access files within your token's permissions. -For supported regions, platform requirements, and current constraints, see [Regions and Limitations](/ai/ti/reference/ti-regions-security-and-limitations.md). +For supported regions, platform requirements, and current constraints, see [Regions and Limitations](/tidb-cloud-filesystem/filesystem-regions-and-limitations.md). ## What's next diff --git a/tidb-cloud-filesystem/filesystem-mount-linux.md b/tidb-cloud-filesystem/filesystem-mount-linux.md index 9d83b2ab1a3dd..11a278c1c9a74 100644 --- a/tidb-cloud-filesystem/filesystem-mount-linux.md +++ b/tidb-cloud-filesystem/filesystem-mount-linux.md @@ -93,7 +93,7 @@ The administrator can then reload the profile: sudo apparmor_parser -r /etc/apparmor.d/fusermount3 ``` -For related errors, see [Troubleshoot TiDB Cloud CLI](/ai/ti/reference/ti-troubleshooting.md). +For related errors, see [Troubleshoot TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-troubleshooting.md). ## What's next diff --git a/tidb-cloud-filesystem/filesystem-quick-start.md b/tidb-cloud-filesystem/filesystem-quick-start.md index ffe4625ad17d3..8a3b9c2082c0d 100644 --- a/tidb-cloud-filesystem/filesystem-quick-start.md +++ b/tidb-cloud-filesystem/filesystem-quick-start.md @@ -86,7 +86,7 @@ Choose one of these Filesystem regions: - `aws-us-west-2` - `alicloud-ap-southeast-1` -Choose a region where you want to store the Filesystem's data. For the provider and location of each region, see [Supported regions](/ai/ti/reference/ti-regions-security-and-limitations.md#supported-regions). +Choose a region where you want to store the Filesystem's data. For the provider and location of each region, see [Supported regions](/tidb-cloud-filesystem/filesystem-regions-and-limitations.md#supported-regions). Configuration saves the inputs locally. Your first remote command verifies the credentials with the service. diff --git a/tidb-cloud-filesystem/filesystem-regions-and-limitations.md b/tidb-cloud-filesystem/filesystem-regions-and-limitations.md new file mode 100644 index 0000000000000..a32579552db9d --- /dev/null +++ b/tidb-cloud-filesystem/filesystem-regions-and-limitations.md @@ -0,0 +1,56 @@ +--- +title: TiDB Cloud Filesystem Regions and Limitations +summary: Review supported TiDB Cloud Filesystem regions, mount platform requirements, durability boundaries, and current product limitations. +--- + +# TiDB Cloud Filesystem Regions and Limitations + +TiDB Cloud Filesystem is available only in the regions and environments listed below. Check these boundaries before creating a Filesystem or choosing how to access its data. + +> **Note:** +> +> TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. + +## Supported regions + +| Provider | Location | Canonical region code | +| --- | --- | --- | +| AWS | N. Virginia | `aws-us-east-1` | +| AWS | Oregon | `aws-us-west-2` | +| AWS | Singapore | `aws-ap-southeast-1` | +| Alibaba Cloud | Singapore | `alicloud-ap-southeast-1` | + +TiDB Cloud Starter also supports some regions where Filesystem is not available. Filesystem commands in those regions fail with an `unsupported endpoint` error. Supported Filesystem regions are built into each `ti` release. If a region was added after your installed version was released, upgrade `ti`; specifying a service URL cannot enable it. + +Filesystem list and describe commands query only the selected region. They do not aggregate resources across regions. For CLI-wide region selection and Starter availability, see [TiDB Cloud CLI Regions, Security, and Limitations](/ai/ti/reference/ti-regions-security-and-limitations.md#supported-regions). + +## Mount platform support + +| Platform | Filesystem mount | Vault mount | Requirements and alternatives | +| --- | --- | --- | --- | +| macOS | WebDAV by default; FUSE with explicit `--driver fuse` | FUSE | The built-in WebDAV helper supports Filesystem mounts. Install macFUSE and approve its system extension for FUSE or Vault mounts. | +| Linux | FUSE | FUSE | Install FUSE3 and provide access to `/dev/fuse`. WebDAV mounting is not supported. | +| Windows | Not supported | Not supported | Use `ti fs` data-plane commands and non-mount Vault commands instead. | + +FUSE and WebDAV are implemented by the bundled [Drive9](https://github.com/mem9-ai/drive9) companion. The CLI does not fall back to a separate native mount implementation. A running mount keeps the companion version loaded when it started; unmount and remount after updating `ti`. + +Ubuntu 26.04 also confines `fusermount3` with AppArmor. Use a mount path under `$HOME` or `/mnt`; `/workspace` requires an explicit local AppArmor rule even when `ti` runs as root. For setup and workarounds, see [Mount TiDB Cloud Filesystem on Linux](/tidb-cloud-filesystem/filesystem-mount-linux.md#ubuntu-2604-mount-path-restrictions). + +## Durability boundaries + +- Default FUSE behavior uses local buffering and asynchronous remote work where the companion permits it. Abruptly killing the mount process or deleting a machine can lose uncommitted memory or write-back state. +- The default coding-agent mount profile stores dependency trees, generated output, caches, and Git internals locally. Local-only data disappears when its disk disappears unless it is packed or otherwise preserved. +- Remote-committed Filesystem data survives client or sandbox deletion; deleting the machine does not delete the remote Filesystem. + +For the supported drain and graceful unmount workflow, see [Finish safely](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). `drain-file-system` is a FUSE-only online durability barrier; WebDAV does not support it. + +## Current limitations + +- Journals are append-only, and the current public command surface has no journal delete command. +- The local credential store keeps one selected token per profile and Filesystem. It does not mirror all remote tokens. Older create or import credentials without a known token ID remain usable, but cannot be correlated with remote token metadata. For credential selection, see [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md#understand-local-selection). +- Filesystem extraction and embedding provider configuration is optional. Leaving it unconfigured does not block resource administration, file access, search, layers, Git, journal, vault, or mount workflows. +- OpenAI provider interfaces are supported for embedding and image, audio, and video extraction. Alibaba Cloud Model Studio Qwen ASR is supported only for audio extraction. Other vendors are conditionally compatible only through the exact OpenAI-compatible contract; native Anthropic, Gemini, Vertex AI, Bedrock, and Azure OpenAI interfaces are not supported. +- App-managed embedding requires a provider model that returns exactly 1024 dimensions. Filesystems that report `source=database_auto` use database-managed embedding and reject app-managed configuration. For configuration steps, see [Configure TiDB Cloud Filesystem AI Providers](/tidb-cloud-filesystem/configure-filesystem-ai-providers.md). +- Filesystem runtime operations, including direct file access, layers, mounts, Git workspaces, journals, and Vault operations, depend on the installed `ti-drive9` companion. + +For token or mount failures, see [Troubleshoot TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-troubleshooting.md). For command syntax and options, see the [Filesystem CLI command reference](/ai/ti/reference/ti-filesystem.md). diff --git a/tidb-cloud-filesystem/filesystem-troubleshooting.md b/tidb-cloud-filesystem/filesystem-troubleshooting.md new file mode 100644 index 0000000000000..8b430f67181b2 --- /dev/null +++ b/tidb-cloud-filesystem/filesystem-troubleshooting.md @@ -0,0 +1,186 @@ +--- +title: Troubleshoot TiDB Cloud Filesystem +summary: Learn how to diagnose TiDB Cloud Filesystem token, region, companion, mount, and access failures and choose a safe recovery path. +--- + +# Troubleshoot TiDB Cloud Filesystem + +Use the symptoms below to diagnose Filesystem access and mount failures. Add `--debug` only when needed, and review redacted output before sharing it. For CLI installation, API key authentication, Starter, or SQL failures, see [Troubleshoot TiDB Cloud CLI](/ai/ti/reference/ti-troubleshooting.md). + +> **Note:** +> +> TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. + +## Filesystem token is missing + +For a clean sandbox, provide the token and region. `ti` derives the Filesystem ID from the token: + +```bash +export TI_FS_TOKEN="" +export TI_REGION_CODE="" +ti fs check-file-system +``` + +The FS token is not the TiDB Cloud API private key. `TI_FS_FILE_SYSTEM_ID` is optional when a token is supplied; set it only when you want `ti` to verify that a separately distributed ID matches the token. + +If the token is known but is not stored on the current machine, import it and then select the derived ID: + +```bash +# Store a known token without requiring TiDB Cloud API keys. +chmod 600 ./fs-token +ti fs import-file-system-token --from-file ./fs-token --region +ti fs list-files --file-system-id --path / +``` + +If every known token is lost or revoked, use TiDB Cloud API keys to generate another owner token: + +```bash +ti fs generate-file-system-token \ + --file-system-id "" \ + --token-name recovery \ + --ttl 24h +``` + +The new plaintext appears once in the response. Store it securely or add `--store-locally` to select it on the current machine. + +## Filesystem token is rejected + +A data-plane HTTP 401 cannot distinguish a token that was disabled, expired, refreshed on another machine, or revoked. Inspect remote metadata with TiDB Cloud API keys: + +```bash +ti fs list-file-system-tokens \ + --file-system-id "" \ + --include-expired \ + --output text +``` + +Token names are not unique. Use the immutable `token_id` from this output for enable, disable, or delete operations. Old credentials created or imported without token lifecycle metadata can remain valid, but `ti` cannot safely identify their list row and never guesses a match. + +After enable, disable, delete, or refresh, allow approximately 10 seconds for authentication caches to converge. If refresh reports `fs.token_refresh_ambiguous`, the server might have rotated the token even though the response was lost. The outcome is unknown: the old token might still work if the refresh did not commit, or it might already be invalid. The replacement token from a committed refresh cannot be recovered because its response was lost. Do not retry the refresh with the old token. Instead, use TiDB Cloud credentials to generate an independent owner token. + +If token mutation reports `fs.token_mount_active`, use the exact mount path in the error: + +```bash +ti fs drain-file-system --mount-path /path/to/workspace +ti fs unmount-file-system --mount-path /path/to/workspace +``` + +Then retry the token operation. A mount on another machine is not visible locally; coordinate rotation with that machine separately. + +## Filesystem selection is missing + +List remote resources in the configured region with TiDB Cloud API keys and select one explicitly: + +```bash +ti fs list-file-systems --output text +ti fs list-files --file-system-id --path / +``` + +Or select the Filesystem for subsequent commands in the current shell: + +```bash +export TI_FS_FILE_SYSTEM_ID="" +``` + +The CLI intentionally does not infer a Filesystem from local credential count, including when only one credential exists. Supply its ID or an FS token whose embedded ID can be derived. + +## Filesystem region is unsupported + +The configured TiDB Cloud region might not be one of the Filesystem endpoints built into the installed `ti` release. Compare it with [supported Filesystem regions](/tidb-cloud-filesystem/filesystem-regions-and-limitations.md#supported-regions). Change placement with a valid profile or command-scoped `--region`; do not configure a raw server URL. + +## Companion is missing or incompatible + +The release installer places `ti-drive9`, the companion runtime for Filesystem commands, next to `ti`. You do not invoke `ti-drive9` directly. Re-run the current installer when the CLI reports a missing companion: + +```bash +curl -fsSL https://github.com/tidbcloud/ti-cli/releases/latest/download/install.sh | sh -s -- --yes +``` + +Verify that `PATH` resolves the expected `ti`: + +```bash +command -v ti +ti --version +``` + +Do not copy an arbitrary standalone Drive9 binary into place. + +## Filesystem creation reaches quota + +If creation returns a quota or capacity error, list existing Filesystems in the configured region before trying again: + +```bash +ti fs list-file-systems --output text +``` + +Do not delete an unrelated Filesystem to make automation pass. If the error links to TiDB Cloud billing because a payment method is required, follow that guidance before retrying. + +## Mount does not become ready + +Background mount success prints the CLI result without Drive9 startup messages. If startup fails or times out, inspect the companion log path in the error. Confirm: + +- the mount path exists and is writable; +- no existing mount covers the path; +- the FS token and region are valid; +- FUSE prerequisites or the WebDAV helper are installed; +- the remote region is reachable. + +macOS defaults to WebDAV. To request FUSE after installing macFUSE: + +```bash +ti fs mount-file-system \ + --mount-path /path/to/workspace \ + --driver fuse +``` + +Linux needs FUSE3 and access to `/dev/fuse`. Filesystem and Vault mounts are not supported on Windows; use `ti fs` data-plane commands or non-mount Vault commands instead. + +## Ubuntu 26.04 rejects a FUSE mount under `/workspace` + +Ubuntu 26.04 applies an AppArmor profile to `fusermount3`. Its default mount-path allowlist does not include `/workspace`, so root and non-root users can both receive: + +```text +/usr/bin/fusermount3: mount failed: Permission denied +``` + +Confirm the denial: + +```bash +sudo journalctl -k --since "10 minutes ago" | + grep 'profile="fusermount3"' +``` + +An entry with `operation="mount"`, `name="/workspace/"`, and `info="failed mntpnt match"` identifies this restriction. Mount under `$HOME` or `/mnt` instead: + +```bash +mkdir -p "$HOME/workspace" +ti fs mount-file-system --mount-path "$HOME/workspace" +``` + +Changing the owner or mode of `/workspace` does not bypass AppArmor. If the path cannot change, add explicit `/workspace` mount and unmount rules to `/etc/apparmor.d/local/fusermount3` as described in [Ubuntu 26.04 mount-path restrictions](/tidb-cloud-filesystem/filesystem-mount-linux.md#ubuntu-2604-mount-path-restrictions). + +## Mount becomes stale after a process crash + +If the companion is killed without graceful unmount, FUSE access can return `EIO` or `Transport endpoint is not connected`. Stop processes with open files, then try: + +```bash +ti fs unmount-file-system \ + --mount-path /path/to/workspace \ + --force +``` + +Use `--ignore-absent` when cleanup should succeed if no locator remains. Abrupt cleanup cannot guarantee recovery of pending writes from a deleted local disk. + +## Unmount reports busy + +Close editors, shells whose working directory is inside the mount, and other open file handles, and then retry: + +```bash +ti fs unmount-file-system --mount-path /path/to/workspace +``` + +Unmount performs the graceful FUSE drain automatically. Running `drain-file-system` separately does not close file descriptors or resolve a busy mount; use it only when you need to flush pending work while leaving the mount online. Drain is not supported for WebDAV. + +## Report a problem + +Include the `ti` version, OS and architecture, command name, stable error code, and redacted logs. Never include API keys, FS or vault tokens, DB passwords, SQL containing private data, or file contents. Report issues at [github.com/tidbcloud/ti-cli/issues](https://github.com/tidbcloud/ti-cli/issues). diff --git a/tidb-cloud-filesystem/manage-git-workspaces.md b/tidb-cloud-filesystem/manage-git-workspaces.md index 707608cd43ab8..f3459c8c3e735 100644 --- a/tidb-cloud-filesystem/manage-git-workspaces.md +++ b/tidb-cloud-filesystem/manage-git-workspaces.md @@ -37,7 +37,7 @@ ti fs-git hydrate-git-workspace \ Hydration fetches missing blob data from the remote repository without discarding your working-tree changes. -If cloning or hydration fails, inspect the CLI error and diagnostic log before retrying. See [Troubleshoot TiDB Cloud CLI](/ai/ti/reference/ti-troubleshooting.md) for common Filesystem and companion issues. +If cloning or hydration fails, inspect the CLI error and diagnostic log before retrying. See [Troubleshoot TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-troubleshooting.md) for common Filesystem and companion issues. ## Add and use a linked worktree diff --git a/tidb-cloud-filesystem/work-with-filesystem-data.md b/tidb-cloud-filesystem/work-with-filesystem-data.md index 6b72ebce4696a..48bfd88c8e08e 100644 --- a/tidb-cloud-filesystem/work-with-filesystem-data.md +++ b/tidb-cloud-filesystem/work-with-filesystem-data.md @@ -73,7 +73,7 @@ ti fs find-files --path /reports --file-name-pattern "*.md" --tag stage=review `find-files` searches beneath the specified path and can filter by file name, type, tags, size, or modification time. See its [options and result limit](/ai/ti/reference/ti-fs-find-files.md). -For supported regions and current CLI and platform limitations, see [Regions and Limitations](/ai/ti/reference/ti-regions-security-and-limitations.md). +For supported regions and current Filesystem and platform limitations, see [Regions and Limitations](/tidb-cloud-filesystem/filesystem-regions-and-limitations.md). ## What's next From 49f34b19c033e0c988923e8e13cf1f0730b915f5 Mon Sep 17 00:00:00 2001 From: qiancai Date: Thu, 17 Sep 2026 17:38:57 +0800 Subject: [PATCH 06/14] revise manage-filesystem-resources.md --- TOC-tidb-cloud-filesystem.md | 23 +++---- .../filesystem-quick-start.md | 1 + .../manage-filesystem-resources.md | 66 +++++++++++-------- 3 files changed, 49 insertions(+), 41 deletions(-) diff --git a/TOC-tidb-cloud-filesystem.md b/TOC-tidb-cloud-filesystem.md index 8e4b16390764e..f73f83392b971 100644 --- a/TOC-tidb-cloud-filesystem.md +++ b/TOC-tidb-cloud-filesystem.md @@ -15,25 +15,20 @@ ## GUIDES -- Basic Operations - - [Manage TiDB Cloud Filesystems](/tidb-cloud-filesystem/manage-filesystem-resources.md) - - [Manage Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) - - [Work with Files and Directories](/tidb-cloud-filesystem/work-with-filesystem-data.md) - - [Share a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-sharing.md) +- [Manage TiDB Cloud Filesystem](/tidb-cloud-filesystem/manage-filesystem-resources.md) +- [Manage Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) +- [Work with Files and Directories](/tidb-cloud-filesystem/work-with-filesystem-data.md) +- [Share a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-sharing.md) - Mount a Filesystem - [Overview](/tidb-cloud-filesystem/filesystem-mount.md) - [Linux](/tidb-cloud-filesystem/filesystem-mount-linux.md) - [macOS](/tidb-cloud-filesystem/filesystem-mount-macos.md) - [Docker and Docker Compose](/tidb-cloud-filesystem/filesystem-mount-docker.md) -- Advanced Capabilities - - [Manage Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) - - [Manage Git Workspaces](/tidb-cloud-filesystem/manage-git-workspaces.md) - - [Use Filesystem Journals](/tidb-cloud-filesystem/use-filesystem-journals.md) - - [Manage Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md) - - [Configure Filesystem AI Providers](/tidb-cloud-filesystem/configure-filesystem-ai-providers.md) - -## USE CASES - +- [Manage Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) +- [Manage Git Workspaces](/tidb-cloud-filesystem/manage-git-workspaces.md) +- [Use Filesystem Journals](/tidb-cloud-filesystem/use-filesystem-journals.md) +- [Manage Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md) +- [Configure Filesystem AI Providers](/tidb-cloud-filesystem/configure-filesystem-ai-providers.md) - [Automation and AI Agent Workflows](/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md) ## REFERENCES diff --git a/tidb-cloud-filesystem/filesystem-quick-start.md b/tidb-cloud-filesystem/filesystem-quick-start.md index 8a3b9c2082c0d..111c297d6509e 100644 --- a/tidb-cloud-filesystem/filesystem-quick-start.md +++ b/tidb-cloud-filesystem/filesystem-quick-start.md @@ -119,6 +119,7 @@ The read returns `Hello from my workspace`. The file remains available after you ## What's next +- [Manage the Filesystem](/tidb-cloud-filesystem/manage-filesystem-resources.md) to inspect, check, select, and delete Filesystem resources. - [Mount the Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) to use it through a local directory. - [Share the workspace](/tidb-cloud-filesystem/filesystem-sharing.md) with another machine or agent. - [Manage layers and checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) to review changes before publishing them. diff --git a/tidb-cloud-filesystem/manage-filesystem-resources.md b/tidb-cloud-filesystem/manage-filesystem-resources.md index 83dcf3ab60608..4d1881a11410b 100644 --- a/tidb-cloud-filesystem/manage-filesystem-resources.md +++ b/tidb-cloud-filesystem/manage-filesystem-resources.md @@ -1,42 +1,43 @@ --- title: Manage TiDB Cloud Filesystems -summary: Learn how to safely create, inspect, check, select, and delete TiDB Cloud Filesystem resources by using TiDB Cloud CLI. +summary: Learn how to create, inspect, check, select, and delete TiDB Cloud Filesystems by using TiDB Cloud CLI. aliases: ['/ai/manage-filesystem-resources'] --- # Manage TiDB Cloud Filesystems -TiDB Cloud Filesystem is a serverless distributed file system designed for AI agents and automation workloads. It provides a persistent, shareable file namespace that remains available independently of the local machine, sandbox, or CI runner that accesses it. +You can use [TiDB Cloud CLI (`ti`)](/ai/ti/ti-overview.md) to create, inspect, check, select, and delete TiDB Cloud Filesystems. -You can access files directly through TiDB Cloud CLI commands or mount a Filesystem into a supported environment and work with it like a local file system. This makes it useful for preserving agent state, sharing files across isolated environments, handing off CI artifacts, and maintaining reusable workspaces. - -This document describes how to use [`ti fs` commands](/ai/ti/reference/ti-filesystem.md) to create, inspect, select, and delete Filesystem resources. +For command syntax, flags, and output fields, see the [`ti fs` command reference](/ai/ti/reference/ti-filesystem.md). ## Prerequisites -- [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli) and [configure access](/tidb-cloud-filesystem/filesystem-quick-start.md#step-2-configure-access) with TiDB Cloud API credentials. -- Install `jq`, or use another JSON processor to extract fields from the CLI's JSON output. +Before you begin, follow [Get Started with TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-quick-start.md) to install TiDB Cloud CLI (`ti`) and configure the access. ## Create a Filesystem -Create a Filesystem and save the returned ID and one-time owner token in a file that is not world-readable. The `--wait` flag tells the CLI to poll until data-plane access is ready before returning: - -`umask 077` restricts the output file to the current user because it contains the owner token. +Create a Filesystem and wait until it is ready: ```shell -umask 077 ti fs create-file-system \ --display-name agent-workspace \ --label environment=development \ - --wait > ./filesystem.json + --wait +``` -export TI_FS_FILE_SYSTEM_ID="$(jq -r '.file_system_id' ./filesystem.json)" -export TI_FS_TOKEN="$(jq -r '.fs_token' ./filesystem.json)" +The response includes the Filesystem ID and an owner FS token in the `fs_token` field. The CLI automatically stores the owner FS token locally. + +Copy the returned `file_system_id` and select the Filesystem for subsequent commands in the current shell: + +```shell +export TI_FS_FILE_SYSTEM_ID="" ``` +Setting `TI_FS_FILE_SYSTEM_ID` lets subsequent commands identify the target Filesystem without requiring `--file-system-id` on every command. For data-access commands, the CLI uses the locally stored FS token for the selected Filesystem. + > **Warning:** > -> The JSON response includes `fs_token` only once. The CLI also stores this token in its local credential directory automatically. However, if the local storage is lost, you cannot retrieve the token again. Store a backup copy in a secret manager, and then delete `filesystem.json`. +> The owner FS token plaintext in `fs_token` is returned only when the token is issued. Treat it as a secret and do not expose it in logs, issues, or source control. If you need to use the token on another machine or store a backup, save it securely in a secret manager. For more information, see [Manage Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md). > **Note:** > @@ -44,48 +45,59 @@ export TI_FS_TOKEN="$(jq -r '.fs_token' ./filesystem.json)" ## List and inspect Filesystems -List the Filesystems available in the effective region: +List the Filesystems available in the current region: ```shell ti fs list-file-systems --output text ``` -Read authoritative metadata for one Filesystem: +View metadata for a Filesystem: ```shell ti fs describe-file-system --file-system-id "" ``` -If you have access to more than one Filesystem, pass `--file-system-id` explicitly or set the `TI_FS_FILE_SYSTEM_ID` environment variable. The CLI does not automatically select a Filesystem for you. +If you work with more than one Filesystem, specify the target Filesystem in one of the following ways: + +- Pass `--file-system-id ""` to an individual command. +- Set `TI_FS_FILE_SYSTEM_ID` to select a Filesystem for subsequent commands in the current shell. + +The CLI does not automatically select a Filesystem based on the number of Filesystems or locally stored credentials. -The current CLI does not provide a command to change a Filesystem's display name or labels after creation. Choose these values when creating the resource. +The current CLI does not provide a command to change a Filesystem's display name or labels after creation. Choose these values when you create the Filesystem. ## Check access -Verify resource selection, endpoint resolution, credentials, and companion access: +Check whether the CLI can access a Filesystem: ```shell ti fs check-file-system --file-system-id "" ``` -The result includes an overall `status` and checks for local credentials, endpoint selection, the companion binary, and remote reachability. A `passed` status means all checks passed; `warning` or `failed` identifies a check that needs attention. +The result includes an overall `status` and checks local credentials, endpoint selection, the bundled Filesystem runtime, and remote connectivity. + +- `passed` means all checks succeeded. +- `warning` or `failed` identifies a check that needs attention. + +For common access and connectivity issues, see [Troubleshoot TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-troubleshooting.md). ## Delete a Filesystem > **Warning:** > -> Before deleting a Filesystem, drain and unmount any active local mounts for it. The CLI does not do this automatically. +> Deleting a Filesystem permanently removes its remote data. Before deletion, stop applications that are using the Filesystem and successfully unmount any active local mounts. For information about finishing pending writes safely, see [Finish safely](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). -Delete a Filesystem by explicit ID: +Delete a Filesystem by its ID: ```shell ti fs delete-file-system --file-system-id "" ``` -Filesystem deletion is asynchronous. After the service accepts the request, the CLI reports the Filesystem status as `deleting` and removes the matching local credential. This output does not mean that remote deletion has finished. +Filesystem deletion is asynchronous. After the service accepts the request, the CLI reports the Filesystem status as `deleting` and removes the matching locally stored credential. This status means that deletion has started, not that the remote Filesystem and its data have already been removed. ## What's next -- [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) -- [Work with Files and Directories](/tidb-cloud-filesystem/work-with-filesystem-data.md) -- [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) +- [Manage Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) to generate, delegate, rotate, or revoke Filesystem access. +- [Work with Files and Directories](/tidb-cloud-filesystem/work-with-filesystem-data.md) to copy, read, organize, and search Filesystem data. +- [Mount a Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) to access remote files through a local directory. +- [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) for command syntax, flags, and output fields. \ No newline at end of file From ec42713b4cfea94ed927bba5c84937a6dde178c4 Mon Sep 17 00:00:00 2001 From: qiancai Date: Thu, 17 Sep 2026 17:57:44 +0800 Subject: [PATCH 07/14] Update manage-filesystem-tokens.md --- .../manage-filesystem-tokens.md | 70 ++++++++++++------- 1 file changed, 46 insertions(+), 24 deletions(-) diff --git a/tidb-cloud-filesystem/manage-filesystem-tokens.md b/tidb-cloud-filesystem/manage-filesystem-tokens.md index c60d6e06eacad..b24059efba10b 100644 --- a/tidb-cloud-filesystem/manage-filesystem-tokens.md +++ b/tidb-cloud-filesystem/manage-filesystem-tokens.md @@ -6,29 +6,38 @@ aliases: ['/ai/manage-filesystem-tokens'] # Manage TiDB Cloud Filesystem Tokens -You can use Filesystem tokens to give users or automation access to a TiDB Cloud Filesystem without sharing TiDB Cloud API credentials. +Filesystem tokens let you give users, applications, and automation access to a TiDB Cloud Filesystem without sharing your TiDB Cloud API credentials. + +You can use an owner token for full access to a Filesystem, or create scoped tokens that limit access to specific paths and operations. For more information about token types and permissions, see [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md). ## Prerequisites +Before you begin: + - [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). -- For owner-token generation and TiDB Cloud-authenticated token management, [configure TiDB Cloud API credentials](/tidb-cloud-filesystem/filesystem-quick-start.md#step-2-configure-access) and obtain the Filesystem ID. -- For scoped-token generation or bearer-authenticated token management, obtain an [owner FS token](/tidb-cloud-filesystem/filesystem-authorization.md#owner-fs-tokens). You can pass it through `--fs-token`, set `TI_FS_TOKEN`, or use the local token stored for an explicitly selected Filesystem. +- Have access to an existing TiDB Cloud Filesystem. If you do not have one, follow [Get Started with TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-quick-start.md) to create one. + +Some token management operations require TiDB Cloud API credentials or an existing owner token. The relevant requirements are described in each section of this guide. > **Note:** > -> To avoid security risks, treat token plaintext as a secret. Token creation and rotation commands return plaintext only once when the token is issued. You cannot retrieve the plaintext later. +> Treat Filesystem tokens as secrets. When a command creates or refreshes a token, the token plaintext is returned only once and cannot be retrieved later. ## Import an existing token -When you run `import-file-system-token`, the CLI validates the token format, extracts the Filesystem ID embedded in it, verifies connectivity by making a remote stat request, and stores the token in the local credential directory: +If you already have a Filesystem token, import it to the local CLI credential store: ```shell ti fs import-file-system-token --from-file ./fs-token --region aws-us-east-1 ``` +The CLI validates the token, extracts the Filesystem ID from it, verifies connectivity, and stores the token locally. + ## Generate an owner token -Generate another owner token by using TiDB Cloud API credentials. The CLI does not store the generated token locally by default, so you must capture its one-time plaintext response securely: +To generate another owner token, configure TiDB Cloud API credentials and obtain the Filesystem ID. + +Generate the token and save its one-time plaintext response securely: ```shell umask 077 @@ -38,11 +47,13 @@ ti fs generate-file-system-token \ --ttl 24h > ./ci-token.json ``` -To have the CLI store the generated token locally, add `--store-locally`. Use `--replace` if a different token is already stored for this Filesystem. +The CLI does not store the generated token locally by default. To store it locally, add `--store-locally`. If a different token is already stored for this Filesystem, also add `--replace`. ## Generate and delegate a scoped token -On a trusted machine with an owner token, generate a path-and-operation-limited token for an agent: +To generate a scoped token, use an existing owner token on a trusted machine. Provide the owner token through `--fs-token` or `TI_FS_TOKEN`, or use the token stored locally for the selected Filesystem. + +The following example uses the locally stored owner token and creates a scoped token that allows an agent to read, list, and write files under `/workspace`: ```shell SCOPED_TOKEN="$(ti fs generate-file-system-scoped-token \ @@ -53,52 +64,63 @@ SCOPED_TOKEN="$(ti fs generate-file-system-scoped-token \ --query fs_token --output text)" ``` -Transfer the token through a secret manager. In the agent's environment, inject the token and the Filesystem's region: +Transfer the token through a secret manager. In the receiving environment, provide the scoped token and the Filesystem region: ```shell export TI_FS_TOKEN="" export TI_REGION_CODE="" + ti fs list-files --path /workspace ``` -The `--allow` value uses `:`. In this example, the token permits `read`, `list`, and `write` under `/workspace`. +The `--allow` value uses the format `:`. Supported operations are `read`, `list`, `search`, `write`, and `delete`; `search` requires `read`. In this example, the token permits `read`, `list`, and `write` operations under `/workspace`. + +The remote `/workspace` directory must already exist. To use this token for a mount, specify `--remote-path /workspace`. A token restricted to `/workspace` cannot mount the Filesystem root `/`. -The remote `/workspace` directory must already exist. For a mount, select the allowed subtree with `--remote-path /workspace`; a token restricted to `/workspace` cannot mount the root `/`. For the permission model and credential precedence, see [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md). +For more information about scoped permissions and credential selection, see [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md). ## Inspect and change token status -List non-secret token metadata: +List non-secret metadata for Filesystem tokens: ```shell -ti fs list-file-system-tokens --file-system-id "" --output text +ti fs list-file-system-tokens \ + --file-system-id "" \ + --output text ``` -The list does not return token plaintext. Preserve newly generated or refreshed tokens in a secret manager. If you lose an owner token, generate a replacement using TiDB Cloud API credentials; you cannot recover the original secret by listing tokens. +The output does not include token plaintext. If you lose an owner token, generate a replacement using TiDB Cloud API credentials. You cannot recover the original token by listing tokens. -Use [`disable-file-system-token`](/ai/ti/reference/ti-fs-disable-file-system-token.md) to suspend a token temporarily and [`enable-file-system-token`](/ai/ti/reference/ti-fs-enable-file-system-token.md) to restore it. +Use [`disable-file-system-token`](/ai/ti/reference/ti-fs-disable-file-system-token.md) to temporarily suspend a token, and [`enable-file-system-token`](/ai/ti/reference/ti-fs-enable-file-system-token.md) to restore it. ## Rotate or revoke a token -Use [`refresh-file-system-token`](/ai/ti/reference/ti-fs-refresh-file-system-token.md) to rotate a token. When you refresh the locally stored token, the CLI automatically updates the local credential file. When you refresh a token provided through `--fs-token` or `TI_FS_TOKEN`, the CLI returns the new token in the command output without storing it. +Use [`refresh-file-system-token`](/ai/ti/reference/ti-fs-refresh-file-system-token.md) to rotate a Filesystem token. + +When you refresh a locally stored token, the CLI automatically updates the local credential. When you refresh a token provided through `--fs-token` or `TI_FS_TOKEN`, the CLI returns the new token in the command output without storing it locally. > **Note:** > -> Refresh is non-idempotent. For example, after a network timeout, the service might have rotated the token even though you did not receive the new value. Do not retry with the old token. Instead, generate a new owner token using TiDB Cloud credentials. +> Refresh is non-idempotent. For example, after a network timeout, the service might have rotated the token even though you did not receive the new value. Do not retry the refresh with the old token. Instead, generate a new owner token using TiDB Cloud API credentials. > **Warning:** > -> Before you rotate, disable, or delete a token used by an active local mount, [stop writers and unmount it safely](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). Drain a FUSE mount before unmounting; WebDAV does not support drain. The CLI checks known local mounts but cannot discover every remote machine using the token. Stop writes and unmount consumers on those machines before changing it. +> Before rotating, disabling, or deleting a token used by an active mount, stop applications that are writing to the mount and successfully unmount it. The CLI can detect known local mounts but cannot discover mounts on other machines. Coordinate with those machines before changing the token. For more information, see [Finish safely](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). -Before retiring a token, distribute and validate a replacement. Then use [`delete-file-system-token`](/ai/ti/reference/ti-fs-delete-file-system-token.md) to revoke the retired token by its token ID: +Before retiring a token, distribute and validate its replacement. Then revoke the old token by its token ID: ```shell -ti fs delete-file-system-token --file-system-id "" --token-id "" +ti fs delete-file-system-token \ + --file-system-id "" \ + --token-id "" ``` -If the deleted token matches the locally stored token, the CLI automatically removes the local credential. Token changes can take time to propagate through authorization caches. Disabling an owner token does not replace reviewing and revoking scoped tokens that it issued. +If the deleted token matches the locally stored token, the CLI automatically removes the local credential. Token changes can take time to propagate through authorization caches. + +Disabling or revoking an owner token does not automatically revoke scoped tokens generated from it. Review and revoke those scoped tokens separately when necessary. ## What's next -- [Share a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-sharing.md). -- [Explore automation and AI agent workflows](/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md). -- [TiDB Cloud Filesystem CLI Command Reference (in TiDB for AI documentation)](/ai/ti/reference/ti-filesystem.md). +- [Share a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-sharing.md) +- [Explore automation and AI agent workflows](/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md) +- [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) From 2f6022ae3486fd8e359f439541ae3be914d64626 Mon Sep 17 00:00:00 2001 From: qiancai Date: Fri, 18 Sep 2026 09:29:22 +0800 Subject: [PATCH 08/14] add access-filesystem.md, and refactor filesystem-authorization.md and filesystem-layers-checkpoints.md --- TOC-tidb-cloud-filesystem.md | 3 +- tidb-cloud-filesystem/access-filesystem.md | 96 ++++++++++++ .../filesystem-authorization.md | 110 +++++++++----- .../filesystem-layers-checkpoints.md | 65 ++++++--- .../filesystem-regions-and-limitations.md | 2 +- .../manage-filesystem-layers.md | 138 ++++++++++++------ .../manage-filesystem-vault-secrets.md | 3 +- .../manage-git-workspaces.md | 2 +- .../use-filesystem-journals.md | 3 +- .../work-with-filesystem-data.md | 2 +- 10 files changed, 313 insertions(+), 111 deletions(-) create mode 100644 tidb-cloud-filesystem/access-filesystem.md diff --git a/TOC-tidb-cloud-filesystem.md b/TOC-tidb-cloud-filesystem.md index f73f83392b971..5044d21f900f3 100644 --- a/TOC-tidb-cloud-filesystem.md +++ b/TOC-tidb-cloud-filesystem.md @@ -16,8 +16,9 @@ ## GUIDES - [Manage TiDB Cloud Filesystem](/tidb-cloud-filesystem/manage-filesystem-resources.md) -- [Manage Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) +- [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md) - [Work with Files and Directories](/tidb-cloud-filesystem/work-with-filesystem-data.md) +- [Manage Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) - [Share a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-sharing.md) - Mount a Filesystem - [Overview](/tidb-cloud-filesystem/filesystem-mount.md) diff --git a/tidb-cloud-filesystem/access-filesystem.md b/tidb-cloud-filesystem/access-filesystem.md new file mode 100644 index 0000000000000..1264cbc39c883 --- /dev/null +++ b/tidb-cloud-filesystem/access-filesystem.md @@ -0,0 +1,96 @@ +--- +title: Access an Existing TiDB Cloud Filesystem +summary: Learn how to access an existing TiDB Cloud Filesystem from your current machine, another machine, CI job, or agent environment. +--- + +# Access an Existing TiDB Cloud Filesystem + +If a TiDB Cloud Filesystem already exists, you can access it from your current machine or another environment. + +How you connect depends on where you are working: + +- If you created or imported a Filesystem on the current machine, TiDB Cloud CLI (`ti`) can use the token already stored locally. +- If you are working from another machine, CI job, or agent environment, provide a Filesystem token and region for that environment. + +## Prerequisites + +Before you begin, [install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli) and make sure you have access to the Filesystem. For information about owner and scoped tokens and their permissions, see [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md). + +## Continue using a Filesystem on the same machine + +If you created the Filesystem by using `ti` on the current machine, or previously imported its token, the CLI already has a token stored locally. + +Select the Filesystem for the current shell: + +```shell +export TI_FS_FILE_SYSTEM_ID="" +``` + +You can then run Filesystem commands without providing the Filesystem ID or token each time: + +```shell +ti fs list-files --path / +``` + +Setting `TI_FS_FILE_SYSTEM_ID` selects the Filesystem for subsequent commands in the current shell. It does not change or revoke any Filesystem tokens. + +Alternatively, you can select the Filesystem for an individual command: + +```shell +ti fs list-files \ + --file-system-id "" \ + --path / +``` + +## Access a Filesystem from another environment + +If you are accessing the Filesystem from another machine, CI job, agent sandbox, or other environment without its locally stored credential, provide a Filesystem token and the Filesystem region: + +```shell +export TI_FS_TOKEN="" +export TI_REGION_CODE="" +``` + +The token identifies its Filesystem, so you do not need to provide the Filesystem ID separately. + +You can then run commands that the token permits. For example: + +```shell +ti fs list-files --path "" +``` + +A scoped token can access only the paths and operations included in its scope. If another user or administrator gave you the token, check which paths and operations you are allowed to use. + +Treat Filesystem tokens as secrets. For CI jobs and agent environments, inject the token from a secret manager instead of storing it in source code, scripts, or container images. + +If you need to create a token for another environment, see [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md). + +## Switch between Filesystems + +If you have access to multiple Filesystems with locally stored tokens, change `TI_FS_FILE_SYSTEM_ID` to select the Filesystem you want to use in the current shell: + +```shell +export TI_FS_FILE_SYSTEM_ID="" +``` + +The CLI uses the locally stored token for the selected Filesystem. + +A Filesystem can have multiple remote tokens, while each CLI profile stores at most one selected local token for each Filesystem. Changing which Filesystem or local token the CLI uses does not disable or revoke other remote tokens. + +## Credential selection + +In most workflows, use one of the approaches above rather than specifying credentials on every command. + +If multiple token sources are available, `ti` selects the token in the following order: + +1. `--fs-token` +2. `TI_FS_TOKEN` +3. The locally stored token for the selected Filesystem + +For the complete Filesystem and credential selection rules, see [TiDB Cloud CLI Configuration and Credentials](/ai/ti/reference/ti-configuration-and-credentials.md#filesystem-credentials-and-remote-inventory). + +## What's next + +- [Work with Files and Directories](/tidb-cloud-filesystem/work-with-filesystem-data.md) +- [Mount TiDB Cloud Filesystem Locally](/tidb-cloud-filesystem/filesystem-mount.md) +- [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) diff --git a/tidb-cloud-filesystem/filesystem-authorization.md b/tidb-cloud-filesystem/filesystem-authorization.md index 5dd92cc872741..6e4c02f84442e 100644 --- a/tidb-cloud-filesystem/filesystem-authorization.md +++ b/tidb-cloud-filesystem/filesystem-authorization.md @@ -1,71 +1,107 @@ --- title: TiDB Cloud Filesystem Authorization -summary: Choose TiDB Cloud API keys, owner tokens, or scoped tokens to separate Filesystem administration from application data access. +summary: Learn how TiDB Cloud API credentials, owner tokens, and scoped tokens provide different levels of access to TiDB Cloud Filesystem. --- # TiDB Cloud Filesystem Authorization -The person who creates a Filesystem and the agent that uses its files do not need the same credentials. Keep account-level management on a trusted machine and give each application only the Filesystem access it needs. +TiDB Cloud Filesystem uses different credentials for managing a Filesystem and accessing its data. This lets you keep high-privilege credentials in a trusted environment while giving users, applications, and agents only the access they need. + +In general, access becomes more limited at each level: + +```plaintext +TiDB Cloud API credentials + ↓ +Manage Filesystem resources + ↓ +Owner token + ↓ +Access one Filesystem + ↓ +Scoped token + ↓ +Access selected paths and operations +``` > **Note:** > > TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. -## Choose the credential type +## Authorization model -| Credential | Use it for | Scope | Who should hold it | -| --- | --- | --- | --- | -| TiDB Cloud API keys | Create and manage Filesystem resources and generate owner tokens | The account's permissions | A trusted administrator or automation environment | -| Owner FS token | Manage files and tokens within one Filesystem | One Filesystem | A trusted machine or secret manager | -| Scoped FS token | Delegate selected file operations | Specified paths and operations in one Filesystem | The user or application that needs that access | +TiDB Cloud Filesystem uses the following credential types: -### TiDB Cloud API keys +| Credential | Access | Typical use | +| -------------------------- | ----------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | +| TiDB Cloud API credentials | Manage Filesystem resources and owner tokens according to the account's permissions | Filesystem administration in a trusted environment | +| Owner token | Broad access to one Filesystem, including its data and scoped-token management | Trusted workflows that need full Filesystem access | +| Scoped token | Access only to specified paths and operations in one Filesystem | Users, applications, agents, or other machines that need limited access | -A TiDB Cloud public/private API key pair authorizes resource-management operations according to the account's permissions. Use it to create, list, describe, and delete Filesystems, generate owner tokens, and configure Filesystem AI providers. +These credentials are not interchangeable. TiDB Cloud API credentials manage the Filesystem resource itself, while Filesystem tokens control access within a Filesystem. -Configure the keys with `ti configure`, or provide `TIDB_CLOUD_PUBLIC_KEY` and `TIDB_CLOUD_PRIVATE_KEY` together. They are not FS tokens and do not directly replace the token used by a mount. +## TiDB Cloud API credentials -### Owner FS tokens +TiDB Cloud API credentials consist of a public and private API key pair. Their permissions are determined by the associated TiDB Cloud account. -An owner token grants broad access within one Filesystem, including reading, writing, and deleting files. It can issue scoped tokens, list token metadata, and revoke tokens in that Filesystem. It can enable or disable scoped tokens, but enabling or disabling owner tokens requires TiDB Cloud API keys. It is a high-privilege secret, not a read-only mount credential. +Use TiDB Cloud API credentials for resource-level operations such as creating, listing, describing, and deleting Filesystems. They are also required to generate owner tokens. -An owner FS token is not interchangeable with TiDB Cloud API keys: it cannot create or delete the Filesystem resource or generate another owner token through `ti`. In particular, deleting a file and deleting its Filesystem are different permissions. +Keep these credentials in a trusted administrative or automation environment rather than distributing them to applications or agents that only need access to Filesystem data. -Filesystem creation returns an owner token and stores it locally for the creating profile. To generate another owner token, use [`ti fs generate-file-system-token`](/tidb-cloud-filesystem/manage-filesystem-tokens.md#generate-an-owner-token) with TiDB Cloud API keys. +Configure these credentials with `ti configure`, or provide `TIDB_CLOUD_PUBLIC_KEY` and `TIDB_CLOUD_PRIVATE_KEY` together. For details, see [TiDB Cloud CLI Configuration and Credentials](/ai/ti/reference/ti-configuration-and-credentials.md). -### Scoped FS tokens +## Owner tokens -A scoped token limits access to specified path prefixes and operations. The supported operations are `read`, `list`, `search`, `write`, and `delete`. The service enforces these permissions, including when requests arrive through a mount. +An owner token provides broad access to one Filesystem. It can read, write, and delete files and can create and manage scoped tokens for that Filesystem. -- A reporting agent might need `read,list` for `/inputs` and `read,list,write` for `/reports`. -- A reviewer might need only `read,list` for `/reports`. -- `search` also requires `read` on the scope. +An owner token applies only to its Filesystem. It does not replace TiDB Cloud API credentials for resource-level operations such as creating or deleting Filesystems. An owner token also cannot create another owner token; generating another owner token requires TiDB Cloud API credentials. -Scoped tokens cannot generate child tokens or manage token inventory. A scoped token can refresh itself while valid; refresh does not turn it into an owner token or broaden its permissions. +Because an owner token has broad privileges within a Filesystem, keep it on a trusted machine or in a secret manager. When another user, application, or agent needs only limited access, create a scoped token instead of sharing the owner token. -Choose a scoped token when delegating access to an agent or another machine. Keep the owner token on a trusted machine, and use a secret manager to deliver only the scoped token. To generate and use one, see [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md#generate-and-delegate-a-scoped-token). +A Filesystem can have multiple active owner tokens. This lets different users, applications, or environments use separate credentials instead of sharing the same token. -## Understand local selection +For instructions on generating and managing owner tokens, see [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md). -One Filesystem can have multiple remote tokens, but one CLI profile stores at most one selected local token for that Filesystem. Local state is a credential selection, not the authoritative remote token inventory. +## Scoped tokens -### Select a token for data access +A scoped token limits access to specified paths and operations within one Filesystem. -- `--fs-token` takes precedence over `TI_FS_TOKEN` for token-based operations. -- Without an explicit token, data-access commands use the locally stored credential for `--file-system-id` or `TI_FS_FILE_SYSTEM_ID`. -- An explicit token contains the Filesystem ID. A clean environment therefore needs only `TI_FS_TOKEN` and `TI_REGION_CODE` for data access. -- Generating a token does not select it locally unless you pass `--store-locally`. Replacing the local selection does not revoke the previous remote token. +The supported operations are: -### Keep management credentials separate +- `read`: read file content and metadata. +- `list`: list entries under a directory. +- `search`: search for files under the allowed path. `search` also requires `read`. +- `write`: create or modify files and directories. +- `delete`: delete files or directories. -For token list, enable, disable, and delete commands, an explicitly supplied owner token selects bearer authentication; otherwise the CLI uses TiDB Cloud API keys and requires a Filesystem ID. +For example, a reporting application could have: -> **Note:** -> -> A scoped token does not gain administrative capability merely because account keys also exist in the profile. +- `read,list` access to `/inputs` +- `read,list,write` access to `/reports` + +Another user that only needs to review generated reports could receive `read,list` access to `/reports`. + +Scoped permissions are enforced by the Filesystem service, including when the Filesystem is accessed through a mount. A scoped token cannot broaden its own permissions or create or manage other tokens. A scoped token can be refreshed while it is valid, but refreshing it does not broaden its permissions. + +After a scoped token is issued, disabling or revoking the owner token that created it does not automatically revoke the scoped token. Revoke scoped tokens separately when their access is no longer needed. + +Use scoped tokens when giving Filesystem access to users, applications, agents, or other machines that do not need full access to the Filesystem. + +For instructions on creating and delegating scoped tokens, see [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md#generate-and-delegate-a-scoped-token). + +## Credential boundaries + +Use the credential with the minimum access required for a workflow: + +* Use **TiDB Cloud API credentials** to manage the Filesystem resource. +* Use an **owner token** when a trusted workflow needs broad access to one Filesystem or needs to delegate access. +* Use a **scoped token** when a user, application, or agent needs access only to specific paths and operations. + +Having multiple credential types available in the same environment does not combine their permissions. For example, a scoped token remains scoped even if TiDB Cloud API credentials are also configured. + +TiDB Cloud CLI can store a selected Filesystem token locally for convenience. Local credential selection does not change, disable, or revoke remote tokens. For details about how `ti` selects Filesystems and credentials, see [TiDB Cloud CLI Configuration and Credentials](/ai/ti/reference/ti-configuration-and-credentials.md). ## What's next -- [Share one Filesystem with multiple machines](/tidb-cloud-filesystem/filesystem-sharing.md). -- [Generate, rotate, or revoke Filesystem tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md). -- [Mount a token-scoped directory](/tidb-cloud-filesystem/filesystem-mount.md). +- [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) +- [Share a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-sharing.md) +- [Mount TiDB Cloud Filesystem Locally](/tidb-cloud-filesystem/filesystem-mount.md) diff --git a/tidb-cloud-filesystem/filesystem-layers-checkpoints.md b/tidb-cloud-filesystem/filesystem-layers-checkpoints.md index 2e14a6f15d108..864f8b36d0fa0 100644 --- a/tidb-cloud-filesystem/filesystem-layers-checkpoints.md +++ b/tidb-cloud-filesystem/filesystem-layers-checkpoints.md @@ -1,46 +1,67 @@ --- title: TiDB Cloud Filesystem Layers and Checkpoints -summary: Understand how Filesystem layers isolate changes, how forks and checkpoints preserve layer history, and when changes become visible in the base Filesystem. +summary: TiDB Cloud Filesystem layers isolate changes from the base Filesystem, while forks and checkpoints preserve layer history until changes are committed. --- # TiDB Cloud Filesystem Layers and Checkpoints -A Filesystem layer gives you a writable view over a base path without immediately changing the shared files. Use layers to prepare independent drafts, inspect changes, and publish a selected result after review. The CLI calls these objects **layers**, not Git branches; no Git repository is required. +A Filesystem layer records changes over a path in the base Filesystem while keeping those changes separate from the shared base data. Changes in a layer become part of the base Filesystem only when the layer is committed. -For commands to create, fork, checkpoint, commit, or roll back a layer, see [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md). +Layers can also be checkpointed or forked, which lets you preserve a point in a layer's history or continue work in an independent child layer. + +Filesystem layers are independent of Git branches and do not require a Git repository. + +For commands to create, inspect, fork, checkpoint, commit, or roll back layers, see [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md). > **Note:** > > TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. -## Understand the model +## Core concepts + +- **Base Filesystem:** the shared Filesystem state outside a layer. A layer records changes over a base path without immediately modifying this state. +- **Layer:** a writable set of changes over a base path. Until the layer is committed, its changes remain separate from the base Filesystem. +- **Checkpoint:** a named point in a layer's durable history. Creating a checkpoint preserves that point in the layer history but does not apply the layer's changes to the base Filesystem. +- **Fork:** a new writable child layer created from the current state of another layer or from one of its checkpoints. Changes made after the fork are independent between the parent and child layers. +- **Commit:** applies the effective changes in a layer to the base Filesystem. +- **Rollback:** discards the layer's uncommitted changes without applying them to the base Filesystem. Rollback does not reset a layer to an earlier checkpoint. + +## Layers and the base Filesystem + +Changes made in a layer are visible through that layer but are not visible in the base Filesystem until the layer is committed. + +Operations that do not specify a layer access the base Filesystem. This means that an uncommitted layer does not change what other users or applications see when they access the base directly. + +A layer is an overlay on a base path rather than a complete snapshot of the Filesystem. A fork or checkpoint preserves the relevant layer history, but it does not freeze the entire base Filesystem. Data that continues to be resolved from the base can therefore reflect later changes to the base. + +## Forks and checkpoints + +A checkpoint preserves a point in one layer's durable history. It does not create another writable layer and does not publish changes to the base Filesystem. -- **Base Filesystem:** the live shared files read by ordinary file commands and mounts without a layer selector. -- **Layer:** a set of changes over a base path. Its writes remain separate from the base until committed. -- **Fork:** a child layer that pins the parent's layer history at its current tip or a specified checkpoint. Later writes to either layer are independent of the other layer's later changes. -- **Checkpoint:** a named point in one layer's durable history. It does not publish changes to the base or flush another client's pending writes. -- **Commit:** apply all of the layer's effective changes to the base Filesystem. This is not a Git commit or a selective per-file publish operation. +A checkpoint can be accessed as a read-only view. To continue making changes from a checkpoint, fork a new writable layer from it. -Use immutable layer IDs in automation because layer names are not guaranteed to be unique. A checkpoint mount is read-only; to continue writing from a checkpoint, fork a new writable layer. Rolling back a layer discards its changes rather than resetting it to a selected checkpoint. +A fork preserves the parent layer history at the point where the fork is created. After that, new changes in the parent and child layers are independent. -## Understand visibility and boundaries +For operational requirements when creating checkpoints or working with mounted layers, see [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md). -Plain `ti fs read-file` and `ti fs list-files` commands read the base, not an uncommitted layer. Use a layer mount to inspect files in that layer. A fork or checkpoint pins layer history, not the entire base Filesystem: paths resolved from the live base can still reflect later base changes. +## Commit and rollback -Before checkpointing a layer with a writable FUSE mount, [drain the mount](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely) so its writes reach the service. Before committing or rolling back the layer, drain and unmount it. Do not mount the same writable layer at multiple local paths concurrently. +Committing a layer applies all of its effective changes to the base Filesystem. A commit is not a Git commit and does not selectively publish individual files. -Layers are not full Filesystem snapshots, backups, or authorization boundaries. Use [scoped tokens](/tidb-cloud-filesystem/filesystem-authorization.md#scoped-fs-tokens) to limit access. A layer commit can fail if the base has conflicting changes; retain the layer and inspect the conflict rather than assuming an automatic merge. +A commit applies changes to the base Filesystem, not to the layer's parent layer. If the base Filesystem contains conflicting changes, the commit can fail rather than automatically merging the changes. -## Limitations +Rolling back a layer discards its uncommitted changes instead of publishing them. It does not reset the layer to an earlier checkpoint. -### Unsupported operations +## Boundaries and limitations -- The CLI does not provide Git-style merge, rebase, or an in-place reset to a checkpoint. -- A layer commit does not merge changes into its parent layer. +Layers provide change isolation, but they are not full Filesystem snapshots, backups, or authorization boundaries. To restrict access to Filesystem paths or operations, use [scoped tokens](/tidb-cloud-filesystem/filesystem-authorization.md#scoped-tokens). -### Other boundaries +The current layer model has the following limitations: -- Deleting a layer logically abandons it but does not guarantee immediate physical removal of history pinned by descendants. -- Repeated changes to layer-created files and inherited metadata can have preview limitations. Test complex edit histories before relying on them. +- Git-style merge and rebase operations are not supported. +- A layer cannot be reset in place to a checkpoint. To continue from a checkpoint, fork a new layer. +- Committing a child layer does not merge its changes into its parent layer. +- Deleting a layer logically abandons it, but history referenced by descendant layers might remain available while those descendants depend on it. +- Complex histories involving repeated changes to layer-created files or inherited metadata might have limitations during public preview. -For command flags and output fields, see the [`ti fs` command reference](/ai/ti/reference/ti-filesystem.md). +For command syntax, flags, and output fields, see the [`ti fs` command reference](/ai/ti/reference/ti-filesystem.md). diff --git a/tidb-cloud-filesystem/filesystem-regions-and-limitations.md b/tidb-cloud-filesystem/filesystem-regions-and-limitations.md index a32579552db9d..fc477e36834bc 100644 --- a/tidb-cloud-filesystem/filesystem-regions-and-limitations.md +++ b/tidb-cloud-filesystem/filesystem-regions-and-limitations.md @@ -47,7 +47,7 @@ For the supported drain and graceful unmount workflow, see [Finish safely](/tidb ## Current limitations - Journals are append-only, and the current public command surface has no journal delete command. -- The local credential store keeps one selected token per profile and Filesystem. It does not mirror all remote tokens. Older create or import credentials without a known token ID remain usable, but cannot be correlated with remote token metadata. For credential selection, see [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md#understand-local-selection). +- The local credential store keeps one selected token per profile and Filesystem. It does not mirror all remote tokens. Older create or import credentials without a known token ID remain usable, but cannot be correlated with remote token metadata. - Filesystem extraction and embedding provider configuration is optional. Leaving it unconfigured does not block resource administration, file access, search, layers, Git, journal, vault, or mount workflows. - OpenAI provider interfaces are supported for embedding and image, audio, and video extraction. Alibaba Cloud Model Studio Qwen ASR is supported only for audio extraction. Other vendors are conditionally compatible only through the exact OpenAI-compatible contract; native Anthropic, Gemini, Vertex AI, Bedrock, and Azure OpenAI interfaces are not supported. - App-managed embedding requires a provider model that returns exactly 1024 dimensions. Filesystems that report `source=database_auto` use database-managed embedding and reject app-managed configuration. For configuration steps, see [Configure TiDB Cloud Filesystem AI Providers](/tidb-cloud-filesystem/configure-filesystem-ai-providers.md). diff --git a/tidb-cloud-filesystem/manage-filesystem-layers.md b/tidb-cloud-filesystem/manage-filesystem-layers.md index eb7f90f919109..79cee28df0e69 100644 --- a/tidb-cloud-filesystem/manage-filesystem-layers.md +++ b/tidb-cloud-filesystem/manage-filesystem-layers.md @@ -1,22 +1,29 @@ --- title: Manage TiDB Cloud Filesystem Layers and Checkpoints -summary: Learn how to safely create, inspect, fork, checkpoint, roll back, commit, pack, and restore TiDB Cloud Filesystem layers. +summary: Learn how to create, inspect, checkpoint, fork, commit, roll back, and delete TiDB Cloud Filesystem layers. aliases: ['/ai/manage-filesystem-layers'] --- # Manage TiDB Cloud Filesystem Layers and Checkpoints -Use layers to record isolated changes over a Filesystem base path before you commit or discard them. +A layer gives you a separate workspace for changing files without immediately affecting the base Filesystem. You can make and review changes in the layer, then decide whether to apply them to the base Filesystem or discard them. -This guide covers individual CLI operations. To understand layer visibility, forks, checkpoints, and their boundaries before using these commands, see [Layers and Checkpoints](/tidb-cloud-filesystem/filesystem-layers-checkpoints.md). +You can also create a checkpoint to preserve a point in the layer's history, or fork a new layer from the current layer or a checkpoint to continue working independently. + +For an overview of how layers, checkpoints, forks, and the base Filesystem relate to each other, see [Layers and Checkpoints](/tidb-cloud-filesystem/filesystem-layers-checkpoints.md). ## Prerequisites +Before you begin: + - [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). -- For the commands below, set `TI_FS_FILE_SYSTEM_ID` to the Filesystem ID and use its locally stored FS token. Alternatively, set `TI_FS_TOKEN` and `TI_REGION_CODE` for token-only access; the token identifies the Filesystem. To select a Filesystem per command instead, add `--file-system-id ""` to each command. Use a token with the required read or write permission. See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md#understand-local-selection) for selection details. +- Have access to an existing TiDB Cloud Filesystem with a token that provides the required read or write permissions. +- Select the Filesystem and make its token available to `ti`. For available access options, see [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). - Choose the base path whose data the layer overlays. -## Create and inspect a layer +## Create a layer + +Choose the base path that you want the layer to overlay, and create a layer: ```shell ti fs create-layer \ @@ -26,101 +33,140 @@ ti fs create-layer \ --tag task=review ``` -`restore-safe` is the only `--durability-mode` value accepted by the current CLI. See the [`create-layer` command reference](/ai/ti/reference/ti-fs-create-layer.md) for the current option contract. +`--base-root-path` determines which part of the base Filesystem the layer overlays. -List the layers in the selected Filesystem: +`restore-safe` is the only `--durability-mode` value accepted by the current CLI. For all available options, see the [`create-layer` command reference](/ai/ti/reference/ti-fs-create-layer.md). -```shell -ti fs list-layers --output text -``` +The command returns a layer ID. Use the layer ID for subsequent operations, especially in automation, because layer names are not guaranteed to be unique. -Use the returned layer ID to write and inspect changes: +## Work with and inspect layer changes + +Write a file to the layer by specifying its layer ID: ```shell ti fs copy-file \ --from-local ./proposal.md \ --to-remote /workspace/proposal.md \ --layer-id "" +``` + +Inspect the layer and its changes: +```shell ti fs describe-layer --layer-id "" ti fs diff-layer --layer-id "" ``` +List all layers in the selected Filesystem: + +```shell +ti fs list-layers --output text +``` + +Changes that have not been committed remain in the layer. File operations that do not select the layer access the base Filesystem and do not show its uncommitted changes. + > **Note:** > -> `copy-file` with `--layer-id` does not support recursive copy. To seed a directory tree into a layer, mount the layer as a writable FUSE mount and copy files through the mount path. +> `copy-file` with `--layer-id` does not support recursive copy. To copy a directory tree into a layer, mount the layer as a writable FUSE mount and copy files through the mount path. +> +> Do not mount the same writable layer at multiple local paths concurrently. Reuse its existing mount, or unmount it before mounting the layer elsewhere. + +## Create a checkpoint + +A checkpoint preserves a point in the layer's durable history. -Do not mount the same writable layer at multiple local paths concurrently. Reuse its existing mount, or unmount it before mounting the layer elsewhere. +If the layer has an active writable FUSE mount, drain pending writes before creating the checkpoint so that the checkpoint includes the changes that have reached the service: + +```shell +ti fs drain-file-system \ + --mount-path "/path/to/workspace" +``` -## Create a checkpoint and fork a layer +Then create the checkpoint: ```shell ti fs create-layer-checkpoint \ --layer-id "" \ --checkpoint-id seed \ --label "before review" +``` + +A checkpoint mount is read-only. To continue making changes from a checkpoint, fork a new writable layer. +## Fork a layer + +Fork a new writable layer from the current layer or one of its checkpoints: + +```shell ti fs fork-layer \ --parent-layer-ref "" \ --layer-name experiment \ --checkpoint-id seed ``` -Use `list-layer-chain` to inspect the pinned ancestry of the fork: +Use the layer ID returned for the fork when you perform subsequent operations on it. + +To inspect the fork's pinned ancestry: ```shell -ti fs list-layer-chain --layer-ref experiment +ti fs list-layer-chain --layer-ref "" ``` -A checkpoint mount is read-only. To continue working from a checkpoint, fork a new writable layer from it. +After a fork is created, changes made to the parent and child layers are independent. -## Finish work in a layer +## Commit or discard layer changes -> **Warning:** -> -> Before you create a checkpoint for a layer with a writable FUSE mount, run [`drain-file-system`](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). A checkpoint includes only changes that have reached the service. Before you roll back or commit the layer, drain and then [`unmount-file-system`](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). The CLI does not perform these steps automatically. +Before committing or rolling back a layer with an active writable FUSE mount, stop applications that are writing to the mount, drain pending writes, and unmount it: -Choose one outcome for a layer: +```shell +ti fs drain-file-system \ + --mount-path "/path/to/workspace" -- Roll back the layer to discard its changes: +ti fs unmount-file-system \ + --mount-path "/path/to/workspace" +``` - ```shell - ti fs rollback-layer --layer-id "" - ``` +For more information about safely finishing mount activity, see [Finish safely](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). -- Commit the layer to apply its changes to the base path: +To apply the layer's changes to the base Filesystem: - ```shell - ti fs commit-layer --layer-id "" - ``` +```shell +ti fs commit-layer --layer-id "" +``` -> **Note:** -> -> Do not run both `rollback-layer` and `commit-layer` in sequence for the same layer. +A commit applies the layer's effective changes to the base Filesystem. If the layer was created by forking another layer, committing it does not merge the changes back into its parent layer. + +If the base Filesystem contains conflicting changes, the commit can fail instead of automatically merging them. Keep the layer and inspect its changes and the base Filesystem before deciding how to proceed. + +To discard the layer's uncommitted changes instead: -## Move local state to another machine (advanced) +```shell +ti fs rollback-layer --layer-id "" +``` + +Rollback discards the current layer changes. It does not reset the layer to an earlier checkpoint. To continue from a checkpoint, fork a new layer from that checkpoint. -When a FUSE mount uses write-back cache, some data can remain in its local overlay directory. To move this local state to another machine, pack it to an explicit remote archive path: +## Delete a layer + +When you no longer need a layer, delete it by its layer ID: ```shell -ti fs pack-file-system \ - --mount-path /path/to/workspace \ - --archive-path /workspace-overlay.tar.gz +ti fs delete-layer --layer-ref "" ``` -On the destination machine, restore the archive into a local overlay root: +Deleting a layer abandons it without immediately erasing all of its history. If the layer has live descendants, the command fails by default. + +To abandon the layer and all of its live descendants, use `--cascade`: ```shell -ti fs unpack-file-system \ - --local-root /path/to/local-overlay \ - --remote-root /workspace \ - --mount-profile portable \ - --archive-path /workspace-overlay.tar.gz +ti fs delete-layer \ + --layer-ref "" \ + --cascade ``` -Use the same local overlay root when you mount the Filesystem on the destination machine. For all pack and unpack options, see the [`pack-file-system`](/ai/ti/reference/ti-fs-pack-file-system.md) and [`unpack-file-system`](/ai/ti/reference/ti-fs-unpack-file-system.md) references. +Use `--cascade` only when you intend to abandon the descendant layers as well. ## What's next -- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) +- [Mount TiDB Cloud Filesystem Locally](/tidb-cloud-filesystem/filesystem-mount.md) - [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) diff --git a/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md b/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md index 9b724add7a1bf..aa26d72e706aa 100644 --- a/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md +++ b/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md @@ -11,7 +11,8 @@ TiDB Cloud Filesystem Vault lets you store secrets, delegate narrowly scoped and ## Prerequisites - [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). -- For the secret commands below, set `TI_FS_FILE_SYSTEM_ID` to the Filesystem ID and use its locally stored FS token. Alternatively, set `TI_FS_TOKEN` and `TI_REGION_CODE` for token-only access; the token identifies the Filesystem. Owner operations require an [owner FS token](/tidb-cloud-filesystem/filesystem-authorization.md#owner-fs-tokens). See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md#understand-local-selection) for selection details. +- Have access to an existing TiDB Cloud Filesystem. Owner operations require an [owner token](/tidb-cloud-filesystem/filesystem-authorization.md#owner-tokens). +- Select the Filesystem and make its token available to `ti`. For available access options, see [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). > **Note:** > diff --git a/tidb-cloud-filesystem/manage-git-workspaces.md b/tidb-cloud-filesystem/manage-git-workspaces.md index f3459c8c3e735..ef11b55d9f8d5 100644 --- a/tidb-cloud-filesystem/manage-git-workspaces.md +++ b/tidb-cloud-filesystem/manage-git-workspaces.md @@ -12,7 +12,7 @@ Use `ti fs-git` when you want a Git workspace on a mounted TiDB Cloud Filesystem - [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). - [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) through FUSE. -- For the commands below, set `TI_FS_FILE_SYSTEM_ID` to the mounted Filesystem ID and use its locally stored FS token. Alternatively, set `TI_FS_TOKEN` and `TI_REGION_CODE` for token-only access; the token identifies the Filesystem. To select a Filesystem per command instead, add `--file-system-id ""` to each command. Use a token with Git workspace permissions. See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md#understand-local-selection) for selection details. +- Select the mounted Filesystem and make a token with the required permissions available to `ti`. For available access options, see [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). - Install Git and configure repository credentials independently. ## Clone a workspace diff --git a/tidb-cloud-filesystem/use-filesystem-journals.md b/tidb-cloud-filesystem/use-filesystem-journals.md index a10f60dfd057f..1cce3d2b0d78c 100644 --- a/tidb-cloud-filesystem/use-filesystem-journals.md +++ b/tidb-cloud-filesystem/use-filesystem-journals.md @@ -13,7 +13,8 @@ Use a journal when you need to trace the order of agent actions or handoffs acro ## Prerequisites - [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). -- For the commands below, set `TI_FS_FILE_SYSTEM_ID` to the Filesystem ID and use its locally stored FS token. Alternatively, set `TI_FS_TOKEN` and `TI_REGION_CODE` for token-only access; the token identifies the Filesystem. To select a Filesystem per command instead, add `--file-system-id ""` to each command. Use a token with journal permissions. See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md#understand-local-selection) for selection details. +- Have access to an existing TiDB Cloud Filesystem with a token that provides the required journal permissions. +- Select the Filesystem and make its token available to `ti`. For available access options, see [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). ## Create a journal diff --git a/tidb-cloud-filesystem/work-with-filesystem-data.md b/tidb-cloud-filesystem/work-with-filesystem-data.md index 48bfd88c8e08e..12e6e30f327d7 100644 --- a/tidb-cloud-filesystem/work-with-filesystem-data.md +++ b/tidb-cloud-filesystem/work-with-filesystem-data.md @@ -12,7 +12,7 @@ In TiDB Cloud CLI, you can use [`ti fs` commands](/ai/ti/reference/ti-filesystem - [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). - [Create a Filesystem](/tidb-cloud-filesystem/manage-filesystem-resources.md) or obtain access to an existing one. -- For the commands below, set `TI_FS_FILE_SYSTEM_ID` to the Filesystem ID and use its locally stored FS token. Alternatively, set `TI_FS_TOKEN` and `TI_REGION_CODE` for token-only access; the token identifies the Filesystem. To select a Filesystem per command instead, add `--file-system-id ""` to each command. Use a token with the permissions required for each operation. See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md#understand-local-selection) for selection details. +- Select the Filesystem and make its token available to `ti`. For available access options, see [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). ## Copy data From b83fbdfe7b198aa8d9aafb0fade5b90ff73bdc91 Mon Sep 17 00:00:00 2001 From: qiancai Date: Fri, 18 Sep 2026 11:09:29 +0800 Subject: [PATCH 09/14] revise mounting docs, work-with-filesystem-data.md, and filesystem-sharing.md --- TOC-ai.md | 3 +- .../filesystem-authorization.md | 2 +- .../filesystem-mount-docker.md | 156 +++++++++---- .../filesystem-mount-linux.md | 188 ++++++++++----- .../filesystem-mount-macos.md | 161 +++++++++---- tidb-cloud-filesystem/filesystem-mount.md | 155 ++++++++----- .../filesystem-regions-and-limitations.md | 4 +- tidb-cloud-filesystem/filesystem-sharing.md | 216 ++++++++++++------ .../filesystem-troubleshooting.md | 4 +- ...filesystem-for-automation-and-ai-agents.md | 1 - .../work-with-filesystem-data.md | 109 ++++++--- 11 files changed, 686 insertions(+), 313 deletions(-) diff --git a/TOC-ai.md b/TOC-ai.md index b492604ab13f4..09e77c170248a 100644 --- a/TOC-ai.md +++ b/TOC-ai.md @@ -45,11 +45,10 @@ - [Overview](/ai/ti/ti-overview.md) - [Install, Configure, and Update TiDB Cloud CLI](/ai/ti/reference/ti-install-configure-update.md) - [Manage TiDB Cloud Starter Instances](/ai/ti/guides/manage-starter-instances.md) - - [Use TiDB Cloud Filesystem with TiDB Cloud CLI](/ai/ti/guides/manage-filesystems-via-cli.md) + - [Manage TiDB Cloud Filesystem](/ai/ti/guides/manage-filesystems-via-cli.md) - Scenarios for Users and Automation - [Run a Daily TiDB Cloud CLI Workflow](/ai/ti/guides/ti-daily-workflow-example.md) - [Query TiDB Cloud Starter with Explicit SQL Roles](/ai/ti/guides/ti-query-sql-with-roles-example.md) - - [Share a TiDB Cloud Filesystem Across Machines](/ai/ti/guides/ti-share-filesystem-across-machines-example.md) - [Hand Off CI Artifacts Between Isolated Jobs with TiDB Cloud Filesystem](/ai/ti/guides/ti-ci-artifact-handoff-example.md) - Scenarios for AI Agents - [Use TiDB Cloud Filesystem in an Agent Sandbox](/ai/ti/guides/ti-agent-sandbox-example.md) diff --git a/tidb-cloud-filesystem/filesystem-authorization.md b/tidb-cloud-filesystem/filesystem-authorization.md index 6e4c02f84442e..13e628a53d3ef 100644 --- a/tidb-cloud-filesystem/filesystem-authorization.md +++ b/tidb-cloud-filesystem/filesystem-authorization.md @@ -103,5 +103,5 @@ TiDB Cloud CLI can store a selected Filesystem token locally for convenience. Lo ## What's next - [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) -- [Share a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-sharing.md) +- [Share a TiDB Cloud Filesystem Across Machines](/tidb-cloud-filesystem/filesystem-sharing.md) - [Mount TiDB Cloud Filesystem Locally](/tidb-cloud-filesystem/filesystem-mount.md) diff --git a/tidb-cloud-filesystem/filesystem-mount-docker.md b/tidb-cloud-filesystem/filesystem-mount-docker.md index 01b9f322bf33e..940afd9639fe7 100644 --- a/tidb-cloud-filesystem/filesystem-mount-docker.md +++ b/tidb-cloud-filesystem/filesystem-mount-docker.md @@ -1,11 +1,13 @@ --- title: Mount TiDB Cloud Filesystem in Docker -summary: Configure Docker or Docker Compose with a FUSE device, mount permissions, and a Filesystem token to use persistent remote files. +summary: Mount a TiDB Cloud Filesystem inside a Docker container by giving the container access to FUSE and the required mount permissions. --- # Mount TiDB Cloud Filesystem in Docker -A container needs both FUSE userspace tools and permission to use the Linux host's FUSE device. Installing `fuse3` inside an otherwise restricted container is not sufficient. +To mount a TiDB Cloud Filesystem inside a Docker container, the container must be able to use FUSE on the Linux host. In addition to installing `fuse3` inside the container, you need to expose `/dev/fuse` and grant the container the permissions required to create a mount. + +These steps are for Docker running on a Linux host. If your environment does not allow FUSE access or the required container privileges, use direct `ti fs` commands such as `copy-file`, `read-file`, and `list-files` instead. > **Note:** > @@ -13,30 +15,42 @@ A container needs both FUSE userspace tools and permission to use the Linux host ## Prerequisites -- A Linux Docker host with an accessible `/dev/fuse` device. -- An FS token for the existing Filesystem and its region code. -- Permission to start a container with the device and capabilities below. +Before you begin: + +- Use a Linux Docker host with `/dev/fuse` available. +- Have permission to start containers with access to `/dev/fuse` and the required mount capabilities. +- Have a Filesystem token and the region code for an existing TiDB Cloud Filesystem. See [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). -These examples target a Linux Docker host. Do not assume Docker Desktop's VM or a managed sandbox exposes the same device and security controls. If mounting is unavailable, use direct `ti fs` file commands instead. +Docker Desktop and managed container or sandbox platforms might not expose `/dev/fuse` or allow the required privileges. > **Warning:** > -> `SYS_ADMIN` grants broad privileges, and `apparmor=unconfined` disables the container's AppArmor profile. Use these settings only in an environment whose security policy permits them. A scoped FS token limits remote data access but does not restore container isolation. Do not grant these privileges to untrusted agent code without an appropriate isolation boundary. +> The examples in this guide grant the container `SYS_ADMIN` and disable its AppArmor profile with `apparmor=unconfined`. These settings give the container broader access than a standard Docker container. +> +> Use them only when permitted by your environment's security policy. Do not run untrusted code with these privileges unless your environment provides an appropriate additional isolation boundary. A scoped Filesystem token can restrict access to Filesystem data, but it does not reduce the container privileges granted by Docker. -## Start with Docker +## Provide Filesystem access to the container -On the host, provide the token through your secret manager and set the matching region: +On the Docker host, set the Filesystem token and region: ```bash -# Do not put a real token in a Dockerfile or commit it to source control. export TI_FS_TOKEN="" -export TI_REGION_CODE="aws-us-east-1" +export TI_REGION_CODE="" ``` -Start an interactive container: +Treat the Filesystem token as a secret. Do not put it in a Dockerfile or commit it to source control. + +The Docker and Docker Compose examples below pass these values from the host into the container. + +## Start the container + +Choose either Docker or Docker Compose. + +### Option 1: Use Docker + +Start an interactive Ubuntu container with access to FUSE: ```bash -# Expose FUSE and the permissions needed to create the mount. docker run --rm -it \ --device /dev/fuse \ --cap-add SYS_ADMIN \ @@ -46,13 +60,17 @@ docker run --rm -it \ ubuntu:24.04 bash ``` -The device option exposes `/dev/fuse`; `SYS_ADMIN` permits the mount operation; the security option removes AppArmor restrictions that could otherwise reject it. A host's other security controls can still prohibit mounting. +This command does the following: -Continue with [Install and mount inside the container](/tidb-cloud-filesystem/filesystem-mount-docker.md#install-and-mount-inside-the-container). +- Exposes the host's `/dev/fuse` device to the container; +- Grants the capability required to create the mount; and +- Passes the Filesystem token and region into the container. -## Start with Docker Compose +After the container starts, continue with [Install and mount inside the container](#install-and-mount-inside-the-container). -As an alternative to `docker run`, use the following `compose.yaml`. It reads the same two environment variables from the host: +### Option 2: Use Docker Compose + +Alternatively, create a `compose.yaml` file: ```yaml services: @@ -70,60 +88,100 @@ services: TI_REGION_CODE: ${TI_REGION_CODE:?Set TI_REGION_CODE} ``` +On the Docker host, start the container and open a shell in it: + ```bash -# Start the container, then open its shell. docker compose up -d docker compose exec agent bash ``` +Then continue with the steps below inside the container. + ## Install and mount inside the container -Run these commands inside the container. This Ubuntu image starts as root, so no `sudo` is needed for package installation: +The following commands use the `ubuntu:24.04` image from the examples above, which runs as `root` by default. -```bash -# Install HTTPS download support and the FUSE3 mount helper. -apt-get update -apt-get install -y --no-install-recommends ca-certificates curl fuse3 -``` +1. Install `fuse3` and the tools required to install `ti`: -Install the CLI: + ```bash + apt-get update + apt-get install -y --no-install-recommends ca-certificates curl fuse3 + ``` -```bash -# The installer includes the Filesystem mount runtime. -curl -fsSL https://github.com/tidbcloud/ti-cli/releases/latest/download/install.sh | sh -s -- --yes -``` +2. Install TiDB Cloud CLI: + + ```bash + curl -fsSL https://github.com/tidbcloud/ti-cli/releases/latest/download/install.sh | sh -s -- --yes + ``` + +3. Add `ti` to the current shell, create a local mount directory, and mount the Filesystem: + + ```bash + export PATH="$HOME/.ti/bin:$PATH" + + mkdir -p "$HOME/workspace" -After installation, prepare the current shell and mount: + ti fs mount-file-system --mount-path "$HOME/workspace" + ``` + + On Linux, `ti` uses the FUSE driver by default. + +4. Verify that you can access the Filesystem through the mounted directory: + + ```bash + ls "$HOME/workspace" + ``` + +If your Filesystem token grants access only to a specific path, mount that path with `--remote-path`. For example: ```bash -# The token identifies the Filesystem; no profile configuration is needed. -export PATH="$HOME/.ti/bin:$PATH" -mkdir -p "$HOME/workspace" -ti fs mount-file-system --mount-path "$HOME/workspace" --driver fuse -ls "$HOME/workspace" +ti fs mount-file-system \ + --remote-path /workspace \ + --mount-path "$HOME/workspace" ``` -For a token restricted to a subtree, add the matching `--remote-path`, for example `/workspace`. Add `--read-only` for a read-only token. Keep the application and mount under the same OS user. If your application image uses a non-root user, install dependencies while building the image and mount as the application user at runtime. +To prevent writes through the local mount, add `--read-only`. To enforce read-only access at the Filesystem service, use a scoped token with read-only permissions. + +Run the mount and the application that accesses it as the same OS user. If your application image uses a non-root user, install the required packages when building the image and create the mount as the application user at runtime. ## Stop the container safely -Stop application writes, close files, and unmount inside the container: +Before stopping or removing the container: -```bash -# Wait for a successful unmount before destroying the container. -ti fs unmount-file-system --mount-path "$HOME/workspace" && exit -``` +1. Inside the container, stop applications that are writing to the mounted directory and close open files. -The `docker run --rm` container is removed after its shell exits. For Compose, run the following on the host only after the unmount succeeds: +2. Unmount the Filesystem: -```bash -# Remove the container after the mount has stopped cleanly. -docker compose down -``` + ```bash + ti fs unmount-file-system --mount-path "$HOME/workspace" + ``` + + Wait for the unmount to succeed before stopping or removing the container. A successful FUSE unmount flushes pending writes. -The remote Filesystem remains available. Removing the container before pending writes reach the service can lose those writes; an automatic container timeout is not a graceful unmount. +3. Stop the container: + + - If you started it with `docker run --rm -it`, exit the shell after the unmount succeeds: + + ```bash + exit + ``` + + Docker removes the container automatically because it was started with `--rm`. + + - If you used Docker Compose, exit the container shell, and then run the following command on the Docker host: + + ```bash + docker compose down + ``` + +Stopping or removing the container does not delete the remote Filesystem. + +> **Warning:** +> +> Do not remove the container after an unmount error. Pending writes might still exist only inside the container. Keep the container running, resolve the error, and verify that the required files have reached the Filesystem first. ## What's next -- [Share data with the next sandbox](/tidb-cloud-filesystem/filesystem-sharing.md). -- [Restrict tokens to the required paths and operations](/tidb-cloud-filesystem/filesystem-authorization.md). +- [Mount TiDB Cloud Filesystem Locally](/tidb-cloud-filesystem/filesystem-mount.md) for read-only mounts, mounting part of a Filesystem, and safe unmount behavior. +- [Share a TiDB Cloud Filesystem Across Machines](/tidb-cloud-filesystem/filesystem-sharing.md) to give another user or environment access to the same Filesystem. +- [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) to restrict access to specific paths and operations. diff --git a/tidb-cloud-filesystem/filesystem-mount-linux.md b/tidb-cloud-filesystem/filesystem-mount-linux.md index 11a278c1c9a74..9253c12440e3e 100644 --- a/tidb-cloud-filesystem/filesystem-mount-linux.md +++ b/tidb-cloud-filesystem/filesystem-mount-linux.md @@ -1,11 +1,11 @@ --- title: Mount TiDB Cloud Filesystem on Linux -summary: Set up FUSE3 on Linux, mount a TiDB Cloud Filesystem as the application user, and troubleshoot mount permission errors. +summary: Install FUSE userspace tools, mount a TiDB Cloud Filesystem on Linux, verify access, and troubleshoot common mount permission issues. --- # Mount TiDB Cloud Filesystem on Linux -Linux mounts use FUSE to route local filesystem operations to TiDB Cloud Filesystem. Applications can use the mounted directory without adopting a storage SDK. +On Linux, TiDB Cloud Filesystem uses FUSE to make Filesystem data available through a local directory. After mounting, your applications and tools can work with the files by using ordinary local file paths. > **Note:** > @@ -13,89 +13,175 @@ Linux mounts use FUSE to route local filesystem operations to TiDB Cloud Filesys ## Prerequisites -- Install `ti` and select a Filesystem using [a local credential or an FS token](/tidb-cloud-filesystem/filesystem-mount.md#select-a-filesystem). -- Use a Linux host with FUSE support. For a container, follow [Docker and Docker Compose](/tidb-cloud-filesystem/filesystem-mount-docker.md) instead. -- Run the mount and application as the same OS user. +Before you begin: -## Install FUSE3 +- [Install TiDB Cloud CLI (`ti`)](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). +- Make the Filesystem and its token available to `ti`. See [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). +- Use a Linux host where you can install the `fuse3` package. For a container, follow [Docker and Docker Compose](/tidb-cloud-filesystem/filesystem-mount-docker.md) instead. -On Ubuntu or Debian: +Run the mount and the application that uses it as the same OS user. -```bash -# Install the mount helper; this administrative step might require sudo. -sudo apt-get update -sudo apt-get install -y --no-install-recommends fuse3 -``` +## Install FUSE userspace tools -Check the helper and device: +1. On Ubuntu or Debian, install the `fuse3` package: -```bash -# Both the helper and the kernel device must be available. -command -v fusermount3 -ls -l /dev/fuse -``` + ```bash + sudo apt-get update + sudo apt-get install -y --no-install-recommends fuse3 + ``` -On another distribution, install its FUSE3 package. If `/dev/fuse` is missing or inaccessible, have the host administrator enable FUSE and grant the mounting user access. Installing a library alone does not provide a usable mount environment. +2. Check that the FUSE mount helper is installed: -## Mount and verify + ```bash + command -v fusermount3 + ``` -Use an empty directory under your home directory: + The command prints the path to `fusermount3`, such as: -```bash -# Keep the mount owned and used by the current OS user. -mkdir -p "$HOME/workspace" -ti fs mount-file-system --mount-path "$HOME/workspace" --driver fuse -``` + ```text + /usr/bin/fusermount3 + ``` -With a writable token, write a test file and confirm it has reached the remote Filesystem: +3. Check that the FUSE device is available: -```bash -# Use a unique filename so the test does not overwrite an existing file. -TEST_FILE="mount-check-$(date +%s).txt" -printf 'Hello from Linux\n' > "$HOME/workspace/$TEST_FILE" -ti fs drain-file-system --mount-path "$HOME/workspace" --timeout 30s -ti fs read-file --path "/$TEST_FILE" -``` + ```bash + ls -l /dev/fuse + ``` -This example mounts the remote root `/`. If you mounted a subtree, include that remote prefix in the `read-file` path. + The command shows an entry for `/dev/fuse`. -When finished, stop writers and unmount: +If `fusermount3` is not found, make sure the `fuse3` package is installed. If `/dev/fuse` does not exist or you cannot use it when mounting, ask the host administrator to enable FUSE and grant your user access to the device. -```bash -# Stop the mount without deleting the remote Filesystem. -ti fs unmount-file-system --mount-path "$HOME/workspace" -``` +On another Linux distribution, install its FUSE package that provides `fusermount3` and perform the same checks. + +## Mount and verify the Filesystem + +1. Create a directory for the mount: + + ```bash + mkdir -p "$HOME/workspace" + ``` + + Use a directory owned by the same OS user that will run the applications accessing the mount. + +2. Mount the Filesystem: + + ```bash + ti fs mount-file-system --mount-path "$HOME/workspace" + ``` + + On Linux, `ti` uses FUSE by default. + + The command waits until the mount is ready before returning. If it succeeds, you can start using `$HOME/workspace` as a local directory. + +3. Verify that you can access the mounted Filesystem: + + ```bash + ls "$HOME/workspace" + ``` + + The command lists the files and directories at the root of the Filesystem. + + If your token has write permission, you can also create and read a test file: + + ```bash + TEST_FILE="mount-check-$(date +%s).txt" + + printf 'Hello from Linux\n' > "$HOME/workspace/$TEST_FILE" + cat "$HOME/workspace/$TEST_FILE" + ``` + + Example output: + + ```text + Hello from Linux + ``` -## Diagnose permission errors +4. When you are finished, stop applications that are writing to the mounted directory, close open files, and unmount the Filesystem: -If `fusermount3` reports `Permission denied`, check the local mount path, `/dev/fuse` access, and the host's security policy. This error is not necessarily a TiDB Cloud credential failure. + ```bash + ti fs unmount-file-system --mount-path "$HOME/workspace" + ``` -Ubuntu 26.04 can apply an AppArmor profile to `fusermount3` that allows paths under home directories but rejects a top-level path such as `/workspace`. Becoming root or running `chown` on `/workspace` does not bypass that policy. Prefer `$HOME/workspace`; if the application requires another path, see [Ubuntu 26.04 mount-path restrictions](#ubuntu-2604-mount-path-restrictions). + A successful unmount flushes pending FUSE writes before stopping the mount. -A root-created FUSE mount is also not automatically usable by an application running as a different user. Mount as the user that will run the application rather than trying to repair access with `chown` afterward. + If you created the test file above, you can optionally verify after unmounting that the file reached the Filesystem: -For startup failures, inspect the diagnostic log path shown by the CLI. Do not repeatedly start mounts at the same path without checking whether a previous mount is still present. + ```bash + ti fs read-file --path "/$TEST_FILE" + ``` + + Example output: + + ```text + Hello from Linux + ``` + +Unmounting removes the local mount but does not delete the Filesystem or its data. + +If you need pending writes to reach the Filesystem while keeping the mount running, see [Make FUSE writes available without unmounting](/tidb-cloud-filesystem/filesystem-mount.md#make-fuse-writes-available-without-unmounting). + +## Troubleshoot mount permission errors + +If the mount fails with `Permission denied`, check the following in order: + +1. Make sure the mount directory is writable by the current user: + + ```bash + ls -ld "$HOME/workspace" + ``` + +2. Make sure `/dev/fuse` exists: + + ```bash + ls -l /dev/fuse + ``` + +3. Make sure you are creating the mount as the same OS user that will run the application. + + Do not create the FUSE mount as `root` and then try to give another user access by changing the ownership of the mount directory. Instead, create the mount as the application user. + +4. Check whether another mount is already using the same path: + + ```bash + mount | grep "$HOME/workspace" + ``` + + If a mount is listed, unmount it with `ti fs unmount-file-system --mount-path "$HOME/workspace"` or `fusermount3 -u "$HOME/workspace"` before creating a new mount. + +5. If `ti` reports a diagnostic log path, inspect that log for the underlying error. + +On systems with additional security controls, such as AppArmor, the operating system can reject a mount even when the directory permissions look correct. + +For additional mount errors, see [Troubleshoot TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-troubleshooting.md). ## Ubuntu 26.04 mount-path restrictions -Ubuntu 26.04 applies an AppArmor profile to `/usr/bin/fusermount3`. By default, mount under the current user's home directory, `/mnt`, `/media`, `/tmp`, or `/run/user/` instead of `/workspace`. +On Ubuntu 26.04, the AppArmor profile for `fusermount3` can prevent FUSE mounts at some paths. In particular, a top-level directory such as `/workspace` can fail with `Permission denied` even when its file permissions appear correct. + +For the commands in this guide, use `$HOME/workspace` as the mount directory instead of `/workspace`. -If an application requires `/workspace`, ask the host administrator to add these rules to `/etc/apparmor.d/local/fusermount3`: +Changing the owner of `/workspace` or running the mount as `root` does not bypass the AppArmor restriction. + +If your application specifically requires `/workspace`, ask the host administrator to allow that path in `/etc/apparmor.d/local/fusermount3`: ```text mount fstype=@{fuse_types} options=(nosuid,nodev) options in (ro,rw,noatime,dirsync,nodiratime,noexec,sync) -> /workspace/{,**/}, umount /workspace/{,**/}, ``` -The administrator can then reload the profile: +The administrator can then reload the AppArmor profile: ```shell sudo apparmor_parser -r /etc/apparmor.d/fusermount3 ``` -For related errors, see [Troubleshoot TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-troubleshooting.md). +After the profile is updated, retry the mount at `/workspace`. + +For help checking whether AppArmor caused the failure, see [Ubuntu 26.04 rejects a FUSE mount under `/workspace`](/tidb-cloud-filesystem/filesystem-troubleshooting.md#ubuntu-2604-rejects-a-fuse-mount-under-workspace). ## What's next -- [Manage layers and checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md). -- [Share files with another machine](/tidb-cloud-filesystem/filesystem-sharing.md). +- [Mount TiDB Cloud Filesystem Locally](/tidb-cloud-filesystem/filesystem-mount.md) for read-only mounts, mounting part of a Filesystem, and safe unmount behavior. +- [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) to mount and work with layers or checkpoints. +- [Share a TiDB Cloud Filesystem Across Machines](/tidb-cloud-filesystem/filesystem-sharing.md) to give another user or environment access to the Filesystem. diff --git a/tidb-cloud-filesystem/filesystem-mount-macos.md b/tidb-cloud-filesystem/filesystem-mount-macos.md index 0392974f5999a..ca31a0a79de6b 100644 --- a/tidb-cloud-filesystem/filesystem-mount-macos.md +++ b/tidb-cloud-filesystem/filesystem-mount-macos.md @@ -1,11 +1,15 @@ --- title: Mount TiDB Cloud Filesystem on macOS -summary: Mount a TiDB Cloud Filesystem with macOS WebDAV, or choose macFUSE when your workflow needs layers and historical checkpoints. +summary: Mount a TiDB Cloud Filesystem as a local directory on macOS, and use macFUSE when you need advanced mount features. --- # Mount TiDB Cloud Filesystem on macOS -On macOS, `ti` uses WebDAV by default, so you can start with a local directory without installing FUSE. Install macFUSE and explicitly select the FUSE driver when you need layer mounts, checkpoint mounts, or the FUSE drain operation. +On macOS, you can mount a TiDB Cloud Filesystem as a local directory and access its files with your usual applications and tools. + +For most workflows, no additional mount software is required. Without macFUSE, TiDB Cloud CLI (`ti`) uses WebDAV. If macFUSE is installed, `ti` prefers FUSE in automatic driver selection. + +If you need features that require FUSE, such as mounting layers or checkpoints or using `drain-file-system`, install macFUSE and use the FUSE driver instead. > **Note:** > @@ -13,70 +17,131 @@ On macOS, `ti` uses WebDAV by default, so you can start with a local directory w ## Prerequisites -Install `ti` and select a Filesystem using [a local credential or an FS token](/tidb-cloud-filesystem/filesystem-mount.md#select-a-filesystem). Use a writable owner or scoped token for the write examples below. +Before you begin: + +- [Install TiDB Cloud CLI (`ti`)](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). +- Make the Filesystem and its token available to `ti`. See [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). + +The write examples below require a token with write permission. + +## Mount with WebDAV + +To explicitly use WebDAV, pass `--driver webdav`. + +1. Create a local directory for the mount: + + ```bash + mkdir -p "$HOME/workspace" + ``` + +2. Mount the Filesystem: + + ```bash + ti fs mount-file-system \ + --mount-path "$HOME/workspace" \ + --driver webdav + ``` + + This command explicitly selects WebDAV. + +3. Access the Filesystem through the mounted directory: + + ```bash + ls "$HOME/workspace" + ``` + + If your token has write permission, you can also create and read a test file: + + ```bash + TEST_FILE="mount-check-$(date +%s).txt" + + printf 'Hello from macOS\n' > "$HOME/workspace/$TEST_FILE" + cat "$HOME/workspace/$TEST_FILE" + ``` + +4. When you are finished, close files that are open in applications and unmount the Filesystem: + + ```bash + ti fs unmount-file-system --mount-path "$HOME/workspace" + ``` + + WebDAV does not support `drain-file-system`. Complete a normal unmount before shutting down the machine or handing updated files to another user or environment. + + If you created the test file above, you can optionally confirm that it is available directly from the Filesystem: + + ```bash + ti fs read-file --path "/$TEST_FILE" + ``` + +Unmounting removes the local mount but does not delete the Filesystem or its data. + +## Use macFUSE when you need FUSE features + +Use FUSE instead of WebDAV when you need to: + +- mount a layer or checkpoint, or +- make pending writes reach the Filesystem while keeping the mount running with `drain-file-system`. + +To use FUSE on macOS: + +1. Install [macFUSE](https://macfuse.github.io/) and complete any installation or security approval steps required by your macOS version. + + Installing `ti` does not install macFUSE. + +2. Prepare the local mount directory: -## Mount with the default WebDAV driver + - If a mount is already active at `$HOME/workspace`, unmount it before reusing the same directory: -```bash -# Mount under your home directory, not at a root-level path such as /workspace. -mkdir -p "$HOME/workspace" -ti fs mount-file-system --mount-path "$HOME/workspace" -``` + ```bash + ti fs unmount-file-system --mount-path "$HOME/workspace" + ``` -The successful result identifies the `webdav` driver. You can explicitly request the same mode with `--driver webdav`. + - Ensure that `$HOME/workspace` exists: -```bash -# Verify a write through the local mount. -TEST_FILE="mount-check-$(date +%s).txt" -printf 'Hello from macOS\n' > "$HOME/workspace/$TEST_FILE" -cat "$HOME/workspace/$TEST_FILE" -``` + ```bash + mkdir -p "$HOME/workspace" + ``` -Before a handoff or shutdown, close application files and unmount: +3. Mount the Filesystem with FUSE: -```bash -# WebDAV has no drain command; finish file operations and unmount normally. -ti fs unmount-file-system --mount-path "$HOME/workspace" -ti fs read-file --path "/$TEST_FILE" -``` + ```bash + ti fs mount-file-system \ + --mount-path "$HOME/workspace" \ + --driver fuse + ``` -The final command reads from the remote service, independently of the mount. WebDAV and FUSE are different filesystem interfaces; do not assume WebDAV has every FUSE or POSIX capability. + Layer and checkpoint mounts require FUSE, and checkpoint mounts are always read-only. For details, see [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md). -## Use macFUSE for layers and checkpoints + If you need pending writes to reach the Filesystem while keeping the mount running, see [Make FUSE writes available without unmounting](/tidb-cloud-filesystem/filesystem-mount.md#make-fuse-writes-available-without-unmounting). -Install [macFUSE](https://macfuse.github.io/) and complete the installation and security approvals required for your macOS version. Installing `ti` does not install macFUSE. Follow the macFUSE installation guidance for your system; do not assume every macFUSE backend is supported by the bundled Filesystem runtime. +4. When you are finished, stop applications from writing to the mount, close open files, and unmount it: -Unmount an existing WebDAV mount before reusing its directory with FUSE: + ```bash + ti fs unmount-file-system --mount-path "$HOME/workspace" + ``` -```bash -# Select FUSE explicitly after installing macFUSE. -ti fs mount-file-system --mount-path "$HOME/workspace" --driver fuse -``` + A successful FUSE unmount flushes pending writes. You do not need to run `drain-file-system` before a normal unmount. -To keep a FUSE mount online while waiting for pending writes to reach the service: +## Troubleshoot mount issues -```bash -# Use this durability barrier before a layer checkpoint or cross-machine handoff. -ti fs drain-file-system --mount-path "$HOME/workspace" --timeout 30s -``` +If a WebDAV mount fails to start: -For layer and checkpoint mount options, see the [`mount-file-system` command reference](/ai/ti/reference/ti-fs-mount-file-system.md). Layer and checkpoint mounts cannot run through WebDAV. +- Make sure the local mount directory exists and is writable. +- Check the diagnostic log path reported by `ti` for the underlying error. +- Make sure another mount is not already using the same local directory. -When finished with the FUSE mount, stop writers and unmount it: +If a FUSE mount fails to start: -```bash -# Release the mount after pending writes have been flushed. -ti fs unmount-file-system --mount-path "$HOME/workspace" -``` +- Make sure macFUSE is installed. +- Complete any macOS security approvals required by macFUSE. +- Check the diagnostic log path reported by `ti`. -## Troubleshoot startup +If unmounting fails, keep the mount process and machine running until you resolve the error and verify that required files have reached the Filesystem. Do not remove local mount data while pending writes might remain. -- Use an empty directory you can write to, such as `$HOME/workspace`. A root-level path might not be writable on macOS. -- For FUSE, verify that macFUSE installation and required approvals are complete. -- If startup fails, inspect the diagnostic log path in the CLI error. The top-level background-process error does not by itself identify the cause. -- If unmount fails, keep the process, local cache, and machine available until you resolve the failure and verify remote data. +For additional mount errors, see [Troubleshoot TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-troubleshooting.md). ## What's next -- [Understand mount lifecycle and durability](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). -- [Share a read-only workspace](/tidb-cloud-filesystem/filesystem-sharing.md). +- [Mount TiDB Cloud Filesystem Locally](/tidb-cloud-filesystem/filesystem-mount.md) for read-only mounts, mounting part of a Filesystem, and safe unmount behavior. +- [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) to work with layers and checkpoints through FUSE. +- [Share a TiDB Cloud Filesystem Across Machines](/tidb-cloud-filesystem/filesystem-sharing.md) to give another user or environment access to the Filesystem. diff --git a/tidb-cloud-filesystem/filesystem-mount.md b/tidb-cloud-filesystem/filesystem-mount.md index d9ea320816bab..9347bd5f6a1b9 100644 --- a/tidb-cloud-filesystem/filesystem-mount.md +++ b/tidb-cloud-filesystem/filesystem-mount.md @@ -1,114 +1,147 @@ --- title: Mount TiDB Cloud Filesystem Locally -summary: Select a Filesystem and mount driver, access remote files from a local directory, and stop a mount without losing pending writes. +summary: Mount an existing TiDB Cloud Filesystem as a local directory, use its files with local tools, and unmount it safely. aliases: ['/ai/mount-filesystem'] --- # Mount TiDB Cloud Filesystem Locally -A mount makes remote files available at a local directory. Use it when your editor, application, or agent expects filesystem paths instead of file-transfer commands. The remote Filesystem persists independently of the mount process. +Mount a TiDB Cloud Filesystem when your editor, application, or agent needs to access Filesystem data through local file paths. + +After mounting, you can use ordinary local tools to read and write files in the mounted directory. Unmounting removes the local access point but does not delete the Filesystem or its data. > **Note:** > > TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. -## Choose your environment +## Before you begin -- [Linux](/tidb-cloud-filesystem/filesystem-mount-linux.md): use FUSE3 and an accessible `/dev/fuse` device. -- [macOS](/tidb-cloud-filesystem/filesystem-mount-macos.md): use the default WebDAV driver, or install macFUSE and explicitly select FUSE for layers and checkpoints. -- [Docker and Docker Compose](/tidb-cloud-filesystem/filesystem-mount-docker.md): expose the Linux host's FUSE device and allow mounting inside the container. +Before mounting a Filesystem: -Native Windows mounting is not supported by `ti`. Use direct commands such as `ti fs copy-file`, `ti fs read-file`, and `ti fs list-files` instead. +- [Install TiDB Cloud CLI (`ti`)](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). +- Make the Filesystem and its token available to `ti`. See [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). +- Complete the mount setup for your environment: [Linux](/tidb-cloud-filesystem/filesystem-mount-linux.md), [macOS](/tidb-cloud-filesystem/filesystem-mount-macos.md), or [Docker and Docker Compose](/tidb-cloud-filesystem/filesystem-mount-docker.md). -With `--driver auto`, the CLI selects WebDAV on macOS and FUSE on Linux. To mount a layer or checkpoint on macOS, install macFUSE and select `--driver fuse`. WebDAV mounting is not supported on Linux. +Native Filesystem mounting is not supported on Windows. On Windows, use direct commands such as `ti fs copy-file`, `ti fs read-file`, and `ti fs list-files` instead. -| Capability | FUSE | WebDAV | -| --- | --- | --- | -| Supported platforms | Linux; macOS with macFUSE | macOS | -| Layer and checkpoint mounts | Supported | Not supported | -| Drain pending writes without unmounting | Supported | Not supported; close files and unmount normally | -| Read-only mount with `--read-only` | Supported | Supported | +You normally do not need to choose a mount driver manually. `ti` uses FUSE on Linux. On macOS without macFUSE, `ti` uses WebDAV; if macFUSE is installed, `ti` prefers FUSE in automatic driver selection. On macOS, use FUSE when you need layers, checkpoints, or the drain operation. -The two drivers do not provide identical filesystem behavior. Use the platform guides to choose a driver rather than assuming full POSIX feature parity. +## Mount and use the Filesystem -## Select a Filesystem +To mount the Filesystem: -### Use a locally stored token +1. Create a local directory for the mount: -After creating a Filesystem with the CLI, select its ID: + ```bash + mkdir -p "$HOME/workspace" + ``` -```bash -# Select the Filesystem whose token is already stored locally. -export TI_FS_FILE_SYSTEM_ID="" -``` +2. Mount the Filesystem: -The CLI uses the selected resource's stored token and region information. You can pass `--file-system-id` on each command instead. There is no default Filesystem selected merely because only one resource exists. + ```bash + ti fs mount-file-system --mount-path "$HOME/workspace" + ``` -### Use a token without configuring a profile + The command waits until the mount is ready before returning. The mount continues running in the background, so closing the terminal does not unmount it. -On a machine with `ti` installed, provide the token through a secret manager or the environment: + If the mount fails to start, check the diagnostic log path reported by the CLI. -```bash -# Inject these values from your secret manager in automation. -export TI_FS_TOKEN="" -export TI_REGION_CODE="aws-us-east-1" -``` +3. Access the Filesystem through the mounted directory using ordinary local tools: + + ```bash + ls "$HOME/workspace" + + printf 'Written through the mount\n' > "$HOME/workspace/mounted.txt" -Set the region to where this Filesystem was created. `ti` derives the Filesystem ID from the token; neither `ti configure` nor `TI_FS_FILE_SYSTEM_ID` is required. A supplied ID must match the token. If your shell already has an ID for another Filesystem, clear it before using the token-only workflow. + cat "$HOME/workspace/mounted.txt" + ``` -Use an owner token for full Filesystem access, or a scoped token with the operations required by the application. For a token restricted to `/workspace`, mount that subtree rather than `/`. See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md). + The write example requires a token with write permission and a writable mount. -## Mount and use the files +If you use FUSE, run the mount and the applications that access it as the same OS user. Changing the ownership of the mount directory does not give another user access through an existing FUSE mount. -After installing the platform dependencies, mount the selected Filesystem: +### Mount only part of the Filesystem + +By default, the Filesystem root `/` is mounted. + +To mount only a specific directory, use `--remote-path`. For example: ```bash -# Use an empty directory owned by the user who will access the mount. -mkdir -p "$HOME/workspace" -ti fs mount-file-system --mount-path "$HOME/workspace" +ti fs mount-file-system \ + --remote-path /workspace \ + --mount-path "$HOME/workspace" ``` -The command waits for readiness and returns a structured result with `status: mounted`. The bundled `ti-drive9` companion keeps the mount alive in the background. Closing the terminal does not unmount it, but terminating that process or the machine interrupts access. If startup fails, inspect the diagnostic log path reported by the CLI. +In this example, the remote `/workspace` directory becomes the root of the local mount. For example, `/workspace/project.md` is available locally as `$HOME/workspace/project.md`. -To expose only one remote directory, pass `--remote-path /workspace`. To make a mount read-only, add `--read-only`. These are client-side mount settings, not substitutes for a scoped token's server-enforced permissions. +If you use a scoped token that grants access only to a specific path, mount that path rather than the Filesystem root. -To mount a layer or a read-only checkpoint, select the FUSE driver and use the layer options in the [`mount-file-system` command reference](/ai/ti/reference/ti-fs-mount-file-system.md). +### Create a read-only mount + +To prevent writes through the local mount, add `--read-only`: ```bash -# These are ordinary local filesystem commands, not CLI subcommands. -ls "$HOME/workspace" -printf 'Written through the mount\n' > "$HOME/workspace/mounted.txt" -cat "$HOME/workspace/mounted.txt" +ti fs mount-file-system \ + --mount-path "$HOME/workspace" \ + --read-only ``` -The write example requires a writable mount and token. Use the same OS user for mounting and file access; changing a directory's ownership does not grant another user access through an existing FUSE mount. +The `--read-only` option prevents writes through this mount, but it does not change the permissions of the Filesystem token. To enforce read-only access at the service, use a scoped token that grants only the required read access. + +### Mount a layer or checkpoint + +Layer and checkpoint mounts require FUSE. On macOS, install macFUSE and specify `--driver fuse`. + +Checkpoint mounts are always read-only. + +For layer and checkpoint workflows, see [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md). For all mount options, see the [`mount-file-system` command reference](/ai/ti/reference/ti-fs-mount-file-system.md). ## Finish safely -Stop applications writing to the mount and close their files. If you need pending writes to reach the service while keeping a FUSE mount online, such as before creating a checkpoint or handing work to another machine, drain it: +In most cases, simply stop writing and unmount the Filesystem. You only need to drain a FUSE mount when you want pending writes to reach the Filesystem while keeping the mount running. -```bash -# FUSE only: wait for pending remote writes without unmounting. -ti fs drain-file-system --mount-path "$HOME/workspace" --timeout 30s -``` +### Unmount when you are finished -The timeout is how long the CLI waits for dirty handles and pending writes to drain. If the command times out or returns an error, it has not confirmed that all writes reached the service. Keep the mount and machine available, resolve the error, and verify remote data before ending the session. +When you no longer need the mount: -When finished, unmount: +1. Stop applications from writing to the mounted directory and close any open files. -```bash -# Graceful unmount flushes pending FUSE work before stopping the mount. -ti fs unmount-file-system --mount-path "$HOME/workspace" -``` +2. Unmount the Filesystem: + + ```bash + ti fs unmount-file-system --mount-path "$HOME/workspace" + ``` + +A successful FUSE unmount flushes pending writes before stopping the mount. You do not need to run `drain-file-system` before a normal unmount. + +For a WebDAV mount, close open files and unmount normally. WebDAV does not support the drain operation. + +Unmounting removes the local mount but does not delete the Filesystem or its data. + +### Make FUSE writes available without unmounting + +If you want to keep a FUSE mount running but need pending writes to reach the Filesystem—for example, before creating a checkpoint or sharing updated files with another machine: + +1. Stop applications from writing to the relevant files and close those files. + +2. Drain the mount: + + ```bash + ti fs drain-file-system \ + --mount-path "$HOME/workspace" \ + --timeout 30s + ``` + +A successful drain confirms that pending writes have reached the service while leaving the mount running. -An explicit drain is not required before every normal unmount. WebDAV does not support drain: close application files and complete a normal unmount before a handoff. Drain and unmount use the local mount locator and do not require you to provide the region or token again. The running mount still needs valid credentials to finish remote work. +If the drain times out or returns an error, some writes might not have reached the Filesystem. Keep the mount and machine available, resolve the error, and verify the files before ending the session or telling another user that the updates are ready. > **Warning:** > -> Do not destroy a sandbox, stop its mount process, or delete its local cache after a drain or unmount error. Pending writes might exist only on that machine. Resolve the error and verify remote data before removing the environment. Unmounting does not delete the remote Filesystem. +> If a drain or unmount fails, do not shut down or destroy the machine or sandbox, stop the mount process, or delete its local data. Some pending writes might still exist only on that machine. Resolve the error and verify that the required files have reached the Filesystem first. ## What's next -- [Share a Filesystem across environments](/tidb-cloud-filesystem/filesystem-sharing.md). -- [Manage layers and checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md). -- [Look up mount options](/ai/ti/reference/ti-fs-mount-file-system.md). +- [Share a TiDB Cloud Filesystem Across Machines](/tidb-cloud-filesystem/filesystem-sharing.md) to give another user or environment access to the same Filesystem. +- [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) to make and review isolated changes before applying them to the base Filesystem. +- See the [`mount-file-system` command reference](/ai/ti/reference/ti-fs-mount-file-system.md) for all mount options. diff --git a/tidb-cloud-filesystem/filesystem-regions-and-limitations.md b/tidb-cloud-filesystem/filesystem-regions-and-limitations.md index fc477e36834bc..cd9c5c694a75d 100644 --- a/tidb-cloud-filesystem/filesystem-regions-and-limitations.md +++ b/tidb-cloud-filesystem/filesystem-regions-and-limitations.md @@ -28,8 +28,8 @@ Filesystem list and describe commands query only the selected region. They do no | Platform | Filesystem mount | Vault mount | Requirements and alternatives | | --- | --- | --- | --- | -| macOS | WebDAV by default; FUSE with explicit `--driver fuse` | FUSE | The built-in WebDAV helper supports Filesystem mounts. Install macFUSE and approve its system extension for FUSE or Vault mounts. | -| Linux | FUSE | FUSE | Install FUSE3 and provide access to `/dev/fuse`. WebDAV mounting is not supported. | +| macOS | WebDAV without macFUSE; FUSE when macFUSE is installed (automatic or explicit) | FUSE | The built-in WebDAV helper supports Filesystem mounts. Install macFUSE and approve its system extension for FUSE or Vault mounts. | +| Linux | FUSE | FUSE | Install the `fuse3` package and provide access to `/dev/fuse`. WebDAV mounting is not supported. | | Windows | Not supported | Not supported | Use `ti fs` data-plane commands and non-mount Vault commands instead. | FUSE and WebDAV are implemented by the bundled [Drive9](https://github.com/mem9-ai/drive9) companion. The CLI does not fall back to a separate native mount implementation. A running mount keeps the companion version loaded when it started; unmount and remount after updating `ti`. diff --git a/tidb-cloud-filesystem/filesystem-sharing.md b/tidb-cloud-filesystem/filesystem-sharing.md index 39804c55113b2..297b03fd93fb5 100644 --- a/tidb-cloud-filesystem/filesystem-sharing.md +++ b/tidb-cloud-filesystem/filesystem-sharing.md @@ -1,114 +1,194 @@ --- -title: Share a TiDB Cloud Filesystem -summary: Share a remote workspace across machines and sandboxes with separate access tokens, and hand off files after writes reach the service. +title: Share a TiDB Cloud Filesystem Across Machines +summary: Learn how to share part of an existing TiDB Cloud Filesystem with another user, machine, CI job, or agent, and remove that access when it is no longer needed. +aliases: ['/ai/ti-share-filesystem-across-machines-example'] --- -# Share a TiDB Cloud Filesystem +# Share a TiDB Cloud Filesystem Across Machines -Sharing a Filesystem gives participants access to the same remote namespace, not independent copies. For example, an agent can produce a report in a sandbox and a reviewer can open that report from a laptop without downloading and redistributing an archive for every revision. +You can share files in a TiDB Cloud Filesystem with another user, machine, CI job, or agent without copying the files between environments. + +Create a separate scoped token for each user or environment you want to share with. Each token can limit access to specific paths and actions, so you can give only the access that is needed and revoke it later without affecting anyone else. > **Note:** > > TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. -## What each participant needs +## Prerequisites -The trusted machine uses TiDB Cloud API keys to create the Filesystem. Other participants need `ti`, an FS token, and the Filesystem region. They do not need a copy of `~/.ti/`, account API keys, or their own Filesystem resource. +Before you begin: -Use separate tokens so you can retire one participant's access without changing every participant's credentials. Choose a scoped token for a restricted task; an owner token grants broad Filesystem access. See [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md). +- Have access to an existing TiDB Cloud Filesystem. +- On a machine you trust, have an owner token for the Filesystem. You need an owner token to create scoped tokens. +- [Install TiDB Cloud CLI (`ti`)](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli) on the machine or environment that needs access to the shared Filesystem. +- Have a secure way, such as a secret manager, to transfer Filesystem tokens. -## Prepare the workspace on machine A +For information about owner and scoped tokens, see [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md). -With `ti` installed and configured, create a Filesystem or use an existing one. This example creates a new resource: +## Give access to specific files -```bash -# Retain the ID; the creator's token is stored locally by the CLI. -FILE_SYSTEM_ID="$(ti fs create-file-system \ - --display-name shared-reports --wait \ - --query file_system_id --output text)" -``` +To share part of a Filesystem with another user or environment: -Create a directory and publish the first report: +1. Decide which paths they need to access and what they need to do with those paths. -```bash -# Write through the direct file interface, without a local mount. -ti fs create-directory --file-system-id "$FILE_SYSTEM_ID" --path /reports -printf 'The first report is ready for review.\n' | ti fs copy-file \ - --file-system-id "$FILE_SYSTEM_ID" \ - --from-stdin --to-remote /reports/summary.txt -``` +2. If the path you want to share does not already exist, create it. The following example creates `/reports`: -Issue a read-only token for the reviewer: + ```bash + ti fs create-directory \ + --file-system-id "" \ + --path /reports + ``` -```bash -# Keep this value in a secret manager, not in a shared log. -REVIEW_TOKEN="$(ti fs generate-file-system-scoped-token \ - --file-system-id "$FILE_SYSTEM_ID" \ - --subject reviewer --ttl 24h \ - --allow /reports:read,list \ - --query fs_token --output text)" -``` +3. Create a scoped token for the user or environment. -Deliver `REVIEW_TOKEN` and the Filesystem's region code securely to machine B. Retain the Filesystem ID on machine A for administration. The token expires after the requested lifetime; a saved environment variable does not extend it. + The following example gives a reviewer read-only access to `/reports` for 24 hours: -## Open the report on machine B + ```bash + REVIEW_TOKEN="$(ti fs generate-file-system-scoped-token \ + --file-system-id "" \ + --subject reviewer \ + --ttl 24h \ + --allow /reports:read,list \ + --query fs_token \ + --output text)" + ``` -Inject the reviewer's token and matching region into the environment: + This token lets the reviewer read and list files under `/reports`, but does not give access to other paths in the Filesystem. -```bash -# In production, inject the token from a secret manager instead of pasting it into a shell. -# No ti configure is needed on the receiving machine. -export TI_FS_TOKEN="" -export TI_REGION_CODE="" -ti fs read-file --path /reports/summary.txt -``` + If the reviewer also needs to add or update files, include `write` for that path. For example, `--allow /reports:read,list,write` lets the reviewer read, list, and write files under `/reports` without giving them owner access to the Filesystem. + +4. Send the token and the Filesystem region code through a secure channel or secret manager. + + The other user or environment does not need your TiDB Cloud API credentials or a copy of your local `~/.ti/` directory. + + The token remains valid until it expires or you revoke it. Saving the token in an environment variable does not extend its lifetime. + +> **Note:** +> +> The token value is shown only when the token is created. Treat it as a secret and do not expose it in logs, issues, chat messages, or source control. + +For more information about token permissions and expiration, see [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md). + +## Access the shared files from another machine + +On the machine or environment that needs access: + +1. Set the scoped token and Filesystem region: + + ```bash + export TI_FS_TOKEN="" + export TI_REGION_CODE="" + ``` + + The token identifies the Filesystem, so you do not need to provide the Filesystem ID. + +2. Verify that you can access the shared path: + + ```bash + ti fs list-files --path /reports + ti fs read-file --path /reports/summary.txt + ``` + + The token can be used only for the paths and actions included in its scope. Other access is rejected by the Filesystem service. + +For other ways to access an existing Filesystem, see [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). + +### Mount the shared directory (optional) -On macOS or Linux with the [mount dependencies](/tidb-cloud-filesystem/filesystem-mount.md#choose-your-environment), expose the allowed directory locally: +On a supported platform, you can also mount the shared directory and access its files through a local path: ```bash -# Mount only the scope allowed by the reviewer token. mkdir -p "$HOME/reports" + ti fs mount-file-system \ --remote-path /reports \ --mount-path "$HOME/reports" \ --read-only -cat "$HOME/reports/summary.txt" ``` -The remote `/reports` prefix becomes the local mount root, so the local file is `$HOME/reports/summary.txt`, not `$HOME/reports/reports/summary.txt`. +The remote `/reports` directory becomes the root of the local mount. For example, `/reports/summary.txt` is available locally at: -The token enforces read-only access at the service. `--read-only` also tells the local mount to reject writes; using that flag with an owner token alone would not restrict the owner's other API or CLI access. +```text +$HOME/reports/summary.txt +``` -## Hand off new data safely +The scoped token limits what you can do in the Filesystem. The `--read-only` option also prevents writes through this local mount. -A successful write to a FUSE-mounted file might still be buffered on the producing machine. Before telling the reviewer that a revision is ready, stop the application's writes and drain its FUSE mount, or unmount it successfully. For WebDAV, close application files and finish a normal unmount. See [Finish safely](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). +Direct `ti fs` commands and mounts access the same files in the Filesystem. For example, a file uploaded with `ti fs copy-file` is also available through a mount. Changes made through a mount become available to direct commands and other users after the writes reach the service. -Use a direct remote read to verify a handoff independently of another mount's cache. Existing open handles and client caches can retain older content; do not assume every reader instantly sees each local write. +For mount requirements and platform-specific setup, see [Mount TiDB Cloud Filesystem Locally](/tidb-cloud-filesystem/filesystem-mount.md). -Coordinate writers to the same path. Shared storage is not a distributed lock or an automatic merge system. Use separate paths or [layers](/tidb-cloud-filesystem/filesystem-layers-checkpoints.md) for independent drafts, and publish only after review. +## Make sure updates are ready to share -## End access without deleting the workspace +When multiple users or environments access the same Filesystem, they work with the same files rather than separate copies. -On machine B, stop readers and unmount: +If files are written through a mount, make sure the latest changes have reached the Filesystem before telling someone else that they are ready. -```bash -# Remove the local mount, not the shared remote data. -ti fs unmount-file-system --mount-path "$HOME/reports" -unset TI_FS_TOKEN TI_REGION_CODE -``` +For a FUSE mount: -On machine A, identify and revoke the reviewer's token when no longer needed: +1. Stop applications from writing to the files and close any files that are still open. -```bash -# Find the reviewer token ID in the metadata, then revoke that token only. -ti fs list-file-system-tokens --file-system-id "$FILE_SYSTEM_ID" --output text -ti fs delete-file-system-token \ - --file-system-id "$FILE_SYSTEM_ID" --token-id "" -``` +2. Make sure pending writes reach the Filesystem: + + - To keep the mount running, drain it. + - If you are finished with the mount, unmount it successfully. + +3. If you need to verify the handoff, read the updated file directly from the Filesystem: + + ```bash + ti fs read-file --path /reports/summary.txt + ``` + +For a WebDAV mount, close open files and unmount normally. WebDAV does not support drain. See [Finish safely](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). + +If multiple users or environments have write access, avoid writing to the same files at the same time. TiDB Cloud Filesystem does not automatically merge conflicting changes. + +If different users or workflows need to make changes independently before applying them to the base Filesystem, see [Layers and Checkpoints](/tidb-cloud-filesystem/filesystem-layers-checkpoints.md). + +## Stop sharing access + +### On the machine using the shared Filesystem + +1. Stop applications that use the shared files. + +2. If the Filesystem is mounted, unmount it: + + ```bash + ti fs unmount-file-system --mount-path "$HOME/reports" + ``` + +3. Remove the token from the local environment: + + ```bash + unset TI_FS_TOKEN TI_REGION_CODE + ``` + +Removing the token from the local environment prevents that environment from using the saved value, but does not revoke the token itself. Anyone who still has the token can continue using it until it expires or is revoked. + +### On the machine where you manage the Filesystem + +1. Find the token you want to revoke: + + ```bash + ti fs list-file-system-tokens \ + --file-system-id "" \ + --output text + ``` + +2. Revoke that token: + + ```bash + ti fs delete-file-system-token \ + --file-system-id "" \ + --token-id "" + ``` + +Revoking one token removes that user's or environment's access without affecting other tokens or deleting the Filesystem. -Do not delete the Filesystem to disconnect one participant: resource deletion affects everyone and removes the shared data. +Do not delete the Filesystem just to stop sharing it with one user or environment. Deleting the Filesystem removes the shared Filesystem and its data for everyone. ## What's next -- [Understand layers and checkpoints for independent drafts](/tidb-cloud-filesystem/filesystem-layers-checkpoints.md). -- [Run the agent sandbox example](/ai/ti/guides/ti-agent-sandbox-example.md). +- [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) to create, rotate, disable, or revoke tokens. +- [TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/filesystem-layers-checkpoints.md) to make changes independently before applying them to the base Filesystem. +- [Automation and AI Agent Workflows](/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md) for workflows that use shared Filesystem data. diff --git a/tidb-cloud-filesystem/filesystem-troubleshooting.md b/tidb-cloud-filesystem/filesystem-troubleshooting.md index 8b430f67181b2..d73b6f03aaf21 100644 --- a/tidb-cloud-filesystem/filesystem-troubleshooting.md +++ b/tidb-cloud-filesystem/filesystem-troubleshooting.md @@ -125,7 +125,7 @@ Background mount success prints the CLI result without Drive9 startup messages. - FUSE prerequisites or the WebDAV helper are installed; - the remote region is reachable. -macOS defaults to WebDAV. To request FUSE after installing macFUSE: +On macOS without macFUSE, `ti` uses WebDAV. If macFUSE is installed, automatic driver selection prefers FUSE. To explicitly request FUSE: ```bash ti fs mount-file-system \ @@ -133,7 +133,7 @@ ti fs mount-file-system \ --driver fuse ``` -Linux needs FUSE3 and access to `/dev/fuse`. Filesystem and Vault mounts are not supported on Windows; use `ti fs` data-plane commands or non-mount Vault commands instead. +Linux needs FUSE support, the `fuse3` package, and access to `/dev/fuse`. Filesystem and Vault mounts are not supported on Windows; use `ti fs` data-plane commands or non-mount Vault commands instead. ## Ubuntu 26.04 rejects a FUSE mount under `/workspace` diff --git a/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md b/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md index e68cd2130c0b7..503462d75e97f 100644 --- a/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md +++ b/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md @@ -13,7 +13,6 @@ To share work across machines and CI jobs, see the following guides in the **TiD | What you want to do | Guide | | --- | --- | -| Access the same workspace from multiple machines | [Share a TiDB Cloud Filesystem Across Machines](/ai/ti/guides/ti-share-filesystem-across-machines-example.md) | | Pass build artifacts between isolated CI jobs | [Hand Off CI Artifacts Between Isolated Jobs with TiDB Cloud Filesystem](/ai/ti/guides/ti-ci-artifact-handoff-example.md) | To support AI agent workflows, see the following guides in the **TiDB for AI** documentation. If you are evaluating Filesystem for agents, start with the sandbox example before moving to longer-running or parallel workflows. diff --git a/tidb-cloud-filesystem/work-with-filesystem-data.md b/tidb-cloud-filesystem/work-with-filesystem-data.md index 12e6e30f327d7..f7aed56fac2a6 100644 --- a/tidb-cloud-filesystem/work-with-filesystem-data.md +++ b/tidb-cloud-filesystem/work-with-filesystem-data.md @@ -1,82 +1,135 @@ --- title: Work with Files and Directories in TiDB Cloud Filesystem -summary: Learn how to copy, read, organize, search, and inspect files and directories in TiDB Cloud Filesystem by using the CLI. +summary: Learn how to upload, download, read, organize, inspect, and search files and directories in TiDB Cloud Filesystem. aliases: ['/ai/work-with-filesystem-data'] --- # Work with Files and Directories in TiDB Cloud Filesystem -In TiDB Cloud CLI, you can use [`ti fs` commands](/ai/ti/reference/ti-filesystem.md) to transfer data between local storage and TiDB Cloud Filesystem and to manage its remote namespace. +You can work with files and directories in a TiDB Cloud Filesystem directly from TiDB Cloud CLI (`ti`) without mounting the Filesystem. + +Use `ti fs` commands to upload and download files, read file contents, list and inspect paths, organize files and directories, and search for data. + +For the complete command reference, see [`ti fs`](/ai/ti/reference/ti-filesystem.md). ## Prerequisites +Before you begin: + - [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). - [Create a Filesystem](/tidb-cloud-filesystem/manage-filesystem-resources.md) or obtain access to an existing one. - Select the Filesystem and make its token available to `ti`. For available access options, see [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). -## Copy data +## Upload and download files + +Upload a local file to the Filesystem: -Upload a local file to a remote path: +```shell +ti fs copy-file \ + --from-local ./report.md \ + --to-remote /reports/report.md +``` + +Download a file from the Filesystem: ```shell -ti fs copy-file --from-local ./report.md --to-remote /reports/report.md +ti fs copy-file \ + --from-remote /reports/report.md \ + --to-local ./downloads/report.md \ + --create-parents ``` -[`copy-file`](/ai/ti/reference/ti-fs-copy-file.md) also supports downloads, streaming, appending, resuming, and recursive copies. +You can also use `copy-file` to copy files or directories within the Filesystem, stream data through standard input or output, append to a file, or resume an interrupted transfer. + +To copy a directory recursively, use `--recursive`. For all supported copy operations and options, see the [`copy-file` reference](/ai/ti/reference/ti-fs-copy-file.md). -## Read and inspect data +## Read and inspect files and directories -Read a file or byte range to standard output: +Read the complete contents of a file: ```shell -ti fs read-file --path /reports/report.md --offset 0 --length 1024 +ti fs read-file --path /reports/report.md ``` -List a directory and inspect one path: +To read only part of a file, use `--offset` and `--length`. For example, the following command reads the first 1024 bytes: + +```shell +ti fs read-file \ + --path /reports/report.md \ + --offset 0 \ + --length 1024 +``` + +List the contents of a directory: ```shell ti fs list-files --path /reports --output text +``` + +Inspect metadata for a file or directory: + +```shell ti fs describe-file --path /reports/report.md ``` -## Organize the namespace +## Organize files and directories -Create a directory, move a file, and remove data with the corresponding commands: +Create a directory: ```shell ti fs create-directory --path /reports/archive -ti fs move-file --from-remote /draft.md --to-remote /reports/final.md -ti fs delete-file --path /scratch --recursive ``` -You can also use [`chmod-file`](/ai/ti/reference/ti-fs-chmod-file.md), [`create-symlink`](/ai/ti/reference/ti-fs-create-symlink.md), and [`create-hardlink`](/ai/ti/reference/ti-fs-create-hardlink.md) to manage POSIX-style metadata and links. +Move a file to another path: + +```shell +ti fs move-file \ + --from-remote /draft.md \ + --to-remote /reports/final.md +``` + +Delete a file or directory: + +```shell +ti fs delete-file --path /scratch --recursive +``` > **Warning:** > -> `delete-file --recursive` permanently deletes the target directory and its contents. Verify the remote path before you run the command. +> `delete-file --recursive` permanently deletes the specified directory and its contents. Verify the path before deleting it. You can use `--dry-run` to validate the request without deleting data. + +For workflows that need POSIX-style metadata or links, you can also use [`chmod-file`](/ai/ti/reference/ti-fs-chmod-file.md), [`create-symlink`](/ai/ti/reference/ti-fs-create-symlink.md), and [`create-hardlink`](/ai/ti/reference/ti-fs-create-hardlink.md). -## Search for data +## Search files and content -Search file content below a path: +Use `search-file-content` when you want to find files based on their content: ```shell -ti fs search-file-content --path /reports --pattern "TODO" +ti fs search-file-content \ + --path /reports \ + --pattern "TODO" ``` -`--pattern` is a text query, not a regular expression or glob. Use `--limit` to cap the number of results; `0` uses the service default. See the [`search-file-content` reference](/ai/ti/reference/ti-fs-search-file-content.md). +`--pattern` is a text query, not a regular expression or glob. Use `--limit` to control the maximum number of results. -Find paths by name, type, tags, size, or timestamps: +For details, see the [`search-file-content` reference](/ai/ti/reference/ti-fs-search-file-content.md). + +If you know something about the file itself rather than its contents, use `find-files`. You can filter by file name, type, tags, size, or modification time. + +For example, find Markdown files tagged `stage=review`: ```shell -ti fs find-files --path /reports --file-name-pattern "*.md" --tag stage=review +ti fs find-files \ + --path /reports \ + --file-name-pattern "*.md" \ + --tag stage=review ``` -`find-files` searches beneath the specified path and can filter by file name, type, tags, size, or modification time. See its [options and result limit](/ai/ti/reference/ti-fs-find-files.md). - -For supported regions and current Filesystem and platform limitations, see [Regions and Limitations](/tidb-cloud-filesystem/filesystem-regions-and-limitations.md). +For all available filters, see the [`find-files` reference](/ai/ti/reference/ti-fs-find-files.md). ## What's next -- [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) -- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) -- [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) +- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) to work with Filesystem data through a local directory and existing local tools. +- [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) to make and review isolated changes before applying them to the base Filesystem. +- [Share a TiDB Cloud Filesystem Across Machines](/tidb-cloud-filesystem/filesystem-sharing.md) to give another machine or environment access to the Filesystem. +- [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) for complete command syntax and options. From 1ba523df3fc007b56499d1a13c741f0b96ef7461 Mon Sep 17 00:00:00 2001 From: qiancai Date: Fri, 18 Sep 2026 13:04:38 +0800 Subject: [PATCH 10/14] revise fours docs from the task-oriented perspective --- .../configure-filesystem-ai-providers.md | 101 +++++++++---- .../manage-filesystem-vault-secrets.md | 135 +++++++++++++----- .../manage-git-workspaces.md | 93 +++++++++--- .../use-filesystem-journals.md | 65 +++++++-- 4 files changed, 291 insertions(+), 103 deletions(-) diff --git a/tidb-cloud-filesystem/configure-filesystem-ai-providers.md b/tidb-cloud-filesystem/configure-filesystem-ai-providers.md index 332cccdbef7c9..5569b31cfae10 100644 --- a/tidb-cloud-filesystem/configure-filesystem-ai-providers.md +++ b/tidb-cloud-filesystem/configure-filesystem-ai-providers.md @@ -1,21 +1,38 @@ --- title: Configure TiDB Cloud Filesystem AI Providers -summary: Learn how to inspect and configure media extraction and embedding providers for a TiDB Cloud Filesystem by using the CLI. +summary: Configure AI providers for media extraction and embeddings in a TiDB Cloud Filesystem. aliases: ['/ai/configure-filesystem-ai-providers'] --- # Configure TiDB Cloud Filesystem AI Providers -A TiDB Cloud Filesystem can optionally extract text from media files and generate embeddings for stored content. To enable these capabilities, you can configure one or more AI providers through the CLI. +Use this guide when you want TiDB Cloud Filesystem to extract searchable content from images, audio, or video, or when you want to configure a custom embedding provider for semantic search. + +- **Media extraction** uses an AI provider to extract text or descriptions from images, audio, or video so that the content can be searched. +- **Embeddings** represent text and extracted descriptions as vectors for semantic search. + +These capabilities are optional and can be configured separately. For example, you can configure media extraction without configuring a custom embedding provider. + +This guide shows you how to check the current configuration, configure providers for media extraction and embeddings, and disable a custom configuration when it is no longer needed. ## Prerequisites -- Follow [Get Started with TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-quick-start.md) to install the TiDB Cloud CLI, create a Filesystem, and obtain the Filesystem ID. -- Obtain the API key required by your selected AI provider. +Before you begin: + +- [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). +- Have access to an existing TiDB Cloud Filesystem and obtain its Filesystem ID. +- Configure TiDB Cloud API credentials. The commands in this guide require TiDB Cloud API credentials and an explicit Filesystem ID; they do not use a Filesystem token. +- If you want to enable or replace a provider configuration, obtain the provider endpoint, model name, and API key. + +> **Note:** +> +> When you configure a custom provider, TiDB Cloud Filesystem sends content to that provider for processing. Make sure that the provider account and its data retention and privacy policies are appropriate for your data. + +## Provide the API key of your AI provider -The configuration commands require TiDB Cloud API credentials and an explicit Filesystem ID. Set the provider key through `TI_FS_AI_PROVIDER_API_KEY`. The CLI does not persist the key locally. The Filesystem service stores it encrypted and returns only a masked value in subsequent configuration output. +When you enable or replace a provider configuration, provide the API key of your AI provider through `TI_FS_AI_PROVIDER_API_KEY`. -For an interactive shell, read and export the provider key without placing it in shell history: +For an interactive shell, read and export the key without placing it in shell history: ```bash printf 'Provider API key: ' >&2 @@ -24,15 +41,21 @@ printf '\n' >&2 export TI_FS_AI_PROVIDER_API_KEY ``` -In CI, inject `TI_FS_AI_PROVIDER_API_KEY` from a masked secret. Unset the variable after you finish configuring providers. +The TiDB Cloud CLI does not store the key locally. The Filesystem service stores it encrypted and returns only a masked value when you inspect the configuration later. + +In CI, provide `TI_FS_AI_PROVIDER_API_KEY` through your CI secret-management mechanism. > **Note:** > -> When you enable, re-enable, or replace a provider configuration, the Filesystem service sends a small built-in request to the provider endpoint to validate the credentials, connectivity, and model response. This validation request might incur a provider charge. A disable-only or prompt-only update does not make a validation request. +> When you enable, re-enable, or replace a provider configuration, the Filesystem service sends a small request to the provider to validate the credentials, connectivity, and model response. The provider might charge for this validation request. Disabling a provider or updating only an extraction prompt does not send a validation request. + +## Configure media extraction + +Media extraction lets TiDB Cloud Filesystem process images, audio, or video and make the extracted text or descriptions available for content search. -## Inspect media extraction configuration +### Check the current configuration -Read the effective extraction configuration for a media type: +Before changing the configuration, check the current extraction configuration for the media type you want to process: ```shell ti fs describe-file-system-extract-configuration \ @@ -40,9 +63,11 @@ ti fs describe-file-system-extract-configuration \ --media-type image ``` -## Update media extraction configuration +Replace `image` with `audio` or `video` to inspect another media type. -Use [`update-file-system-extract-configuration`](/ai/ti/reference/ti-fs-update-file-system-extract-configuration.md) to enable, update, or disable image, audio, or video extraction. For example, configure image extraction through an OpenAI-compatible provider: +### Enable or update media extraction + +For example, to enable image extraction with an OpenAI-compatible provider: ```shell ti fs update-file-system-extract-configuration \ @@ -54,7 +79,13 @@ ti fs update-file-system-extract-configuration \ --provider-protocol openai ``` -The `openai` protocol supports image, audio, and video extraction. The `qwen-asr` protocol is supported only for audio extraction through Alibaba Cloud Model Studio. An endpoint from another provider might work if it implements the required OpenAI-compatible API contract. Native interfaces for Anthropic, Gemini, Vertex AI, Amazon Bedrock, and Azure OpenAI are not supported. +The `openai` protocol supports image, audio, and video extraction. For audio extraction, you can also use the `qwen-asr` protocol with Alibaba Cloud Model Studio. + +Other provider endpoints can be used only if they implement the required OpenAI-compatible API contract. Native interfaces for Anthropic, Gemini, Vertex AI, Amazon Bedrock, and Azure OpenAI are not supported. + +For all available options, see [`update-file-system-extract-configuration`](/ai/ti/reference/ti-fs-update-file-system-extract-configuration.md). + +### Disable media extraction To disable extraction for a media type: @@ -65,36 +96,40 @@ ti fs update-file-system-extract-configuration \ --enabled false ``` -## Inspect embedding configuration +## Configure embeddings + +Embeddings represent text content and extracted media descriptions as vectors for semantic search. -Read whether embeddings are managed by the application or database: +### Check the current configuration + +Before configuring a custom embedding provider, check the current embedding configuration: ```shell ti fs describe-file-system-embedding-configuration \ --file-system-id "" ``` -## Update embedding configuration +Check the `source` field in the output. If it is `database_auto`, embeddings are managed by the service and you cannot replace the configuration with a custom provider. + +### Enable or update a custom embedding provider -Use [`update-file-system-embedding-configuration`](/ai/ti/reference/ti-fs-update-file-system-embedding-configuration.md) to update the optional application-managed embedding configuration. For example: +If the current configuration allows a custom provider, configure an OpenAI-compatible embedding endpoint: ```shell ti fs update-file-system-embedding-configuration \ --file-system-id "" \ --enabled true \ --provider-api-base https://api.openai.com/v1 \ - --provider-model text-embedding-3-small + --provider-model "" ``` -Application-managed embeddings require an OpenAI-compatible endpoint that returns exactly 1024-dimensional vectors. Choose a model that supports this width; models that return a different width are not supported. Before updating the configuration, run `describe-file-system-embedding-configuration` to check the effective source. If it reports `source=database_auto`, the service manages embeddings and rejects an application-managed update. +The provider must implement the required OpenAI-compatible embeddings API and return exactly 1024-dimensional vectors. Make sure that the selected model and provider configuration return 1024 dimensions; other vector dimensions are not supported. -After you finish configuring providers, remove the key from the current shell: +For all available options, see [`update-file-system-embedding-configuration`](/ai/ti/reference/ti-fs-update-file-system-embedding-configuration.md). -```shell -unset TI_FS_AI_PROVIDER_API_KEY -``` +### Disable the custom embedding configuration -To disable that configuration: +To disable the custom embedding configuration: ```shell ti fs update-file-system-embedding-configuration \ @@ -102,13 +137,21 @@ ti fs update-file-system-embedding-configuration \ --enabled false ``` -## Data flow after configuration +## Finish configuring providers + +After you finish configuring providers, remove the provider API key from the current shell: + +```shell +unset TI_FS_AI_PROVIDER_API_KEY +``` + +When custom media extraction is enabled, TiDB Cloud Filesystem sends the relevant media content to the configured extraction provider. When a custom embedding provider is enabled, text content and extracted media descriptions are sent to the embedding provider. -After you enable extraction, the Filesystem service sends media content to the configured extraction provider. It sends the extracted text or descriptions to the configured embedding provider. Choose provider accounts and retention policies appropriate for your data. +If an update fails because of a timeout, lost response, or another error where you cannot tell whether the update succeeded, do not immediately retry the command. Run the corresponding `describe-file-system-*-configuration` command first. -If an update fails because of a timeout, lost response, or another ambiguous network error, run the matching `describe-file-system-*-configuration` command before retrying. The provider validation request might have succeeded and incurred a charge even if the CLI did not receive the response. +The Filesystem service might already have saved the configuration and sent the provider validation request even if the CLI did not receive the response. ## What's next -- [Work with Files and Directories](/tidb-cloud-filesystem/work-with-filesystem-data.md) -- [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) +- [Work with Files and Directories](/tidb-cloud-filesystem/work-with-filesystem-data.md) to search Filesystem content. +- [TiDB Cloud Filesystem CLI Command Reference](/ai/ti/reference/ti-filesystem.md) for complete command syntax and options. diff --git a/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md b/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md index aa26d72e706aa..feb93c72dba66 100644 --- a/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md +++ b/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md @@ -1,121 +1,178 @@ --- title: Manage TiDB Cloud Filesystem Vault Secrets -summary: Learn how to store, read, delegate, inject, audit, revoke, and mount secrets securely with TiDB Cloud Filesystem Vault. +summary: Learn how to store and rotate secrets, delegate temporary access, inject secrets into processes, audit and revoke access, and optionally mount secrets as read-only files. aliases: ['/ai/manage-filesystem-vault-secrets'] --- # Manage TiDB Cloud Filesystem Vault Secrets -TiDB Cloud Filesystem Vault lets you store secrets, delegate narrowly scoped and time-limited access to users or agents, and inject credentials into processes without writing plaintext to disk. +Use TiDB Cloud Filesystem Vault when an application, automation, or agent needs credentials or other sensitive values, but you do not want to store those values in regular Filesystem files or give the workflow broad access to the Filesystem. + +With Vault, a trusted owner can store a secret once and grant access to only the secret or field that a user, application, or agent needs, for a limited time. The delegated workflow can then read the permitted value, inject it into a process, or access it through a read-only mount. The owner can audit the access and revoke the grant when it is no longer needed. + +This guide shows you how to store and rotate secrets, delegate limited access, use delegated secrets, audit and revoke access, and optionally mount secrets as files. ## Prerequisites +Before you begin: + - [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). -- Have access to an existing TiDB Cloud Filesystem. Owner operations require an [owner token](/tidb-cloud-filesystem/filesystem-authorization.md#owner-tokens). -- Select the Filesystem and make its token available to `ti`. For available access options, see [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). +- Have access to an existing TiDB Cloud Filesystem. +- Make the Filesystem and its owner token available to `ti`. See [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). + +An owner token is used to create and replace secrets, create and revoke grants, and view audit events. A delegated Vault token provides only the secret access allowed by its grant. -> **Note:** -> -> To avoid security risks, never print, log, or commit owner or delegated tokens. +Treat both owner tokens and delegated Vault tokens as credentials. Do not expose them in logs, source control, shared terminal output, or command-line arguments. -## Create and read a secret +## Create a secret + +A Vault secret can contain multiple named fields. For example, a database secret might contain a connection URL and a password. + +Create a secret named `db-prod`: ```shell ti fs-vault create-secret \ --secret-name db-prod \ --field DB_URL=mysql://example \ --field PASSWORD=@./password.txt +``` + +In `PASSWORD=@./password.txt`, the `@` prefix tells `ti` to read the field value from the local file instead of treating the file path as the value. -ti fs-vault read-secret --secret-name db-prod +Some Vault commands identify a secret by name, such as `db-prod`. Commands that operate on a specific secret path, such as `replace-secret` and `run-with-secret`, use its full Vault path instead. For example, the Vault path of `db-prod` is `/n/vault/db-prod`. + +### Read a secret value + +`read-secret` returns plaintext secret values. Use it only when you need the value directly, and make sure its output is not written to logs or other unintended destinations. + +For example, to read only the `DB_URL` field: + +```shell +ti fs-vault read-secret \ + --secret-name db-prod \ + --field DB_URL \ + --format raw ``` -In `--field PASSWORD=@./password.txt`, the `@` prefix reads the field value from the local file instead of using the literal text as the value. +When an application needs the secret, prefer [injecting it into the process](#inject-a-secret-into-a-process) instead of reading and handling the plaintext value yourself. + +## Rotate a secret + +`replace-secret` replaces all fields in the secret, not just the field whose value changed. -> **Note:** -> -> All `read-secret` output formats, including the default JSON format, contain plaintext secret values. Direct the output only to the intended process. +To rotate `DB_URL`, create a local directory containing the new `DB_URL` value and the current `PASSWORD` value that you want to keep: -## Replace a secret +```text +./secret-fields/ +├── DB_URL +└── PASSWORD +``` -To rotate a stored value, put each replacement field in a separate file in a local directory. For example, to change `DB_URL`, write its new value to `./secret-fields/DB_URL` and include a `./secret-fields/PASSWORD` file if you want to retain that field. Then replace the secret: +Then replace the secret: ```shell -ti fs-vault replace-secret --secret-path /n/vault/db-prod --from-directory ./secret-fields +ti fs-vault replace-secret \ + --secret-path /n/vault/db-prod \ + --from-directory ./secret-fields ``` -`replace-secret` replaces **all** fields, not only the changed field. Keep the local files out of source control and remove them after use. See the [`replace-secret` reference](/ai/ti/reference/ti-fs-vault-replace-secret.md) for the full command contract. +Each file in the directory becomes a field in the replacement secret. Any existing field that is not included in the directory is not retained. + +Keep these local files out of source control and remove them when they are no longer needed. For details, see the [`replace-secret` reference](/ai/ti/reference/ti-fs-vault-replace-secret.md). ## Delegate limited access -Create a short-lived read grant and capture its token: +Instead of sharing the Filesystem owner token, create a short-lived grant for only the secret fields that another user, application, or agent needs. + +For example, the following grant allows `deploy-agent` to read only the `DB_URL` field for 10 minutes: ```shell -export TI_VAULT_TOKEN="$(ti fs-vault create-grant \ +ti fs-vault create-grant \ --agent-id deploy-agent \ --scope db-prod/DB_URL \ --permission read \ - --ttl 10m \ - --token-only)" + --ttl 10m ``` -Prefer `TI_VAULT_TOKEN` to a command-line token because command-line values can remain in process listings or shell history. +The command returns a delegated Vault token and a grant ID. Give the delegated token only to the workflow that needs the secret, and retain the grant ID so that you can revoke the grant before it expires if necessary. -## Inject a secret into a process +In the environment that uses the delegated secret, make the token available as `TI_VAULT_TOKEN`. Avoid putting the token directly in a command-line argument because command arguments can appear in shell history or process listings. -The CLI can inject secret fields as environment variables into a child process without writing plaintext to disk. When you run the following command, the CLI reads the secret, sets each field as an environment variable (for example, `DB_URL`, `PASSWORD`), removes its own credential environment variables from the child, and then executes the specified command: +## Inject a secret into a process -`/n/vault/db-prod` is the canonical Vault path for the secret named `db-prod`; it is not a local mount path. +If an application can receive credentials through environment variables, use `run-with-secret` to make the permitted secret fields available only to the child process: ```shell -ti fs-vault run-with-secret --secret-path /n/vault/db-prod -- +ti fs-vault run-with-secret \ + --secret-path /n/vault/db-prod \ + -- ``` -Prefer process injection to writing plaintext to disk. +Each permitted secret field becomes an environment variable with the same name. For example, the `DB_URL` and `PASSWORD` fields become the `DB_URL` and `PASSWORD` environment variables. + +The Vault credential used by `ti` is not passed to the child process. This lets the application use the secret without writing its plaintext value to a file. -Field names injected by `run-with-secret` must match `[A-Z_][A-Z0-9_]*`. The command rejects the entire injection if any field name does not match this pattern or any field value contains an unsupported control character. Use uppercase environment-variable-style names when creating fields that you plan to inject. +Field names used with `run-with-secret` must match `[A-Z_][A-Z0-9_]*`. Use uppercase environment-variable-style field names for secrets that you plan to inject into a process. ## Audit and revoke access +To review recent access to `db-prod` by `deploy-agent`, run: + ```shell ti fs-vault list-audit-events \ --secret-name db-prod \ --agent-id deploy-agent \ --since 24h \ --limit 20 +``` +When the delegated access is no longer needed, revoke the grant using the grant ID returned by `create-grant`: + +```shell ti fs-vault delete-grant \ --grant-id "" \ --revoked-by operator \ - --reason rotated + --reason task-complete ``` -Revocation prevents new authorized operations but cannot erase a value that a process already read. +Revoking a grant prevents the delegated token from authorizing new operations. It cannot remove a secret value that a process has already read. -## Mount a read-only Vault view +## Mount secrets as read-only files -On macOS or Linux with FUSE support, you can mount a read-only FUSE view of Vault secrets. The CLI creates the mount and serves secret fields as files under the mount path (for example, `/path/to/vault/db-prod/DB_URL`): +If an application expects credentials as files instead of environment variables, you can optionally expose permitted Vault fields through a read-only FUSE mount on Linux or macOS. -Before mounting, set `TI_VAULT_TOKEN` to a delegated Vault token, such as the token created in [Delegate limited access](#delegate-limited-access). The mount command requires either `TI_VAULT_TOKEN` or `--vault-token`. +For delegated access, first make the delegated Vault token available as `TI_VAULT_TOKEN`. Then create a local mount directory and mount the Vault: ```shell mkdir -p /path/to/vault + ti fs-vault mount-vault \ --mount-path /path/to/vault ``` -Stop any processes that use the mount before you unmount it: +The permitted secret fields are available as files under the mount path. For example: + +```text +/path/to/vault/db-prod/DB_URL +``` + +Processes that can access the mount can read the permitted secret values, so keep access to the mount limited to the intended workload. + +Before unmounting, stop processes that are using the mounted secrets: ```shell -ti fs-vault unmount-vault --mount-path /path/to/vault +ti fs-vault unmount-vault \ + --mount-path /path/to/vault ``` -Vault mounts are unavailable on Windows. Direct secret reads and process injection do not require a mount. +Vault mounts require FUSE and are not available on Windows. Direct secret reads and `run-with-secret` do not require a mount. ## Security recommendations -- Grant the narrowest field scope and shortest practical TTL. -- Do not store delegated tokens in CLI configuration or operation logs. -- Revoke grants after their tasks finish. +- Grant access only to the secret fields required by the workflow and use the shortest practical TTL. +- Prefer `run-with-secret` when an application can receive credentials through environment variables. +- Do not expose owner or delegated tokens in logs, source control, or command-line arguments. +- Revoke grants when their tasks finish or access is no longer needed. ## What's next diff --git a/tidb-cloud-filesystem/manage-git-workspaces.md b/tidb-cloud-filesystem/manage-git-workspaces.md index ef11b55d9f8d5..21998035cab76 100644 --- a/tidb-cloud-filesystem/manage-git-workspaces.md +++ b/tidb-cloud-filesystem/manage-git-workspaces.md @@ -1,21 +1,31 @@ --- title: Manage Git Workspaces on TiDB Cloud Filesystem -summary: Learn how to clone, hydrate, create linked worktrees, and remove Git workspaces on a mounted TiDB Cloud Filesystem. +summary: Learn how to clone Git repositories into a mounted TiDB Cloud Filesystem, speed up large repository setup, and manage linked worktrees. aliases: ['/ai/manage-git-workspaces'] --- # Manage Git Workspaces on TiDB Cloud Filesystem -Use `ti fs-git` when you want a Git workspace on a mounted TiDB Cloud Filesystem without waiting for all file content to download before work begins. Unlike a regular `git clone` into the mount, the CLI can combine a blobless clone with background hydration. Continue to use ordinary Git commands for daily work after setup. +If you want to work with a Git repository directly on a mounted TiDB Cloud Filesystem, you can follow this guide to set up and manage the Git workspace. + +This is especially useful for large repositories, where you can start working before all file contents finish downloading, or when you need separate working directories for different branches without cloning the repository multiple times. This guide covers cloning a repository, completing background downloads when needed, and creating and removing linked Git worktrees. + +After the workspace is set up, you can continue to use ordinary Git commands such as `git status`, `git add`, `git commit`, `git fetch`, and `git push` for your daily Git work. ## Prerequisites +Before you begin: + - [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). -- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) through FUSE. -- Select the mounted Filesystem and make a token with the required permissions available to `ti`. For available access options, see [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). -- Install Git and configure repository credentials independently. +- [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) through FUSE. Git workspaces are not supported on WebDAV mounts. +- Make the mounted Filesystem and a token with the required permissions available to `ti`. See [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). +- Install Git and configure authentication for the repository you want to use. + +All Git workspaces in this guide are created inside the mounted Filesystem path. + +## Clone a Git repository -## Clone a workspace +To clone a Git repository into your mounted Filesystem directory, use the `ti fs-git clone-git-workspace` command and specify the repository URL and target path as follows: ```shell ti fs-git clone-git-workspace \ @@ -23,11 +33,32 @@ ti fs-git clone-git-workspace \ --target-path /path/to/workspace/tidb ``` -For a large repository, add `--blobless --hydrate background` to make the directory tree available immediately. A blobless clone initially fetches Git history and tree metadata without downloading all file contents. The CLI starts a background process that downloads clean file content and Git objects after the clone command returns. Use `--hydrate sync` when your workflow requires hydration to finish before the command returns. +The repository is cloned to `/path/to/workspace/tidb`. + +For a large repository, you can start working before all file contents finish downloading by specifying `--blobless`: + +```shell +ti fs-git clone-git-workspace \ + --repo-url https://github.com/pingcap/tidb.git \ + --target-path /path/to/workspace/tidb \ + --blobless +``` -## Hydrate an existing workspace +With `--blobless`, `ti` downloads the repository structure and Git metadata first. When `--hydrate` is `auto` (the default), `ti` then continues downloading the remaining file contents in the background. You can start working with the repository while this download continues. To disable background hydration, specify `--hydrate off`. -If a workspace was cloned with `--blobless`, you can explicitly fetch the missing Git objects by running `hydrate-git-workspace`: +If you need all file contents to finish downloading before the clone command returns, also specify `--hydrate sync`: + +```shell +ti fs-git clone-git-workspace \ + --repo-url https://github.com/pingcap/tidb.git \ + --target-path /path/to/workspace/tidb \ + --blobless \ + --hydrate sync +``` + +## Finish downloading a blobless workspace + +If you created a workspace with `--blobless`, you can later wait for any remaining file contents to finish downloading: ```shell ti fs-git hydrate-git-workspace \ @@ -35,11 +66,17 @@ ti fs-git hydrate-git-workspace \ --timeout 30m ``` -Hydration fetches missing blob data from the remote repository without discarding your working-tree changes. +Run this command when you want to make sure the remaining Git file data has finished downloading before you continue. For example, you might do this before a task that needs repository file contents to be available. + +This process, called hydration, downloads the Git data that is still missing from the workspace without discarding changes you have already made to files. + +If cloning or hydration fails, check the CLI error and diagnostic log before retrying. See [Troubleshoot TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-troubleshooting.md). + +## Create a linked worktree -If cloning or hydration fails, inspect the CLI error and diagnostic log before retrying. See [Troubleshoot TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-troubleshooting.md) for common Filesystem and companion issues. +Use a linked worktree when you want a separate working directory for another branch without cloning the repository again. The new worktree shares Git data with the base workspace. -## Add and use a linked worktree +For example, create a worktree for a new `feature-x` branch: ```shell ti fs-git add-git-worktree \ @@ -48,22 +85,38 @@ ti fs-git add-git-worktree \ --branch-name feature-x ``` -After creation, use ordinary Git commands in the linked worktree. +After the worktree is created, use ordinary Git commands in `/path/to/workspace/tidb-feature`: + +```shell +git -C /path/to/workspace/tidb-feature status +``` + +For other options, such as creating a detached worktree at a specific commit, see the [`add-git-worktree` command reference](/ai/ti/reference/ti-fs-git-add-git-worktree.md). -## Remove a worktree +## Remove a linked worktree + +When you no longer need a linked worktree, remove it: ```shell ti fs-git remove-git-worktree \ --worktree-path /path/to/workspace/tidb-feature ``` -The CLI checks for uncommitted changes and rejects the removal if the worktree is dirty. Use `--force` only after you decide that local changes in the worktree can be discarded. +Removing a linked worktree does not remove the base workspace or the Git data shared by other worktrees. + +If the worktree contains uncommitted changes, the command refuses to remove it. Commit or preserve any changes you need before removing the worktree. + +Use `--force` only when you are sure that the uncommitted changes can be discarded: + +```shell +ti fs-git remove-git-worktree \ + --worktree-path /path/to/workspace/tidb-feature \ + --force +``` -> **Note:** -> -> Before terminating an ephemeral machine, preserve required changes, remove unused worktrees, and gracefully unmount the Filesystem. +When you are finished using the mounted Filesystem, commit or push any Git changes you want to preserve and [unmount the Filesystem safely](/tidb-cloud-filesystem/filesystem-mount.md#unmount-when-you-are-finished). ## What's next -- [Prepare a Git Workspace for Agents on TiDB Cloud Filesystem](/ai/ti/guides/ti-git-workspace-for-agents-example.md) -- [TiDB Cloud Filesystem Git CLI Command Reference](/ai/ti/reference/ti-filesystem-git.md) +- [Prepare a Git Workspace for Agents on TiDB Cloud Filesystem](/ai/ti/guides/ti-git-workspace-for-agents-example.md) for an agent workflow that uses blobless cloning and background hydration. +- [TiDB Cloud Filesystem Git CLI Command Reference](/ai/ti/reference/ti-filesystem-git.md) for all `ti fs-git` commands and options. diff --git a/tidb-cloud-filesystem/use-filesystem-journals.md b/tidb-cloud-filesystem/use-filesystem-journals.md index 1cce3d2b0d78c..feecdc019c183 100644 --- a/tidb-cloud-filesystem/use-filesystem-journals.md +++ b/tidb-cloud-filesystem/use-filesystem-journals.md @@ -1,23 +1,35 @@ --- title: Use TiDB Cloud Filesystem Journals -summary: Learn how to create, append, read, search, and verify append-only journals for agent and automation events in a Filesystem. +summary: Learn how to record, read, search, and verify ordered events from agent and automation workflows in a TiDB Cloud Filesystem. aliases: ['/ai/use-filesystem-journals'] --- # Use TiDB Cloud Filesystem Journals -Journals provide append-only, hash-chained event logs for agent workflows and automation pipelines running on a TiDB Cloud Filesystem. Use [`ti fs-journal` commands](/ai/ti/reference/ti-filesystem-journal.md) to create a journal, append ordered events, search or read them, and verify the hash chain. +Use a journal when you need an ordered, persistent record of events from an agent or automation workflow. For example, a journal can record when a task starts or finishes, which agent performed an action, and when work is handed off between agents or processes. You can later read or search these events to understand what happened during the workflow. -Use a journal when you need to trace the order of agent actions or handoffs across sessions. The hash chain lets you check the integrity and order of recorded entries. A journal records events; it does not replay actions or replace the files produced by a workflow. +Journal entries are append-only: new events are added as new entries, and existing entries cannot be modified. The entries are also linked through a hash chain, which lets you verify that the recorded history remains intact and in order. + +This guide shows you how to create a journal, record events, read and search recorded events, and verify the journal history. + +Journals are intended for workflow events and history. Store artifacts, working files, and other workflow outputs as regular files in the Filesystem. A journal records what happened; it does not replay workflow actions or replace the files produced by the workflow. + +> **Note:** +> +> The current `ti` CLI does not provide a command to delete individual journals. Do not record secrets or other data that you might need to remove later. ## Prerequisites +Before you begin: + - [Install TiDB Cloud CLI](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). -- Have access to an existing TiDB Cloud Filesystem with a token that provides the required journal permissions. -- Select the Filesystem and make its token available to `ti`. For available access options, see [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). +- Have access to an existing TiDB Cloud Filesystem. +- Make the Filesystem and its token available to `ti`. See [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). ## Create a journal +Create a journal for the workflow you want to record: + ```shell ti fs-journal create-journal \ --journal-kind agent \ @@ -25,27 +37,45 @@ ti fs-journal create-journal \ --actor agent:reviewer ``` -Save the returned journal ID. +Because no journal ID is specified, the service generates one. Save the returned journal ID—you will use it to append, read, and verify entries in this journal. + +The journal kind, title, and actor provide context that can also help you find related workflow records later. ## Append entries +Append an event to the journal: + +```shell +ti fs-journal append-journal-entries \ + --journal-id "" \ + --entry-json '{"type":"review_started"}' +``` + +Each entry needs a `type`, unless you provide one with `--entry-type`. You can also include fields such as `summary`, `actor`, and `occurred_at`. + +The example above omits `--idempotency-key` for brevity. If your workflow might retry the same append operation, provide an idempotency key and reuse the same key for every retry. This prevents the retry from recording the same event more than once: + ```shell ti fs-journal append-journal-entries \ --journal-id "" \ + --idempotency-key review-started \ --entry-json '{"type":"review_started"}' ``` -`--entry-json` accepts a JSON object. Each entry needs a `type`, unless you supply `--entry-type`; optional fields include `summary`, `actor`, and `occurred_at`. For the complete fields and input forms, see the [`append-journal-entries` reference](/ai/ti/reference/ti-fs-journal-append-journal-entries.md). +For all supported fields and input formats, see the [`append-journal-entries` reference](/ai/ti/reference/ti-fs-journal-append-journal-entries.md). ## Read and search entries -Read entries in sequence order: +To review the history of one journal, read its entries: ```shell -ti fs-journal read-journal-entries --journal-id "" +ti fs-journal read-journal-entries \ + --journal-id "" ``` -Search across journals and entries: +Entries are returned in sequence order, so you can follow the workflow in the order it was recorded. + +To find events across journals in the selected Filesystem, use `search-journal-entries`. For example, the following command finds `review_started` events and returns their entry contents: ```shell ti fs-journal search-journal-entries \ @@ -53,17 +83,22 @@ ti fs-journal search-journal-entries \ --include-entries ``` +Unlike `read-journal-entries`, the search command is not limited to one journal. Use filters such as journal kind, actor, entry type, labels, or time range to narrow the results. + ## Verify a journal -Verify that the journal's hash chain is intact: +To check that the stored journal history is internally consistent, verify its hash chain: ```shell -ti fs-journal verify-journal --journal-id "" +ti fs-journal verify-journal \ + --journal-id "" ``` -The current public CLI has no journal delete command. Keep retention needs in mind before recording sensitive or high-volume events. +A successful verification confirms that the stored sequence and hash chain are consistent. + +Hash-chain verification checks the integrity of the recorded journal history. It does not prove that the original event information recorded by an agent or application was accurate. ## What's next -- [Record an Agent Workflow in a TiDB Cloud Filesystem Journal](/ai/ti/guides/ti-journal-agent-workflow-example.md) -- [TiDB Cloud Filesystem Journal CLI Command Reference](/ai/ti/reference/ti-filesystem-journal.md) +- [Record an Agent Workflow in a TiDB Cloud Filesystem Journal](/ai/ti/guides/ti-journal-agent-workflow-example.md) for an end-to-end agent workflow example. +- [TiDB Cloud Filesystem Journal CLI Command Reference](/ai/ti/reference/ti-filesystem-journal.md) for all journal commands and options. From e4faf8b143c432f41e185085a2e202c96dce300a Mon Sep 17 00:00:00 2001 From: qiancai Date: Fri, 18 Sep 2026 13:05:27 +0800 Subject: [PATCH 11/14] move intro from _index.md to filesystem-overview.md --- TOC-tidb-cloud-filesystem.md | 2 +- tidb-cloud-filesystem/_index.md | 76 ++++++++++++-------- tidb-cloud-filesystem/filesystem-overview.md | 64 +++++++++++++++++ 3 files changed, 110 insertions(+), 32 deletions(-) create mode 100644 tidb-cloud-filesystem/filesystem-overview.md diff --git a/TOC-tidb-cloud-filesystem.md b/TOC-tidb-cloud-filesystem.md index 5044d21f900f3..5475cea7e845b 100644 --- a/TOC-tidb-cloud-filesystem.md +++ b/TOC-tidb-cloud-filesystem.md @@ -5,7 +5,7 @@ ## GET STARTED -- [Introduction](/tidb-cloud-filesystem/_index.md) +- [Overview](/tidb-cloud-filesystem/filesystem-overview.md) - [Quick Start](/tidb-cloud-filesystem/filesystem-quick-start.md) ## CONCEPTS diff --git a/tidb-cloud-filesystem/_index.md b/tidb-cloud-filesystem/_index.md index 5c3503903e476..5a926ca3d3a9b 100644 --- a/tidb-cloud-filesystem/_index.md +++ b/tidb-cloud-filesystem/_index.md @@ -1,52 +1,66 @@ --- -title: TiDB Cloud Filesystem -summary: Learn how TiDB Cloud Filesystem keeps agent files available across sessions, shares workspaces, and isolates changes with layers. +title: TiDB Cloud Filesystem Documentation +hide_sidebar: true +hide_commit: true +summary: TiDB Cloud Filesystem provides persistent, shared file storage for applications, automation, and AI agents across machines and sessions. --- -# TiDB Cloud Filesystem + -TiDB Cloud Filesystem is persistent, shared file storage for applications and AI agents. Keep a workspace in the cloud, access it from different machines, and retain its files after a sandbox or process ends. + -Use TiDB Cloud CLI (`ti`) to create a Filesystem, upload and download files, or mount a remote directory for tools that work with local paths. You do not need to provision or manage a separate database to get started. +[TiDB Cloud Filesystem Overview](/tidb-cloud-filesystem/filesystem-overview.md) -> **Note:** -> -> TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. +[Layers and Checkpoints](/tidb-cloud-filesystem/filesystem-layers-checkpoints.md) -## Keep work beyond one session + -An agent might collect source documents in one sandbox, generate a report in another, and hand the result to a person on a laptop. Files stored only on the sandbox's local disk disappear when that environment is removed. Repeatedly exporting and importing archives also creates separate copies that can drift apart. + -With a Filesystem, these participants use the same remote files. To access an existing Filesystem from a new environment, provide an access token and the Filesystem's region code. You do not need to copy the creator's TiDB Cloud API keys or local configuration directory. +[Quick Start](/tidb-cloud-filesystem/filesystem-quick-start.md) -- **Continue a task in another environment.** Mount the existing workspace instead of recreating its input files. -- **Share results with a reviewer.** Give the reviewer a read-only scoped token for a report directory. -- **Compare parallel approaches.** Fork layers for independent drafts, create checkpoints, and commit a selected result to the shared base. + -## Choose how to work with files + -The CLI provides two ways to use the same remote Filesystem: +[Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md) -- **Direct file commands:** [Copy, read, organize, and search files](/tidb-cloud-filesystem/work-with-filesystem-data.md) with `ti fs`, without setting up a mount. -- **Local mounts:** [Expose remote files at a local directory](/tidb-cloud-filesystem/filesystem-mount.md) so existing tools can use filesystem paths. Mount support depends on your operating system. +[Share a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-sharing.md) -## Control access and changes +[Mount a Filesystem](/tidb-cloud-filesystem/filesystem-mount.md) -- [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md) explains when to use API keys, owner tokens, or scoped tokens. -- [Share a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-sharing.md) shows how to give another machine access without sharing account credentials. -- [Layers and Checkpoints](/tidb-cloud-filesystem/filesystem-layers-checkpoints.md) explains how to isolate drafts and publish selected changes. + -## Get started + -If you need to create a Filesystem, follow [Get Started with TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-quick-start.md). You will need TiDB Cloud API keys with permission to create one. +[Manage TiDB Cloud Filesystem](/tidb-cloud-filesystem/manage-filesystem-resources.md) -If someone has given you access to an existing Filesystem, [use its FS token and region without configuring a profile](/tidb-cloud-filesystem/filesystem-mount.md#use-a-token-without-configuring-a-profile). You do not need the creator's API keys to access files within your token's permissions. +[Work with Files and Directories](/tidb-cloud-filesystem/work-with-filesystem-data.md) -For supported regions, platform requirements, and current constraints, see [Regions and Limitations](/tidb-cloud-filesystem/filesystem-regions-and-limitations.md). +[Manage Git Workspaces](/tidb-cloud-filesystem/manage-git-workspaces.md) -## What's next +[Automation and AI Agent Workflows](/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md) -- [Create a Filesystem and write your first file](/tidb-cloud-filesystem/filesystem-quick-start.md). -- [Share a Filesystem across machines](/tidb-cloud-filesystem/filesystem-sharing.md). -- [Explore automation and AI agent workflows](/tidb-cloud-filesystem/use-filesystem-for-automation-and-ai-agents.md). -- [Try the interactive agent sandbox lab](https://labs.tidb.io/labs/demo_901). + + + + +[Authorization](/tidb-cloud-filesystem/filesystem-authorization.md) + +[Manage Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) + +[Manage Filesystem Vault Secrets](/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md) + + + + + +[Filesystem CLI Commands](/ai/ti/reference/ti-filesystem.md) + +[Regions and Limitations](/tidb-cloud-filesystem/filesystem-regions-and-limitations.md) + +[Troubleshooting](/tidb-cloud-filesystem/filesystem-troubleshooting.md) + + + + diff --git a/tidb-cloud-filesystem/filesystem-overview.md b/tidb-cloud-filesystem/filesystem-overview.md new file mode 100644 index 0000000000000..3c012e2be0b30 --- /dev/null +++ b/tidb-cloud-filesystem/filesystem-overview.md @@ -0,0 +1,64 @@ +--- +title: TiDB Cloud Filesystem Overview +summary: Learn what TiDB Cloud Filesystem is, when to use it, and how to access and share persistent files across applications, automation, and AI agents. +--- + +# TiDB Cloud Filesystem Overview + +TiDB Cloud Filesystem is persistent, shared file storage that applications, automation, and AI agents can access across machines and sessions. + +Use a Filesystem when files need to remain available after a local process or temporary environment ends, or when multiple environments need to work with the same files without repeatedly copying them between machines. + +For example, you can use a Filesystem to: + +- continue working with the same files from another machine or environment; +- share files or results with another user, application, CI job, or agent; +- mount shared files as a local directory for tools that expect local file paths; and +- isolate changes in layers before applying selected changes to shared files. + +> **Note:** +> +> TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. + +## How TiDB Cloud Filesystem works + +Files in a TiDB Cloud Filesystem are stored independently of the machine or process that accesses them. When you move to another environment, you can access the same Filesystem instead of copying or recreating its files. + +You use TiDB Cloud CLI (`ti`) to create and manage Filesystems and to work with their files. You do not need to provision or manage a separate database to use TiDB Cloud Filesystem. + +You can work with Filesystem data in two main ways: + +- **Use `ti fs` commands** to upload, download, read, organize, and search files directly without mounting the Filesystem. +- **Mount the Filesystem as a local directory** so applications and tools can work with its files through normal filesystem paths. + +For more information, see [Work with Files and Directories](/tidb-cloud-filesystem/work-with-filesystem-data.md) and [Mount a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-mount.md). + +## Share files across environments + +A Filesystem can be accessed from multiple machines or environments. Each environment can use its own credential instead of sharing the credentials of the user who created the Filesystem. + +For example: + +- a CI job can write build output to a Filesystem and another environment can read it later; +- an agent can continue working with files created during an earlier session; or +- a reviewer can receive read-only access to a specific directory. + +TiDB Cloud Filesystem provides owner tokens and scoped tokens so you can control which files and operations each user or workload can access. + +For more information, see [TiDB Cloud Filesystem Authorization](/tidb-cloud-filesystem/filesystem-authorization.md) and [Share a TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-sharing.md). + +## Isolate and manage changes + +When multiple tasks need to work from the same files without immediately changing the shared base, you can create Filesystem layers. + +Each layer provides an isolated view where changes can be made independently. You can create checkpoints of a layer and later commit selected changes to the base Filesystem. + +For more information, see [Layers and Checkpoints](/tidb-cloud-filesystem/filesystem-layers-checkpoints.md). + +## Get started + +Choose the path that matches what you want to do: + +- **Create a new Filesystem:** follow [Get Started with TiDB Cloud Filesystem](/tidb-cloud-filesystem/filesystem-quick-start.md). +- **Use a Filesystem that someone else has shared with you:** see [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). +- **Check supported regions, platforms, and current limitations:** see [Regions and Limitations](/tidb-cloud-filesystem/filesystem-regions-and-limitations.md). From 672cc3fa304d59bd6cb28898a354bac6c2f5b088 Mon Sep 17 00:00:00 2001 From: qiancai Date: Fri, 18 Sep 2026 16:01:57 +0800 Subject: [PATCH 12/14] update filesystem-mount.md to make it an overview and move necessary steps to individual guides --- .../filesystem-mount-docker.md | 88 +++++++---- .../filesystem-mount-linux.md | 56 +++++-- .../filesystem-mount-macos.md | 86 +++++++++-- tidb-cloud-filesystem/filesystem-mount.md | 143 ++++++------------ 4 files changed, 219 insertions(+), 154 deletions(-) diff --git a/tidb-cloud-filesystem/filesystem-mount-docker.md b/tidb-cloud-filesystem/filesystem-mount-docker.md index 940afd9639fe7..e7295c1d6d9d6 100644 --- a/tidb-cloud-filesystem/filesystem-mount-docker.md +++ b/tidb-cloud-filesystem/filesystem-mount-docker.md @@ -1,13 +1,13 @@ --- title: Mount TiDB Cloud Filesystem in Docker -summary: Mount a TiDB Cloud Filesystem inside a Docker container by giving the container access to FUSE and the required mount permissions. +summary: Mount a TiDB Cloud Filesystem inside a Docker container on a Linux host by enabling FUSE and the required container permissions. --- # Mount TiDB Cloud Filesystem in Docker -To mount a TiDB Cloud Filesystem inside a Docker container, the container must be able to use FUSE on the Linux host. In addition to installing `fuse3` inside the container, you need to expose `/dev/fuse` and grant the container the permissions required to create a mount. +To mount a TiDB Cloud Filesystem inside a Docker container, the container must have access to FUSE on the Linux host. In addition to installing `fuse3` inside the container, you need to expose `/dev/fuse` and grant the container permission to create the mount. -These steps are for Docker running on a Linux host. If your environment does not allow FUSE access or the required container privileges, use direct `ti fs` commands such as `copy-file`, `read-file`, and `list-files` instead. +This guide covers both Docker and Docker Compose on a Linux host. If your environment does not provide FUSE access or allow the required container privileges, use direct `ti fs` commands such as `copy-file`, `read-file`, and `list-files` instead. > **Note:** > @@ -17,21 +17,21 @@ These steps are for Docker running on a Linux host. If your environment does not Before you begin: -- Use a Linux Docker host with `/dev/fuse` available. -- Have permission to start containers with access to `/dev/fuse` and the required mount capabilities. -- Have a Filesystem token and the region code for an existing TiDB Cloud Filesystem. See [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). +- Use a Linux Docker host where `/dev/fuse` is available. +- Have permission to start containers with access to `/dev/fuse` and the capabilities required to create a mount. +- Have a Filesystem token and region code for an existing TiDB Cloud Filesystem. See [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). -Docker Desktop and managed container or sandbox platforms might not expose `/dev/fuse` or allow the required privileges. +Docker Desktop and managed container or sandbox environments might not expose `/dev/fuse` or allow the required privileges. > **Warning:** > -> The examples in this guide grant the container `SYS_ADMIN` and disable its AppArmor profile with `apparmor=unconfined`. These settings give the container broader access than a standard Docker container. +> The examples in this guide grant the container `SYS_ADMIN` and disable its AppArmor profile with `apparmor=unconfined`. These settings give the container broader privileges than a standard Docker container. > -> Use them only when permitted by your environment's security policy. Do not run untrusted code with these privileges unless your environment provides an appropriate additional isolation boundary. A scoped Filesystem token can restrict access to Filesystem data, but it does not reduce the container privileges granted by Docker. +> Use them only when permitted by your environment's security policy. Do not run untrusted code with these privileges unless your environment provides an appropriate additional isolation boundary. A scoped Filesystem token can restrict access to Filesystem data, but it does not reduce the privileges granted to the container by Docker. ## Provide Filesystem access to the container -On the Docker host, set the Filesystem token and region: +On the Docker host, set the Filesystem token and region code: ```bash export TI_FS_TOKEN="" @@ -40,15 +40,15 @@ export TI_REGION_CODE="" Treat the Filesystem token as a secret. Do not put it in a Dockerfile or commit it to source control. -The Docker and Docker Compose examples below pass these values from the host into the container. +The Docker and Docker Compose examples below pass these environment variables from the host into the container. ## Start the container Choose either Docker or Docker Compose. -### Option 1: Use Docker +### Use Docker -Start an interactive Ubuntu container with access to FUSE: +On the Docker host, start an interactive Ubuntu container with access to FUSE: ```bash docker run --rm -it \ @@ -64,13 +64,13 @@ This command does the following: - Exposes the host's `/dev/fuse` device to the container; - Grants the capability required to create the mount; and -- Passes the Filesystem token and region into the container. +- Passes the Filesystem token and region code into the container. After the container starts, continue with [Install and mount inside the container](#install-and-mount-inside-the-container). -### Option 2: Use Docker Compose +### Use Docker Compose -Alternatively, create a `compose.yaml` file: +Alternatively, create the following `compose.yaml` file on the Docker host: ```yaml services: @@ -88,7 +88,7 @@ services: TI_REGION_CODE: ${TI_REGION_CODE:?Set TI_REGION_CODE} ``` -On the Docker host, start the container and open a shell in it: +Start the container and open a shell in it: ```bash docker compose up -d @@ -99,7 +99,7 @@ Then continue with the steps below inside the container. ## Install and mount inside the container -The following commands use the `ubuntu:24.04` image from the examples above, which runs as `root` by default. +The following steps run inside the container. They use the `ubuntu:24.04` image from the preceding examples, which runs as `root` by default. 1. Install `fuse3` and the tools required to install `ti`: @@ -114,35 +114,61 @@ The following commands use the `ubuntu:24.04` image from the examples above, whi curl -fsSL https://github.com/tidbcloud/ti-cli/releases/latest/download/install.sh | sh -s -- --yes ``` -3. Add `ti` to the current shell, create a local mount directory, and mount the Filesystem: +3. Add `ti` to the current shell and create a local directory for the mount: ```bash export PATH="$HOME/.ti/bin:$PATH" mkdir -p "$HOME/workspace" + ``` + +4. Mount the Filesystem: + ```bash ti fs mount-file-system --mount-path "$HOME/workspace" ``` - On Linux, `ti` uses the FUSE driver by default. + On Linux, `ti` uses FUSE by default. After the command succeeds, you can access the Filesystem through `$HOME/workspace`. If the mount fails to start, inspect the diagnostic log path reported by `ti`. + + If your Filesystem token grants access only to a specific remote path, use the following command instead of the preceding mount command: + + ```bash + ti fs mount-file-system \ + --remote-path /workspace \ + --mount-path "$HOME/workspace" + ``` + + In this example, the remote `/workspace` directory becomes the root of the local mount. For more information, see [Mount only part of the Filesystem](/tidb-cloud-filesystem/filesystem-mount.md#mount-only-part-of-the-filesystem). + + To prevent writes through the local mount, add `--read-only` to the mount command. For example, to mount the Filesystem root as read-only: + + ```bash + ti fs mount-file-system \ + --mount-path "$HOME/workspace" \ + --read-only + ``` + + The `--read-only` option affects this local mount only. Use a scoped token with read-only permissions to enforce read-only access at the Filesystem service. -4. Verify that you can access the Filesystem through the mounted directory: +5. Verify that you can access the mounted Filesystem: ```bash ls "$HOME/workspace" ``` -If your Filesystem token grants access only to a specific path, mount that path with `--remote-path`. For example: +Run the mount and the application that accesses it as the same OS user. If your application image uses a non-root user, install the required packages when building the image and create the mount as the application user at runtime. + +## Flush FUSE writes without unmounting + +If you need pending writes to reach the Filesystem while keeping the mount running, stop applications from writing to the relevant files and close those files first. Then drain the mount inside the container: ```bash -ti fs mount-file-system \ - --remote-path /workspace \ - --mount-path "$HOME/workspace" +ti fs drain-file-system \ + --mount-path "$HOME/workspace" \ + --timeout 30s ``` -To prevent writes through the local mount, add `--read-only`. To enforce read-only access at the Filesystem service, use a scoped token with read-only permissions. - -Run the mount and the application that accesses it as the same OS user. If your application image uses a non-root user, install the required packages when building the image and create the mount as the application user at runtime. +A successful drain confirms that pending writes have reached the Filesystem while leaving the mount running. If the drain times out or returns an error, keep the container and Docker host available, resolve the error, and verify the files before stopping or removing the container. For command syntax, see [`drain-file-system`](/ai/ti/reference/ti-fs-drain-file-system.md). ## Stop the container safely @@ -174,14 +200,14 @@ Before stopping or removing the container: docker compose down ``` -Stopping or removing the container does not delete the remote Filesystem. +Stopping or removing the container does not delete the remote Filesystem or its data. > **Warning:** > -> Do not remove the container after an unmount error. Pending writes might still exist only inside the container. Keep the container running, resolve the error, and verify that the required files have reached the Filesystem first. +> If unmounting fails, do not stop or remove the container. Pending writes might still exist only inside the container. Keep the container running, resolve the error, and verify that the required files have reached the Filesystem first. ## What's next -- [Mount TiDB Cloud Filesystem Locally](/tidb-cloud-filesystem/filesystem-mount.md) for read-only mounts, mounting part of a Filesystem, and safe unmount behavior. +- [Mount TiDB Cloud Filesystem Locally](/tidb-cloud-filesystem/filesystem-mount.md) for read-only mounts, mounting layers or checkpoints, and other common mount options. - [Share a TiDB Cloud Filesystem Across Machines](/tidb-cloud-filesystem/filesystem-sharing.md) to give another user or environment access to the same Filesystem. - [Manage TiDB Cloud Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md) to restrict access to specific paths and operations. diff --git a/tidb-cloud-filesystem/filesystem-mount-linux.md b/tidb-cloud-filesystem/filesystem-mount-linux.md index 9253c12440e3e..093ae624a73a4 100644 --- a/tidb-cloud-filesystem/filesystem-mount-linux.md +++ b/tidb-cloud-filesystem/filesystem-mount-linux.md @@ -5,7 +5,7 @@ summary: Install FUSE userspace tools, mount a TiDB Cloud Filesystem on Linux, v # Mount TiDB Cloud Filesystem on Linux -On Linux, TiDB Cloud Filesystem uses FUSE to make Filesystem data available through a local directory. After mounting, your applications and tools can work with the files by using ordinary local file paths. +On Linux, TiDB Cloud Filesystem uses FUSE to make Filesystem data available through a local directory. After mounting, your applications and tools can access the Filesystem by using ordinary local file paths. > **Note:** > @@ -19,7 +19,7 @@ Before you begin: - Make the Filesystem and its token available to `ti`. See [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). - Use a Linux host where you can install the `fuse3` package. For a container, follow [Docker and Docker Compose](/tidb-cloud-filesystem/filesystem-mount-docker.md) instead. -Run the mount and the application that uses it as the same OS user. +Run the mount and the application that accesses it as the same OS user. ## Install FUSE userspace tools @@ -52,11 +52,11 @@ Run the mount and the application that uses it as the same OS user. If `fusermount3` is not found, make sure the `fuse3` package is installed. If `/dev/fuse` does not exist or you cannot use it when mounting, ask the host administrator to enable FUSE and grant your user access to the device. -On another Linux distribution, install its FUSE package that provides `fusermount3` and perform the same checks. +On another Linux distribution, install the FUSE package that provides `fusermount3` and perform the same checks. ## Mount and verify the Filesystem -1. Create a directory for the mount: +1. Create a local directory for the mount: ```bash mkdir -p "$HOME/workspace" @@ -72,7 +72,29 @@ On another Linux distribution, install its FUSE package that provides `fusermoun On Linux, `ti` uses FUSE by default. - The command waits until the mount is ready before returning. If it succeeds, you can start using `$HOME/workspace` as a local directory. + The command waits until the mount is ready before returning. The mount continues running in the background after the command returns, so closing the terminal does not unmount it. + + After the mount succeeds, you can access the Filesystem through `$HOME/workspace`. + + If your Filesystem token grants access only to a specific remote path, use the following command instead of the preceding mount command: + + ```bash + ti fs mount-file-system \ + --remote-path /workspace \ + --mount-path "$HOME/workspace" + ``` + + In this example, the remote `/workspace` directory becomes the root of the local mount. For more information, see [Mount only part of the Filesystem](/tidb-cloud-filesystem/filesystem-mount.md#mount-only-part-of-the-filesystem). + + To prevent writes through the local mount, add `--read-only` to the mount command. For example, to mount the Filesystem root as read-only: + + ```bash + ti fs mount-file-system \ + --mount-path "$HOME/workspace" \ + --read-only + ``` + + The `--read-only` option affects this local mount only. Use a scoped token with read-only permissions to enforce read-only access at the Filesystem service. 3. Verify that you can access the mounted Filesystem: @@ -80,9 +102,9 @@ On another Linux distribution, install its FUSE package that provides `fusermoun ls "$HOME/workspace" ``` - The command lists the files and directories at the root of the Filesystem. + The command lists the files and directories at the root of the mounted path. - If your token has write permission, you can also create and read a test file: + If you used a writable mount and your token has write permission, you can also create and read a test file: ```bash TEST_FILE="mount-check-$(date +%s).txt" @@ -105,7 +127,7 @@ On another Linux distribution, install its FUSE package that provides `fusermoun A successful unmount flushes pending FUSE writes before stopping the mount. - If you created the test file above, you can optionally verify after unmounting that the file reached the Filesystem: + If you created the test file above, you can optionally verify after unmounting that the file is available directly from the Filesystem: ```bash ti fs read-file --path "/$TEST_FILE" @@ -119,7 +141,21 @@ On another Linux distribution, install its FUSE package that provides `fusermoun Unmounting removes the local mount but does not delete the Filesystem or its data. -If you need pending writes to reach the Filesystem while keeping the mount running, see [Make FUSE writes available without unmounting](/tidb-cloud-filesystem/filesystem-mount.md#make-fuse-writes-available-without-unmounting). +> **Warning:** +> +> If unmounting fails, keep the mount and machine running until you resolve the error and verify that the required files have reached the Filesystem. Some pending writes might still exist only on that machine. + +## Flush FUSE writes without unmounting + +If you need pending writes to reach the Filesystem while keeping the mount running, stop applications from writing to the relevant files and close those files first. Then drain the mount: + +```bash +ti fs drain-file-system \ + --mount-path "$HOME/workspace" \ + --timeout 30s +``` + +A successful drain confirms that pending writes have reached the Filesystem while leaving the mount running. If the drain times out or returns an error, keep the mount and machine available, resolve the error, and verify the files before ending the session or telling another user that the updates are ready. For command syntax, see [`drain-file-system`](/ai/ti/reference/ti-fs-drain-file-system.md). ## Troubleshoot mount permission errors @@ -182,6 +218,6 @@ For help checking whether AppArmor caused the failure, see [Ubuntu 26.04 rejects ## What's next -- [Mount TiDB Cloud Filesystem Locally](/tidb-cloud-filesystem/filesystem-mount.md) for read-only mounts, mounting part of a Filesystem, and safe unmount behavior. +- [Mount TiDB Cloud Filesystem Locally](/tidb-cloud-filesystem/filesystem-mount.md) for read-only mounts, mounting layers or checkpoints, and other common mount options. - [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) to mount and work with layers or checkpoints. - [Share a TiDB Cloud Filesystem Across Machines](/tidb-cloud-filesystem/filesystem-sharing.md) to give another user or environment access to the Filesystem. diff --git a/tidb-cloud-filesystem/filesystem-mount-macos.md b/tidb-cloud-filesystem/filesystem-mount-macos.md index ca31a0a79de6b..ddf1676969ecf 100644 --- a/tidb-cloud-filesystem/filesystem-mount-macos.md +++ b/tidb-cloud-filesystem/filesystem-mount-macos.md @@ -1,15 +1,17 @@ --- title: Mount TiDB Cloud Filesystem on macOS -summary: Mount a TiDB Cloud Filesystem as a local directory on macOS, and use macFUSE when you need advanced mount features. +summary: Mount a TiDB Cloud Filesystem as a local directory on macOS, and use macFUSE when you need FUSE-specific features. --- # Mount TiDB Cloud Filesystem on macOS On macOS, you can mount a TiDB Cloud Filesystem as a local directory and access its files with your usual applications and tools. -For most workflows, no additional mount software is required. Without macFUSE, TiDB Cloud CLI (`ti`) uses WebDAV. If macFUSE is installed, `ti` prefers FUSE in automatic driver selection. +For most workflows, you can mount a TiDB Cloud Filesystem with WebDAV. It lets you access Filesystem files through normal local file paths and does not require additional mount software. -If you need features that require FUSE, such as mounting layers or checkpoints or using `drain-file-system`, install macFUSE and use the FUSE driver instead. +Use FUSE with macFUSE when you also need FUSE-specific features, such as mounting a layer or checkpoint, or making pending writes reach the Filesystem without unmounting it. + +Without macFUSE, TiDB Cloud CLI (`ti`) uses WebDAV. If macFUSE is installed, `ti` prefers FUSE when the mount driver is selected automatically. The commands in this guide specify the driver explicitly so that you know which mount method is being used. > **Note:** > @@ -26,7 +28,7 @@ The write examples below require a token with write permission. ## Mount with WebDAV -To explicitly use WebDAV, pass `--driver webdav`. +For most workflows, you can use WebDAV without installing additional mount software. 1. Create a local directory for the mount: @@ -34,23 +36,47 @@ To explicitly use WebDAV, pass `--driver webdav`. mkdir -p "$HOME/workspace" ``` -2. Mount the Filesystem: +2. Mount the Filesystem with WebDAV: + + ```bash + ti fs mount-file-system \ + --mount-path "$HOME/workspace" \ + --driver webdav + ``` + + The mount continues running in the background after the command returns, so closing the terminal does not unmount it. + + After the command succeeds, you can access the Filesystem through `$HOME/workspace`. + + If your Filesystem token grants access only to a specific remote path, use the following command instead of the preceding mount command: ```bash ti fs mount-file-system \ + --remote-path /workspace \ --mount-path "$HOME/workspace" \ --driver webdav ``` - This command explicitly selects WebDAV. + In this example, the remote `/workspace` directory becomes the root of the local mount. For more information, see [Mount only part of the Filesystem](/tidb-cloud-filesystem/filesystem-mount.md#mount-only-part-of-the-filesystem). -3. Access the Filesystem through the mounted directory: + To prevent writes through the local mount, add `--read-only` to the mount command. For example, to mount the Filesystem root as read-only: + + ```bash + ti fs mount-file-system \ + --mount-path "$HOME/workspace" \ + --driver webdav \ + --read-only + ``` + + The `--read-only` option affects this local mount only. Use a scoped token with read-only permissions to enforce read-only access at the Filesystem service. + +3. Verify that you can access the mounted Filesystem: ```bash ls "$HOME/workspace" ``` - If your token has write permission, you can also create and read a test file: + If you used a writable mount and your token has write permission, you can also create and read a test file: ```bash TEST_FILE="mount-check-$(date +%s).txt" @@ -59,6 +85,12 @@ To explicitly use WebDAV, pass `--driver webdav`. cat "$HOME/workspace/$TEST_FILE" ``` + Example output: + + ```text + Hello from macOS + ``` + 4. When you are finished, close files that are open in applications and unmount the Filesystem: ```bash @@ -67,19 +99,25 @@ To explicitly use WebDAV, pass `--driver webdav`. WebDAV does not support `drain-file-system`. Complete a normal unmount before shutting down the machine or handing updated files to another user or environment. - If you created the test file above, you can optionally confirm that it is available directly from the Filesystem: + If you created the test file above, you can optionally verify after unmounting that the file is available directly from the Filesystem: ```bash ti fs read-file --path "/$TEST_FILE" ``` + Example output: + + ```text + Hello from macOS + ``` + Unmounting removes the local mount but does not delete the Filesystem or its data. ## Use macFUSE when you need FUSE features Use FUSE instead of WebDAV when you need to: -- mount a layer or checkpoint, or +- mount a layer or checkpoint; or - make pending writes reach the Filesystem while keeping the mount running with `drain-file-system`. To use FUSE on macOS: @@ -110,9 +148,15 @@ To use FUSE on macOS: --driver fuse ``` + The mount continues running in the background after the command returns, so closing the terminal does not unmount it. + + Run the FUSE mount and the applications that access it as the same OS user. + + If your Filesystem token grants access only to a specific remote path, add `--remote-path` as described in [Mount only part of the Filesystem](/tidb-cloud-filesystem/filesystem-mount.md#mount-only-part-of-the-filesystem) instead of mounting the Filesystem root. + Layer and checkpoint mounts require FUSE, and checkpoint mounts are always read-only. For details, see [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md). - If you need pending writes to reach the Filesystem while keeping the mount running, see [Make FUSE writes available without unmounting](/tidb-cloud-filesystem/filesystem-mount.md#make-fuse-writes-available-without-unmounting). + If you need pending writes to reach the Filesystem while keeping the mount running, see [FUSE write behavior](/tidb-cloud-filesystem/filesystem-mount.md#fuse-write-behavior). For command syntax, see [`drain-file-system`](/ai/ti/reference/ti-fs-drain-file-system.md). 4. When you are finished, stop applications from writing to the mount, close open files, and unmount it: @@ -122,18 +166,34 @@ To use FUSE on macOS: A successful FUSE unmount flushes pending writes. You do not need to run `drain-file-system` before a normal unmount. +Unmounting removes the local mount but does not delete the Filesystem or its data. + +## Flush FUSE writes without unmounting + +If you need pending writes to reach the Filesystem while keeping the FUSE mount running, stop applications from writing to the relevant files and close those files first. Then drain the mount: + +```bash +ti fs drain-file-system \ + --mount-path "$HOME/workspace" \ + --timeout 30s +``` + +A successful drain confirms that pending writes have reached the Filesystem while leaving the mount running. If the drain times out or returns an error, keep the mount and machine available, resolve the error, and verify the files before ending the session or telling another user that the updates are ready. WebDAV does not support `drain-file-system`. For command syntax, see [`drain-file-system`](/ai/ti/reference/ti-fs-drain-file-system.md). + ## Troubleshoot mount issues If a WebDAV mount fails to start: - Make sure the local mount directory exists and is writable. -- Check the diagnostic log path reported by `ti` for the underlying error. - Make sure another mount is not already using the same local directory. +- Check the diagnostic log path reported by `ti` for the underlying error. If a FUSE mount fails to start: - Make sure macFUSE is installed. - Complete any macOS security approvals required by macFUSE. +- Make sure the mount and the application that accesses it run as the same OS user. +- Make sure another mount is not already using the same local directory. - Check the diagnostic log path reported by `ti`. If unmounting fails, keep the mount process and machine running until you resolve the error and verify that required files have reached the Filesystem. Do not remove local mount data while pending writes might remain. @@ -142,6 +202,6 @@ For additional mount errors, see [Troubleshoot TiDB Cloud Filesystem](/tidb-clou ## What's next -- [Mount TiDB Cloud Filesystem Locally](/tidb-cloud-filesystem/filesystem-mount.md) for read-only mounts, mounting part of a Filesystem, and safe unmount behavior. +- [Mount TiDB Cloud Filesystem Locally](/tidb-cloud-filesystem/filesystem-mount.md) for read-only mounts, mounting layers or checkpoints, and other common mount options. - [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) to work with layers and checkpoints through FUSE. - [Share a TiDB Cloud Filesystem Across Machines](/tidb-cloud-filesystem/filesystem-sharing.md) to give another user or environment access to the Filesystem. diff --git a/tidb-cloud-filesystem/filesystem-mount.md b/tidb-cloud-filesystem/filesystem-mount.md index 9347bd5f6a1b9..be9abd3712c10 100644 --- a/tidb-cloud-filesystem/filesystem-mount.md +++ b/tidb-cloud-filesystem/filesystem-mount.md @@ -1,147 +1,90 @@ --- -title: Mount TiDB Cloud Filesystem Locally -summary: Mount an existing TiDB Cloud Filesystem as a local directory, use its files with local tools, and unmount it safely. +title: Mount TiDB Cloud Filesystem Overview +summary: Learn when to mount a TiDB Cloud Filesystem, which mount method to use, and the capabilities and limitations of Filesystem mounts. aliases: ['/ai/mount-filesystem'] --- -# Mount TiDB Cloud Filesystem Locally +# Mount TiDB Cloud Filesystem Overview -Mount a TiDB Cloud Filesystem when your editor, application, or agent needs to access Filesystem data through local file paths. +You can work with files in TiDB Cloud Filesystem either by using `ti fs` commands directly or by mounting the Filesystem as a local directory. -After mounting, you can use ordinary local tools to read and write files in the mounted directory. Unmounting removes the local access point but does not delete the Filesystem or its data. +Mount your TiDB Cloud Filesystem when an editor, application, agent, or other tool needs to access its files through local file paths. After mounting, the Filesystem appears as a local directory, so the tool can read and write its files using normal filesystem operations. + +If you only need to perform file operations through the CLI, such as reading, copying, organizing, or searching files, use `ti fs` commands directly instead of mounting the Filesystem. > **Note:** > > TiDB Cloud Filesystem is currently in public preview. Its features and interfaces are subject to change without notice. -## Before you begin +## Choose a mount method -Before mounting a Filesystem: +The available mount method depends on your environment: -- [Install TiDB Cloud CLI (`ti`)](/tidb-cloud-filesystem/filesystem-quick-start.md#step-1-install-the-cli). -- Make the Filesystem and its token available to `ti`. See [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). -- Complete the mount setup for your environment: [Linux](/tidb-cloud-filesystem/filesystem-mount-linux.md), [macOS](/tidb-cloud-filesystem/filesystem-mount-macos.md), or [Docker and Docker Compose](/tidb-cloud-filesystem/filesystem-mount-docker.md). +| Environment | Mount method | Detailed Guide | +| --- | --- | --- | +| Linux | FUSE | [Mount TiDB Cloud Filesystem on Linux](/tidb-cloud-filesystem/filesystem-mount-linux.md) | +| macOS | WebDAV for general file access, or FUSE with macFUSE for FUSE-specific features | [Mount TiDB Cloud Filesystem on macOS](/tidb-cloud-filesystem/filesystem-mount-macos.md) | +| Docker on Linux | FUSE with access to `/dev/fuse` and additional container privileges | [Mount TiDB Cloud Filesystem in Docker](/tidb-cloud-filesystem/filesystem-mount-docker.md) | Native Filesystem mounting is not supported on Windows. On Windows, use direct commands such as `ti fs copy-file`, `ti fs read-file`, and `ti fs list-files` instead. -You normally do not need to choose a mount driver manually. `ti` uses FUSE on Linux. On macOS without macFUSE, `ti` uses WebDAV; if macFUSE is installed, `ti` prefers FUSE in automatic driver selection. On macOS, use FUSE when you need layers, checkpoints, or the drain operation. - -## Mount and use the Filesystem - -To mount the Filesystem: - -1. Create a local directory for the mount: - - ```bash - mkdir -p "$HOME/workspace" - ``` - -2. Mount the Filesystem: - - ```bash - ti fs mount-file-system --mount-path "$HOME/workspace" - ``` - - The command waits until the mount is ready before returning. The mount continues running in the background, so closing the terminal does not unmount it. - - If the mount fails to start, check the diagnostic log path reported by the CLI. - -3. Access the Filesystem through the mounted directory using ordinary local tools: - - ```bash - ls "$HOME/workspace" - - printf 'Written through the mount\n' > "$HOME/workspace/mounted.txt" - - cat "$HOME/workspace/mounted.txt" - ``` - - The write example requires a token with write permission and a writable mount. - -If you use FUSE, run the mount and the applications that access it as the same OS user. Changing the ownership of the mount directory does not give another user access through an existing FUSE mount. - -### Mount only part of the Filesystem +On macOS, WebDAV is sufficient for general file access and does not require additional mount software. FUSE is required for features such as mounting layers or checkpoints and using `drain-file-system`. -By default, the Filesystem root `/` is mounted. +## Use a token without configuring a profile -To mount only a specific directory, use `--remote-path`. For example: +If another user or system administrator gives you a Filesystem token, you can use that token from the current machine without configuring a `ti` profile or using the creator's API keys: ```bash -ti fs mount-file-system \ - --remote-path /workspace \ - --mount-path "$HOME/workspace" +export TI_FS_TOKEN="" +export TI_REGION_CODE="" ``` -In this example, the remote `/workspace` directory becomes the root of the local mount. For example, `/workspace/project.md` is available locally as `$HOME/workspace/project.md`. +Then follow the guide for your environment. The token identifies the Filesystem and limits the paths and operations available to the current environment. Treat the token as a secret. -If you use a scoped token that grants access only to a specific path, mount that path rather than the Filesystem root. +## Common mount capabilities -### Create a read-only mount +Filesystem mounts support several options that change what is exposed through the local mount: -To prevent writes through the local mount, add `--read-only`: +### Mount only part of the Filesystem -```bash -ti fs mount-file-system \ - --mount-path "$HOME/workspace" \ - --read-only -``` +By default, a mount exposes the Filesystem root `/`. Use `--remote-path` to expose a specific remote directory instead. This is also useful when a scoped token grants access only to a specific path. -The `--read-only` option prevents writes through this mount, but it does not change the permissions of the Filesystem token. To enforce read-only access at the service, use a scoped token that grants only the required read access. +### Create a read-only mount -### Mount a layer or checkpoint +Use `--read-only` to prevent writes through a particular mount. This option does not change the permissions of the Filesystem token. To enforce read-only access at the service level, use a scoped token with read-only permissions. -Layer and checkpoint mounts require FUSE. On macOS, install macFUSE and specify `--driver fuse`. +### Mount layers and checkpoints -Checkpoint mounts are always read-only. +Layers and checkpoints require FUSE. Checkpoint mounts are always read-only. -For layer and checkpoint workflows, see [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md). For all mount options, see the [`mount-file-system` command reference](/ai/ti/reference/ti-fs-mount-file-system.md). +For layer and checkpoint workflows, see [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md). ## Finish safely -In most cases, simply stop writing and unmount the Filesystem. You only need to drain a FUSE mount when you want pending writes to reach the Filesystem while keeping the mount running. - ### Unmount when you are finished -When you no longer need the mount: - -1. Stop applications from writing to the mounted directory and close any open files. - -2. Unmount the Filesystem: +Stop applications from writing to the mounted directory, close open files, and follow your platform guide to run `ti fs unmount-file-system`. A successful FUSE unmount flushes pending writes before stopping the mount. Unmounting removes the local mount but does not delete the Filesystem or its data. - ```bash - ti fs unmount-file-system --mount-path "$HOME/workspace" - ``` +### FUSE write behavior -A successful FUSE unmount flushes pending writes before stopping the mount. You do not need to run `drain-file-system` before a normal unmount. +FUSE mounts can temporarily have writes that have not yet reached the remote Filesystem. -For a WebDAV mount, close open files and unmount normally. WebDAV does not support the drain operation. +A normal successful unmount flushes pending writes, so you do not need to run `drain-file-system` before unmounting. -Unmounting removes the local mount but does not delete the Filesystem or its data. +Use `drain-file-system` only when you need pending writes to reach the Filesystem while keeping the mount running, such as before creating a checkpoint or making updated files available to another environment. -### Make FUSE writes available without unmounting - -If you want to keep a FUSE mount running but need pending writes to reach the Filesystem—for example, before creating a checkpoint or sharing updated files with another machine: - -1. Stop applications from writing to the relevant files and close those files. - -2. Drain the mount: - - ```bash - ti fs drain-file-system \ - --mount-path "$HOME/workspace" \ - --timeout 30s - ``` - -A successful drain confirms that pending writes have reached the service while leaving the mount running. - -If the drain times out or returns an error, some writes might not have reached the Filesystem. Keep the mount and machine available, resolve the error, and verify the files before ending the session or telling another user that the updates are ready. +WebDAV mounts do not support `drain-file-system`. > **Warning:** > -> If a drain or unmount fails, do not shut down or destroy the machine or sandbox, stop the mount process, or delete its local data. Some pending writes might still exist only on that machine. Resolve the error and verify that the required files have reached the Filesystem first. +> If a FUSE drain or unmount fails, keep the mount and machine available until you resolve the error and verify that required files have reached the Filesystem. Some pending writes might still exist only on that machine. ## What's next -- [Share a TiDB Cloud Filesystem Across Machines](/tidb-cloud-filesystem/filesystem-sharing.md) to give another user or environment access to the same Filesystem. -- [Manage TiDB Cloud Filesystem Layers and Checkpoints](/tidb-cloud-filesystem/manage-filesystem-layers.md) to make and review isolated changes before applying them to the base Filesystem. -- See the [`mount-file-system` command reference](/ai/ti/reference/ti-fs-mount-file-system.md) for all mount options. +Choose the guide for your environment: + +- [Mount TiDB Cloud Filesystem on Linux](/tidb-cloud-filesystem/filesystem-mount-linux.md) +- [Mount TiDB Cloud Filesystem on macOS](/tidb-cloud-filesystem/filesystem-mount-macos.md) +- [Mount TiDB Cloud Filesystem in Docker](/tidb-cloud-filesystem/filesystem-mount-docker.md) + +For all mount options, see the [`mount-file-system` command reference](/ai/ti/reference/ti-fs-mount-file-system.md). From 22780c1b2fa2c96852efad0dfb8602bc08f8dc33 Mon Sep 17 00:00:00 2001 From: qiancai Date: Sun, 20 Sep 2026 10:02:35 +0800 Subject: [PATCH 13/14] update manage-filesystem-vault-secrets.md --- tidb-cloud-filesystem/filesystem-quick-start.md | 4 ++-- tidb-cloud-filesystem/filesystem-regions-and-limitations.md | 2 +- tidb-cloud-filesystem/manage-filesystem-vault-secrets.md | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/tidb-cloud-filesystem/filesystem-quick-start.md b/tidb-cloud-filesystem/filesystem-quick-start.md index 111c297d6509e..3efba4ea8c496 100644 --- a/tidb-cloud-filesystem/filesystem-quick-start.md +++ b/tidb-cloud-filesystem/filesystem-quick-start.md @@ -13,9 +13,9 @@ Create a workspace when your application's or agent's files need to outlive the ## Prerequisites -Obtain a TiDB Cloud API public key and private key from the [API Keys page](https://tidbcloud.com/org-settings/api-keys). The keys must have permission to create a Filesystem in your organization. +To create a Filesystem, obtain a TiDB Cloud API public key and private key from the [API Keys page](https://tidbcloud.com/org-settings/api-keys). The keys must have permission to create a Filesystem in your organization. -If someone has already supplied you with an FS token, skip resource creation and follow [Mounting Locally](/tidb-cloud-filesystem/filesystem-mount.md#use-a-token-without-configuring-a-profile). +If someone has already supplied you with an FS token, skip resource creation and follow [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). ## Step 1. Install the CLI diff --git a/tidb-cloud-filesystem/filesystem-regions-and-limitations.md b/tidb-cloud-filesystem/filesystem-regions-and-limitations.md index cd9c5c694a75d..9c4ee30b4ca30 100644 --- a/tidb-cloud-filesystem/filesystem-regions-and-limitations.md +++ b/tidb-cloud-filesystem/filesystem-regions-and-limitations.md @@ -39,7 +39,7 @@ Ubuntu 26.04 also confines `fusermount3` with AppArmor. Use a mount path under ` ## Durability boundaries - Default FUSE behavior uses local buffering and asynchronous remote work where the companion permits it. Abruptly killing the mount process or deleting a machine can lose uncommitted memory or write-back state. -- The default coding-agent mount profile stores dependency trees, generated output, caches, and Git internals locally. Local-only data disappears when its disk disappears unless it is packed or otherwise preserved. +- When you select the `coding-agent` mount profile with `--mount-profile coding-agent`, dependency trees, generated output, caches, and Git internals are stored locally. Local-only data disappears when its disk disappears unless it is packed or otherwise preserved. - Remote-committed Filesystem data survives client or sandbox deletion; deleting the machine does not delete the remote Filesystem. For the supported drain and graceful unmount workflow, see [Finish safely](/tidb-cloud-filesystem/filesystem-mount.md#finish-safely). `drain-file-system` is a FUSE-only online durability barrier; WebDAV does not support it. diff --git a/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md b/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md index feb93c72dba66..af70cdf12ae5b 100644 --- a/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md +++ b/tidb-cloud-filesystem/manage-filesystem-vault-secrets.md @@ -96,7 +96,7 @@ ti fs-vault create-grant \ The command returns a delegated Vault token and a grant ID. Give the delegated token only to the workflow that needs the secret, and retain the grant ID so that you can revoke the grant before it expires if necessary. -In the environment that uses the delegated secret, make the token available as `TI_VAULT_TOKEN`. Avoid putting the token directly in a command-line argument because command arguments can appear in shell history or process listings. +In the environment that uses the delegated secret, make the token available as `TI_VAULT_TOKEN`. Also set `TI_FS_FILE_SYSTEM_ID` to the Filesystem ID and `TI_REGION_CODE` to its region code. The delegated Vault token alone does not identify the Filesystem. Avoid putting the token directly in a command-line argument because command arguments can appear in shell history or process listings. ## Inject a secret into a process @@ -108,7 +108,7 @@ ti fs-vault run-with-secret \ -- ``` -Each permitted secret field becomes an environment variable with the same name. For example, the `DB_URL` and `PASSWORD` fields become the `DB_URL` and `PASSWORD` environment variables. +Each permitted secret field becomes an environment variable with the same name. With the `db-prod/DB_URL` grant in this example, `ti` injects `DB_URL` into the child process, but does not inject `PASSWORD`. The Vault credential used by `ti` is not passed to the child process. This lets the application use the secret without writing its plaintext value to a file. From caad6d0039eb711d0fa3b8f829e8cdc81f6e5c36 Mon Sep 17 00:00:00 2001 From: qiancai Date: Sun, 20 Sep 2026 12:10:07 +0800 Subject: [PATCH 14/14] unify the usage of FS tokens --- ai/ti/guides/manage-filesystems-via-cli.md | 2 +- ai/ti/guides/ti-daily-workflow-example.md | 2 +- ai/ti/guides/ti-journal-agent-workflow-example.md | 2 +- ai/ti/guides/ti-persistent-agent-state-example.md | 2 +- .../guides/ti-share-filesystem-across-machines-example.md | 2 +- ai/ti/reference/ti-cli-reference.md | 2 +- ai/ti/reference/ti-configuration-and-credentials.md | 8 ++++---- ai/ti/reference/ti-fs-delete-file-system-token.md | 2 +- ai/ti/reference/ti-fs-delete-file-system.md | 2 +- ai/ti/reference/ti-fs-disable-file-system-token.md | 2 +- ai/ti/reference/ti-fs-enable-file-system-token.md | 2 +- .../reference/ti-fs-generate-file-system-scoped-token.md | 2 +- ai/ti/reference/ti-fs-generate-file-system-token.md | 2 +- ai/ti/reference/ti-fs-list-file-system-tokens.md | 2 +- ai/ti/reference/ti-fs-vault-list-secrets.md | 2 +- ai/ti/reference/ti-fs-vault-mount-vault.md | 2 +- ai/ti/reference/ti-fs-vault-read-secret.md | 2 +- ai/ti/reference/ti-fs-vault-run-with-secret.md | 2 +- tidb-cloud-filesystem/filesystem-quick-start.md | 2 +- tidb-cloud-filesystem/filesystem-troubleshooting.md | 6 +++--- tidb-cloud-filesystem/manage-filesystem-resources.md | 6 +++--- tidb-cloud-filesystem/manage-filesystem-tokens.md | 8 +++++--- 22 files changed, 33 insertions(+), 31 deletions(-) diff --git a/ai/ti/guides/manage-filesystems-via-cli.md b/ai/ti/guides/manage-filesystems-via-cli.md index 696fbda84a297..f27a64a0601bc 100644 --- a/ai/ti/guides/manage-filesystems-via-cli.md +++ b/ai/ti/guides/manage-filesystems-via-cli.md @@ -17,7 +17,7 @@ Create a Filesystem and wait until it is ready: ti fs create-file-system --display-name my-workspace --wait ``` -Copy the returned `file_system_id` and replace `` in the following commands. The CLI stores the Filesystem token locally. Treat the returned `fs_token` as a secret; do not share the command output publicly. +Copy the returned `file_system_id` and replace `` in the following commands. The CLI stores the Filesystem token locally. Treat the returned `fs_token` (Filesystem token) as a secret; do not share the command output publicly. ```shell echo "Hello from my workspace" | ti fs copy-file --file-system-id "" --from-stdin --to-remote /hello.txt diff --git a/ai/ti/guides/ti-daily-workflow-example.md b/ai/ti/guides/ti-daily-workflow-example.md index 2ced8bd8863fa..baa9d43431477 100644 --- a/ai/ti/guides/ti-daily-workflow-example.md +++ b/ai/ti/guides/ti-daily-workflow-example.md @@ -115,7 +115,7 @@ ti db delete-db-cluster \ ## Security notes -- Do not echo FS tokens or formatted database connection strings. +- Do not echo Filesystem tokens or formatted database connection strings. - Use unique automation prefixes and delete only resources created by that run. - Preview destructive operations with `--dry-run`. diff --git a/ai/ti/guides/ti-journal-agent-workflow-example.md b/ai/ti/guides/ti-journal-agent-workflow-example.md index f1ef2b58efbf1..632930a97b575 100644 --- a/ai/ti/guides/ti-journal-agent-workflow-example.md +++ b/ai/ti/guides/ti-journal-agent-workflow-example.md @@ -17,7 +17,7 @@ A Filesystem journal stores structured append-only entries with sequence informa ## Prerequisites -Select a Filesystem through a configured profile or FS token environment. +Select a Filesystem through a configured profile or Filesystem token environment. ## Step 1. Create the journal diff --git a/ai/ti/guides/ti-persistent-agent-state-example.md b/ai/ti/guides/ti-persistent-agent-state-example.md index 983395793e16c..9ea9f91d7e9cf 100644 --- a/ai/ti/guides/ti-persistent-agent-state-example.md +++ b/ai/ti/guides/ti-persistent-agent-state-example.md @@ -87,7 +87,7 @@ Deleting the Filesystem also deletes its task files and journals. ## Security and operational notes -- The FS token is an owner credential. Keep it in a runtime secret store and do not include it in images or task prompts. +- The Filesystem token is an owner credential. Keep it in a runtime secret store and do not include it in images or task prompts. - A completed direct data-plane write is remotely visible. For mounted FUSE writes, unmount gracefully before deleting the sandbox. - Journals preserve ordered workflow evidence; task files preserve mutable working state. Use both when you need state and history. diff --git a/ai/ti/guides/ti-share-filesystem-across-machines-example.md b/ai/ti/guides/ti-share-filesystem-across-machines-example.md index 5a0a56b1752ad..7fee99c3300db 100644 --- a/ai/ti/guides/ti-share-filesystem-across-machines-example.md +++ b/ai/ti/guides/ti-share-filesystem-across-machines-example.md @@ -108,7 +108,7 @@ ti fs delete-file-system \ ## Security notes -- Each FS token grants owner access. Transfer it as a secret, not in chat or command history, and use a separate token for each machine. +- Each Filesystem token grants owner access. Transfer it as a secret, not in chat or command history, and use a separate token for each machine. - Concurrent writers can overwrite the same paths; coordinate ownership at the workflow level. - Do not terminate a machine before graceful unmount completes. Use an explicit drain only when you need remote durability while keeping the FUSE mount online. diff --git a/ai/ti/reference/ti-cli-reference.md b/ai/ti/reference/ti-cli-reference.md index 50465b2cf2256..30c7588b94d70 100644 --- a/ai/ti/reference/ti-cli-reference.md +++ b/ai/ti/reference/ti-cli-reference.md @@ -148,7 +148,7 @@ ti [ERROR]: Errors are written to stderr and successful command output is written to stdout. Usage and configuration failures return a nonzero exit code before remote mutation. Runtime and remote API failures also return nonzero. An interrupted interactive configuration returns exit code `130`. -`--debug` can show redacted request and resolution context. It must not show API keys, FS tokens, DB passwords, SQL text, file contents, or connection strings. +`--debug` can show redacted request and resolution context. It must not show API keys, Filesystem tokens, DB passwords, SQL text, file contents, or connection strings. ## Related documentation diff --git a/ai/ti/reference/ti-configuration-and-credentials.md b/ai/ti/reference/ti-configuration-and-credentials.md index 320d960a212cb..b8f0a51e3e635 100644 --- a/ai/ti/reference/ti-configuration-and-credentials.md +++ b/ai/ti/reference/ti-configuration-and-credentials.md @@ -92,13 +92,13 @@ One remote Filesystem can have multiple tokens, but each profile stores at most `ti fs generate-file-system-scoped-token` accepts only an owner token and can store its authoritative path scopes locally. The token JWT itself contains the Filesystem ID but not the token kind, token ID, or scopes. Therefore, an explicit or environment token is passed to the service for authorization instead of being classified locally. `TI_FS_TOKEN` can contain either an owner token or a scoped token; available operations depend on its server-side capability. -Owner FS tokens authorize Filesystem data access and token inventory or lifecycle operations. They do not authorize TiDB Cloud Filesystem resource creation, listing, description, or deletion, and they cannot generate another owner token. Those operations require TiDB Cloud API credentials. `ti fs delete-file-system` additionally requires an explicit `--file-system-id`; the ID embedded in `TI_FS_TOKEN` is never used to select a Filesystem for deletion. +Filesystem owner tokens authorize Filesystem data access and token inventory or lifecycle operations. They do not authorize TiDB Cloud Filesystem resource creation, listing, description, or deletion, and they cannot generate another owner token. Those operations require TiDB Cloud API credentials. `ti fs delete-file-system` additionally requires an explicit `--file-system-id`; the ID embedded in `TI_FS_TOKEN` is never used to select a Filesystem for deletion. Resource selection is: 1. explicit `--file-system-id`; 2. `TI_FS_FILE_SYSTEM_ID`; -3. derive the ID from an explicitly supplied FS token; +3. derive the ID from an explicitly supplied Filesystem token; 4. otherwise fail with `fs.missing_file_system_id`. `ti` never infers a Filesystem from a saved default or from the number of local credentials. Use `--file-system-id` for one command or `TI_FS_FILE_SYSTEM_ID` for a shell, sandbox, or automation environment. @@ -120,7 +120,7 @@ export TI_FS_TOKEN="" export TI_REGION_CODE="aws-us-east-1" ``` -These values form an in-memory namespace only. `ti` derives the ID from the token and does not write either value to `~/.ti/`. `TI_FS_FILE_SYSTEM_ID` is optional and, when present, must match the token. Remote Filesystem inventory, description, provisioning, and deletion require TiDB Cloud API credentials. An FS token is neither required nor accepted as authorization for Filesystem deletion. +These values form an in-memory namespace only. `ti` derives the ID from the token and does not write either value to `~/.ti/`. `TI_FS_FILE_SYSTEM_ID` is optional and, when present, must match the token. Remote Filesystem inventory, description, provisioning, and deletion require TiDB Cloud API credentials. A Filesystem token is neither required nor accepted as authorization for Filesystem deletion. ## DB SQL credentials @@ -172,7 +172,7 @@ A successful background FS or vault mount writes a non-secret locator: ~/.ti/mounts/.locator.json ``` -The locator records the placement and companion-home information required for drain and unmount from the same `HOME`. It does not contain the FS token. Successful unmount removes it. +The locator records the placement and companion-home information required for drain and unmount from the same `HOME`. It does not contain the Filesystem token. Successful unmount removes it. ## Operation logs diff --git a/ai/ti/reference/ti-fs-delete-file-system-token.md b/ai/ti/reference/ti-fs-delete-file-system-token.md index 2eba42e4a74cc..e5e62b4426c37 100644 --- a/ai/ti/reference/ti-fs-delete-file-system-token.md +++ b/ai/ti/reference/ti-fs-delete-file-system-token.md @@ -27,7 +27,7 @@ ti fs delete-file-system-token - `--file-system-id `: Specify the Filesystem that owns the token. Required when using TiDB Cloud API credentials; optional when an owner token supplies the ID. - `--token-id `: Specify the immutable token ID returned by the list command. This option is required. -- `--fs-token `: Authorize the request with an owner Filesystem token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. If no Filesystem token is available, the command uses the configured TiDB Cloud API keys. +- `--fs-token `: Authorize the request with a Filesystem owner token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. If no Filesystem token is available, the command uses the configured TiDB Cloud API keys. - `--dry-run`: Validate credentials, identifiers, and known local mount conflicts without revoking the token. - `--help`: Display help information. - `--version`: Display version information. diff --git a/ai/ti/reference/ti-fs-delete-file-system.md b/ai/ti/reference/ti-fs-delete-file-system.md index 744df92789464..6b8ada41589c7 100644 --- a/ai/ti/reference/ti-fs-delete-file-system.md +++ b/ai/ti/reference/ti-fs-delete-file-system.md @@ -23,7 +23,7 @@ ti fs delete-file-system ## Options -- `--file-system-id `: Set the immutable Filesystem ID. FS tokens cannot replace this option or authorize Filesystem deletion. \[required] +- `--file-system-id `: Set the immutable Filesystem ID. Filesystem tokens cannot replace this option or authorize Filesystem deletion. \[required] - `--dry-run`: Validate the request without applying changes. - `--help`: Display help information. - `--version`: Display version information. diff --git a/ai/ti/reference/ti-fs-disable-file-system-token.md b/ai/ti/reference/ti-fs-disable-file-system-token.md index 60f3fc5fc44f7..9e842b03530e9 100644 --- a/ai/ti/reference/ti-fs-disable-file-system-token.md +++ b/ai/ti/reference/ti-fs-disable-file-system-token.md @@ -27,7 +27,7 @@ ti fs disable-file-system-token - `--file-system-id `: Specify the Filesystem that owns the token. Required when using TiDB Cloud API credentials; optional when an owner token supplies the ID. - `--token-id `: Specify the immutable token ID returned by the list command. This option is required. -- `--fs-token `: Authorize the request with an owner Filesystem token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. If no Filesystem token is available, the command uses the configured TiDB Cloud API keys. +- `--fs-token `: Authorize the request with a Filesystem owner token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. If no Filesystem token is available, the command uses the configured TiDB Cloud API keys. - `--dry-run`: Validate credentials, identifiers, and known local mount conflicts without disabling the token. - `--help`: Display help information. - `--version`: Display version information. diff --git a/ai/ti/reference/ti-fs-enable-file-system-token.md b/ai/ti/reference/ti-fs-enable-file-system-token.md index b643e55e0b25b..f53b134aa8862 100644 --- a/ai/ti/reference/ti-fs-enable-file-system-token.md +++ b/ai/ti/reference/ti-fs-enable-file-system-token.md @@ -27,7 +27,7 @@ ti fs enable-file-system-token - `--file-system-id `: Specify the Filesystem that owns the token. Required when using TiDB Cloud API credentials; optional when an owner token supplies the ID. - `--token-id `: Specify the immutable token ID returned by the list command. This option is required. -- `--fs-token `: Authorize the request with an owner Filesystem token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. If no Filesystem token is available, the command uses the configured TiDB Cloud API keys. +- `--fs-token `: Authorize the request with a Filesystem owner token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. If no Filesystem token is available, the command uses the configured TiDB Cloud API keys. - `--dry-run`: Validate the request without changing remote token state. - `--help`: Display help information. - `--version`: Display version information. diff --git a/ai/ti/reference/ti-fs-generate-file-system-scoped-token.md b/ai/ti/reference/ti-fs-generate-file-system-scoped-token.md index 4c8f9b4a7bd40..576c491d24784 100644 --- a/ai/ti/reference/ti-fs-generate-file-system-scoped-token.md +++ b/ai/ti/reference/ti-fs-generate-file-system-scoped-token.md @@ -48,7 +48,7 @@ ti fs generate-file-system-scoped-token - `--ttl `: Set a finite positive token lifetime that resolves to whole seconds. This option is required. - `--allow `: Allow operations under one remote path prefix. Repeat this option for multiple prefixes. Operations are `read`, `list`, `search`, `write`, and `delete`; `search` requires `read`. This option is required. - `--file-system-id `: Assert the Filesystem ID embedded in the owner token. This option is required only when loading a locally stored owner token. -- `--fs-token `: Supply the owner Filesystem token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. +- `--fs-token `: Supply the Filesystem owner token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. - `--subject `: Set an optional server-side audit label of at most 64 bytes. It is not a unique selector. - `--store-locally`: Store and select the generated scoped token for this profile and Filesystem. - `--replace`: Replace an existing selected local token. Requires `--store-locally` and does not revoke the previous remote token. diff --git a/ai/ti/reference/ti-fs-generate-file-system-token.md b/ai/ti/reference/ti-fs-generate-file-system-token.md index 020cc35b7c58e..9868d49c62759 100644 --- a/ai/ti/reference/ti-fs-generate-file-system-token.md +++ b/ai/ti/reference/ti-fs-generate-file-system-token.md @@ -27,7 +27,7 @@ ti fs generate-file-system-token ## Options -- `--file-system-id `: Specify the Filesystem that owns the token. FS tokens cannot replace this option or authorize owner-token generation. This option is required. +- `--file-system-id `: Specify the Filesystem that owns the token. Filesystem tokens cannot replace this option or authorize owner-token generation. This option is required. - `--token-name `: Set an operational token name of at most 64 bytes. Names are not unique. This option is required. - `--ttl `: Set a positive lifetime in whole seconds, up to 365 days. Specify exactly one of `--ttl` and `--no-expiration`. - `--no-expiration`: Create a token without an expiry. Specify exactly one of `--ttl` and `--no-expiration`. diff --git a/ai/ti/reference/ti-fs-list-file-system-tokens.md b/ai/ti/reference/ti-fs-list-file-system-tokens.md index 2b34915c7d6e0..ad97f9611e441 100644 --- a/ai/ti/reference/ti-fs-list-file-system-tokens.md +++ b/ai/ti/reference/ti-fs-list-file-system-tokens.md @@ -27,7 +27,7 @@ ti fs list-file-system-tokens ## Options - `--file-system-id `: Specify the Filesystem whose tokens are listed. Required when using TiDB Cloud API credentials; optional when `--fs-token` or `TI_FS_TOKEN` supplies an owner token, because `ti` derives the ID from that token. -- `--fs-token `: Authorize the request with an owner Filesystem token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. If no Filesystem token is available, the command uses the configured TiDB Cloud API keys. Scoped tokens cannot list token metadata. +- `--fs-token `: Authorize the request with a Filesystem owner token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. If no Filesystem token is available, the command uses the configured TiDB Cloud API keys. Scoped tokens cannot list token metadata. - `--include-expired`: Include expired token metadata. Revoked tokens are not returned by the service. - `--help`: Display help information. - `--offset `: Set the zero-based token offset [default: 0]. diff --git a/ai/ti/reference/ti-fs-vault-list-secrets.md b/ai/ti/reference/ti-fs-vault-list-secrets.md index d7e3a03696ee2..6bd056362a48e 100644 --- a/ai/ti/reference/ti-fs-vault-list-secrets.md +++ b/ai/ti/reference/ti-fs-vault-list-secrets.md @@ -27,7 +27,7 @@ ti fs-vault list-secrets ## Options - `--file-system-id `: Select the file system. You can also set `TI_FS_FILE_SYSTEM_ID`. -- `--fs-token `: Set the owner Filesystem token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. For delegated authentication, use `--vault-token` or `TI_VAULT_TOKEN` instead. +- `--fs-token `: Set the Filesystem owner token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. For delegated authentication, use `--vault-token` or `TI_VAULT_TOKEN` instead. - `--help`: Display help information. - `--vault-token `: Delegated `ti fs-vault` token; prefer `TI_VAULT_TOKEN`. - `--version`: Display version information. diff --git a/ai/ti/reference/ti-fs-vault-mount-vault.md b/ai/ti/reference/ti-fs-vault-mount-vault.md index 5986069e33bb0..45b06bec26d60 100644 --- a/ai/ti/reference/ti-fs-vault-mount-vault.md +++ b/ai/ti/reference/ti-fs-vault-mount-vault.md @@ -32,7 +32,7 @@ ti fs-vault mount-vault - `--mount-path `: Local mount path. \[required] - `--dry-run`: Validate the request without applying changes. - `--file-system-id `: Select the file system. You can also set `TI_FS_FILE_SYSTEM_ID`. -- `--fs-token `: Set the owner Filesystem token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. For delegated authentication, use `--vault-token` or `TI_VAULT_TOKEN` instead. +- `--fs-token `: Set the Filesystem owner token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. For delegated authentication, use `--vault-token` or `TI_VAULT_TOKEN` instead. - `--help`: Display help information. - `--ready-timeout `: Time to wait for a background mount to become ready. \[default: `30s`] - `--vault-token `: Delegated `ti fs-vault` token; prefer `TI_VAULT_TOKEN`. diff --git a/ai/ti/reference/ti-fs-vault-read-secret.md b/ai/ti/reference/ti-fs-vault-read-secret.md index 2a4b6a17d49af..92141cc8fb508 100644 --- a/ai/ti/reference/ti-fs-vault-read-secret.md +++ b/ai/ti/reference/ti-fs-vault-read-secret.md @@ -31,7 +31,7 @@ ti fs-vault read-secret - `--field `: Optional field name to read. - `--file-system-id `: Select the file system. You can also set `TI_FS_FILE_SYSTEM_ID`. - `--format `: Read output format: `json`, `raw`, or `env`. \[default: json] -- `--fs-token `: Set the owner Filesystem token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. For delegated authentication, use `--vault-token` or `TI_VAULT_TOKEN` instead. +- `--fs-token `: Set the Filesystem owner token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. For delegated authentication, use `--vault-token` or `TI_VAULT_TOKEN` instead. - `--help`: Display help information. - `--vault-token `: Delegated `ti fs-vault` token; prefer `TI_VAULT_TOKEN`. - `--version`: Display version information. diff --git a/ai/ti/reference/ti-fs-vault-run-with-secret.md b/ai/ti/reference/ti-fs-vault-run-with-secret.md index f30215f3b6365..70a554059bc2d 100644 --- a/ai/ti/reference/ti-fs-vault-run-with-secret.md +++ b/ai/ti/reference/ti-fs-vault-run-with-secret.md @@ -30,7 +30,7 @@ ti fs-vault run-with-secret - `--secret-path `: Canonical Vault path in the form `/n/vault/`. For example, the secret created as `db-prod` has the path `/n/vault/db-prod`. \[required] - `--file-system-id `: Select the file system. You can also set `TI_FS_FILE_SYSTEM_ID`. -- `--fs-token `: Set the owner Filesystem token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. For delegated authentication, use `--vault-token` or `TI_VAULT_TOKEN` instead. +- `--fs-token `: Set the Filesystem owner token. If omitted, the command uses the `TI_FS_TOKEN` environment variable. If neither is provided, the command uses the local token stored for the selected Filesystem. For delegated authentication, use `--vault-token` or `TI_VAULT_TOKEN` instead. - `--help`: Display help information. - `--vault-token `: Delegated `ti fs-vault` token; prefer `TI_VAULT_TOKEN`. - `--version`: Display version information. diff --git a/tidb-cloud-filesystem/filesystem-quick-start.md b/tidb-cloud-filesystem/filesystem-quick-start.md index 3efba4ea8c496..c7962a15c4356 100644 --- a/tidb-cloud-filesystem/filesystem-quick-start.md +++ b/tidb-cloud-filesystem/filesystem-quick-start.md @@ -15,7 +15,7 @@ Create a workspace when your application's or agent's files need to outlive the To create a Filesystem, obtain a TiDB Cloud API public key and private key from the [API Keys page](https://tidbcloud.com/org-settings/api-keys). The keys must have permission to create a Filesystem in your organization. -If someone has already supplied you with an FS token, skip resource creation and follow [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). +If someone has already supplied you with a Filesystem token, skip resource creation and follow [Access an Existing TiDB Cloud Filesystem](/tidb-cloud-filesystem/access-filesystem.md). ## Step 1. Install the CLI diff --git a/tidb-cloud-filesystem/filesystem-troubleshooting.md b/tidb-cloud-filesystem/filesystem-troubleshooting.md index d73b6f03aaf21..419ff9346d8b8 100644 --- a/tidb-cloud-filesystem/filesystem-troubleshooting.md +++ b/tidb-cloud-filesystem/filesystem-troubleshooting.md @@ -21,7 +21,7 @@ export TI_REGION_CODE="" ti fs check-file-system ``` -The FS token is not the TiDB Cloud API private key. `TI_FS_FILE_SYSTEM_ID` is optional when a token is supplied; set it only when you want `ti` to verify that a separately distributed ID matches the token. +The Filesystem token is not the TiDB Cloud API private key. `TI_FS_FILE_SYSTEM_ID` is optional when a token is supplied; set it only when you want `ti` to verify that a separately distributed ID matches the token. If the token is known but is not stored on the current machine, import it and then select the derived ID: @@ -82,7 +82,7 @@ Or select the Filesystem for subsequent commands in the current shell: export TI_FS_FILE_SYSTEM_ID="" ``` -The CLI intentionally does not infer a Filesystem from local credential count, including when only one credential exists. Supply its ID or an FS token whose embedded ID can be derived. +The CLI intentionally does not infer a Filesystem from local credential count, including when only one credential exists. Supply its ID or a Filesystem token whose embedded ID can be derived. ## Filesystem region is unsupported @@ -121,7 +121,7 @@ Background mount success prints the CLI result without Drive9 startup messages. - the mount path exists and is writable; - no existing mount covers the path; -- the FS token and region are valid; +- the Filesystem token and region are valid; - FUSE prerequisites or the WebDAV helper are installed; - the remote region is reachable. diff --git a/tidb-cloud-filesystem/manage-filesystem-resources.md b/tidb-cloud-filesystem/manage-filesystem-resources.md index 4d1881a11410b..dc5e817c28fd3 100644 --- a/tidb-cloud-filesystem/manage-filesystem-resources.md +++ b/tidb-cloud-filesystem/manage-filesystem-resources.md @@ -25,7 +25,7 @@ ti fs create-file-system \ --wait ``` -The response includes the Filesystem ID and an owner FS token in the `fs_token` field. The CLI automatically stores the owner FS token locally. +From the output, you can get the Filesystem ID in the `file_system_id` field and the Filesystem owner token in the `fs_token` field. The CLI automatically stores the Filesystem owner token locally. Copy the returned `file_system_id` and select the Filesystem for subsequent commands in the current shell: @@ -33,11 +33,11 @@ Copy the returned `file_system_id` and select the Filesystem for subsequent comm export TI_FS_FILE_SYSTEM_ID="" ``` -Setting `TI_FS_FILE_SYSTEM_ID` lets subsequent commands identify the target Filesystem without requiring `--file-system-id` on every command. For data-access commands, the CLI uses the locally stored FS token for the selected Filesystem. +Setting `TI_FS_FILE_SYSTEM_ID` lets subsequent commands identify the target Filesystem without requiring `--file-system-id` on every command. For data-access commands, the CLI uses the locally stored Filesystem token for the selected Filesystem. > **Warning:** > -> The owner FS token plaintext in `fs_token` is returned only when the token is issued. Treat it as a secret and do not expose it in logs, issues, or source control. If you need to use the token on another machine or store a backup, save it securely in a secret manager. For more information, see [Manage Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md). +> The Filesystem owner token plaintext in `fs_token` is returned only when the token is issued. Treat it as a secret and do not expose it in logs, issues, or source control. If you need to use the token on another machine or store a backup, save it securely in a secret manager. For more information, see [Manage Filesystem Tokens](/tidb-cloud-filesystem/manage-filesystem-tokens.md). > **Note:** > diff --git a/tidb-cloud-filesystem/manage-filesystem-tokens.md b/tidb-cloud-filesystem/manage-filesystem-tokens.md index b24059efba10b..a533d52bf6b64 100644 --- a/tidb-cloud-filesystem/manage-filesystem-tokens.md +++ b/tidb-cloud-filesystem/manage-filesystem-tokens.md @@ -8,7 +8,7 @@ aliases: ['/ai/manage-filesystem-tokens'] Filesystem tokens let you give users, applications, and automation access to a TiDB Cloud Filesystem without sharing your TiDB Cloud API credentials. -You can use an owner token for full access to a Filesystem, or create scoped tokens that limit access to specific paths and operations. For more information about token types and permissions, see [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md). +You can use an [owner token](/tidb-cloud-filesystem/filesystem-authorization.md#owner-tokens) for full access to a Filesystem, or create [scoped tokens](/tidb-cloud-filesystem/filesystem-authorization.md#scoped-tokens) that limit access to specific paths and operations. For more information about token types and permissions, see [Authorization](/tidb-cloud-filesystem/filesystem-authorization.md). ## Prerequisites @@ -35,7 +35,9 @@ The CLI validates the token, extracts the Filesystem ID from it, verifies connec ## Generate an owner token -To generate another owner token, configure TiDB Cloud API credentials and obtain the Filesystem ID. +When you create a Filesystem, TiDB Cloud creates an owner token for it and returns it to you. You can generate additional owner tokens when another trusted environment or workflow needs full access to the Filesystem. + +To generate an additional owner token, configure TiDB Cloud API credentials and obtain the Filesystem ID. Generate the token and save its one-time plaintext response securely: @@ -47,7 +49,7 @@ ti fs generate-file-system-token \ --ttl 24h > ./ci-token.json ``` -The CLI does not store the generated token locally by default. To store it locally, add `--store-locally`. If a different token is already stored for this Filesystem, also add `--replace`. +The CLI does not store the generated token locally by default. To store it locally, add `--store-locally` to the preceding command. If a different token is already stored for this Filesystem, also add `--replace`. ## Generate and delegate a scoped token