Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions .github/workflows/publish-brew-tap.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ jobs:
repository: ${{ inputs.tap_repo }}
token: ${{ secrets.tap_token }}
path: tap
fetch-depth: 0
- name: Write formula
shell: bash
working-directory: tap
Expand All @@ -73,8 +74,12 @@ jobs:
version "${VERSION}"
sha256 "${SHA256}"

# The archive is the complete publish output: the Native AOT migration CLI
# loads SQLite's native library from its own directory. libexec keeps the
# archive together and the bin script runs the binary from there.
def install
bin.install "${BINARY_NAME}"
libexec.install Dir["*"]
bin.write_exec_script libexec/"${BINARY_NAME}"
end

test do
Expand All @@ -94,4 +99,14 @@ jobs:
git config user.email "github-actions[bot]@users.noreply.github.com"
git add "Formula/${FORMULA_NAME}.rb"
git commit -m "Update ${FORMULA_NAME} to ${VERSION}" || exit 0
git push
# lql-lsp and DataProviderMigrate publish to the same repository from parallel
# jobs, so a push can lose the race; rebase onto the other job's commit and retry.
for attempt in 1 2 3 4 5; do
if git push; then
exit 0
fi
sleep $((attempt * 2))
git pull --rebase
done
echo "::error::Could not push ${FORMULA_NAME} to the tap after 5 attempts." >&2
exit 1
13 changes: 12 additions & 1 deletion .github/workflows/publish-scoop-bucket.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ jobs:
repository: ${{ inputs.bucket_repo }}
token: ${{ secrets.bucket_token }}
path: bucket
fetch-depth: 0
- uses: actions/setup-node@2028fbc5c25fe9cf00d9f06a71cc4710d4507903 # v6.0.0
with:
node-version: 22
Expand Down Expand Up @@ -92,4 +93,14 @@ jobs:
git config user.email "github-actions[bot]@users.noreply.github.com"
git add "bucket/${MANIFEST_NAME}.json"
git commit -m "Update ${MANIFEST_NAME} to ${VERSION}" || exit 0
git push
# lql-lsp and DataProviderMigrate publish to the same repository from parallel
# jobs, so a push can lose the race; rebase onto the other job's commit and retry.
for attempt in 1 2 3 4 5; do
if git push; then
exit 0
fi
sleep $((attempt * 2))
git pull --rebase
done
echo "::error::Could not push ${MANIFEST_NAME} to the bucket after 5 attempts." >&2
exit 1
67 changes: 48 additions & 19 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,18 +98,42 @@ jobs:
actions: read
contents: read

# ── NuGet trusted-publishing preflight [CI-RELEASE-NUGET-OIDC] ─────────────
# Proves the nuget.org trusted-publishing policy accepts this workflow BEFORE any
# job publishes, so a missing or mismatched policy stops the release (and the
# manual preflight) instead of leaving binaries released without packages.
nuget-oidc-preflight:
needs: prepare
runs-on: ubuntu-latest
timeout-minutes: 5
# The policy is scoped to Environment=release; the OIDC token carries that
# claim only when the job is bound to the environment.
environment: release
permissions:
contents: read
id-token: write
steps:
- name: NuGet login (OIDC trusted publishing)
uses: NuGet/login@8d196754b4036150537f80ac539e15c2f1028841 # v1.2.0
with:
# The nuget.org organization that owns the packages and the policy.
user: Nimblesite

# ── NuGet packages ────────────────────────────────────────────────────────
# KEYLESS: exchanges a short-lived GitHub OIDC token for a ~1h nuget.org API key.
# No long-lived NuGet API key is stored. Implements [CI-RELEASE-NUGET-OIDC].
publish-nuget:
if: >-
github.event_name == 'push' && startsWith(github.ref, 'refs/tags/') &&
needs.prepare.result == 'success' && needs.ci-gate.result == 'success' &&
needs.codeql-gate.result == 'success'
needs: [prepare, ci-gate, codeql-gate]
needs.codeql-gate.result == 'success' && needs.nuget-oidc-preflight.result == 'success'
needs: [prepare, ci-gate, codeql-gate, nuget-oidc-preflight]
runs-on: ubuntu-latest
timeout-minutes: 15
environment: release
permissions:
contents: read
id-token: write
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
Expand All @@ -125,16 +149,18 @@ jobs:
- name: Pack solution
shell: bash
run: dotnet pack DataProvider.sln -c Release -o ./nupkgs
# Minted after packing, immediately before the push, so it cannot expire in transit.
- name: NuGet login (OIDC trusted publishing)
id: nuget_login
uses: NuGet/login@8d196754b4036150537f80ac539e15c2f1028841 # v1.2.0
with:
user: Nimblesite
- name: Push packages to NuGet
shell: bash
env:
NUGET_API_KEY: ${{ secrets.NUGET_API_KEY }}
NUGET_API_KEY: ${{ steps.nuget_login.outputs.NUGET_API_KEY }}
run: |
set -euo pipefail
if [ -z "${NUGET_API_KEY:-}" ]; then
echo "::error::NUGET_API_KEY secret is not set." >&2
exit 1
fi
shopt -s nullglob
packages=(./nupkgs/*.nupkg)
if [ ${#packages[@]} -eq 0 ]; then
Expand All @@ -153,8 +179,8 @@ jobs:
if: >-
github.event_name == 'push' && startsWith(github.ref, 'refs/tags/') &&
needs.prepare.result == 'success' && needs.ci-gate.result == 'success' &&
needs.codeql-gate.result == 'success'
needs: [prepare, ci-gate, codeql-gate]
needs.codeql-gate.result == 'success' && needs.nuget-oidc-preflight.result == 'success'
needs: [prepare, ci-gate, codeql-gate, nuget-oidc-preflight]
uses: ./.github/workflows/release-binary-multiplatform.yml
with:
binary_name: lql-lsp
Expand All @@ -169,8 +195,8 @@ jobs:
if: >-
github.event_name == 'push' && startsWith(github.ref, 'refs/tags/') &&
needs.prepare.result == 'success' && needs.ci-gate.result == 'success' &&
needs.codeql-gate.result == 'success'
needs: [prepare, ci-gate, codeql-gate]
needs.codeql-gate.result == 'success' && needs.nuget-oidc-preflight.result == 'success'
needs: [prepare, ci-gate, codeql-gate, nuget-oidc-preflight]
uses: ./.github/workflows/release-dotnet-aot-multiplatform.yml
with:
binary_name: DataProviderMigrate
Expand Down Expand Up @@ -207,14 +233,17 @@ jobs:
echo "win32_x64=${win}" >> "$GITHUB_OUTPUT"

# ── Homebrew tap ──────────────────────────────────────────────────────────
# Implements [CI-RELEASE-BREW-SCOOP]. GitHub has no OIDC path for pushing to another
# repository, so the tap and bucket use the Nimblesite org secret BREW_SCOOP_PAT,
# shared with every Nimblesite repository that publishes to them.
publish-brew:
needs: [prepare, checksums]
if: >-
github.event_name == 'push' && startsWith(github.ref, 'refs/tags/') &&
needs.checksums.result == 'success' && needs.prepare.outputs.prerelease == 'false'
uses: ./.github/workflows/publish-brew-tap.yml
with:
tap_repo: Nimblesite/homebrew-lql
tap_repo: Nimblesite/homebrew-tap
formula_name: lql-lsp
version: ${{ needs.prepare.outputs.version }}
asset_url: https://github.com/Nimblesite/DataProvider/releases/download/v${{ needs.prepare.outputs.version }}/lql-lsp-${{ needs.prepare.outputs.version }}-darwin-arm64.tar.gz
Expand All @@ -223,7 +252,7 @@ jobs:
homepage: https://github.com/Nimblesite/DataProvider
description: Lambda Query Language LSP server
secrets:
tap_token: ${{ secrets.HOMEBREW_TAP_TOKEN }}
tap_token: ${{ secrets.BREW_SCOOP_PAT }}

# ── Scoop bucket ──────────────────────────────────────────────────────────
publish-scoop:
Expand All @@ -233,7 +262,7 @@ jobs:
needs.checksums.result == 'success' && needs.prepare.outputs.prerelease == 'false'
uses: ./.github/workflows/publish-scoop-bucket.yml
with:
bucket_repo: Nimblesite/scoop-lql
bucket_repo: Nimblesite/scoop-bucket
manifest_name: lql-lsp
version: ${{ needs.prepare.outputs.version }}
url: https://github.com/Nimblesite/DataProvider/releases/download/v${{ needs.prepare.outputs.version }}/lql-lsp-${{ needs.prepare.outputs.version }}-win32-x64.zip
Expand All @@ -242,7 +271,7 @@ jobs:
homepage: https://github.com/Nimblesite/DataProvider
description: Lambda Query Language LSP server
secrets:
bucket_token: ${{ secrets.SCOOP_BUCKET_TOKEN }}
bucket_token: ${{ secrets.BREW_SCOOP_PAT }}

# ── Migration CLI checksums (from its own signed SHA256SUMS) [MIG-AOT-CI] ──
dotnet-checksums:
Expand Down Expand Up @@ -278,7 +307,7 @@ jobs:
needs.dotnet-checksums.result == 'success' && needs.prepare.outputs.prerelease == 'false'
uses: ./.github/workflows/publish-brew-tap.yml
with:
tap_repo: Nimblesite/homebrew-lql
tap_repo: Nimblesite/homebrew-tap
formula_name: dataprovider-migrate
version: ${{ needs.prepare.outputs.version }}
asset_url: https://github.com/Nimblesite/DataProvider/releases/download/v${{ needs.prepare.outputs.version }}/DataProviderMigrate-${{ needs.prepare.outputs.version }}-darwin-arm64.tar.gz
Expand All @@ -287,7 +316,7 @@ jobs:
homepage: https://github.com/Nimblesite/DataProvider
description: DataProvider database schema migration CLI (YAML → SQLite/PostgreSQL)
secrets:
tap_token: ${{ secrets.HOMEBREW_TAP_TOKEN }}
tap_token: ${{ secrets.BREW_SCOOP_PAT }}

# ── Scoop bucket (migration CLI) ──────────────────────────────────────────
publish-scoop-migrate:
Expand All @@ -297,7 +326,7 @@ jobs:
needs.dotnet-checksums.result == 'success' && needs.prepare.outputs.prerelease == 'false'
uses: ./.github/workflows/publish-scoop-bucket.yml
with:
bucket_repo: Nimblesite/scoop-lql
bucket_repo: Nimblesite/scoop-bucket
manifest_name: dataprovider-migrate
version: ${{ needs.prepare.outputs.version }}
url: https://github.com/Nimblesite/DataProvider/releases/download/v${{ needs.prepare.outputs.version }}/DataProviderMigrate-${{ needs.prepare.outputs.version }}-win32-x64.zip
Expand All @@ -306,7 +335,7 @@ jobs:
homepage: https://github.com/Nimblesite/DataProvider
description: DataProvider database schema migration CLI (YAML → SQLite/PostgreSQL)
secrets:
bucket_token: ${{ secrets.SCOOP_BUCKET_TOKEN }}
bucket_token: ${{ secrets.BREW_SCOOP_PAT }}

# ── Per-platform VSIX → VS Code Marketplace ───────────────────────────────
# Disabled at the user's request. [SWR-VSIX-PUBLISH-AUTH]
Expand Down
20 changes: 20 additions & 0 deletions Migration/DataProviderMigrate/Program.cs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
using System.Collections.Immutable;
using System.Reflection;
using Microsoft.Data.Sqlite;
using Nimblesite.DataProvider.Migration.Core;
using Nimblesite.DataProvider.Migration.Postgres;
Expand Down Expand Up @@ -40,6 +41,11 @@ public static int Main(string[] args)
return ShowTopLevelUsage();
}

if (args[0] is "--version" or "-V")
{
return ShowVersion();
}

var command = args[0];
var remainingArgs = args[1..];

Expand Down Expand Up @@ -542,6 +548,19 @@ private static int ShowProviderError(string provider)

// ── Usage / Errors ───────────────────────────────────────────────────

// Implements [MIG-CLI-COMMANDS]: prints the release version (the tag stamped into
// <Version>) without build metadata. The Homebrew formula test asserts it.
private static int ShowVersion()
{
var version =
typeof(Program)
.Assembly.GetCustomAttribute<AssemblyInformationalVersionAttribute>()
?.InformationalVersion
?? "unknown";
Console.WriteLine(version.Split('+')[0]);
return 0;
}

private static int ShowTopLevelUsage()
{
Console.WriteLine(
Expand All @@ -555,6 +574,7 @@ export Export C# schema class to YAML file
Usage:
DataProviderMigrate migrate --schema schema.yaml --output database.db [--provider sqlite|postgres|sqlserver]
DataProviderMigrate export --assembly assembly.dll --type Namespace.SchemaClass --output schema.yaml
DataProviderMigrate --version

Run 'DataProviderMigrate <command> --help' for command-specific options.
"""
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
using System.Reflection;

namespace Nimblesite.DataProvider.Migration.Tests;

/// <summary>
/// Implements [MIG-CLI-COMMANDS]: <c>--version</c> prints the stamped release version
/// without build metadata. The Homebrew formula test runs it against the native binary.
/// </summary>
public sealed class DataProviderMigrateVersionTests
{
private const string ReleaseVersionPattern = @"^\d+\.\d+\.\d+(-[0-9A-Za-z.-]+)?$";

[Theory]
[InlineData("--version")]
[InlineData("-V")]
public void Main_VersionFlag_PrintsReleaseVersion(string flag)
{
var informational = typeof(DataProviderMigrate.Program)
.Assembly.GetCustomAttribute<AssemblyInformationalVersionAttribute>()
?.InformationalVersion;

var (exitCode, output) = MigrationCliConsole.Run(flag);
var printed = output.Trim();

Assert.Equal(expected: 0, actual: exitCode);
Assert.Matches(ReleaseVersionPattern, printed);
Assert.NotNull(informational);
Assert.StartsWith(printed, informational, StringComparison.Ordinal);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ internal static (int ExitCode, string Output) Migrate(
)
);

internal static (int ExitCode, string Output) Run(params string[] args) =>
Invoke(run: () => DataProviderMigrate.Program.Main(args: args));

internal static string WriteTempSchemaFile(string prefix, string contents)
{
var schemaPath = Path.Combine(Path.GetTempPath(), $"{prefix}-{Guid.NewGuid():N}.yaml");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,18 @@ public void NativeBinary_SqlServerProvider_ReportsUnsupported()
);
}

[SkippableFact]
public void NativeBinary_Version_PrintsReleaseVersion()
{
var bin = RequireNativeBinary();

// Implements [MIG-CLI-COMMANDS]: the Homebrew formula test runs `--version`.
var result = RunRaw(bin, ["--version"]);

Assert.True(result.ExitCode == 0, userMessage: result.Output);
Assert.Matches(@"^\d+\.\d+\.\d+(-[0-9A-Za-z.-]+)?$", result.Output.Trim());
}

private static string RequireNativeBinary()
{
var bin = NativeBinaryPath();
Expand Down
26 changes: 24 additions & 2 deletions docs/specs/ci-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,32 @@ every version assertion and dry-run immutability check.

### Publishing Preflight [CI-RELEASE-PREFLIGHT]

Manual Release runs take a version and verify manifests,
CI, and the CodeQL security gate on the selected commit. Publishing jobs require
Manual Release runs take a version and verify manifests, NuGet trusted
publishing, CI, and the CodeQL security gate on the selected commit. Publishing jobs require
a tag push, so manual preflight cannot upload packages or release assets.

### NuGet Trusted Publishing [CI-RELEASE-NUGET-OIDC]

NuGet packages publish without a stored API key. `publish-nuget` uses the pinned
`NuGet/login` action to exchange a short-lived GitHub OIDC token for a nuget.org
API key that lasts about an hour. The nuget.org trusted-publishing policy belongs
to the `Nimblesite` organization and matches repository `Nimblesite/DataProvider`,
workflow `release.yml` and environment `release`. The `nuget-oidc-preflight` job
performs the same exchange before any job publishes. A missing or mismatched policy
therefore fails the release, and the manual preflight, before NuGet packages or
GitHub Release binaries ship.

### Homebrew and Scoop [CI-RELEASE-BREW-SCOOP]

Stable tags (no prerelease suffix) update `lql-lsp` and `dataprovider-migrate` in
`Nimblesite/homebrew-tap` (`Formula/`) and `Nimblesite/scoop-bucket` (`bucket/`).
GitHub has no OIDC path for pushing to another repository, so these jobs use the
Nimblesite org secret `BREW_SCOOP_PAT`, which the other Nimblesite repositories
that publish there also use. Formulae install the complete archive into `libexec`
and expose the binary through an exec script, because the Native AOT migration CLI
loads SQLite's native library from its own directory. Both products push to the
same repositories from parallel jobs, so each push rebases and retries.

### Marketplace Authentication [SWR-VSIX-PUBLISH-AUTH]

Marketplace publication and its credential preflight are commented out in the
Expand Down
4 changes: 4 additions & 0 deletions docs/specs/migration-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -690,6 +690,10 @@ dotnet run --project Migration/DataProviderMigrate/DataProviderMigrate.csproj --
| `--type`, `-t` | Yes | Fully qualified schema type name |
| `--output`, `-o` | Yes | YAML file path to write |

`DataProviderMigrate --version` (or `-V`) prints the release version stamped from the tag,
for example `0.11.0`, without build metadata. The Homebrew formula test runs it against the
native binary.

Schema export types MUST expose either a static `Definition` property returning `SchemaDefinition` or a static `Build()` method returning `SchemaDefinition`.

#### 7.4.2 YAML-Only Migration Input [MIG-CLI-YAML-ONLY]
Expand Down
Loading