Skip to content

FIX: Finalize Conda build and release workflows #842

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Gaurav Sharma (bewithgaurav) wants to merge 3 commits into
base: main
Choose a base branch
from
Open

FIX: Finalize Conda build and release workflows #842

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9bdb9ae
FIX: Correct Conda Python matrix and Windows ARM64 cross-build
bewithgaurav Oct 13, 2026
3dbf8fa
FIX: Sync finalized Conda release and publisher workflow
jahnvi480 Oct 13, 2026
ca6ceac
FIX: Enforce disabled redirects during Conda rehearsal
jahnvi480 Oct 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
152 changes: 39 additions & 113 deletions OneBranchPipelines/conda-release-pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,11 @@
# step: required subdirs + full Python matrix + one version), and -- when publishToConda=true -- publishes the SELF-CONTAINED
# mssql-python conda package to Anaconda.org via anaconda-client (conda-publish-step; ESRP
# has no Conda ContentType). There is a single self-contained package (it vendors the ODBC
# Driver 18 payload); no companion. When publishToConda=false (the default), the same gate
# runs and a DRY-RUN step prints the exact `anaconda upload` plan WITHOUT uploading -- so a
# validate-only run of THIS pipeline replaces the old separate dummy pipeline.
#
# TEMPLATE by mode: publishToConda=false (dry run / dummy) extends the NonOfficial template
# (no Production Service-Tree classification required, so a validate-only run passes the
# inventory-compliance gate); publishToConda=true (real release) extends the Official template
# (validated production publish -- the definition must be classified Production to
# upload). Review the dry-run output, then explicitly select publishToConda=true.
# Driver 18 payload); no companion. When publishToConda=false (the default), the same
# official container runs publisher provisioning and audits without publishing secrets,
# then deliberately fails at a read-only Anaconda authentication probe with a random
# invalid token. Only a confirmed HTTP 401 is the expected rehearsal outcome.
# No upload, promotion or cleanup is attempted in rehearsal, even if authentication succeeds.
# An administrator must enable the native Exclusive lock check on 'Anaconda Publishing'.
# The production stage uses that shared resource throughout publication and recovery.
# Keep its credentials, pipeline authorizations and other checks restricted.
Expand All @@ -31,7 +27,7 @@ parameters:
- name: publishToConda
displayName: 'Publish Conda Packages to Anaconda.org (PRODUCTION)'
type: boolean
default: false # Safety: default to a validate-only dry run.
default: false # Safety: default to a secretless, invalid-token publisher rehearsal.
- name: condaChannel
displayName: 'Anaconda.org channel/org to publish to'
type: string
Expand Down Expand Up @@ -72,13 +68,8 @@ resources:
trigger: none

extends:
# NonOfficial for a validate-only dry run (publishToConda=false): no Production Service-Tree
# classification is required, so the inventory-compliance gate passes. Official for a real
# publish (publishToConda=true): the governed production path.
${{ if eq(parameters.publishToConda, true) }}:
template: 'v2/OneBranch.Official.CrossPlat.yml@templates'
${{ else }}:
template: 'v2/OneBranch.NonOfficial.CrossPlat.yml@templates'
# Rehearse the actual managed container/network path, not a NonOfficial substitute.
template: 'v2/OneBranch.Official.CrossPlat.yml@templates'
parameters:
featureFlags:
WindowsHostVersion:
Expand All @@ -98,8 +89,7 @@ extends:
exclusionFile: '$(REPO_ROOT)/.config/PolicheckExclusions.xml'
publishLogs:
enabled: true
# Official-only heavy gates (binskim break + TSA). The NonOfficial dry run skips them,
# matching the old dummy pipeline's minimal SDL.
# Publication-only compliance outputs; both modes retain managed container isolation.
${{ if eq(parameters.publishToConda, true) }}:
binskim:
enabled: true
Expand Down Expand Up @@ -167,97 +157,33 @@ extends:
releaseVersions: '$(Provenance.releaseVersions)'
rsTransportVersion: '$(Provenance.rsTransportVersion)'

# Dry run (publishToConda=false): enumerate every package and print the exact
# `anaconda upload` plan WITHOUT uploading, so a validate-only run shows precisely
# what a real publish WOULD do. Reuses the conda tree the gate already downloaded
# into $(Build.SourcesDirectory)/conda-artifacts. This replaces the old dummy pipeline.
- ${{ if eq(parameters.publishToConda, false) }}:
- task: PowerShell@2
displayName: 'Dry-run publish plan (validate-only): enumerate packages + print the anaconda upload plan (NO upload)'
env:
CONDA_CHANNEL: '${{ parameters.condaChannel }}'
CONDA_LABEL: '${{ parameters.condaLabel }}'
inputs:
targetType: 'inline'
script: |
$ErrorActionPreference = 'Stop'

$root = "$(Build.SourcesDirectory)/conda-artifacts/conda"
if (-not (Test-Path $root)) {
Write-Error "Consolidated conda tree not found at $root. Did the readiness gate download it?"
exit 1
}

$pkgs = @(Get-ChildItem -Path $root -Recurse -Include *.conda, *.tar.bz2)
if ($pkgs.Count -eq 0) { Write-Error "No conda packages found under $root."; exit 1 }
$identifierPattern = '^[A-Za-z0-9][A-Za-z0-9._-]*$'
if ($env:CONDA_CHANNEL -notmatch $identifierPattern) {
Write-Error "Invalid condaChannel '$env:CONDA_CHANNEL'. Use letters, digits, dot, underscore, or hyphen; the first character must be alphanumeric."
exit 1
}
if ($env:CONDA_LABEL -notmatch $identifierPattern) {
Write-Error "Invalid condaLabel '$env:CONDA_LABEL'. Use letters, digits, dot, underscore, or hyphen; the first character must be alphanumeric."
exit 1
}

# mssql-python bindings only (the self-contained model ships no companion).
function Get-Binding($items) { @($items | Where-Object { $_.Name -like 'mssql-python-*' -and $_.Name -notlike 'mssql-python-odbc-*' }) }
$bySubdir = $pkgs | Group-Object { $_.Directory.Name } | Sort-Object Name

Write-Host "====================================="
Write-Host "[DRY-RUN CONDA RELEASE] discovered $($pkgs.Count) package(s) across $($bySubdir.Count) subdir(s):"
foreach ($g in $bySubdir) {
Write-Host (" {0,-14} {1} mssql-python package(s)" -f $g.Name, (Get-Binding $g.Group).Count)
}

# A self-contained release ships ONLY mssql-python; flag (don't fail) a stray companion.
$stray = @($pkgs | Where-Object { $_.Name -like 'mssql-python-odbc-*' })
if ($stray.Count -gt 0) {
Write-Host "##vso[task.logissue type=warning]$($stray.Count) stray mssql-python-odbc package(s) present; the self-contained model ships only mssql-python."
}

Write-Host "`nanaconda upload PLAN (DRY-RUN - NOT executed):"
Write-Host " target channel/org : $env:CONDA_CHANNEL"
Write-Host " target label : $env:CONDA_LABEL"
foreach ($p in ($pkgs | Sort-Object FullName)) {
$subdir = Split-Path -Leaf (Split-Path -Parent $p.FullName)
$sha256 = (Get-FileHash -LiteralPath $p.FullName -Algorithm SHA256).Hash.ToLowerInvariant()
Write-Host " SHA256: $sha256 $subdir/$($p.Name)"
Write-Host " WOULD RUN: anaconda upload --user $env:CONDA_CHANNEL --label $env:CONDA_LABEL --skip-existing $subdir/$($p.Name)"
}

Write-Host "`n====================================="
Write-Host "DRY-RUN COMPLETE: no 'anaconda upload' was executed (publishToConda=false)."
Write-Host "Review this dry run and verify the shared publishing-resource checks before selecting publishToConda=true."
Write-Host "====================================="

# PRODUCTION publish (releaseJob) -- runs ONLY when publishToConda=true and
# ONLY after the ValidateConda gate succeeds, so an incomplete set is never
# uploaded. anaconda-client reads ANACONDA_API_TOKEN from the env (never the
# command line), so the token never appears in the logs.
- ${{ if eq(parameters.publishToConda, true) }}:
- job: PublishConda
displayName: 'Publish conda packages to Anaconda.org'
dependsOn: ValidateConda
timeoutInMinutes: 60
cancelTimeoutInMinutes: 5
templateContext:
type: releaseJob
isProduction: true
pool:
type: windows
variables:
ob_outputDirectory: '$(Build.ArtifactStagingDirectory)'
mssqlPythonVersion: $[ dependencies.ValidateConda.outputs['Provenance.mssqlPythonVersion'] ]
releaseVersions: $[ dependencies.ValidateConda.outputs['Provenance.releaseVersions'] ]
rsTransportVersion: $[ dependencies.ValidateConda.outputs['Provenance.rsTransportVersion'] ]
steps:
- checkout: self
fetchDepth: 1
- template: /OneBranchPipelines/steps/conda-publish-step.yml@self
parameters:
condaChannel: '${{ parameters.condaChannel }}'
condaLabel: '${{ parameters.condaLabel }}'
mssqlPythonVersion: '$(mssqlPythonVersion)'
releaseVersions: '$(releaseVersions)'
rsTransportVersion: '$(rsTransportVersion)'
# Both modes share provisioning and audits; only production receives the group.
Comment thread
jahnvi480 marked this conversation as resolved.
- job: PublishConda
displayName: 'Publish or rehearse conda packages at Anaconda.org'
dependsOn: ValidateConda
timeoutInMinutes: 60
cancelTimeoutInMinutes: 5
templateContext:
type: releaseJob
isProduction: ${{ parameters.publishToConda }}
pool:
type: windows
variables:
ob_outputDirectory: '$(Build.ArtifactStagingDirectory)'
WindowsContainerImage: 'onebranch.azurecr.io/windows/ltsc2022/vse2022:latest'
mssqlPythonVersion: $[ dependencies.ValidateConda.outputs['Provenance.mssqlPythonVersion'] ]
releaseVersions: $[ dependencies.ValidateConda.outputs['Provenance.releaseVersions'] ]
rsTransportVersion: $[ dependencies.ValidateConda.outputs['Provenance.rsTransportVersion'] ]
steps:
- checkout: self
# Match OneBranch's injected checkout; restricted commands cannot relocate it.
path: s
fetchDepth: 1
- template: /OneBranchPipelines/steps/conda-publish-step.yml@self
parameters:
publishToConda: ${{ parameters.publishToConda }}
condaChannel: '${{ parameters.condaChannel }}'
condaLabel: '${{ parameters.condaLabel }}'
mssqlPythonVersion: '$(mssqlPythonVersion)'
releaseVersions: '$(releaseVersions)'
rsTransportVersion: '$(rsTransportVersion)'
5 changes: 3 additions & 2 deletions OneBranchPipelines/steps/conda-build-validate-step-posix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,12 +93,13 @@ steps:
--stage-dir "$(ob_outputDirectory)/conda" \
--conda-subdir "${{ parameters.condaSubdir }}" \
--conda-target-subdir "${CONDA_TARGET_SUBDIR:-}" \
--python-versions "${PYTHON_VERSIONS:-}"
--python-versions "${MSSQL_CONDA_PYTHON_VERSIONS:-}"
workingDirectory: $(Build.SourcesDirectory)
env:
# Surface the caller's value inputs as env vars (validated in-script above) rather than
# interpolating them into the script text -- repo secret-handling convention.
PYTHON_VERSIONS: ${{ parameters.pythonVersions }}
# Avoid the shared PYTHON_VERSIONS pipeline variable overwriting this input.
MSSQL_CONDA_PYTHON_VERSIONS: ${{ parameters.pythonVersions }}
CONDA_TARGET_SUBDIR: ${{ parameters.condaTargetSubdir }}
displayName: 'Conda build + validate (${{ parameters.condaSubdir }})'
continueOnError: ${{ parameters.continueOnError }}
Loading
Loading