Repository navigation
614 lines (583 loc) · 30.6 KB
/
Copy pathtutorial_scripts.yml
File metadata and controls
614 lines (583 loc) · 30.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
name: Run the tutorial scripts on a fresh wheel
# The companion of test_wheel_suite.yml: that one builds one wheel per OS and runs the pytest
# suite against it, this one builds the same wheel and runs every script of the tutorial against
# it - the same set citools/test_all_tutorial_scripts.py runs nightly, assembled from
# docs/source/tutorial by tutorial_bundle.export_tree(), so it tests the scripts as the
# documentation currently shows them.
#
# Why it is worth a separate workflow: the tests in tests/ are written to be fast and to pin down
# one mechanism each, while the tutorial scripts are what a reader actually types in, and they use
# far more of the API - the plotting interface, the mesh generators, bifurcation tracking,
# multi-component flow. A wheel can pass the suite and still fail on the first thing a new user
# tries.
#
# The per-script verdict and timing are put on the run's SUMMARY page, not just in the log: the log
# is over a hundred scripts long and nobody scrolls it. The harness writes --report-json and
# citools/tutorial_report_to_summary.py turns that into a table (failures first, the green ones
# folded away), and that script's exit code is what makes the job red - the harness itself always
# exits 0, because the nightly parses its printed output instead.
#
# Eigensolvers, per platform. macOS arm64 downloads the real+complex PETSc/SLEPc pair built by
# prebuild_petsc_slepc.yml and runs the harness WITHOUT --no-petsc, so the SLEPc path - azimuthal
# stability, Floquet analysis, bifurcation tracking - is exercised in CI rather than only on the
# nightly machine. It is the platform that needs it most: there is no MKL on Apple silicon, so
# PETSc/MUMPS is what pyoomph's autodetection picks there for the linear solver as well.
# Every other platform keeps --no-petsc and runs on the built-in spectra eigensolver over Pardiso,
# which is exactly what a `pip install pyoomph` user gets.
#
# Either way NOTHING is skipped for a missing SLEPc: no tutorial script names an eigensolver any
# more (they leave the choice to the autodetection), and the harness treats a ModuleNotFoundError
# for petsc4py as a real failure rather than forgiving it - see the _OPTIONAL_MODULES comment in
# citools/test_all_tutorial_scripts.py.
#
# What is deliberately NOT covered here:
# * PETSc/SLEPc on Linux, macOS x86_64 and Windows - see above.
# * preCICE, shapely and the interactive bifurcation GUI. The harness reports these as skips, by
# name, and they are listed in the summary table.
#
# NOTE: GitHub only offers the "Run workflow" button for workflows that exist on the DEFAULT
# branch, which is why this file lives on main even though development happens on develop. Pick the
# branch to test via the `ref` input.
# The title of the run carries the parameters that decide whether it COUNTS as a release
# prerequisite. wheels.yml's gate looks for a successful run of this workflow on the exact commit
# being released whose title contains "[omp=3]" and "[platforms=all]"; the run's inputs themselves
# are not exposed by the Actions API, so they are put here instead. Keep the markers verbatim if
# you touch this line - the gate matches them as plain substrings.
run-name: >-
Tutorials [omp=${{ inputs.omp }}] [platforms=${{ inputs.platforms }}]
on ${{ inputs.ref || github.ref_name }}
on:
workflow_dispatch:
inputs:
ref:
description: Branch, tag or SHA to build and test (defaults to the branch this is dispatched on)
required: false
type: string
default: ""
python_version:
description: Python version to build the wheel for and run the scripts with
required: false
type: string
default: "3.12"
quick_test:
description: Pass --quick-test, i.e. stop each script after its first Newton solve (a full run takes hours)
required: false
type: boolean
default: true
omp:
description: Assemble the elements on this many threads (0 = serial). Standard runners have 4 vCPU, macos-14 arm64 has 3
required: false
type: string
default: "0"
platforms:
description: Which platforms to run on
required: false
type: choice
default: all
options: [all, linux, macos, windows, linux-and-macos,
linux-x86_64, macos-intel-x86_64, macos-arm64, windows-msys2-ucrt64]
script_timeout:
description: Kill any single script that runs longer than this many seconds (0 waits forever)
required: false
type: string
# 1800 is enough for a --quick-test pass and not for a full one: the full run of 30th August
# 2026 lost exactly two scripts to it, both on macOS arm64 - Moving_Mesh/droplet_spread_3d.py
# and Advanced_Linear_Dynamics/rayleigh_benard_azimuthal_stability.py, each killed at 1800.0 s
# with nothing wrong with them. That job took 138 minutes of the 360 the workflow allows, so
# an hour per script costs headroom it has: the slowest platform would have to lose two hours
# to a single script before the cap became the binding constraint rather than this.
default: "3600"
skip_folders:
description: Bundle folders to skip, space separated (e.g. "PreCICE_Coupling Plotting_Interface")
required: false
type: string
default: ""
only:
description: Run ONLY the scripts whose "Folder/script.py" path contains one of these substrings, space separated (e.g. "rayleigh_plateau_pinchoff"). Empty runs everything. For chasing one flaky script across the platforms without paying for a full pass
required: false
type: string
default: ""
prebuild_run_id:
description: Optional "Prebuild static CLN/GiNaC" run id to take the CLN/GiNaC artifacts from (defaults to the latest successful run)
required: false
type: string
petsc_prebuild_run_id:
description: Optional "Prebuild PETSc/SLEPc" run id to take the macOS arm64 PETSc artifact from (defaults to the latest successful run)
required: false
type: string
# Called by wheels.yml when the release gate finds no qualifying run of this workflow for the
# commit being released; see the "prereqs" job there. Same inputs as above, minus the
# descriptions and with the `platforms` choice spelled as a plain string, which is all
# workflow_call supports.
workflow_call:
inputs:
ref:
required: false
type: string
default: ""
python_version:
required: false
type: string
default: "3.12"
quick_test:
required: false
type: boolean
default: true
omp:
required: false
type: string
default: "0"
platforms:
required: false
type: string
default: all
script_timeout:
required: false
type: string
default: "3600"
skip_folders:
required: false
type: string
default: ""
only:
required: false
type: string
default: ""
prebuild_run_id:
required: false
type: string
petsc_prebuild_run_id:
required: false
type: string
permissions:
contents: read
# gh run list/download below reads another workflow's run artifacts.
actions: read
jobs:
tutorials:
name: ${{ matrix.label }}
runs-on: ${{ matrix.runner }}
# The wheel build is ~40 minutes on a cold macOS runner and the --quick-test tutorial pass about
# the same; a full (non-quick) pass is what the upper end is for.
timeout-minutes: 360
strategy:
fail-fast: false
matrix:
include:
- label: linux-x86_64
runner: ubuntu-latest
platform: linux
prebuild_artifact: static-ginac-cln-linux-x86_64
- label: macos-intel-x86_64
runner: macos-15-intel
platform: macos
prebuild_artifact: static-ginac-cln-macos-intel-x86_64
- label: macos-arm64
runner: macos-14
platform: macos
prebuild_artifact: static-ginac-cln-macos-arm64
# The one platform that gets PETSc/SLEPc; see the header. The cpXY suffix is appended
# at run time, since petsc4py/slepc4py are compiled extension modules.
petsc_artifact: petsc-slepc-macos-arm64
- label: windows-msys2-ucrt64
runner: windows-latest
platform: windows
prebuild_artifact: static-ginac-cln-windows-msys2-ucrt64
steps:
- name: Decide whether this platform was asked for
id: want
shell: bash
env:
WANTED: ${{ inputs.platforms }}
PLATFORM: ${{ matrix.platform }}
LABEL: ${{ matrix.label }}
run: |
# A LABEL as well as a platform group, so a single job can be re-run on its own. Both Mac
# jobs share platform "macos", so "macos" could never mean just one of them - and when the
# arm64 job alone fails, which is what the PETSc/SLEPc build makes likely, repeating the
# Intel one costs another forty minutes to learn nothing.
case "${WANTED}" in
all) yes=1 ;;
linux-and-macos) [[ "${PLATFORM}" == "linux" || "${PLATFORM}" == "macos" ]] && yes=1 || yes=0 ;;
*) [[ "${WANTED}" == "${PLATFORM}" || "${WANTED}" == "${LABEL}" ]] && yes=1 || yes=0 ;;
esac
echo "run=${yes}" >> "${GITHUB_OUTPUT}"
[[ "${yes}" == "1" ]] || echo "Skipping ${LABEL}: only '${WANTED}' was requested"
- uses: actions/checkout@v4
if: steps.want.outputs.run == '1'
with:
ref: ${{ inputs.ref || github.ref }}
- name: Set up the cibuildwheel driver interpreter
if: steps.want.outputs.run == '1'
id: driver
uses: actions/setup-python@v5
with:
python-version: '3.14'
# The interpreter the wheel is installed into and the tutorial scripts are run with. Last, so
# that $pythonLocation (which the MSYS2 step below inherits) points at it.
- name: Set up the interpreter the scripts run on
if: steps.want.outputs.run == '1'
id: runpy
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python_version }}
# CLN and GiNaC are not rebuilt here, for the same reason as in test_wheel_suite.yml:
# compiling both from source costs more than everything else together, and these are the very
# static libraries the released wheels are built against. Windows cannot do without them at
# all (there is no MSVC port of CLN).
- name: Resolve prebuild run id
if: steps.want.outputs.run == '1'
id: prebuild
env:
GH_TOKEN: ${{ github.token }}
MANUAL_RUN_ID: ${{ inputs.prebuild_run_id }}
shell: bash
run: |
if [[ -n "${MANUAL_RUN_ID}" ]]; then
RUN_ID="${MANUAL_RUN_ID}"
else
RUN_ID="$(gh run list \
--workflow prebuild_static_ginac_cln.yml \
--repo "${GITHUB_REPOSITORY}" \
--status success \
--limit 1 \
--json databaseId \
--jq '.[0].databaseId')"
fi
if [[ -z "${RUN_ID}" || "${RUN_ID}" == "null" ]]; then
echo "Could not resolve a successful prebuild run id - run the 'Prebuild static CLN/GiNaC' workflow first" >&2
exit 1
fi
# WHICH COMMIT the artifact was built from, not just which run - "latest successful run"
# says nothing about that, and on 31st August 2026 the newest artifact (30th August) had
# been built from a commit predating citools/patches/ginac-deterministic-*.patch, so the
# wheels linked an unpatched GiNaC whose term ordering moves between processes. The build
# now catches that itself (cmake/ThirdPartyGiNaC.cmake runs a probe and fails), but that
# failure is far easier to read with the artifact's provenance already in the log.
PRE_SHA="$(gh run view "${RUN_ID}" --repo "${GITHUB_REPOSITORY}" \
--json headSha --jq '.headSha' 2>/dev/null || echo unknown)"
PRE_AT="$(gh run view "${RUN_ID}" --repo "${GITHUB_REPOSITORY}" \
--json createdAt --jq '.createdAt' 2>/dev/null || echo unknown)"
echo "Using prebuild run id ${RUN_ID} (built from ${PRE_SHA} at ${PRE_AT})"
{
echo "### Static CLN/GiNaC artifact"
echo "- run \`${RUN_ID}\`, built from commit \`${PRE_SHA}\` at ${PRE_AT}"
} >> "${GITHUB_STEP_SUMMARY}"
echo "run_id=${RUN_ID}" >> "${GITHUB_OUTPUT}"
- name: Download the static CLN/GiNaC artifact
if: steps.want.outputs.run == '1'
env:
GH_TOKEN: ${{ github.token }}
shell: bash
run: |
rm -rf prebuilt
mkdir -p prebuilt
gh run download "${{ steps.prebuild.outputs.run_id }}" \
--repo "${GITHUB_REPOSITORY}" \
--name "${{ matrix.prebuild_artifact }}" \
--dir prebuilt
find prebuilt -maxdepth 4 -type d | sort
- name: Compute the cibuildwheel build selector
if: steps.want.outputs.run == '1'
id: cibw
shell: bash
env:
PYVER: ${{ inputs.python_version }}
run: echo "selector=cp${PYVER//./}-*" >> "${GITHUB_OUTPUT}"
# The tutorials draw with matplotlib and mesh with gmsh (through pygmsh), both of which want
# X libraries present even when they never open a window.
- name: Install runtime libraries (Linux)
if: steps.want.outputs.run == '1' && matrix.platform == 'linux'
run: |
sudo apt-get update
sudo apt-get install -y libglu1-mesa libglu1-mesa-dev libxcursor1 libxinerama1 libxft2 libxrender1 libopengl0 libgl1
- name: Build the wheel (Linux/macOS)
if: steps.want.outputs.run == '1' && matrix.platform != 'windows'
env:
CIBW_BUILD: ${{ steps.cibw.outputs.selector }}
CIBW_SKIP: "*-musllinux_*"
CIBW_ARCHS: native
CIBW_BUILD_FRONTEND: build
# No pytest here - the tutorial scripts below are this workflow's test. Left empty rather
# than left at pyproject.toml's test-command, which would silently add ~25 minutes of a
# suite that test_wheel_suite.yml already runs.
CIBW_TEST_COMMAND: ""
CIBW_TEST_SOURCES: ""
CIBW_BEFORE_ALL_LINUX: >
bash -lc 'if command -v dnf >/dev/null 2>&1; then dnf -y install wget bzip2 patch;
elif command -v yum >/dev/null 2>&1; then yum -y install wget bzip2 patch;
elif command -v apk >/dev/null 2>&1; then apk add --no-cache wget bzip2 patch;
elif command -v apt-get >/dev/null 2>&1; then apt-get update && apt-get install -y wget bzip2 patch;
else echo "No supported package manager for before-all" >&2; exit 1; fi'
CIBW_ENVIRONMENT: >
PYOOMPH_GINAC_INCLUDE_DIR=$PWD/prebuilt/GiNaC_static/install/include
PYOOMPH_GINAC_LIB_DIR=$PWD/prebuilt/GiNaC_static/install/lib
PYOOMPH_CLN_INCLUDE_DIR=$PWD/prebuilt/GiNaC_static/install/include
PYOOMPH_CLN_LIB_DIR=$PWD/prebuilt/GiNaC_static/install/lib
MACOSX_DEPLOYMENT_TARGET=10.13
# The threading backend is pinned ON, because the omp input is worthless without it: with
# CMake's AUTO default a runner that cannot find one produces a serial wheel, and --omp N
# then does nothing at all while the report looks like a threaded run. OpenMP on Linux,
# GCD on macOS - never both, and never OpenMP on macOS, where it would load a second
# runtime beside MKL's libiomp5 (that collision is what src/mac_accelerate.cpp and the GCD
# backend exist for). This does tie the workflow to a `ref` that HAS the GCD backend, which
# is why it was left out before; anything from 93d014c8 onwards has it.
CIBW_CONFIG_SETTINGS: >
cmake.define.PYOOMPH_DOWNLOAD_CLN=OFF
cmake.define.PYOOMPH_DOWNLOAD_GINAC=OFF
cmake.define.PYOOMPH_ASSUME_GINAC_HASH_PATCHED=ON
cmake.define.PYOOMPH_USE_MPI=OFF
cmake.define.PYOOMPH_PARANOID=OFF
cmake.define.PYOOMPH_HAS_SPECTRA=ON
${{ matrix.platform == 'macos' && 'cmake.define.PYOOMPH_USE_GCD=ON' || 'cmake.define.PYOOMPH_USE_OPENMP=ON' }}
uses: pypa/cibuildwheel@v4.1.0
with:
output-dir: wheelhouse
- name: Set up MSYS2
if: steps.want.outputs.run == '1' && matrix.platform == 'windows'
uses: msys2/setup-msys2@v2
with:
msystem: UCRT64
update: true
install: >-
make
patch
git
pkgconf
base-devel
mingw-w64-ucrt-x86_64-toolchain
- name: Build the wheel (Windows/MSYS2)
# Windows builds through MSYS2/MinGW rather than MSVC - CLN has no MSVC port, and its
# MinGW-built libstdc++ could not be mixed into an MSVC extension anyway.
if: steps.want.outputs.run == '1' && matrix.platform == 'windows'
shell: msys2 {0}
env:
CC: gcc
CXX: g++
run: |
# cibuildwheel needs >= 3.11, so it is driven by the 3.14 donor rather than by whatever
# inputs.python_version happens to be.
DRIVER="$(cygpath -u '${{ steps.driver.outputs.python-path }}')"
"$DRIVER" -m pip install --upgrade pip
"$DRIVER" -m pip install --upgrade "cibuildwheel==4.1.0" delvewheel
PREBUILT_WIN="$(cygpath -m "$PWD/prebuilt/GiNaC_static/install")"
export CIBW_BUILD="${{ steps.cibw.outputs.selector }}"
export CIBW_ARCHS="AMD64"
export CIBW_BUILD_FRONTEND="build"
export CIBW_TEST_COMMAND=""
export CIBW_TEST_SOURCES=""
export CIBW_ENVIRONMENT_WINDOWS="CC=gcc CXX=g++ CXXFLAGS='-static-libgcc -static-libstdc++' LDFLAGS='-static-libgcc -static-libstdc++' PYOOMPH_GINAC_INCLUDE_DIR=${PREBUILT_WIN}/include PYOOMPH_GINAC_LIB_DIR=${PREBUILT_WIN}/lib PYOOMPH_CLN_INCLUDE_DIR=${PREBUILT_WIN}/include PYOOMPH_CLN_LIB_DIR=${PREBUILT_WIN}/lib"
export CIBW_REPAIR_WHEEL_COMMAND_WINDOWS="delvewheel repair --add-path C:/msys64/ucrt64/bin -w {dest_dir} {wheel}"
export CIBW_CONFIG_SETTINGS="cmake.args=-GUnix\ Makefiles cmake.define.PYOOMPH_DOWNLOAD_CLN=OFF cmake.define.PYOOMPH_DOWNLOAD_GINAC=OFF cmake.define.PYOOMPH_ASSUME_GINAC_HASH_PATCHED=ON cmake.define.PYOOMPH_USE_MPI=OFF cmake.define.PYOOMPH_PARANOID=OFF cmake.define.PYOOMPH_HAS_SPECTRA=ON"
"$DRIVER" -m cibuildwheel --output-dir wheelhouse
- name: Install the wheel into a fresh venv (Linux/macOS)
if: steps.want.outputs.run == '1' && matrix.platform != 'windows'
shell: bash
run: |
set -euo pipefail
ls -l wheelhouse
# A venv, not the runner's interpreter: nothing the tutorial scripts pip-installed can
# then leak into the runner's own Python. The harness runs each script with
# sys.executable, so this venv's python is what every script sees.
"${{ steps.runpy.outputs.python-path }}" -m venv "${RUNNER_TEMP}/tutorialvenv"
VENV="${RUNNER_TEMP}/tutorialvenv/bin/python"
"$VENV" -m pip install --upgrade pip
# [topology] pulls shapely, which the axisymmetric pinch-off scripts need; without it the
# harness would report them as skipped-for-missing-dependency, which is not what a
# tutorial run should be proving.
WHEEL=$(ls wheelhouse/*.whl | head -n 1)
"$VENV" -m pip install "${WHEEL}[topology]"
# From RUNNER_TEMP, NOT from the workspace: `python -c` puts the current directory on
# sys.path, so run in the checkout this imports the SOURCE ./pyoomph - which has no
# compiled extension next to it - and dies on "No module named pyoomph._pyoomph_core"
# while the installed wheel next door is perfectly fine. The tutorial scripts themselves
# are unaffected: the harness runs each one from inside its own bundle folder.
(cd "${RUNNER_TEMP}" && "$VENV" -c "import importlib.metadata as m,pyoomph,pyoomph._pyoomph_core as c; print('pyoomph',m.version('pyoomph'),'from',pyoomph.__file__); print('extension from',c.__file__)")
echo "TUTORIAL_PYTHON=$VENV" >> "${GITHUB_ENV}"
- name: Install the wheel into a fresh venv (Windows)
if: steps.want.outputs.run == '1' && matrix.platform == 'windows'
shell: msys2 {0}
run: |
set -euo pipefail
cd "$(cygpath -u "$GITHUB_WORKSPACE")"
ls -l wheelhouse
RUNPY="$(cygpath -u '${{ steps.runpy.outputs.python-path }}')"
"$RUNPY" -m venv "$(cygpath -u "$RUNNER_TEMP")/tutorialvenv"
VENV="$(cygpath -u "$RUNNER_TEMP")/tutorialvenv/Scripts/python.exe"
"$VENV" -m pip install --upgrade pip
WHEEL=$(ls wheelhouse/*.whl | head -n 1)
"$VENV" -m pip install "${WHEEL}[topology]"
# See the Linux/macOS step: from RUNNER_TEMP, so that the checkout's own ./pyoomph
# directory is not what gets imported.
(cd "$(cygpath -u "$RUNNER_TEMP")" && "$VENV" -c "import importlib.metadata as m,pyoomph,pyoomph._pyoomph_core as c; print('pyoomph',m.version('pyoomph'),'from',pyoomph.__file__); print('extension from',c.__file__)")
echo "TUTORIAL_PYTHON=$(cygpath -m "$VENV")" >> "${GITHUB_ENV}"
- name: Resolve the PETSc/SLEPc prebuild run id
if: steps.want.outputs.run == '1' && matrix.petsc_artifact
id: petsc_prebuild
env:
GH_TOKEN: ${{ github.token }}
MANUAL_RUN_ID: ${{ inputs.petsc_prebuild_run_id }}
shell: bash
run: |
if [[ -n "${MANUAL_RUN_ID}" ]]; then
RUN_ID="${MANUAL_RUN_ID}"
else
RUN_ID="$(gh run list \
--workflow prebuild_petsc_slepc.yml \
--repo "${GITHUB_REPOSITORY}" \
--status success \
--limit 1 \
--json databaseId \
--jq '.[0].databaseId')"
fi
[[ -n "${RUN_ID}" && "${RUN_ID}" != "null" ]] || {
echo "Could not resolve a successful PETSc prebuild run id - run the 'Prebuild PETSc/SLEPc' workflow first" >&2
exit 1; }
echo "Using PETSc prebuild run id ${RUN_ID}"
echo "run_id=${RUN_ID}" >> "${GITHUB_OUTPUT}"
- name: Install the PETSc/SLEPc prebuild
if: steps.want.outputs.run == '1' && matrix.petsc_artifact
env:
GH_TOKEN: ${{ github.token }}
PYVER: ${{ inputs.python_version }}
shell: bash
run: |
set -euo pipefail
# petsc4py's extension dlopens Homebrew's libgfortran - MUMPS and ScaLAPACK are Fortran, and
# the prebuild runner had it only because it installed gcc to compile them. The image this
# job runs on provides no unversioned `gcc` prefix, so the import died with "Library not
# loaded: /opt/homebrew/opt/gcc/lib/gcc/current/libgfortran.5.dylib" - and, since every
# SLEPc test only asks whether petsc4py imports, as a wall of failures half an hour into
# the suite rather than here. Checked, not assumed, for the same reason.
brew install gcc || brew link --overwrite gcc
test -e "$(brew --prefix gcc)/lib/gcc/current/libgfortran.5.dylib"
rm -rf petsc-prebuilt && mkdir -p petsc-prebuilt
gh run download "${{ steps.petsc_prebuild.outputs.run_id }}" \
--repo "${GITHUB_REPOSITORY}" \
--name "${{ matrix.petsc_artifact }}-cp${PYVER//./}" \
--dir petsc-prebuilt
# Unpacked at the absolute path it was CONFIGURED with: PETSc bakes the prefix into
# petscvariables, the RPATHs and the macOS install_names, so anywhere else fails to import.
sudo mkdir -p /opt
sudo tar xzf petsc-prebuilt/pyoomph-petsc.tar.gz -C /opt
cat /opt/pyoomph-petsc/env.sh
set -a; source /opt/pyoomph-petsc/env.sh; set +a
# The harness reads PETSC_DIR/PETSC_ARCH_REAL/PETSC_ARCH_COMPLEX and puts the matching
# arch on each script's PYTHONPATH *and* loader path itself, so what is exported here only
# has to make a plain `python -c "import petsc4py"` work. env.sh names the REAL arch, and
# until env_with_petsc() also moved DYLD_LIBRARY_PATH the complex scripts inherited it:
# petsc4py reported complex scalars (a compile-time constant) while having mapped
# real/lib/libpetsc, and slepc4py died on a symbol only the complex build exports.
for v in PETSC_DIR PETSC_ARCH_REAL PETSC_ARCH_COMPLEX PATH PYTHONPATH \
LD_LIBRARY_PATH DYLD_LIBRARY_PATH; do
echo "${v}=${!v}" >> "${GITHUB_ENV}"
done
- name: Run the tutorial scripts (Linux/macOS)
if: steps.want.outputs.run == '1' && matrix.platform != 'windows'
shell: bash
env:
# No display on a runner, and several scripts call plt.show()-adjacent code paths.
MPLBACKEND: Agg
# The harness itself only imports docs/source/tutorial/tutorial_bundle.py; it never needs
# pyoomph, and running it with the venv's python keeps the two consistent anyway.
QUICK: ${{ inputs.quick_test && '--quick-test' || '' }}
SKIPS: ${{ inputs.skip_folders }}
ONLY: ${{ inputs.only }}
TIMEOUT: ${{ inputs.script_timeout }}
OMP: ${{ inputs.omp }}
run: |
set -o pipefail
# --no-petsc unless the PETSc prebuild was installed above (macOS arm64), which is what
# sets PETSC_DIR. With it the harness runs each script against the real or the complex
# build, whichever that script needs; without it every eigenproblem uses spectra.
# An if, not `[[ ... ]] && NO_PETSC=`: this step runs under `bash -e`, where the
# trailing && of a false test is the step's exit status and every non-arm64 job would
# stop right here.
NO_PETSC="--no-petsc"
if [[ -n "${PETSC_DIR:-}" ]]; then NO_PETSC=""; fi
# Only when asked for: --omp 0 would still take the threaded code path with one thread,
# which is not the serial element loop every previous run of this workflow measured.
OMP_FLAG=""
if [[ "${OMP:-0}" != "0" ]]; then OMP_FLAG="--omp ${OMP}"; fi
# Word splitting is the point: each substring becomes its own repeatable --only.
ONLY_FLAGS=""
for o in ${ONLY:-}; do ONLY_FLAGS="${ONLY_FLAGS} --only ${o}"; done
# The harness exits 0 whatever happens - the verdict comes from the report in the next
# step - so nothing here can hide a failing script.
"${TUTORIAL_PYTHON}" -u citools/test_all_tutorial_scripts.py \
${NO_PETSC} ${QUICK} --timeout "${TIMEOUT:-0}" ${OMP_FLAG} \
${ONLY_FLAGS} --report-json tutorial_report.json \
${SKIPS} 2>&1 | tail -n 4000
- name: Run the tutorial scripts (Windows)
if: steps.want.outputs.run == '1' && matrix.platform == 'windows'
shell: msys2 {0}
env:
MPLBACKEND: Agg
QUICK: ${{ inputs.quick_test && '--quick-test' || '' }}
SKIPS: ${{ inputs.skip_folders }}
ONLY: ${{ inputs.only }}
TIMEOUT: ${{ inputs.script_timeout }}
OMP: ${{ inputs.omp }}
run: |
set -o pipefail
cd "$(cygpath -u "$GITHUB_WORKSPACE")"
# Run from the MSYS2 shell so that the UCRT64 gcc is on PATH: it is the JIT compiler
# pyoomph picks for the generated element code, and without it every single script would
# fall back to tccbox (or fail outright).
export PATH="/ucrt64/bin:$PATH"
OMP_FLAG=""
if [[ "${OMP:-0}" != "0" ]]; then OMP_FLAG="--omp ${OMP}"; fi
# Word splitting is the point: each substring becomes its own repeatable --only.
ONLY_FLAGS=""
for o in ${ONLY:-}; do ONLY_FLAGS="${ONLY_FLAGS} --only ${o}"; done
"$(cygpath -u "${TUTORIAL_PYTHON}")" -u citools/test_all_tutorial_scripts.py \
--no-petsc ${QUICK} --timeout "${TIMEOUT:-0}" ${OMP_FLAG} \
${ONLY_FLAGS} --report-json tutorial_report.json \
${SKIPS} 2>&1 | tail -n 4000
# The point of the whole workflow: the per-script verdict and timing, on the run's summary
# page rather than at line 40000 of the log. Its exit code is what fails the job.
- name: Summarise
if: always() && steps.want.outputs.run == '1'
shell: bash
run: |
# In the workspace rather than under RUNNER_TEMP: on the Windows runner this step's bash
# is git-bash, which is handed RUNNER_TEMP as a backslashed Windows path it cannot then
# use as a path at all.
REPORT="tutorial_report.json"
if [[ ! -f "${REPORT}" ]]; then
echo "## ${{ matrix.label }} - no report" >> "${GITHUB_STEP_SUMMARY}"
echo "" >> "${GITHUB_STEP_SUMMARY}"
echo "The harness did not get as far as writing its report; see the job log." >> "${GITHUB_STEP_SUMMARY}"
exit 1
fi
# Once, into a file: it is both appended to the summary page and echoed into the log,
# and its exit code (non-zero if any script failed) is the job's verdict.
# Not a hardcoded "python3": the Windows runner's bash has python.exe and no python3.
PY="$(command -v python3 || command -v python)"
set +e
"${PY}" citools/tutorial_report_to_summary.py "${REPORT}" \
--title "${{ matrix.label }}" --out tutorial_summary.md
STATUS=$?
set -e
cat tutorial_summary.md >> "${GITHUB_STEP_SUMMARY}"
cat tutorial_summary.md
exit ${STATUS}
# The report plus the log of every script that failed - the harness writes <script>.log next
# to the script it belongs to, and only for failures unless --keep-logs is given.
- name: Upload the report and the failure logs
if: always() && steps.want.outputs.run == '1'
uses: actions/upload-artifact@v4
with:
name: tutorial-report-${{ matrix.label }}
path: |
tutorial_report.json
citools/pyoomph_tutorial_scripts/**/*.log
if-no-files-found: warn
# Kept so a failing script can be reproduced locally against the exact same binary.
- name: Upload the wheel
if: always() && steps.want.outputs.run == '1'
uses: actions/upload-artifact@v4
with:
name: tutorial-wheel-${{ matrix.label }}
path: wheelhouse/*.whl
if-no-files-found: warn