Repository navigation
RunTests - test discovery using ALTool instead of BcContainerHelper - #2426
Draft
spetersenms wants to merge 61 commits into
Draft
spetersenms wants to merge 61 commits into
spetersenms wants to merge 61 commits into
Conversation
Introduces a new opt-in RunTests action that moves normal test execution (testFolders) out of the RunPipeline action. When useSeparateTestAction is true, RunPipeline compiles, publishes and installs apps and keeps the build container alive, but skips normal tests; the RunTests action then runs those tests against the same container and produces TestResults.xml. BCPT and page scripting tests are unchanged. Behavior is unchanged when the setting is false. Includes the new setting (ReadSettings, settings.schema.json, settings.md), the RunPipeline keep-alive guard, template _BuildALGoProject wiring, release notes and Pester tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The separate RunTests action now runs normal tests through Microsoft's headless al runtests (AlTool) CLI by default instead of BcContainerHelper. The runner resolves the kept-alive container's connection settings host-side from the container name, installs the AL developer tools as a dotnet global tool, runs the test codeunits batched (with an isolated fallback/rerun pass) and emits the same TestResults.xml (JUnit) schema, so downstream test result analysis is unchanged. No RunPipeline change is required. BcContainerHelper remains available via a RunTestsInBcContainer override script, which fully replaces the built-in AlTool runner (this is how, for example, BCApps supplies its own runner and how Legacy test-type tests are handled). The AlTool runner does not run Legacy test types or tests that require UI/client-callback interaction; this is documented as a known limitation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The official released al runtests tool does not support --testplan, so the batch-first path in AlToolTestRunner hard-set �llPassed=false whenever the batch failed to connect and never reset it, making every run falsely report FAILED. Remove the batching path entirely and run each test codeunit in its own �l runtests <codeunitId> --testmethods --raw invocation, with the existing rerun-on-failure pass. �llPassed now derives solely from real per-method outcomes via Add-JUnitTestSuite. - Delete Invoke-AlBatchRunTests, ConvertFrom-AlBatchOutput, ConvertTo-AlTestPlanJson and their Export-ModuleMember entries. - Update README and RELEASENOTES to per-codeunit wording. - Drop the two batch-specific unit test contexts. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…. Using -recurse flag for ConvertTo-HashTable
…tersen/separateTestAction
…tersen/separateTestAction
…tersen/separateTestAction
…tersen/separateTestAction
…to spetersen/separateTestAction
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…tersen/separateTestAction
…tersen/separateTestAction
… restoreDatabases is set, as that is currently not supported.
… around parsing callstack for failed AlTool test runs.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
AlTool is resolved before the no-test-apps return, so projects with no test packages can fail on unnecessary tool installation instead of skipping test execution.
1 open issue
What changed in this PR
Migrates the separate RunTests action from BcContainerHelper test discovery to ALTool package discovery while preserving disabled-test filtering and batched execution.
Changes:
- Discovers app metadata and test methods directly from
.apppackages. - Warns that built-in
testTypefiltering is unsupported while preserving the value for custom overrides. - Updates unit tests, end-to-end assertions, and documentation for the discovery flow.
| File | Description |
|---|---|
| Tests/RunTests.Test.ps1 | Updates discovery mocks and runner integration assertions. |
| Tests/AlToolTestRunner.Test.ps1 | Tests discovery parsing, validation, filtering, and runner inputs. |
| Scenarios/settings.md | Documents the testType filtering limitation. |
| RELEASENOTES.md | Describes package discovery and updated runner limitations. |
| e2eTests/scenarios/SeparateTestAction/runtest.ps1 | Checks discovery and selected-method log output. |
| Actions/RunTests/RunTests.psm1 | Integrates package discovery and passes codeunits to execution. |
| Actions/RunTests/README.md | Explains ALTool discovery and remaining BcContainerHelper responsibilities. |
| Actions/RunTests/AlToolTestRunner.psm1 | Adds package discovery and filters discovered tests for execution. |
| Actions/.Modules/settings.schema.json | Aligns the testType description with current behavior. |
Review effort: Balanced
Comment on lines
+294
to
+299
| if ([string]::IsNullOrWhiteSpace($alToolPath)) { | ||
| if ($getAlToolPath) { | ||
| $alToolPath = & $getAlToolPath | ||
| } | ||
| else { | ||
| $alToolPath = GetAlToolPath |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
❔What, Why & How
The initial implementation of RunTests used the legacy BcContainerHelper function for test discovery. ALTool has since added support for this functionality. This PR updates RunTests to use the ALTool test discovery.
There is one current limitation which is that ALTool does not support filtering by test type. This means that the test type setting for testing is ignored for RunTests until this is implemented in ALTool.
Disabled test logic is still somewhat long since ALTool doesn't support a list of tests not to run, only a list of tests to run. This means AL-Go has to first find the full list of tests, then the list of tests to skip and then filter based on that and finally pass that list to the ALTool test runner.
✅ Checklist