Skip to content

improvement update test levels - #853

Merged
pahmann merged 14 commits into
eclipse-score:mainfrom
etas-contrib:improvement_test_levels
Nov 3, 2026
Merged

pahmann merged 14 commits into
eclipse-score:mainfrom
etas-contrib:improvement_test_levels

Conversation

@RolandJentschETAS

Copy link
Copy Markdown
Contributor

This pull request updates the requirements and guidance for linking tests to requirements and clarifies the allowed metadata for test cases at different verification levels. The main changes include stricter definitions for test metadata, explicit mapping of allowed link types per test level, and updates to documentation to ensure consistency across the verification process.

Process requirements and metadata guidance updates:

  • The Linking Requirements to Tests and Verification Documentation Checks requirements (gd_req__verification_link_tests and gd_req__verification_checks) are updated to version 2, reflecting stricter and more detailed guidance for linking tests to requirements. [1] [2]
  • Test metadata fields such as TestType and DerivationTechnique now specify which test levels (unit, component, feature, platform) each value applies to, and fields may contain multiple comma-separated values.
  • The allowed types for Verifies links are now explicitly defined for each test level, determined by folder structure, and minimum coverage requirements are enforced for each level.

Documentation and workflow consistency:

  • References in the verification concept documentation are updated to require use of only the supported identifiers and test levels for Sphinx-Needs metadata, ensuring alignment with the new requirements. [1] [2]
  • All relevant verification workflow definitions now reference the updated version (2) of the gd_req__verification_link_tests requirement, ensuring process consistency. [1] [2] [3] [4]

@github-actions

Copy link
Copy Markdown

The created documentation from the pull request is available at: docu-html

PandaeDo
PandaeDo previously approved these changes Nov 3, 2026

@PandaeDo PandaeDo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine for me

* Interface Test (interface-test)
* Requirements-based Test (requirements-based)
* Resource Usage Evaluation (resource-usage)
* Requirements-based Test (requirements-based): unit, component, feature, platform -> for testing functional, process, tool or other non-functional requirements which are not resource-related (units have no requirements of their own, but unit tests can verify component or feature requirements)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to keep the part with "->" as additional explanation or is this maybe already covered in the "quick guide"

* Analysis of equivalence classes (equivalence-classes): all levels -> for interfaces, interface requirements or functional requirements containing values; can be combined with boundary-values
* Fuzz testing (fuzz-testing): unit, component, feature -> for interfaces, interface requirements or functional requirements containing values which are tested with random values
* Error guessing based on knowledge or experience (error-guessing): all levels -> for testing known error-prone areas; applicable for all test types
* Explorative testing (explorative-testing): feature, platform -> for exploring the system and its boundaries without predefined test cases

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could consider to use the statement from another section also here instead of "->"

Allowed values for *TestType* and *DerivationTechnique* and a short guide when to use which are available in the :ref:`verification_quick_guide`. Multiple values can be given for both.

…ds.rst

Co-authored-by: Philipp Ahmann <2428012+pahmann@users.noreply.github.com>
Signed-off-by: RolandJentschETAS <135332348+RolandJentschETAS@users.noreply.github.com>
RolandJentschETAS and others added 8 commits November 3, 2026 08:28
Co-authored-by: Philipp Ahmann <2428012+pahmann@users.noreply.github.com>
Signed-off-by: RolandJentschETAS <135332348+RolandJentschETAS@users.noreply.github.com>
Co-authored-by: Philipp Ahmann <2428012+pahmann@users.noreply.github.com>
Signed-off-by: RolandJentschETAS <135332348+RolandJentschETAS@users.noreply.github.com>
Co-authored-by: Philipp Ahmann <2428012+pahmann@users.noreply.github.com>
Signed-off-by: RolandJentschETAS <135332348+RolandJentschETAS@users.noreply.github.com>
Co-authored-by: Philipp Ahmann <2428012+pahmann@users.noreply.github.com>
Signed-off-by: RolandJentschETAS <135332348+RolandJentschETAS@users.noreply.github.com>
Co-authored-by: Philipp Ahmann <2428012+pahmann@users.noreply.github.com>
Signed-off-by: RolandJentschETAS <135332348+RolandJentschETAS@users.noreply.github.com>
Co-authored-by: Philipp Ahmann <2428012+pahmann@users.noreply.github.com>
Signed-off-by: RolandJentschETAS <135332348+RolandJentschETAS@users.noreply.github.com>
@pahmann
pahmann merged commit c313459 into eclipse-score:main Nov 3, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants