Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .readme-partials/USING.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,36 @@ You can use the following commands to control the tests:
* `composer phpunit` - Run only the unit test suite.
* `composer behat` - Run only the functional test suite.

### Behat 3 and Behat 4

The testing framework works with both Behat 3 and Behat 4. Composer installs Behat 4 on PHP 8.2 and
later, and Behat 3 on older PHP versions, and a package that only uses the steps of the framework
does not need to change anything for either.

Behat 4 dropped a few things that the framework bridges, within limits:

* Behat 4 only reads its configuration from a `behat.php` (or `behat.dist.php`) file. For a package
that only has a `behat.yml`, `composer behat` hands the `default` profile of that file to Behat 4
through the `BEHAT_PARAMS` environment variable, with anything that variable already holds
merged over it. Other profiles and `imports` are not carried over; a package that needs those
adds a `behat.php` next to its `behat.yml`, or names a configuration file with `--config`.
* Behat 4 only reads step definitions and hooks from attributes (`#[Given( '...' )]`), not from
annotations (`@Given ...`) anymore. This matters to packages with a context class of their own.
Behat 3 reads attributes as well, but only on PHP 8, and it defines a step twice, which is an
error, when a method has both. A context that has to run on PHP 7 too can do what
`WP_CLI\Tests\Context\FeatureContext` does: declare everything with attributes, and repeat the
declarations as annotations on an interface that the context only implements on PHP 7. Behat
3.15, the last release for PHP 8.0, has no attributes for the `BeforeSuite` and `AfterSuite`
hooks yet, so there the interface still has to carry the annotations of those two.
* Behat 4 fails a step whose regular expression captures more values than its method has
parameters, where Behat 3 dropped the extra ones. Use non-capturing groups (`(?:...)`) for those.

To keep a package on Behat 3 for the time being, require it explicitly:

```bash
composer require --dev behat/behat:^3.15
```

### Analysing the PHP blocks in feature files

Feature files embed PHP snippets in docstrings, which none of the static analysis tools normally
Expand Down
30 changes: 30 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,36 @@ You can use the following commands to control the tests:
* `composer phpunit` - Run only the unit test suite.
* `composer behat` - Run only the functional test suite.

### Behat 3 and Behat 4

The testing framework works with both Behat 3 and Behat 4. Composer installs Behat 4 on PHP 8.2 and
later, and Behat 3 on older PHP versions, and a package that only uses the steps of the framework
does not need to change anything for either.

Behat 4 dropped a few things that the framework bridges, within limits:

* Behat 4 only reads its configuration from a `behat.php` (or `behat.dist.php`) file. For a package
that only has a `behat.yml`, `composer behat` hands the `default` profile of that file to Behat 4
through the `BEHAT_PARAMS` environment variable, with anything that variable already holds
merged over it. Other profiles and `imports` are not carried over; a package that needs those
adds a `behat.php` next to its `behat.yml`, or names a configuration file with `--config`.
* Behat 4 only reads step definitions and hooks from attributes (`#[Given( '...' )]`), not from
annotations (`@Given ...`) anymore. This matters to packages with a context class of their own.
Behat 3 reads attributes as well, but only on PHP 8, and it defines a step twice, which is an
error, when a method has both. A context that has to run on PHP 7 too can do what
`WP_CLI\Tests\Context\FeatureContext` does: declare everything with attributes, and repeat the
declarations as annotations on an interface that the context only implements on PHP 7. Behat
3.15, the last release for PHP 8.0, has no attributes for the `BeforeSuite` and `AfterSuite`
hooks yet, so there the interface still has to carry the annotations of those two.
* Behat 4 fails a step whose regular expression captures more values than its method has
parameters, where Behat 3 dropped the extra ones. Use non-capturing groups (`(?:...)`) for those.

To keep a package on Behat 3 for the time being, require it explicitly:

```bash
composer require --dev behat/behat:^3.15
```

### Analysing the PHP blocks in feature files

Feature files embed PHP snippets in docstrings, which none of the static analysis tools normally
Expand Down
21 changes: 20 additions & 1 deletion bin/run-behat-tests
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/bash

# Run the Behat tests only if a Behat config file is found.
if [ ! -f "behat.yml" ]; then
if [ ! -f "behat.yml" ] && [ ! -f "behat.php" ] && [ ! -f "behat.dist.php" ]; then
echo 'Did not detect "behat.yml" file, skipping Behat tests.'
exit 0;
fi
Expand Down Expand Up @@ -293,6 +293,25 @@ done
WP_CLI_TESTS_ROOT="$( cd -P "$( dirname "$SOURCE" )/.." && pwd )"
export WP_CLI_TESTS_ROOT

# Behat 4 only reads PHP configuration. Unless the project has one, or the run names a configuration
# file, hand Behat the default profile of behat.yml through BEHAT_PARAMS, which it reads in every
# version. What BEHAT_PARAMS already holds takes precedence over the file, as it does in Behat 3.
BEHAT_CONFIG_ARG=""
for arg in "$@"; do
case "$arg" in
-c|-c*|--config|--config=*) BEHAT_CONFIG_ARG=1;;
esac
done

if [ -f "behat.yml" ] && [ ! -f "behat.php" ] && [ ! -f "behat.dist.php" ] && [ -z "${BEHAT_CONFIG_ARG}" ]; then
if ! BEHAT_YML_PARAMS=$(php "$WP_CLI_TESTS_ROOT"/utils/behat-config.php); then
exit 1
fi
if [ -n "${BEHAT_YML_PARAMS}" ]; then
export BEHAT_PARAMS="${BEHAT_YML_PARAMS}"
fi
fi

# Generate the tags to apply environment-specific filters.
BEHAT_TAGS=$(php "$WP_CLI_TESTS_ROOT"/utils/behat-tags.php)

Expand Down
2 changes: 1 addition & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
"require": {
"php": ">=7.2.24",
"ext-tokenizer": "*",
"behat/behat": "^v3.15.0",
"behat/behat": "^v3.15.0 || ^v4.0.0",
"dealerdirect/phpcodesniffer-composer-installer": "^0.4.3 || ^0.5 || ^0.6.2 || ^0.7.1 || ^1.0.0",
"johnbillion/wp-compat": "^2.0 || ^3.0",
"php-parallel-lint/php-console-highlighter": "^1.0",
Expand Down
9 changes: 9 additions & 0 deletions phpcs.xml.dist
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,9 @@

<exclude-pattern>tests/phpstan/scan-files.php</exclude-pattern>

<!-- Generated by utils/generate-legacy-annotations.php. -->
<exclude-pattern>src/Context/legacy/*</exclude-pattern>

<rule ref="WordPress.NamingConventions">
<exclude-pattern>tests/data/*</exclude-pattern>
<exclude-pattern>src/PHPStan/*</exclude-pattern>
Expand All @@ -75,6 +78,12 @@
<exclude-pattern>src/PHPStan/*</exclude-pattern>
</rule>

<!-- The contexts declare their steps and hooks with attributes for Behat 4.
PHP 7 gets them as annotations instead, see src/Context/LegacyAnnotations.php. -->
<rule ref="PHPCompatibility.Attributes.NewAttributes.Found">
<exclude-pattern>src/Context/*</exclude-pattern>
</rule>

<!-- This is a procedural stand-alone file that is never loaded in a WordPress context,
so this file does not have to comply with WP naming conventions. -->
<rule ref="WordPress.NamingConventions.PrefixAllGlobals">
Expand Down
4 changes: 4 additions & 0 deletions phpstan.neon.dist
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,16 @@ parameters:
# Templates are copied into the test run directory and executed by the
# child WP-CLI process, where Requests v1 classes may still exist.
- src/Context/templates
# Generated, and declaring the interface that src/Context/LegacyAnnotations.php
# declares empty where the attributes are read.
- src/Context/legacy
scanDirectories:
- vendor/wp-cli/wp-cli
- vendor/phpunit/php-code-coverage
- vendor/behat/behat
scanFiles:
- tests/phpstan/scan-files.php
- utils/generate-legacy-annotations.php
treatPhpDocTypesAsCertain: false
dynamicConstantNames:
- WP_DEBUG
Expand Down
45 changes: 17 additions & 28 deletions src/Context/FeatureContext.php
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,13 @@
use Behat\Behat\Hook\Scope\AfterFeatureScope;
use Behat\Behat\Hook\Scope\BeforeFeatureScope;
use Behat\Behat\Hook\Scope\BeforeStepScope;
use Behat\Hook\AfterFeature;
use Behat\Hook\AfterScenario;
use Behat\Hook\AfterSuite;
use Behat\Hook\BeforeFeature;
use Behat\Hook\BeforeScenario;
use Behat\Hook\BeforeStep;
use Behat\Hook\BeforeSuite;
use SebastianBergmann\CodeCoverage\Report\Clover;
use SebastianBergmann\CodeCoverage\Driver\Selector;
use SebastianBergmann\CodeCoverage\Driver\Xdebug;
Expand All @@ -31,7 +38,7 @@
/**
* Features context.
*/
class FeatureContext implements Context {
class FeatureContext implements Context, LegacyAnnotations {

use GivenStepDefinitions;
use ThenStepDefinitions;
Expand Down Expand Up @@ -261,38 +268,28 @@ class FeatureContext implements Context {
*/
private $step_line = 0;

/**
* @BeforeFeature
*/
#[BeforeFeature]
public static function store_feature( BeforeFeatureScope $scope ): void {
self::$feature = $scope->getFeature();
}

/**
* @BeforeScenario
*/
#[BeforeScenario]
public function store_scenario( BeforeScenarioScope $scope ): void {
$this->scenario = $scope->getScenario();
}

/**
* @BeforeStep
*/
#[BeforeStep]
public function store_step( BeforeStepScope $scope ): void {
$this->step_line = $scope->getStep()->getLine();
}

/**
* @AfterScenario
*/
#[AfterScenario]
public function forget_scenario( AfterScenarioScope $scope ): void {
$this->step_line = 0;
$this->scenario = null;
}

/**
* @AfterFeature
*/
#[AfterFeature]
public static function forget_feature( AfterFeatureScope $scope ): void {
self::$feature = null;
}
Expand Down Expand Up @@ -1105,9 +1102,7 @@ private static function cache_wp_files( $version = '' ): void {
self::$cache_dir = $cache_dir;
}

/**
* @BeforeSuite
*/
#[BeforeSuite]
public static function prepare( BeforeSuiteScope $scope ): void {
self::bootstrap_feature_context();

Expand Down Expand Up @@ -1141,9 +1136,7 @@ public static function prepare( BeforeSuiteScope $scope ): void {
}
}

/**
* @AfterSuite
*/
#[AfterSuite]
public static function afterSuite( AfterSuiteScope $scope ): void {
if ( self::$composer_local_repository ) {
self::remove_dir( self::$composer_local_repository );
Expand All @@ -1157,9 +1150,7 @@ public static function afterSuite( AfterSuiteScope $scope ): void {
self::merge_coverage_reports();
}

/**
* @BeforeScenario
*/
#[BeforeScenario]
public function beforeScenario( BeforeScenarioScope $scope ): void {
if ( self::$log_run_times ) {
self::log_run_times_before_scenario( $scope );
Expand All @@ -1182,9 +1173,7 @@ public function beforeScenario( BeforeScenarioScope $scope ): void {
}
}

/**
* @AfterScenario
*/
#[AfterScenario]
public function afterScenario( AfterScenarioScope $scope ): void {

if ( self::$run_dir ) {
Expand Down
Loading
Loading