feat: add an orchestration layer for testing packages

docs/.nav.yml

@@ -329,6 +329,7 @@ nav:
       - testing/benchmark.md
       - testing/debugging.md
       - "Session Testing": testing/session_testing.md
+      - testing/packages.md
   - "Command Line Usage":
       - cli/index.md
       - cli/cli_overview.md

docs/changelogs/v4.8.0.md

@@ -249,6 +249,7 @@ Release Date: Unreleased
 
 - Added `assertSameSql()` to `CIUnitTestCase` to compare generated SQL while ignoring newlines in the actual SQL.
 - Added `FeatureTestTrait::withFiles()` and `MockUploadedFile` for testing routes that validate and move uploaded files.
+- Added `CodeIgniter\Test\Orchestrator\AbstractOrchestrator` to build a throwaway application skeleton that packages test against. See [Testing Packages](../testing/packages.md) for details.
 
 ### Database
 

docs/index.md

@@ -171,6 +171,7 @@
     - [Benchmarking](testing/benchmark.md)
     - [Debugging Your Application](testing/debugging.md)
     - [Session Testing](testing/session_testing.md)
+    - [Testing Packages](testing/packages.md)
 
 - [Command Line Usage](cli/index.md)
     - [CLI Overview](cli/cli_overview.md)

docs/testing/index.md

@@ -13,3 +13,4 @@ CodeIgniter ships with a number of tools to help you test and debug your applica
 - [Benchmarking](benchmark.md)
 - [Debugging Your Application](debugging.md)
 - [Session Testing](session_testing.md)
+- [Testing Packages](packages.md)

docs/testing/packages.md

@@ -0,0 +1,115 @@
+# Testing Packages
+
+!!! success "Available from version 4.8.0"
+
+A package that supplements CodeIgniter has no `app/`, `public/`, `writable/` or `spark` of its own. **AbstractOrchestrator** builds a throwaway application skeleton from the framework that your package is installed with, so that your tests run against real paths, configuration and commands instead of stubs.
+
+## Defining an Orchestrator
+
+Extend `CodeIgniter\Test\Orchestrator\AbstractOrchestrator` and implement `getPackageRoot()`:
+
+```php
+<?php
+
+namespace Acme\Blog\Tests;
+
+use CodeIgniter\Test\Orchestrator\AbstractOrchestrator;
+
+final class BlogOrchestrator extends AbstractOrchestrator
+{
+    protected function getPackageRoot(): string
+    {
+        return dirname(__DIR__);
+    }
+
+    protected function getFixtures(): array
+    {
+        return [__DIR__ . '/Fixtures'];
+    }
+
+    protected function getEnv(): array
+    {
+        return ['app.baseURL' => 'http://localhost/'];
+    }
+}
+```
+
+The package root is the directory holding `vendor/autoload.php`. The skeleton shares that autoloader, so your package and its dev dependencies are available inside it.
+
+The following methods can be overridden:
+
+- `getFixtures()` returns a list of directories that are mirrored onto the root of the skeleton, in order, after the framework files. A later directory overwrites an earlier one. Use them to add controllers, routes, configuration or a `Registrar`, for example `Fixtures/app/Config/Routes.php`.
+- `getEnv()` returns settings for the skeleton's `.env`. Values are quoted, so they may contain spaces, `#` and quotes, but a value with a line break is refused. Names may only contain letters, digits, `.`, `_`, `-` and `\`, which covers forms such as `app.baseURL` and `Config\App.baseURL`. `CI_ENVIRONMENT` defaults to `development`, because `spark` does not run in the `testing` environment.
+- `getFrameworkPath()` returns the framework that the skeleton is copied from, which is the one that provides the orchestrator. Expose it from your subclass if your tests need the framework's location.
+- `afterBuild()` runs when the skeleton is complete, for example to create a database.
+- `getPath()` returns the skeleton's location, `<packageRoot>/build/orchestrator` by default. A token is appended to it so that parallel processes do not share a skeleton. The token is read from `ORCHESTRATOR_TEST_TOKEN`, or else from `TEST_TOKEN`, which parallel runners such as ParaTest set for each process. It may only contain letters, digits, hyphens and underscores: an invalid `ORCHESTRATOR_TEST_TOKEN` throws an exception, and an invalid `TEST_TOKEN` is ignored. Set `ORCHESTRATOR_TEST_TOKEN` to an empty string to opt out of `TEST_TOKEN`.
+
+!!! note "Note"
+    The skeleton is generated, so add its location to your `.gitignore`, which is `build/` unless `getPath()` is overridden.
+
+## Testing in the PHPUnit Process
+
+Call `boot()` from your PHPUnit bootstrap file. It builds the skeleton, defines the path constants against it, and loads the framework's `system/Test/bootstrap.php`:
+
+```php
+<?php
+
+// tests/bootstrap.php
+
+require_once __DIR__ . '/../vendor/autoload.php';
+
+(new Acme\Blog\Tests\BlogOrchestrator())->boot();
+```
+
+Point `bootstrap` at this file in your `phpunit.dist.xml`. Since it replaces Composer's autoloader as the bootstrap, it loads `vendor/autoload.php` from the package root itself, so that your orchestrator class can be found. Your tests then extend `CIUnitTestCase` as usual.
+
+!!! important "Important"
+    The path constants and the framework's static state last for the whole process, so one process hosts one skeleton.
+
+## Testing Through spark
+
+`spark()` runs the skeleton's `spark` in a separate process and returns a `SparkResult` with the `exitCode`, `stdout`, `stderr` and `timedOut` properties:
+
+```php
+$orchestrator = new BlogOrchestrator();
+$orchestrator->build();
+
+$result = $orchestrator->spark(['blog:publish', '--dry-run'], stdin: "yes\n", environment: ['APP_DEBUG' => '1']);
+
+$this->assertSame(0, $result->exitCode);
+$this->assertStringContainsString('Published', $result->stdout);
+```
+
+`stdin` is written to the process, and `environment` adds to the inherited environment. A `null` value unsets a variable. Pass `timeout` in seconds to terminate a process that runs longer, in which case `timedOut` is `true`.
+
+!!! warning "Warning"
+    A terminated process usually reports `-1` as its `exitCode`, so check `timedOut` first. Only the `spark` process is terminated, so processes that it started itself may keep running.
+
+`stdin` is not a terminal, which affects interactive commands:
+
+- Each line of `stdin` answers the next prompt asked from `execute()`, such as `ask()`. `select()` and `multiselect()` fall back to numbered choices, so pass the number instead of key presses.
+- The command runs as non-interactive, so `interact()` and the prompts for missing arguments are skipped. Pass those values as arguments and options instead.
+- Some prompt labels are not written to `stdout`, so assert on the command's own output instead.
+
+!!! warning "Warning"
+    Once `stdin` runs out, the remaining prompts take their defaults instead of failing, so a missing answer does not stop the command.
+
+## Changing the Skeleton
+
+`write()` creates a file in the skeleton, and `delete()` removes a file or a directory tree from it. Both take a path relative to the skeleton, and refuse a path that would leave it, whether through `..` or a symlink. `build()` removes the previous skeleton before it creates a new one, and `clean()` removes the skeleton, for example from `tearDownAfterClass()`. A skeleton booted with `boot()` is read until the process ends, so it is left in place.
+
+## Worker Mode
+
+The skeleton has no [Worker Mode](../installation/worker_mode.md) files of its own, because `worker:install` generates them. Run it from `afterBuild()` to publish `public/frankenphp-worker.php` and the `Caddyfile` from the template of the framework that the skeleton is copied from:
+
+```php
+protected function afterBuild(): void
+{
+    $this->spark(['worker:install']);
+}
+```
+
+Your tests can then inspect or change the published files with `write()` and `delete()`.
+
+!!! note "Note"
+    The orchestrator does not run FrankenPHP. `spark()` waits for its process to exit, so it cannot host a server, and `boot()` sets up the regular per-process test environment instead of the worker's request loop. To send real requests to a worker, start `frankenphp run` yourself with the skeleton as its working directory, and stop it when your tests finish.