Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/architecture/guards.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ hand-edited; `--check` refuses drift (`X_GUARDS_DOC_DRIFT`). A guard is a runnab
| `bun run dead-docs-host` | Enforce, as a ratchet, that no shipped source builds a URL on `ultimate.dev`. | `X_DEAD_DOCS_HOST` `X_DEAD_DOCS_HOST_PIN_STALE` `X_DEAD_DOCS_HOST_UNSCANNED` |
| `bun run declaration-readers-callers` | Enforce that every `@ultimat3/core` export that THROWS has a non-test caller — in shipped framework source or in a tracked app. | `X_CORE_THROWERS_UNSCANNED` `X_CORE_THROWER_PIN_STALE` `X_CORE_THROWER_UNCALLED` |
| `bun run declaration-readers` | Enforce that every leaf key of every DECLARATION an app writes — the type each primitive and each `define*` factory takes — is read by something in `packages/*/src`. | `X_DECLARATION_READERS_UNSCANNED` `X_DECLARATION_READER_PIN_STALE` `X_DECLARED_KEY_UNREAD` |
| `bun run dependency-allowlist` | Enforce that every external workspace dependency is an explicit row in the checked-in allowlist. | `X_DEPENDENCY_ALLOWLIST_STALE` `X_DEPENDENCY_NOT_ALLOWLISTED` `X_DEPENDENCY_UNSCANNED` |
| `bun run doc-commands` | Enforce, as a gate step, that a command a doc page hands a reader is a command this build ships. | `X_DOC_COMMAND_ALLOWANCE_STALE` `X_DOC_COMMAND_PIN_EXCEEDED` `X_DOC_COMMAND_PIN_STALE` `X_DOC_COMMAND_UNKNOWN` `X_DOC_COMMAND_UNSCANNED` `X_GUARD_LOAD_FAILED` |
| `bun run scripts/doc-config-keys.ts` | Enforce, as a gate step, that documentation citing an `app.config.ts` key names a key that EXISTS. | `X_DOC_CONFIG_ALLOWANCE_STALE` `X_DOC_CONFIG_KEY_UNKNOWN` |
| `bun run scripts/doc-fixes.ts` | Enforce, as a gate step, that the `Fix` column of `wiki/Error-Codes.md` is runnable — the same rule `x verify`'s `errors` step already applies to every `fix:` in shipped source. | `X_DOC_CAUSE_ECHOES_TITLE` `X_DOC_FIX_UNRUNNABLE` `X_DOC_FIX_UNSCANNED` `X_NEW_ERROR_CODE_EXISTS` `X_NEW_ERROR_CODE_INVALID` `X_NEW_ERROR_CODE_PATTERN_UNKNOWN` |
Expand Down Expand Up @@ -85,6 +86,7 @@ hand-edited; `--check` refuses drift (`X_GUARDS_DOC_DRIFT`). A guard is a runnab
| `bun run scripts/setup-commands.ts` | Enforce, as a gate step, that every page enumerating what a scaffolded app's `bun run setup` runs enumerates the script `x new` actually writes. | `X_DOC_SETUP_COMMANDS_UNSCANNED` `X_DOC_SETUP_COMMAND_STALE` |
| `bun run side-effects` | Two rules, and both are about what a bundler that HONOURS the array does with it. | `X_CLI_BAD_FLAG` `X_SIDE_EFFECTS_ANCHOR_STALE` `X_SIDE_EFFECTS_BY_USE_STALE` `X_SIDE_EFFECTS_ENTRY_STALE` `X_SIDE_EFFECTS_MISSING` `X_SIDE_EFFECTS_PIN_STALE` `X_SIDE_EFFECTS_UNANCHORED` `X_SIDE_EFFECTS_UNDECLARED` `X_SIDE_EFFECTS_UNSCANNED` |
| `bun run skip-if-cleanup` | Refuse a test file that resets a PROCESS-GLOBAL registry from inside a skippable block — a `.skipIf(` or a `describe.skip` alias — where the reset never runs in the configuration the block is skipped in. | `X_SKIP_CLEANUP_UNREACHED` `X_SKIP_CLEANUP_UNSCANNED` |
| `bun run source-headers` | Enforce that every shipped source file opens with a 1–4 line header stating its one responsibility. | `X_SOURCE_HEADER_MISSING` `X_SOURCE_HEADER_OUT_OF_CONTRACT` `X_SOURCE_HEADER_PIN_STALE` `X_SOURCE_HEADER_UNSCANNED` |
| `bun run spawn-timeout` | Enforce, as a ratchet, that a child process this tree waits on carries a deadline: a `timeout` (or an abort `signal`) in its options. | `X_CORPUS_UNSCANNED` `X_SPAWN_TIMEOUT_PIN_STALE` `X_SPAWN_TIMEOUT_PIN_UNEXPLAINED` `X_SPAWN_UNTIMED` |
| `bun run sql-export-readers` | Enforce that every `SQL_*` statement a package's public entry exports has a reader outside it. | `X_SQL_EXPORT_UNREAD` `X_SQL_EXPORT_UNSCANNED` |
| `bun run sql-literal-copies` | Enforce that ONE module in this tree turns a value into a SQL string literal. | `X_SQL_LITERAL_COPY` `X_SQL_LITERAL_UNSCANNED` |
Expand Down
37 changes: 36 additions & 1 deletion framework.manifest.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
"declaration-readers": "bun --preload ./scripts/lib/guard-preload.ts scripts/declaration-readers.ts",
"doc-commands": "bun --preload ./scripts/lib/guard-preload.ts scripts/doc-commands.ts",
"doc-paths": "bun run scripts/doc-paths.ts",
"dependency-allowlist": "bun run scripts/dependency-allowlist.ts",
"flight-copies": "bun run scripts/flight-copies.ts",
"finite-bounds": "bun run scripts/finite-bounds.ts",
"dev": "bun run --filter @ultimat3/cli dev",
Expand Down Expand Up @@ -70,6 +71,7 @@
"side-effects": "bun run scripts/side-effects.ts",
"skip-if-cleanup": "bun run scripts/skip-if-cleanup.ts",
"test-discovery": "bun run scripts/test-discovery.ts",
"source-headers": "bun run scripts/source-headers.ts",
"spawn-timeout": "bun run scripts/spawn-timeout.ts",
"sql-literal-copies": "bun run scripts/sql-literal-copies.ts",
"test": "bun test --isolate --path-ignore-patterns='**/dist/**' --path-ignore-patterns='**/examples/**' --path-ignore-patterns='**/dummy/**'",
Expand Down
16 changes: 15 additions & 1 deletion packages/cli/src/templates/emitted-contract.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -86,14 +86,28 @@ describe('unit · every emitted file survives the linter the scaffold configures
sandboxes.push(root);
try {
for (const file of files) await Bun.write(join(dir, file.path), file.contents);
// A config of the sandbox's OWN, whenever the files under lint carry none. Biome resolves a
// config by walking UP from the directory it runs in, so a sandbox with no `biome.json` of
// its own reads the one above it — and the temp root is shared with everything else on the
// machine. A stray `/tmp/biome.json` left by another run is not a config Biome can parse,
// so the lint failed on the AMBIENT file rather than on the emitted one, and the answer
// depended on what else had been in the temp root that day. The scaffold carries its own
// config, so this only ever fires for the probe cases below.
if (!files.some((file) => file.path === 'biome.json')) {
await Bun.write(join(dir, 'biome.json'), '{}\n');
}
const biome = join(REPO_ROOT, 'node_modules', '.bin', 'biome');
// AWAITED, through the CLI's one subprocess boundary — never `Bun.spawnSync`. A synchronous
// wait holds the test worker's only thread, so a Biome that does not come back is a worker
// that does not either: this file is the one CI named when the `unit` step ran out its
// eight minutes (run 36992691094, `worker crashed: SIGKILL` under this file's header, after
// `killed 1 dangling process` and no result line). Awaited, a slow or stuck Biome is one
// test failing at its own timeout while the worker moves on to the next.
const run = await exec([biome, 'check', '.'], { cwd: dir });
// `--config-path` names the config to use and DISABLES the upward walk, so the run reads
// the sandbox's own config and nothing above it: the sandbox is the whole world the lint
// answers about. Without it a sandbox that carries no config still inherits one from the
// temp root, and the verdict stops being a fact about the emitted files.
const run = await exec([biome, 'check', '--config-path=.', '.'], { cwd: dir });
return run.ok ? '' : `${run.stdout}${run.stderr}`;
} finally {
// The findings are already a string. Left behind, each run kept a whole scaffold under the
Expand Down
2 changes: 2 additions & 0 deletions scripts/corpus-unscanned.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ import { probeDatabaseScan } from './probe-databases';
import { protoIndexFindings } from './proto-index';
import { vocabularyFindings } from './render-modes';
import { secretCompareFindings } from './secret-compare';
import { sourceHeaderFindings } from './source-headers';
import { spawnSites } from './spawn-timeout';
import { bareErrorFindings } from './test-bare-error';
import { testFixFindings } from './test-fix-citations';
Expand All @@ -56,6 +57,7 @@ const GUARDS: readonly (readonly [string, (root: string) => Promise<unknown>])[]
['proto-index', protoIndexFindings],
['render-modes', vocabularyFindings],
['secret-compare', secretCompareFindings],
['source-headers', sourceHeaderFindings],
['spawn-timeout', spawnSites],
['test-bare-error', bareErrorFindings],
['test-fix-citations', testFixFindings],
Expand Down
137 changes: 137 additions & 0 deletions scripts/dependency-allowlist.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
// The enforcement half of `scripts/dependency-allowlist.ts`: this file IS the build error. The
// gate's `unit` step runs every `scripts/**/*.test.ts`, so a new dependency fails the gate.

import { describe, expect, test } from 'bun:test';
// why: Bun ships no temporary-directory API, so the fixture root needs node:os.
import { mkdir, mkdtemp, rm } from 'node:fs/promises';
// why: Bun ships no temporary-directory API, so the fixture root needs node:os.
import { tmpdir } from 'node:os';
// why: Bun exposes no path-join primitive.
import { join } from 'node:path';
import {
checkDependencyAllowlist,
type DependencyDeclaration,
dependencyAllowlistFindings,
} from './dependency-allowlist';

const declaration = (
name: string,
field: DependencyDeclaration['field'] = 'dependencies',
): DependencyDeclaration => ({ manifest: 'packages/core/package.json', field, name });

describe('an external dependency', () => {
test('an allowlisted dependency passes', () => {
const findings = checkDependencyAllowlist({
declarations: [declaration('typescript')],
workspaceNames: new Set(['@ultimat3/core']),
allowlist: {
'packages/core/package.json#dependencies#typescript': { reason: 'the compiler' },
},
});
expect(findings).toEqual([]);
});

test('an unallowlisted dependencies entry fails', () => {
const findings = checkDependencyAllowlist({
declarations: [declaration('left-pad')],
workspaceNames: new Set(['@ultimat3/core']),
allowlist: {},
});
expect(findings[0]?.code).toBe('X_DEPENDENCY_NOT_ALLOWLISTED');
});

test('an unallowlisted devDependencies entry fails', () => {
const findings = checkDependencyAllowlist({
declarations: [declaration('left-pad', 'devDependencies')],
workspaceNames: new Set(['@ultimat3/core']),
allowlist: {},
});
expect(findings[0]?.code).toBe('X_DEPENDENCY_NOT_ALLOWLISTED');
});

test('an internal dependency needs no allowlist row', () => {
const findings = checkDependencyAllowlist({
declarations: [declaration('@ultimat3/core')],
workspaceNames: new Set(['@ultimat3/core']),
allowlist: {},
});
expect(findings).toEqual([]);
});

test('an allowlist row for a dependency that no longer exists is stale', () => {
const findings = checkDependencyAllowlist({
declarations: [],
workspaceNames: new Set(['@ultimat3/core']),
allowlist: { 'packages/core/package.json#dependencies#gone': { reason: 'was here' } },
});
expect(findings[0]?.code).toBe('X_DEPENDENCY_ALLOWLIST_STALE');
});
});

describe('a broken manifest', () => {
test('is refused as a finding, never thrown', async () => {
const root = await mkdtemp(join(tmpdir(), 'ultimate-dependency-allowlist-'));
try {
await Bun.write(join(root, 'package.json'), '{ this is not json');
const findings = await dependencyAllowlistFindings(root);
expect(findings[0]?.code).toBe('X_DEPENDENCY_UNSCANNED');
expect(findings[0]?.fix).toContain('bun run');
} finally {
await rm(root, { recursive: true, force: true });
}
});

/**
* A workspace manifest that does not parse is a dependency this rule never saw, not a manifest
* with no dependencies. Skipping it reported a clean tree over a dependency nobody had checked,
* which is what `scripts/lib/corpus.ts` calls out for a whole tree — "a guard whose glob matched
* nothing answered 'no findings', which is exactly what a clean tree answers".
*/
test('a workspace manifest that does not parse is refused, never skipped', async () => {
const root = await mkdtemp(join(tmpdir(), 'ultimate-dependency-allowlist-'));
try {
// A readable root that DOES declare a dependency, so the root branch passes and the scan
// reaches the workspace manifest — the branch under test is the workspace one.
await Bun.write(
join(root, 'package.json'),
'{"name":"root","version":"1.0.0","devDependencies":{"typescript":"^5.0.0"}}',
);
await mkdir(join(root, 'packages/core'), { recursive: true });
await Bun.write(join(root, 'packages/core/package.json'), '{ this is not json');
const findings = await dependencyAllowlistFindings(root);
expect(findings[0]?.code).toBe('X_DEPENDENCY_UNSCANNED');
expect(findings[0]?.at).toBe('packages/core/package.json');
expect(findings[0]?.fix).toContain('bun run');
} finally {
await rm(root, { recursive: true, force: true });
}
});

/**
* A tree whose manifests are all readable but declare no dependency at all enforces nothing —
* the same false clean the unreadable manifest produces, reached the other way. It must answer
* `X_DEPENDENCY_UNSCANNED` rather than an empty list.
*/
test('a readable tree that declares no dependency is refused, never a clean list', async () => {
const root = await mkdtemp(join(tmpdir(), 'ultimate-dependency-allowlist-'));
try {
await Bun.write(join(root, 'package.json'), '{"name":"root","version":"1.0.0"}');
await mkdir(join(root, 'packages/core'), { recursive: true });
await Bun.write(
join(root, 'packages/core/package.json'),
'{"name":"@ultimat3/core","version":"1.0.0"}',
);
const findings = await dependencyAllowlistFindings(root);
expect(findings[0]?.code).toBe('X_DEPENDENCY_UNSCANNED');
expect(findings[0]?.fix).toContain('bun run');
} finally {
await rm(root, { recursive: true, force: true });
}
});
});

describe('the repository tree', () => {
test('is clean', async () => {
expect(await dependencyAllowlistFindings(process.cwd())).toEqual([]);
});
});
Loading
Loading