diff --git a/.agents/skills/contribution-triage/SKILL.md b/.agents/skills/contribution-triage/SKILL.md index 21ba3b7a6566a..e5325c398bc48 100644 --- a/.agents/skills/contribution-triage/SKILL.md +++ b/.agents/skills/contribution-triage/SKILL.md @@ -71,13 +71,12 @@ from evidence you could not access. Do not run untrusted PR code merely to decid Read the guide's linked sections before applying these checks. Inspect enough source to substantiate scope and behavioral claims; leave the full correctness, security, and performance audit to code review. -- Under [prior approval](../../../CONTRIBUTING.md#prior-approval), identify the actual failure and - intended behavior. Inspect maintainer responses in the linked issue or discussion, including what - direction and scope they approved. A link, label, or acknowledgment alone is not approval. Judge a - claimed obvious-bug exception by purpose, impact, and necessary changes, without numeric cutoffs. - Separately assess focused configuration of an established capability: identify what already exists, - what the option controls, and its effects and necessary scope. This route does not require a proven - obvious bug. Adding a setting alone establishes neither a new feature nor an approval exemption. +- Under [fix a bug](../../../CONTRIBUTING.md#prior-approval), identify the actual failure and + intended behavior. A feature, or anything on the guide's not-interested list, warrants closure + unless a maintainer in the trusted exemption list asked for that PR. Inspect the request and the + direction and scope it covers. A link, label, or acknowledgment alone is not a request. Judge a + claimed bug fix by purpose, impact, and necessary changes, without numeric cutoffs. A setting that + adds a workflow or changes a default is a feature. Check whether an alleged fix intentionally changes product behavior or overlooks an existing workflow. Preserve any useful documentation, onboarding, or discoverability problem when declining the solution. - Under [one problem](../../../CONTRIBUTING.md#one-problem), trace how each material change contributes @@ -90,27 +89,18 @@ scope and behavioral claims; leave the full correctness, security, and performan as the guide requires them. Do not demand unrelated evidence or repo-wide tests. An unavailable local platform does not defeat adequate contributor evidence. Do not infer fabrication or AI authorship from writing style, suspicion, or a check you cannot reproduce. +- Under [models](../../../CONTRIBUTING.md#models), use only the model and harness the PR names, + usually in its closing attribution line. -### Configuration and workflow examples - -- Hosting CLI-path configuration in [#11653](https://github.com/pingdotgg/t3code/pull/11653) is eligible - for deeper review under the maintainer's ruling: it makes an existing capability configurable. - It need not qualify as an obvious-bug repair or obtain prior feature approval on that basis. - Still assess one underlying problem, necessary scope and credible verification. Deeper review can - reject the configuration mechanism or its implementation. -- Preserving Files as an independent tab in [#14436](https://github.com/pingdotgg/t3code/pull/14436) - changes tab lifetime and navigation. The maintainer classified it as a broader workflow change - requiring prior product-direction approval, which is absent. Propose closure for missing approval - in a dry run, or carry out closure in authorized enforcement. Its good evidence does not make it - eligible or justify keeping it pending after that ruling. The remedy is to obtain scope approval. - -Use these examples to distinguish effects, not to exempt every configuration option. Apply current -trusted policy and reassess changed submission evidence; neither example grants permanent eligibility. +Bans in the guide are a maintainer decision. Report suspected spam to maintainers; do not ban or +threaten a ban. ## Apply the outcome Do not automatically close a PR if a maintainer in the trusted `TRIAGE_EXEMPTIONS.td` list has commented or submitted a review, including on earlier heads, or if `triage:keep-open` is present. +Comments that triage itself posted, such as an earlier notice or question, are not maintainer +involvement, even when they were posted from a maintainer's account on that maintainer's behalf. Leave protected PRs open for maintainer decision. This does not grant an author exemption, eligibility, or review approval. @@ -126,10 +116,20 @@ comment text but do not write to GitHub. Keep the eligibility finding separate f This is a routing exception, not a claim that the PR passed eligibility or correctness review. - **Closure warranted.** The assessment is complete and establishes a specific policy violation. Prepare a clear explanation under [closure and reconsideration](../../../CONTRIBUTING.md#closure-and-reconsideration), - post it, and close the PR. Verify that the explanation is present and the PR is closed. The comment + post it, then read the repository variable `PR_LIMITS_ENFORCE` + (`gh variable get PR_LIMITS_ENFORCE -R pingdotgg/t3code`). When it prints `true`, apply + `triage:closing` as the last thing you do to the PR, verify that the explanation and the label + are present, and do not close the PR yourself: + [pr-limits.yml](../../../.github/workflows/pr-limits.yml) closes it 72 hours after the label. Any + activity on the PR after the label, including a further comment or label from you, removes the + label; treat an author's reply or push as new evidence, reassess, and apply the label again only + if the finding still holds. When the command prints any other value, or fails saying the variable + was not found, the variable is off and nothing would close a labeled PR: close it yourself and + verify that the explanation is present and the PR is closed. Only another failure, such as a + permission error, means the variable cannot be read; then report the closure as incomplete. The comment must name the violated rule, cite supporting submission evidence, link the maintained guide section, and give a concrete remedy. Missing contributor evidence can warrant closure; explain what must be - established. Missing product approval calls for maintainer discussion, not an agent's product decision. + established. The remedy for a feature is an Ideas discussion, not an agent's product decision. Close for multiple problems only when the independence of those changes is supported. Closing a PR does not require a configured Macroscope label. - **Needs explanation or maintainer decision.** Name the unresolved question and who can resolve it. @@ -137,9 +137,9 @@ comment text but do not write to GitHub. Keep the eligibility finding separate f was posted. If tracing the source leaves an extra diff's necessity unclear, request the causal explanation needed. If established facts leave a product-direction choice to maintainers, identify that choice. Leave the PR open and pending that answer; do not mark it eligible. Do not substitute - "the agent was uncertain" for a violated rule. Once maintainers establish that a workflow change - requires approval and that approval is absent, apply the closure outcome instead of retaining a - pending classification. Good verification does not cure missing approval. + "the agent was uncertain" for a violated rule. Once a change is established as a feature with no + maintainer request, apply the closure outcome instead of retaining a pending classification. Good + verification does not make a feature eligible. - **Incomplete; retry required.** State the failed retrieval, missing access, truncated diff, or unfinished assessment and what is needed to resume. Do not convert operational failures into policy violations or hand the PR off as having passed. An inaccessible artifact is different from an omitted artifact. @@ -173,9 +173,10 @@ description, evidence, approvals, existing triage comments, PR open/closed state label when relevant. Reassess changes that could invalidate the finding. Reuse an existing explanation only if it still matches the current finding; avoid duplicate comments, closures, or label applications. -For closure, establish that the required explanation is posted before closing. If posting fails or its -result is ambiguous, read back the comments before retrying or proceeding. If the comment succeeds but -closure fails, preserve the comment and retry only the unfinished closure after checking current state. +For closure, establish that the required explanation is posted before applying `triage:closing` or +closing. If posting fails or its result is ambiguous, read back the comments before retrying or +proceeding. If the comment succeeds but the label or the closure fails, preserve the comment and retry +only that unfinished step after checking current state. Handle a failed or ambiguous label application the same way: inspect labels before any retry. If access or state still cannot be verified, report an incomplete action and the step needed to resume. Do not retry blindly or claim success for an unverified action. diff --git a/.agents/skills/test-t3-mobile/SKILL.md b/.agents/skills/test-t3-mobile/SKILL.md index f89e13ab68241..ace1f9637ec86 100644 --- a/.agents/skills/test-t3-mobile/SKILL.md +++ b/.agents/skills/test-t3-mobile/SKILL.md @@ -36,8 +36,9 @@ vp run mobile:install This installs the dev client for the checkout's native fingerprint: from the device, the local cache, or the shared cache in seconds, building once only -when no machine has that fingerprint yet (5-15 minutes, logged to the file it -names). The device id may be omitted when exactly one device is booted. It +when no machine has that fingerprint yet: in the `Mobile Dev Client` workflow +on Blacksmith when HEAD is pushed and `gh` may dispatch it (the command waits), +otherwise here (5-15 minutes, logged to the file it names). The device id may be omitted when exactly one device is booted. It prints one JSON line; on failure it names the next command. Mobile verification includes that step unless the user prohibits it. diff --git a/.github/VOUCHED.td b/.github/VOUCHED.td index 1c4ae817060a9..69c1ea6575a4a 100644 --- a/.github/VOUCHED.td +++ b/.github/VOUCHED.td @@ -83,6 +83,7 @@ github:tris203 github:tsouth89 github:UtkarshUsername github:vitalyiegorov +-github:wSedlacek Opened over 130 unapproved PRs in 12 days, far past the open PR limit github:Yash-Singh1 github:yashranaway github:Ymit24 diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 4a66b5d268103..94157c4135fac 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -15,16 +15,11 @@ reproduction steps or environment where relevant. --> -## Scope and approval - - +## Scope + + ## Verification diff --git a/.github/scripts/mobile-preview-comment.cjs b/.github/scripts/mobile-preview-comment.cjs new file mode 100644 index 0000000000000..e4876b0842eaa --- /dev/null +++ b/.github/scripts/mobile-preview-comment.cjs @@ -0,0 +1,126 @@ +// The Android half of the mobile preview comment. The android job writes the fingerprint and +// the update; the android-build job adds the client's install link a few minutes later. Each +// run merges what it knows into the state kept in the comment's first line. +const MARKER = "mobile-preview-android"; +const PROJECT_ID = "d763fcb8-d37c-41ea-a773-b54a0ab4a454"; +const APP_SCHEME = "t3-code"; + +function readState(body) { + const match = new RegExp(`^`).exec(body ?? ""); + if (!match) return null; + try { + const state = JSON.parse(match[1]); + const text = (value) => (typeof value === "string" ? value : ""); + return { + fingerprint: text(state.fingerprint), + client: text(state.client), + group: text(state.group), + }; + } catch { + return null; + } +} + +/** + * New values win; a new fingerprint drops the old fingerprint's client. A build that finishes + * after the PR moved to another fingerprint changes nothing: its client cannot run the update + * the comment now shows. + */ +function mergeState(previous, next) { + const buildOnly = Boolean(next.client) && !next.group; + if ( + buildOnly && + previous?.fingerprint && + next.fingerprint && + previous.fingerprint !== next.fingerprint + ) { + return previous; + } + const fingerprint = next.fingerprint || previous?.fingerprint || ""; + const sameFingerprint = !previous?.fingerprint || previous.fingerprint === fingerprint; + return { + fingerprint, + client: next.client || (sameFingerprint ? (previous?.client ?? "") : ""), + group: next.group || previous?.group || "", + }; +} + +function render(state, runUrl) { + const qr = `https://qr.expo.dev/eas-update?appScheme=${APP_SCHEME}&projectId=${PROJECT_ID}&groupId=${state.group}`; + const rows = [ + ``, + "🤖 Android preview", + "", + "  | 🤖 Android", + "--- | ---", + `Fingerprint | ${state.fingerprint || "n/a"}`, + `Dev client | ${state.client ? `[Install](${state.client})` : `Building on Blacksmith ([run](${runUrl})); this comment gets the install link when it finishes`}`, + ]; + if (state.group) { + rows.push( + `Update Details | [Update Permalink](https://expo.dev/projects/${PROJECT_ID}/updates/${state.group})`, + `Update QR | `, + ); + } + return rows.join("\n"); +} + +async function github(path, init = {}) { + const response = await fetch(`https://api.github.com${path}`, { + ...init, + headers: { + authorization: `Bearer ${process.env.GH_TOKEN}`, + accept: "application/vnd.github+json", + "content-type": "application/json", + ...init.headers, + }, + }); + if (!response.ok) { + throw new Error( + `GitHub ${init.method ?? "GET"} ${path}: ${response.status} ${await response.text()}`, + ); + } + return response.json(); +} + +async function main() { + const repository = process.env.GITHUB_REPOSITORY; + const pr = process.env.PR; + if (!repository || !pr || !process.env.GH_TOKEN) { + throw new Error("GITHUB_REPOSITORY, PR and GH_TOKEN are required."); + } + const runUrl = `${process.env.GITHUB_SERVER_URL}/${repository}/actions/runs/${process.env.GITHUB_RUN_ID}`; + let existing = null; + for (let page = 1; existing === null; page++) { + const comments = await github( + `/repos/${repository}/issues/${pr}/comments?per_page=100&page=${page}`, + ); + // Only this workflow's own comment: anyone can post the marker, and nobody else's is editable. + existing = + comments.find( + (comment) => + comment.user?.login === "github-actions[bot]" && readState(comment.body) !== null, + ) ?? null; + if (comments.length < 100) break; + } + const state = mergeState(readState(existing?.body), { + fingerprint: process.env.FINGERPRINT ?? "", + client: process.env.CLIENT ?? "", + group: process.env.GROUP ?? "", + }); + const body = JSON.stringify({ body: render(state, runUrl) }); + if (existing) { + await github(`/repos/${repository}/issues/comments/${existing.id}`, { method: "PATCH", body }); + } else { + await github(`/repos/${repository}/issues/${pr}/comments`, { method: "POST", body }); + } +} + +if (require.main === module) { + main().catch((error) => { + console.error(error); + process.exitCode = 1; + }); +} + +module.exports = { mergeState, readState, render }; diff --git a/.github/scripts/mobile-preview-comment.test.cjs b/.github/scripts/mobile-preview-comment.test.cjs new file mode 100644 index 0000000000000..99bfc45d4d2b9 --- /dev/null +++ b/.github/scripts/mobile-preview-comment.test.cjs @@ -0,0 +1,62 @@ +const assert = require("node:assert/strict"); +const test = require("node:test"); + +const { mergeState, readState, render } = require("./mobile-preview-comment.cjs"); + +const fingerprint = "c31608b37a8a2dca35ccee39cfd2ec7305042629"; +const run = "https://github.com/pingdotgg/t3code/actions/runs/1"; + +test("the comment round-trips its state", () => { + const state = { fingerprint, client: "", group: "8e413a0a" }; + const body = render(state, run); + assert.deepEqual(readState(body), state); + assert.match( + body, + /Building on Blacksmith \(\[run\]\(https:\/\/github\.com\/pingdotgg\/t3code\/actions\/runs\/1\)\)/, + ); + assert.match( + body, + /qr\.expo\.dev\/eas-update\?appScheme=t3-code&projectId=[0-9a-f-]+&groupId=8e413a0a/, + ); + assert.equal(readState("🚀 Expo continuous deployment is ready!"), null); +}); + +test("the build job adds the install link without losing the update", () => { + const first = mergeState(null, { fingerprint, client: "", group: "8e413a0a" }); + const built = mergeState(first, { fingerprint, client: "https://expo.dev/a/b", group: "" }); + assert.deepEqual(built, { fingerprint, client: "https://expo.dev/a/b", group: "8e413a0a" }); + assert.match(render(built, run), /Dev client \| \[Install\]\(https:\/\/expo\.dev\/a\/b\)/); +}); + +test("a new fingerprint drops the old client; the same one keeps it", () => { + const built = { fingerprint, client: "https://expo.dev/a/b", group: "old" }; + assert.deepEqual(mergeState(built, { fingerprint, client: "", group: "new" }), { + fingerprint, + client: "https://expo.dev/a/b", + group: "new", + }); + assert.deepEqual(mergeState(built, { fingerprint: "f".repeat(40), client: "", group: "new" }), { + fingerprint: "f".repeat(40), + client: "", + group: "new", + }); +}); + +test("a build that finishes after the PR moved to another fingerprint changes nothing", () => { + const current = { fingerprint: "f".repeat(40), client: "", group: "new" }; + assert.deepEqual( + mergeState(current, { fingerprint, client: "https://expo.dev/a/b", group: "" }), + current, + ); +}); + +test("state from the comment is always three strings", () => { + assert.deepEqual( + readState('\nx'), + { + fingerprint: "", + client: "", + group: "", + }, + ); +}); diff --git a/.github/scripts/pr-limits.cjs b/.github/scripts/pr-limits.cjs new file mode 100644 index 0000000000000..77c09785e4104 --- /dev/null +++ b/.github/scripts/pr-limits.cjs @@ -0,0 +1,320 @@ +const fs = require("node:fs"); + +const HOUR_MS = 60 * 60 * 1000; +const OPEN_LIMIT = 3; +const CONFLICT_IDLE_MS = 7 * 24 * HOUR_MS; +const NOTICE_MS = 72 * HOUR_MS; +// A first run meets the whole backlog. These keep one run from flooding watchers or GitHub's +// limit on how fast a token may comment. +const MAX_REQUESTS_PER_RUN = 100; +const MAX_CLOSES_PER_RUN = 50; +// A last look that ends in "not today" closes nothing, so the looks need a cap of their own. +const MAX_RECHECKS_PER_RUN = 150; +const WRITE_PAUSE_MS = 1_000; +// Measured here: about two and a half minutes until every open PR has an answer. +const MERGEABLE_WAIT_MS = 150_000; +const READ_ATTEMPTS = 4; +const READ_RETRY_MS = 5_000; +// GitHub stamps a label event and the PR's updatedAt a second or two apart. +const SAME_MOMENT_MS = 15_000; + +const REBASE_LABEL = "needs-rebase"; +const CLOSING_LABEL = "triage:closing"; +const KEEP_OPEN_LABEL = "triage:keep-open"; +const UNVOUCHED_LABEL = "vouch:unvouched"; +const WRITE_PERMISSIONS = new Set(["admin", "maintain", "write"]); +const EXEMPTIONS_PATH = ".github/TRIAGE_EXEMPTIONS.td"; +const VOUCHED_PATH = ".github/VOUCHED.td"; +const LOGIN_PATTERN = /^github:([A-Za-z0-9](?:[A-Za-z0-9]|-(?=[A-Za-z0-9])){0,38})$/; + +// `labels` matches a PR that carries any one of them. +const SWEEP_QUERY = ` + query ($owner: String!, $repo: String!, $labels: [String!], $cursor: String) { + repository(owner: $owner, name: $repo) { + pullRequests(states: OPEN, labels: $labels, first: 50, after: $cursor) { + pageInfo { hasNextPage endCursor } + nodes { + number + mergeable + baseRefName + updatedAt + author { login } + labels(first: 100) { nodes { name } } + timelineItems(itemTypes: [LABELED_EVENT], last: 20) { + nodes { ... on LabeledEvent { createdAt label { name } } } + } + } + } + } + }`; + +const RECHECK_QUERY = ` + query ($owner: String!, $repo: String!, $number: Int!) { + repository(owner: $owner, name: $repo) { + pullRequest(number: $number) { + state + mergeable + updatedAt + labels(first: 100) { nodes { name } } + } + } + }`; + +const AUTHOR_OPEN_PRS_QUERY = ` + query ($q: String!) { + search(query: $q, type: ISSUE, first: 50) { + nodes { ... on PullRequest { number } } + } + }`; + +const pause = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); + +// GitHub answers a heavy list request with a 502 now and then, and one of those must not end the +// run. Only reads go through here: repeating a read is safe, repeating a comment is not. +async function read(github, query, variables) { + for (let attempt = 1; ; attempt += 1) { + try { + return await github.graphql(query, variables); + } catch (error) { + const transient = typeof error.status === "number" && error.status >= 500; + if (attempt === READ_ATTEMPTS || !transient) throw error; + await pause(attempt * READ_RETRY_MS); + } + } +} + +// Fails closed: a missing, empty, or malformed list must never read as "nobody is exempt". +function parseExemptions(text) { + const logins = new Set(); + for (const raw of text.split("\n")) { + const line = raw.trim(); + if (line === "" || line.startsWith("#")) continue; + const match = LOGIN_PATTERN.exec(line); + if (!match) throw new Error(`${EXEMPTIONS_PATH}: malformed entry "${line}"`); + const login = match[1].toLowerCase(); + if (logins.has(login)) throw new Error(`${EXEMPTIONS_PATH}: duplicate entry "${line}"`); + logins.add(login); + } + if (logins.size === 0) throw new Error(`${EXEMPTIONS_PATH}: no entries`); + return logins; +} + +// VOUCHED.td: one `github:login` per line, `#` comments, and `-github:login reason` to denounce. +function vouchedLogins(text) { + const logins = new Set(); + for (const raw of text.split("\n")) { + const entry = raw.trim().split(/\s+/)[0]; + if (entry.startsWith("github:")) logins.add(entry.slice("github:".length).toLowerCase()); + } + if (logins.size === 0) throw new Error(`${VOUCHED_PATH}: no entries`); + return logins; +} + +function labeledAt(pr, name) { + const last = pr.timelineItems.nodes.filter((event) => event.label?.name === name).at(-1); + return last ? Date.parse(last.createdAt) : undefined; +} + +// The whole policy for the two windows. Activity is anything that moves the PR's updatedAt: a +// push, a comment, a review, a label. Nobody can set that time, so nothing here trusts the author. +function decide(pr, exemptions, now) { + const labels = new Set(pr.labels.nodes.map((label) => label.name)); + if (labels.has(KEEP_OPEN_LABEL)) return "skip"; + const updatedAt = Date.parse(pr.updatedAt); + + if (labels.has(CLOSING_LABEL)) { + const since = labeledAt(pr, CLOSING_LABEL); + if (since === undefined) return "unknown-notice"; + // Any activity after the notice sends the PR back to triage. + if (updatedAt - since > SAME_MOMENT_MS) return "clear-notice"; + return now - since >= NOTICE_MS ? "close-notice" : "skip"; + } + + const exempt = exemptions.has((pr.author?.login ?? "").toLowerCase()); + const subject = labels.has(UNVOUCHED_LABEL) && !exempt; + if (labels.has(REBASE_LABEL)) { + if (!subject || pr.mergeable === "MERGEABLE") return "clear-rebase"; + // Activity restarts the seven days and keeps the label, so a PR is asked to rebase once. + const idle = now - updatedAt >= CONFLICT_IDLE_MS; + return idle && pr.mergeable === "CONFLICTING" ? "close-conflict" : "skip"; + } + return subject && pr.mergeable === "CONFLICTING" ? "request-rebase" : "skip"; +} + +function guideLink(context, anchor) { + return `https://github.com/${context.repo.owner}/${context.repo.repo}/blob/main/CONTRIBUTING.md#${anchor}`; +} + +async function removeLabel(github, context, number, name) { + try { + await github.rest.issues.removeLabel({ ...context.repo, issue_number: number, name }); + } catch (error) { + if (error.status !== 404) throw error; + } +} + +async function closeWithComment(github, context, number, body) { + await github.rest.issues.createComment({ ...context.repo, issue_number: number, body }); + await github.rest.pulls.update({ ...context.repo, pull_number: number, state: "closed" }); +} + +async function sweep({ github, context, core, dryRun }) { + const exemptions = parseExemptions(fs.readFileSync(EXEMPTIONS_PATH, "utf8")); + const repo = { owner: context.repo.owner, repo: context.repo.repo }; + + const list = async () => { + const prs = []; + for (let cursor = null; ;) { + const page = ( + await read(github, SWEEP_QUERY, { + ...repo, + cursor, + labels: [UNVOUCHED_LABEL, REBASE_LABEL, CLOSING_LABEL], + }) + ).repository.pullRequests; + prs.push(...page.nodes); + if (!page.pageInfo.hasNextPage) return prs; + cursor = page.pageInfo.endCursor; + } + }; + // GitHub computes mergeability lazily and forgets it whenever the base moves. Asking starts the + // computation, so ask once more. A PR that is still unknown waits for tomorrow's run. + let prs = await list(); + if (prs.some((pr) => pr.mergeable === "UNKNOWN")) { + await pause(MERGEABLE_WAIT_MS); + prs = await list(); + } + + const now = Date.now(); + // Start further down the list each day. The step is no larger than a run's cap, so a cap that + // is reached cannot leave the same PRs waiting behind the same others every day. + const start = (Math.floor(now / (24 * HOUR_MS)) * MAX_RECHECKS_PER_RUN) % (prs.length || 1); + prs = [...prs.slice(start), ...prs.slice(0, start)]; + const count = {}; + let rechecks = 0; + for (const pr of prs) { + const { number } = pr; + let action = decide(pr, exemptions, now); + if (action === "skip") continue; + if (action === "unknown-notice") { + core.warning(`#${number}: cannot tell when ${CLOSING_LABEL} was applied, leaving it open.`); + continue; + } + if (action === "request-rebase" && (count[action] ?? 0) >= MAX_REQUESTS_PER_RUN) continue; + if (action.startsWith("close")) { + if ((count["close-conflict"] ?? 0) + (count["close-notice"] ?? 0) >= MAX_CLOSES_PER_RUN) { + continue; + } + if (rechecks >= MAX_RECHECKS_PER_RUN) continue; + rechecks += 1; + // The list is minutes old by now. Closing is the one step worth a second look. + const fresh = (await read(github, RECHECK_QUERY, { ...repo, number })).repository.pullRequest; + if (fresh.state !== "OPEN") continue; + // A merge to the base can resolve a conflict without touching the PR, so this is re-read + // too. An answer that is not ready yet reads as "not conflicting" and waits for tomorrow. + action = decide({ ...pr, ...fresh }, exemptions, now); + if (!action.startsWith("close")) continue; + } + + count[action] = (count[action] ?? 0) + 1; + core.info(`#${number} ${action}${dryRun ? " (dry run)" : ""}`); + if (dryRun) continue; + + if (action === "request-rebase") { + // The request goes first: a label without it would start a clock nobody was told about. + await github.rest.issues.createComment({ + ...context.repo, + issue_number: number, + body: [ + `This PR conflicts with \`${pr.baseRefName}\`. Rebase it or merge \`${pr.baseRefName}\` into it. If it still conflicts after 7 days with no activity, it will be closed automatically.`, + "", + `See the [conflict window](${guideLink(context, "conflict-window")}).`, + ].join("\n"), + }); + await github.rest.issues.addLabels({ + ...context.repo, + issue_number: number, + labels: [REBASE_LABEL], + }); + } else if (action === "clear-rebase") { + await removeLabel(github, context, number, REBASE_LABEL); + } else if (action === "clear-notice") { + await removeLabel(github, context, number, CLOSING_LABEL); + } else { + const conflict = action === "close-conflict"; + await closeWithComment( + github, + context, + number, + conflict + ? [ + `Closing: this PR still conflicts with \`${pr.baseRefName}\` and has had no activity for 7 days since the rebase request.`, + "", + `To continue, rebase the branch and open a new PR that links this one. See the [conflict window](${guideLink(context, "conflict-window")}).`, + ].join("\n") + : [ + "Closing: 72 hours have passed since the triage notice above with no activity on this PR.", + "", + `To ask for reconsideration, address the notice and comment here, or open a focused replacement PR that links this one. See [closure and reconsideration](${guideLink(context, "closure-and-reconsideration")}).`, + ].join("\n"), + ); + // Without this a maintainer's reopen would be closed again by the next run. + await removeLabel(github, context, number, conflict ? REBASE_LABEL : CLOSING_LABEL); + } + await pause(WRITE_PAUSE_MS); + } + + const summary = Object.entries(count).map(([action, n]) => `${n} ${action}`); + core.info( + `${dryRun ? "Dry run. " : ""}${prs.length} PRs read: ${summary.join(", ") || "nothing to do"}.`, + ); + return count; +} + +async function enforceOpenLimit({ github, context, core, dryRun }) { + const pr = context.payload.pull_request; + const author = pr.user.login; + // Who is trusted is decided here rather than by the vouch action, which reports its own failed + // lookups as "not vouched". Every read below throws on failure, and a throw closes nothing. + if (pr.user.type === "Bot") return; + if (parseExemptions(fs.readFileSync(EXEMPTIONS_PATH, "utf8")).has(author.toLowerCase())) return; + if (vouchedLogins(fs.readFileSync(VOUCHED_PATH, "utf8")).has(author.toLowerCase())) return; + const { data: access } = await github.rest.repos.getCollaboratorPermissionLevel({ + ...context.repo, + username: author, + }); + if (WRITE_PERMISSIONS.has(access.permission)) return; + if (pr.labels.some((label) => label.name === KEEP_OPEN_LABEL)) return; + // Someone else reopening a PR is a maintainer overriding the limit. + const reopened = context.payload.action === "reopened"; + if (reopened && context.payload.sender.login.toLowerCase() !== author.toLowerCase()) return; + + const q = `repo:${context.repo.owner}/${context.repo.repo} is:pr is:open author:${author} sort:created-asc`; + const { search } = await read(github, AUTHOR_OPEN_PRS_QUERY, { q }); + // A new PR counts only the older ones, so two opened together cannot close each other. + const others = search.nodes.filter( + (node) => node.number && (reopened ? node.number !== pr.number : node.number < pr.number), + ).length; + if (others < OPEN_LIMIT) { + core.info(`PR #${pr.number}: ${author} has ${others} other open PRs.`); + return; + } + if (dryRun) { + core.notice(`Dry run. PR #${pr.number} would close: ${author} has ${others} other open PRs.`); + return; + } + + await closeWithComment( + github, + context, + pr.number, + [ + `Closing: ${author} already has ${OPEN_LIMIT} or more open PRs here, which is the limit for contributors who are not vouched.`, + "", + `Once one of them is merged or closed, open this again. See the [open PR limit](${guideLink(context, "open-pr-limit")}).`, + ].join("\n"), + ); + core.info(`PR #${pr.number}: closed, ${author} has ${others} other open PRs.`); +} + +module.exports = { decide, enforceOpenLimit, parseExemptions, sweep }; diff --git a/.github/scripts/test-delta.cjs b/.github/scripts/test-delta.cjs new file mode 100644 index 0000000000000..c9789f644d272 --- /dev/null +++ b/.github/scripts/test-delta.cjs @@ -0,0 +1,78 @@ +// Counts the test cases a pull request adds and removes, from a zero-context diff, so PR Size can +// say when a change grows the suite. Counting declarations keeps the number deterministic, unlike +// CI timings, which move with runner load. +const MARKER = ""; +const NET_CASES_TO_REPORT = 20; +const TEST_FILE = /\.(?:test|spec)\.[cm]?[jt]sx?$/; +// `<` admits type arguments, as in `it.each(...)`, which may span lines. +const TEST_CASE = /^\s*(?:it|test)(?:\.\w+)*\s*[(<]/; +const isBrowserTest = (path) => path.includes(".browser.test."); + +function kindOf(path) { + if (isBrowserTest(path)) return "browser"; + if (path.startsWith("apps/server/") || path.includes(".integration.")) return "server"; + return "unit"; +} + +/** @param {string} diff output of `git diff -U0 base...head` */ +function countTestDelta(diff) { + const kinds = { browser: 0, server: 0, unit: 0 }; + const newBrowserFiles = []; + let path = null; + // A browser test file is new when it is added, or renamed from a file that was not one. + let file = null; + const flush = () => { + if (file?.isNew && isBrowserTest(file.target)) newBrowserFiles.push(file.target); + }; + for (const line of diff.split("\n")) { + if (line.startsWith("diff --git ")) { + flush(); + path = null; + file = { target: line.replace(/^diff --git a\/\S+ b\//, ""), isNew: false }; + continue; + } + if (line.startsWith("new file mode") && file) file.isNew = true; + if (line.startsWith("rename from ") && file) file.isNew = !isBrowserTest(line.slice(12)); + if (line.startsWith("--- ")) { + const source = line.slice(4).replace(/^a\//, ""); + path = TEST_FILE.test(source) ? source : null; + continue; + } + if (line.startsWith("+++ ")) { + const target = line.slice(4).replace(/^b\//, ""); + // A deleted file's target is /dev/null; keep its old path to count the removed cases. + if (target !== "/dev/null") path = TEST_FILE.test(target) ? target : null; + continue; + } + if (line.startsWith("new file mode") || !path) continue; + const sign = line[0] === "+" ? 1 : line[0] === "-" ? -1 : 0; + if (sign !== 0 && TEST_CASE.test(line.slice(1))) kinds[kindOf(path)] += sign; + } + flush(); + return { kinds, net: kinds.browser + kinds.server + kinds.unit, newBrowserFiles }; +} + +/** Returns the comment body, or null when the change does not grow the suite enough to mention. */ +function render(delta) { + if (delta.net < NET_CASES_TO_REPORT && delta.newBrowserFiles.length === 0) return null; + const signed = (n) => (n > 0 ? `+${n}` : String(n)); + const lines = [ + MARKER, + `This PR changes the test suite by **${signed(delta.net)} test cases** ` + + `(browser ${signed(delta.kinds.browser)}, unit ${signed(delta.kinds.unit)}, server ${signed(delta.kinds.server)}).`, + ]; + if (delta.newBrowserFiles.length > 0) { + lines.push( + "", + `New browser test files: ${delta.newBrowserFiles.map((file) => `\`${file}\``).join(", ")}.`, + "Each browser test file costs CI seconds of module loading before its first test runs; a unit test costs milliseconds.", + ); + } + lines.push( + "", + "Keep tests that would fail for a real reason, and test logic below the UI where it can be. See [Testing](https://github.com/pingdotgg/t3code/blob/main/docs/internals/testing.md).", + ); + return lines.join("\n"); +} + +module.exports = { MARKER, countTestDelta, render }; diff --git a/.github/scripts/test-delta.test.cjs b/.github/scripts/test-delta.test.cjs new file mode 100644 index 0000000000000..82a3e00862d2f --- /dev/null +++ b/.github/scripts/test-delta.test.cjs @@ -0,0 +1,59 @@ +const assert = require("node:assert/strict"); +const test = require("node:test"); + +const { countTestDelta, render } = require("./test-delta.cjs"); + +const diff = `diff --git a/apps/web/src/Panel.browser.test.tsx b/apps/web/src/Panel.browser.test.tsx +new file mode 100644 +--- /dev/null ++++ b/apps/web/src/Panel.browser.test.tsx +@@ -0,0 +1,4 @@ ++it("opens", async () => {}); ++ it.each<{ n: number }>([{ n: 1 }])("renders $n", async () => {}); ++// it("is a comment, still counted only when it starts the line") +diff --git a/apps/server/src/thing.test.ts b/apps/server/src/thing.test.ts +--- a/apps/server/src/thing.test.ts ++++ b/apps/server/src/thing.test.ts +@@ -3,2 +3,1 @@ +-it.effect("old", () => {}); +-test("older", () => {}); ++ it.effect("new", () => {}); +diff --git a/packages/shared/src/gone.test.ts b/packages/shared/src/gone.test.ts +deleted file mode 100644 +--- a/packages/shared/src/gone.test.ts ++++ /dev/null +@@ -1,1 +0,0 @@ +-it("goes", () => {}); +diff --git a/apps/web/src/Menu.test.tsx b/apps/web/src/Menu.browser.test.tsx +similarity index 100% +rename from apps/web/src/Menu.test.tsx +rename to apps/web/src/Menu.browser.test.tsx +diff --git a/apps/web/src/Old.browser.test.tsx b/apps/web/src/New.browser.test.tsx +similarity index 100% +rename from apps/web/src/Old.browser.test.tsx +rename to apps/web/src/New.browser.test.tsx +diff --git a/apps/web/src/Panel.tsx b/apps/web/src/Panel.tsx +--- a/apps/web/src/Panel.tsx ++++ b/apps/web/src/Panel.tsx +@@ -1 +1 @@ ++it("is not a test file", () => {}); +`; + +test("counts cases by kind, including deleted files, and finds new or renamed browser files", () => { + assert.deepEqual(countTestDelta(diff), { + kinds: { browser: 2, server: -1, unit: -1 }, + net: 0, + newBrowserFiles: ["apps/web/src/Panel.browser.test.tsx", "apps/web/src/Menu.browser.test.tsx"], + }); +}); + +test("reports only growth worth a reviewer's attention", () => { + const quiet = { kinds: { browser: 0, server: 3, unit: 5 }, net: 8, newBrowserFiles: [] }; + assert.equal(render(quiet), null); + const growth = { kinds: { browser: 4, server: 10, unit: 12 }, net: 26, newBrowserFiles: [] }; + assert.match(render(growth), /\*\*\+26 test cases\*\* \(browser \+4, unit \+12, server \+10\)/); + assert.match( + render(countTestDelta(diff)), + /New browser test files: `apps\/web\/src\/Panel\.browser\.test\.tsx`/, + ); +}); diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index bac2f8f541c89..6f75c92562d03 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -140,10 +140,16 @@ jobs: - name: Start installing browser secret helper build libraries if: needs.changes.outputs.build != 'false' run: | - sudo sed -i 's|http://|https://|g' /etc/apt/blacksmith-ubuntu-mirrors.txt /etc/apt/sources.list.d/ubuntu.sources + # Fetch only the indexes these main and universe packages need. The full set is ~42 MB, + # which a slow mirror can stretch past the wait below. + sudo sed -i -e 's|http://|https://|g' -e 's/^Components: .*/Components: main universe/' \ + /etc/apt/blacksmith-ubuntu-mirrors.txt /etc/apt/sources.list.d/ubuntu.sources ( set +e - sudo apt-get update && sudo apt-get install -y libsecret-1-dev pkg-config xvfb + sudo apt-get update -o Acquire::Languages=none \ + -o Acquire::IndexTargets::deb::DEP-11::DefaultEnabled=false \ + -o Acquire::IndexTargets::deb::CNF::DefaultEnabled=false && + sudo apt-get install -y libsecret-1-dev pkg-config xvfb echo "$?" > "$RUNNER_TEMP/apt-status" ) > "$RUNNER_TEMP/apt.log" 2>&1 < /dev/null & @@ -287,10 +293,16 @@ jobs: # the background while Vite+ installs; the step before the tests waits for it. - name: Start installing browser secret helper build libraries run: | - sudo sed -i 's|http://|https://|g' /etc/apt/blacksmith-ubuntu-mirrors.txt /etc/apt/sources.list.d/ubuntu.sources + # Fetch only the indexes these main and universe packages need. The full set is ~42 MB, + # which a slow mirror can stretch past the wait below. + sudo sed -i -e 's|http://|https://|g' -e 's/^Components: .*/Components: main universe/' \ + /etc/apt/blacksmith-ubuntu-mirrors.txt /etc/apt/sources.list.d/ubuntu.sources ( set +e - sudo apt-get update && sudo apt-get install -y libsecret-1-dev pkg-config + sudo apt-get update -o Acquire::Languages=none \ + -o Acquire::IndexTargets::deb::DEP-11::DefaultEnabled=false \ + -o Acquire::IndexTargets::deb::CNF::DefaultEnabled=false && + sudo apt-get install -y libsecret-1-dev pkg-config echo "$?" > "$RUNNER_TEMP/apt-status" ) > "$RUNNER_TEMP/apt.log" 2>&1 < /dev/null & @@ -314,7 +326,7 @@ jobs: run: python3 -B .github/scripts/stage-preview-bundle.test.py - name: Test release checks - run: node --test .github/scripts/check-nightly-release.test.cjs .github/scripts/vercel-deployment-output.test.cjs + run: node --test .github/scripts/check-nightly-release.test.cjs .github/scripts/vercel-deployment-output.test.cjs .github/scripts/mobile-preview-comment.test.cjs .github/scripts/test-delta.test.cjs - name: Finish installing browser secret helper build libraries timeout-minutes: 5 @@ -343,16 +355,22 @@ jobs: # The web suite is the largest outside the server. In the shared Test job it # started last and ran alone for about 40s after everything else finished. - # release.yml mirrors this test, test_web, and test_server split. + # Browser tests are bound by the Vite server transforming modules for each + # file, not by Chromium, so a second runner (and Vite server) halves the job + # that gated every run. release.yml mirrors this test, test_web, and test_server split. test_web: permissions: contents: read id-token: write - name: Test Web + name: Test Web ${{ matrix.shard }} needs: changes if: ${{ !cancelled() && needs.changes.outputs.test_web != 'false' }} runs-on: blacksmith-8vcpu-ubuntu-2404 timeout-minutes: 10 + strategy: + fail-fast: false + matrix: + shard: [1, 2] steps: - name: Checkout uses: actions/checkout@v6 @@ -394,7 +412,7 @@ jobs: fi - name: Test - run: vp run --filter @t3tools/web test + run: vp run --filter @t3tools/web test --shard ${{ matrix.shard }}/${{ strategy.job-total }} # apps/server sets `fileParallelism: false`, so its test files run strictly # one at a time. Sharding spreads them over separate runners instead of @@ -469,6 +487,17 @@ jobs: node apps/server/node_modules/playwright-core/cli.js install-deps chromium fi + # Each server test file runs in its own Vitest process, and a fresh install starts + # Vitest's content-keyed module cache empty, so every file re-transformed its imports. + # Only pushes to main save it, so pull requests never evict each other's entries. + - name: Restore Vitest module cache + if: steps.changes.outputs.test_server != 'false' + uses: actions/cache/restore@v6 + with: + path: node_modules/.vitest-cache + key: vitest-modules-server-${{ matrix.shard }}-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}-${{ github.sha }} + restore-keys: vitest-modules-server-${{ matrix.shard }}-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}- + - name: Test if: steps.changes.outputs.test_server != 'false' env: @@ -477,15 +506,14 @@ jobs: run: >- vp run test:cached -- --shard ${{ matrix.shard }}/${{ strategy.job-total }} --uncached apps/server/integration/transferBudgetV2.integration.test.ts - --verify-hits 1 --report "${{ runner.temp }}/test-cache-${{ matrix.shard }}.json" apps/server + --verify-hits 1 apps/server - - name: Upload test cache measurements - if: always() && steps.changes.outputs.test_server != 'false' - uses: actions/upload-artifact@v7 + - name: Save Vitest module cache + if: github.event_name == 'push' && steps.changes.outputs.test_server != 'false' + uses: actions/cache/save@v6 with: - name: test-cache-${{ matrix.shard }} - path: ${{ runner.temp }}/test-cache-${{ matrix.shard }}.json - if-no-files-found: error + path: node_modules/.vitest-cache + key: vitest-modules-server-${{ matrix.shard }}-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}-${{ github.sha }} # integration/transferBudgetV2.integration.test.ts writes the report; exactly one shard # produces these files. Gating the upload on their presence keeps a @@ -592,7 +620,7 @@ jobs: needs: changes if: ${{ !cancelled() && needs.changes.outputs.release_smoke != 'false' }} runs-on: blacksmith-8vcpu-ubuntu-2404 - timeout-minutes: 10 + timeout-minutes: 15 steps: - name: Checkout uses: actions/checkout@v6 diff --git a/.github/workflows/mobile-dev-client.yml b/.github/workflows/mobile-dev-client.yml new file mode 100644 index 0000000000000..5f8cb86befb59 --- /dev/null +++ b/.github/workflows/mobile-dev-client.yml @@ -0,0 +1,315 @@ +name: Mobile Dev Client +run-name: ${{ github.event_name == 'workflow_dispatch' && format('Dev client {0}{1} {2}', inputs.platform, inputs.platform == 'android' && format(' {0}', inputs.abi) || '', inputs.fingerprint) || '' }} + +# Builds the simulator and emulator development client for a commit's native +# fingerprint and shares it through the native-client cache, the one +# `vp run mobile:install` reads on every machine. Android builds with Gradle, +# ccache, and pnpm state on a sticky disk; iOS builds on a macOS runner, so no +# machine needs Xcode or an EAS build for it. +# +# - push to main: main's client is cached before anyone asks for it. +# - workflow_dispatch: `mobile:install` starts this for the checkout's HEAD on a +# cache miss, when the person running it may dispatch workflows here, and +# waits for the client in the cache. The run name is how it finds its run. +# - pull_request (this file or the script): builds without sharing, as a check. +on: + push: + branches: [main] + paths: + - apps/mobile/** + - packages/client-runtime/** + - packages/contracts/** + - packages/shared/** + - assets/** + - patches/** + - scripts/mobile-native-client.ts + - package.json + - pnpm-lock.yaml + - pnpm-workspace.yaml + - .github/workflows/mobile-dev-client.yml + pull_request: + paths: + - .github/workflows/mobile-dev-client.yml + - scripts/mobile-native-client.ts + workflow_dispatch: + inputs: + ref: + description: Commit to build (default main) + required: false + type: string + platform: + description: Platform + required: true + default: android + type: choice + options: [android, ios] + abi: + description: Android ABI of the device + required: true + default: x86_64 + type: choice + options: [x86_64, arm64-v8a] + fingerprint: + description: Native fingerprint the caller expects; the run fails when the commit's differs + required: false + type: string + +permissions: + contents: read + +jobs: + android: + name: Android ${{ matrix.abi }} + if: github.repository == 'pingdotgg/t3code' && (github.event_name != 'workflow_dispatch' || inputs.platform == 'android') + runs-on: blacksmith-32vcpu-ubuntu-2404 + timeout-minutes: 45 + strategy: + fail-fast: false + matrix: + # Pool emulators are x86_64; a dispatch builds the ABI it asks for. + abi: ${{ github.event_name == 'workflow_dispatch' && fromJSON(format('["{0}"]', inputs.abi)) || fromJSON('["x86_64"]') }} + concurrency: + # One run per client: a second request for the same fingerprint waits and finds it + # in the cache. A shared group would drop every waiting run but the newest. + group: mobile-dev-client-android-${{ matrix.abi }}-${{ github.event_name == 'pull_request' && github.event.pull_request.number || inputs.fingerprint || github.sha }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + env: + NODE_OPTIONS: --max-old-space-size=8192 + ABI: ${{ matrix.abi }} + EXPECTED_FINGERPRINT: ${{ inputs.fingerprint }} + # A pull request checks that the build works: it builds even on a cache hit and shares nothing. + T3_NATIVE_CLIENT_CACHE: ${{ github.event_name == 'pull_request' && 'off' || '' }} + # Gradle and Xcode output goes to the job log, not a file on the runner. + T3_NATIVE_CLIENT_VERBOSE: "1" + steps: + - name: Checkout + uses: actions/checkout@v6 + with: + ref: ${{ inputs.ref || github.sha }} + + # A dispatch names any commit GitHub has, including a fork's. Only a commit that + # collaborators pushed here builds with the cache login and the shared disk. + - name: Check the commit belongs to this repository + if: github.event_name == 'workflow_dispatch' + env: + GH_TOKEN: ${{ github.token }} + run: | + sha="$(git rev-parse HEAD)" + if git ls-remote --heads origin | grep -q "^$sha"; then + exit 0 + fi + status="$(gh api "repos/$GITHUB_REPOSITORY/compare/$sha...main" --jq .status || true)" + if [ "$status" = "ahead" ] || [ "$status" = "identical" ]; then + exit 0 + fi + echo "::error::$sha is not the head of a branch in $GITHUB_REPOSITORY and not on main; build it locally." + exit 1 + + # The build runs dependency scripts; it gets no token. checkout keeps the token in a + # file under RUNNER_TEMP that .git/config includes, so both go. `persist-credentials: + # false` fails here: its cleanup runs `git submodule foreach`, which stops at the + # vendored gitlinks under .repos (no .gitmodules entry). + - name: Drop checkout credentials + run: | + git config --local --get-regexp '^includeif\..*\.path$' | while read -r key file; do + rm -f "$file" + git config --local --unset-all "$key" + done + git config --local --name-only --get-regexp '^http\..*\.extraheader$' | while read -r key; do + git config --local --unset-all "$key" + done + rm -f "$RUNNER_TEMP"/git-credentials-*.config + if git config --local --get-regexp 'extraheader|includeif' || ls "$RUNNER_TEMP"/git-credentials-*.config 2>/dev/null; then + echo "::error::checkout credentials are still present" + exit 1 + fi + + - name: Prepare build directories + run: | + cache="$HOME/.cache/t3-android-dev-client" + { + echo "GRADLE_USER_HOME=$cache/gradle" + echo "CCACHE_DIR=$cache/ccache" + echo "CCACHE_MAXSIZE=5G" + # Gradle installs the NDK fresh each run, so key on the compiler's version, not its mtime. + echo "CCACHE_COMPILERCHECK=%compiler% -v" + # The checkout path is the same on every run, so ccache keys stay stable. + echo "CCACHE_BASEDIR=$GITHUB_WORKSPACE" + # CMake (3.17+) reads these, and AGP passes Gradle's environment through. + echo "CMAKE_C_COMPILER_LAUNCHER=ccache" + echo "CMAKE_CXX_COMPILER_LAUNCHER=ccache" + } >> "$GITHUB_ENV" + + # Pull requests build cold: their code must not write Gradle or ccache state that + # the runs holding the cache login restore. + - name: Mount build cache + if: github.event_name != 'pull_request' + uses: useblacksmith/stickydisk@v1 + with: + key: ${{ github.repository }}-android-dev-client-${{ matrix.abi }} + path: ~/.cache/t3-android-dev-client + + - name: Configure Gradle and ccache + run: | + mkdir -p "$GRADLE_USER_HOME" "$CCACHE_DIR" + # No daemon: nothing should hold the sticky disk open when it is committed. + printf 'org.gradle.caching=true\norg.gradle.daemon=false\n' > "$GRADLE_USER_HOME/gradle.properties" + sudo sed -i 's|http://|https://|g' /etc/apt/blacksmith-ubuntu-mirrors.txt /etc/apt/sources.list.d/ubuntu.sources + sudo apt-get update -qq && sudo apt-get install -y -qq ccache + ccache --zero-stats + + - name: Setup dependencies + uses: ./.github/actions/setup-ci-deps + with: + task-cache-read-token: ${{ secrets.T3_TASK_CACHE_READ_TOKEN }} + install-args: --filter=@t3tools/mobile... --filter=@t3tools/scripts... + + - name: Setup Java + uses: actions/setup-java@v5 + with: + distribution: temurin + java-version: 17 + + # Only this step's login can write to the cache; pull requests never get it. + # Only this login can write to the cache; pull requests never get it. + - id: login + name: Log in to the native client cache + if: github.event_name != 'pull_request' + env: + WRITE_TOKEN: ${{ secrets.T3_NATIVE_CLIENT_WRITE_TOKEN }} + run: | + if [ -z "$WRITE_TOKEN" ]; then + echo "::warning::T3_NATIVE_CLIENT_WRITE_TOKEN is not set; nothing can be shared, so nothing is built." + exit 0 + fi + node --input-type=module <<'NODE' + import { mkdirSync, readFileSync, writeFileSync } from "node:fs"; + import { homedir } from "node:os"; + import { join } from "node:path"; + const { endpoint } = JSON.parse(readFileSync(".task-cache.json", "utf8")); + const directory = join(homedir(), ".config/t3-task-cache"); + mkdirSync(directory, { recursive: true, mode: 0o700 }); + writeFileSync( + join(directory, "credentials.json"), + JSON.stringify({ [endpoint]: { name: "ci-native-clients", token: process.env.WRITE_TOKEN } }), + { mode: 0o600 }, + ); + NODE + echo "T3_NATIVE_CLIENT_SHARE=required" >> "$GITHUB_ENV" + echo "ready=true" >> "$GITHUB_OUTPUT" + + - name: Build and share + if: github.event_name == 'pull_request' || steps.login.outputs.ready == 'true' + shell: bash # pipefail: a failed build fails the step through tee + run: | + node scripts/mobile-native-client.ts ensure android --prepare --abi "$ABI" ${EXPECTED_FINGERPRINT:+--expect-fingerprint "$EXPECTED_FINGERPRINT"} | tee "$RUNNER_TEMP/dev-client.log" + result="$(tail -n 1 "$RUNNER_TEMP/dev-client.log")" + echo "### android dev client: \`$(jq -r '.source' <<<"$result")\` for fingerprint \`$(jq -r '.fingerprint' <<<"$result")\`" >> "$GITHUB_STEP_SUMMARY" + + - name: Cache statistics + if: success() || failure() + continue-on-error: true + run: | + { + echo '```' + ccache --show-stats + echo '```' + } >> "$GITHUB_STEP_SUMMARY" + + ios: + name: iOS simulator + if: github.repository == 'pingdotgg/t3code' && (github.event_name != 'workflow_dispatch' || inputs.platform == 'ios') + runs-on: blacksmith-12vcpu-macos-26 + timeout-minutes: 60 + concurrency: + group: mobile-dev-client-ios-${{ github.event_name == 'pull_request' && github.event.pull_request.number || inputs.fingerprint || github.sha }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + env: + NODE_OPTIONS: --max-old-space-size=8192 + EXPECTED_FINGERPRINT: ${{ inputs.fingerprint }} + T3_NATIVE_CLIENT_CACHE: ${{ github.event_name == 'pull_request' && 'off' || '' }} + # Gradle and Xcode output goes to the job log, not a file on the runner. + T3_NATIVE_CLIENT_VERBOSE: "1" + steps: + - name: Checkout + uses: actions/checkout@v6 + with: + ref: ${{ inputs.ref || github.sha }} + + # A dispatch names any commit GitHub has, including a fork's. Only a commit that + # collaborators pushed here builds with the cache login and the shared disk. + - name: Check the commit belongs to this repository + if: github.event_name == 'workflow_dispatch' + env: + GH_TOKEN: ${{ github.token }} + run: | + sha="$(git rev-parse HEAD)" + if git ls-remote --heads origin | grep -q "^$sha"; then + exit 0 + fi + status="$(gh api "repos/$GITHUB_REPOSITORY/compare/$sha...main" --jq .status || true)" + if [ "$status" = "ahead" ] || [ "$status" = "identical" ]; then + exit 0 + fi + echo "::error::$sha is not the head of a branch in $GITHUB_REPOSITORY and not on main; build it locally." + exit 1 + + # The build runs dependency scripts; it gets no token. checkout keeps the token in a + # file under RUNNER_TEMP that .git/config includes, so both go. `persist-credentials: + # false` fails here: its cleanup runs `git submodule foreach`, which stops at the + # vendored gitlinks under .repos (no .gitmodules entry). + - name: Drop checkout credentials + run: | + git config --local --get-regexp '^includeif\..*\.path$' | while read -r key file; do + rm -f "$file" + git config --local --unset-all "$key" + done + git config --local --name-only --get-regexp '^http\..*\.extraheader$' | while read -r key; do + git config --local --unset-all "$key" + done + rm -f "$RUNNER_TEMP"/git-credentials-*.config + if git config --local --get-regexp 'extraheader|includeif' || ls "$RUNNER_TEMP"/git-credentials-*.config 2>/dev/null; then + echo "::error::checkout credentials are still present" + exit 1 + fi + + - name: Setup dependencies + uses: ./.github/actions/setup-ci-deps + with: + task-cache-read-token: ${{ secrets.T3_TASK_CACHE_READ_TOKEN }} + install-args: --filter=@t3tools/mobile... --filter=@t3tools/scripts... + + # Only this login can write to the cache; pull requests never get it. + - id: login + name: Log in to the native client cache + if: github.event_name != 'pull_request' + env: + WRITE_TOKEN: ${{ secrets.T3_NATIVE_CLIENT_WRITE_TOKEN }} + run: | + if [ -z "$WRITE_TOKEN" ]; then + echo "::warning::T3_NATIVE_CLIENT_WRITE_TOKEN is not set; nothing can be shared, so nothing is built." + exit 0 + fi + node --input-type=module <<'NODE' + import { mkdirSync, readFileSync, writeFileSync } from "node:fs"; + import { homedir } from "node:os"; + import { join } from "node:path"; + const { endpoint } = JSON.parse(readFileSync(".task-cache.json", "utf8")); + const directory = join(homedir(), ".config/t3-task-cache"); + mkdirSync(directory, { recursive: true, mode: 0o700 }); + writeFileSync( + join(directory, "credentials.json"), + JSON.stringify({ [endpoint]: { name: "ci-native-clients", token: process.env.WRITE_TOKEN } }), + { mode: 0o600 }, + ); + NODE + echo "T3_NATIVE_CLIENT_SHARE=required" >> "$GITHUB_ENV" + echo "ready=true" >> "$GITHUB_OUTPUT" + + - name: Build and share + if: github.event_name == 'pull_request' || steps.login.outputs.ready == 'true' + shell: bash # pipefail: a failed build fails the step through tee + run: | + node scripts/mobile-native-client.ts ensure ios --prepare ${EXPECTED_FINGERPRINT:+--expect-fingerprint "$EXPECTED_FINGERPRINT"} | tee "$RUNNER_TEMP/dev-client.log" + result="$(tail -n 1 "$RUNNER_TEMP/dev-client.log")" + echo "### ios dev client: \`$(jq -r '.source' <<<"$result")\` for fingerprint \`$(jq -r '.fingerprint' <<<"$result")\`" >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/mobile-eas-preview-fork.yml b/.github/workflows/mobile-eas-preview-fork.yml index e7615d46011bd..a8bf5725af70c 100644 --- a/.github/workflows/mobile-eas-preview-fork.yml +++ b/.github/workflows/mobile-eas-preview-fork.yml @@ -1,8 +1,8 @@ name: Mobile EAS Preview (Fork) # EAS previews for fork PRs. This runs the PR's code with an Expo token: the -# app config and plugins run on this runner, and native builds run on EAS -# with preview signing credentials. The label is the maintainer's sign-off +# app config and plugins run on this runner, the iOS client builds on EAS and +# the Android one on Blacksmith (android-build), with preview signing credentials. The label is the maintainer's sign-off # that they reviewed that code, so it covers exactly the commit it was applied # to: this pins that commit and consumes the label before running anything. A # later push deploys nothing until the label is applied again. @@ -29,6 +29,9 @@ jobs: permissions: contents: read pull-requests: write + outputs: + build: ${{ steps.client.outputs.build }} + fingerprint: ${{ steps.client.outputs.fingerprint }} env: APP_VARIANT: preview NODE_OPTIONS: --max-old-space-size=8192 @@ -76,9 +79,43 @@ jobs: with: ref: ${{ github.event.pull_request.head.sha }} fetch-depth: 0 - persist-credentials: false # No sparse-checkout: see mobile-eas-preview.yml. + # The comment script comes from main, not from the PR. + - name: Checkout trusted scripts + if: steps.expo-token.outputs.present == 'true' + uses: actions/checkout@v6 + with: + path: .trusted + sparse-checkout: .github/scripts + + # The steps below run the PR's dependency and build scripts; they get no token. + # checkout keeps it in a file under RUNNER_TEMP that .git/config includes, so both + # go. Asking checkout not to persist it fails here: that cleanup runs `git submodule + # foreach`, which stops at the vendored gitlinks under .repos (no .gitmodules entry). + - name: Drop checkout credentials + if: steps.expo-token.outputs.present == 'true' + run: | + for dir in . .trusted; do + [ -e "$dir/.git" ] || continue + git -C "$dir" config --local --get-regexp '^includeif\..*\.path$' | while read -r key file; do + rm -f "$file" + git -C "$dir" config --local --unset-all "$key" + done + git -C "$dir" config --local --name-only --get-regexp '^http\..*\.extraheader$' | while read -r key; do + git -C "$dir" config --local --unset-all "$key" + done + if git -C "$dir" config --local --get-regexp 'extraheader|includeif'; then + echo "::error::checkout credentials are still configured in $dir" + exit 1 + fi + done + rm -f "$RUNNER_TEMP"/git-credentials-*.config + if ls "$RUNNER_TEMP"/git-credentials-*.config 2>/dev/null; then + echo "::error::a checkout credential file is still present" + exit 1 + fi + # No caches: pull_request_target saves them under main's scope, where # trusted workflows with other secrets would restore what PR code wrote. - name: Setup Vite+ @@ -139,7 +176,180 @@ jobs: with: profile: preview:dev branch: pr-${{ github.event.pull_request.number }} - platform: all + # Android is below: its client builds on Blacksmith (android-build). + platform: ios environment: preview working-directory: apps/mobile github-token: ${{ secrets.GITHUB_TOKEN }} + + # Android, as in mobile-eas-preview.yml: the same fingerprint command the Expo + # action runs. A client built for a same-repo PR (its tag) or by EAS is reused. + - id: client + name: Find the Android client for this fingerprint + if: steps.expo-token.outputs.present == 'true' + working-directory: apps/mobile + shell: bash + env: + EXPO_TOKEN: ${{ secrets.EXPO_PREVIEW_TOKEN }} + run: | + hash="$(eas env:exec --non-interactive preview "npx expo-updates fingerprint:generate --platform android" | sed -n '/^{/,$p' | jq -er '.hash | select(type == "string" and length > 0)')" + echo "fingerprint=$hash" >> "$GITHUB_OUTPUT" + tag="mobile-android-preview-build/$hash" + link="" + if git fetch --quiet origin "refs/tags/$tag:refs/tags/$tag" 2>/dev/null; then + link="$(git tag -l --format='%(contents:subject)' "$tag")" + else + link="$(eas build:list --platform android --build-profile preview:dev --runtime-version "$hash" --status finished --limit 5 --json --non-interactive \ + | sed -n '/^\[/,$p' \ + | jq -r --arg now "$(date -u +%Y-%m-%dT%H:%M:%SZ)" '[.[] | select((.expirationDate // "9999") > $now)][0] | select(. != null) | "https://expo.dev/accounts/pingdotgg/projects/t3-code/builds/\(.id)"')" + fi + echo "link=$link" >> "$GITHUB_OUTPUT" + if [ -z "$link" ]; then echo "build=true" >> "$GITHUB_OUTPUT"; fi + + - id: update + name: Publish the Android update + if: steps.expo-token.outputs.present == 'true' + working-directory: apps/mobile + shell: bash + env: + EXPO_TOKEN: ${{ secrets.EXPO_PREVIEW_TOKEN }} + PR: ${{ github.event.pull_request.number }} + run: | + group="$(eas update --auto --branch "pr-$PR" --platform android --environment preview --non-interactive --json | sed -n '/^\[/,$p' | jq -er '.[0].group')" + echo "group=$group" >> "$GITHUB_OUTPUT" + + - name: Comment + if: steps.expo-token.outputs.present == 'true' + env: + GH_TOKEN: ${{ github.token }} + PR: ${{ github.event.pull_request.number }} + FINGERPRINT: ${{ steps.client.outputs.fingerprint }} + CLIENT: ${{ steps.client.outputs.link }} + GROUP: ${{ steps.update.outputs.group }} + run: node .trusted/.github/scripts/mobile-preview-comment.cjs + + # The reviewed commit's Android client, built cold: no sticky disk, because PR + # code could leave Gradle or ccache state that trusted builds would restore, and + # no build record tag, so nobody else is served an APK a fork's code built. + android-build: + name: Android preview client + needs: preview + if: needs.preview.outputs.build == 'true' + runs-on: blacksmith-32vcpu-ubuntu-2404 + timeout-minutes: 60 + permissions: + contents: read + pull-requests: write + env: + APP_VARIANT: preview + NODE_OPTIONS: --max-old-space-size=8192 + MOBILE_VERSION_POLICY: fingerprint + FINGERPRINT: ${{ needs.preview.outputs.fingerprint }} + # Phones only: emulators use the development client from the native-client cache. + ORG_GRADLE_PROJECT_reactNativeArchitectures: armeabi-v7a,arm64-v8a + steps: + - name: Checkout reviewed commit + uses: actions/checkout@v6 + with: + ref: ${{ github.event.pull_request.head.sha }} + fetch-depth: 0 + + # The comment script comes from main, not from the PR. + - name: Checkout trusted scripts + uses: actions/checkout@v6 + with: + path: .trusted + sparse-checkout: .github/scripts + + # The steps below run the PR's dependency and build scripts; they get no token. + # checkout keeps it in a file under RUNNER_TEMP that .git/config includes, so both + # go. Asking checkout not to persist it fails here: that cleanup runs `git submodule + # foreach`, which stops at the vendored gitlinks under .repos (no .gitmodules entry). + - name: Drop checkout credentials + run: | + for dir in . .trusted; do + [ -e "$dir/.git" ] || continue + git -C "$dir" config --local --get-regexp '^includeif\..*\.path$' | while read -r key file; do + rm -f "$file" + git -C "$dir" config --local --unset-all "$key" + done + git -C "$dir" config --local --name-only --get-regexp '^http\..*\.extraheader$' | while read -r key; do + git -C "$dir" config --local --unset-all "$key" + done + if git -C "$dir" config --local --get-regexp 'extraheader|includeif'; then + echo "::error::checkout credentials are still configured in $dir" + exit 1 + fi + done + rm -f "$RUNNER_TEMP"/git-credentials-*.config + if ls "$RUNNER_TEMP"/git-credentials-*.config 2>/dev/null; then + echo "::error::a checkout credential file is still present" + exit 1 + fi + + - name: Setup Node + uses: actions/setup-node@v6 + with: + node-version-file: package.json + + - name: Enable corepack + run: | + npm install --global corepack@latest + corepack enable + pnpm --version + + - name: Install mobile dependencies + run: pnpm install --frozen-lockfile --filter=@t3tools/mobile... + + - name: Setup Java + uses: actions/setup-java@v5 + with: + distribution: temurin + java-version: 17 + + - name: Setup EAS + uses: expo/expo-github-action@v8 + with: + eas-version: latest + eas-cache: false + token: ${{ secrets.EXPO_PREVIEW_TOKEN }} + packager: npm + + - name: Pull preview environment variables + working-directory: apps/mobile + env: + EXPO_TOKEN: ${{ secrets.EXPO_PREVIEW_TOKEN }} + run: | + eas env:pull preview --non-interactive + node --input-type=module <<'NODE' + import { appendFileSync, readFileSync } from "node:fs"; + import { parseEnv } from "node:util"; + const env = parseEnv(readFileSync(".env.local", "utf8")); + const googleServicesFile = env.T3CODE_ANDROID_GOOGLE_SERVICES_FILE; + if (googleServicesFile) { + appendFileSync(process.env.GITHUB_ENV, `T3CODE_ANDROID_GOOGLE_SERVICES_FILE=${googleServicesFile}\n`); + } + NODE + + - id: build + name: Build and upload + working-directory: apps/mobile + shell: bash + env: + EXPO_TOKEN: ${{ secrets.EXPO_PREVIEW_TOKEN }} + run: | + eas build --local --platform android --profile preview:dev --non-interactive --output "$RUNNER_TEMP/t3code-preview.apk" + eas upload --platform android --build-path "$RUNNER_TEMP/t3code-preview.apk" --fingerprint "$FINGERPRINT" --non-interactive --json | tee "$RUNNER_TEMP/upload.json" + link="$(sed -n '/^{/,$p' "$RUNNER_TEMP/upload.json" | jq -r '.url // empty')" + if [ -z "$link" ]; then + echo "::error::eas upload returned no install link" + exit 1 + fi + echo "link=$link" >> "$GITHUB_OUTPUT" + + - name: Comment + env: + GH_TOKEN: ${{ github.token }} + PR: ${{ github.event.pull_request.number }} + CLIENT: ${{ steps.build.outputs.link }} + run: node .trusted/.github/scripts/mobile-preview-comment.cjs diff --git a/.github/workflows/mobile-eas-preview.yml b/.github/workflows/mobile-eas-preview.yml index 08183d0385811..5989d7a0c0cc8 100644 --- a/.github/workflows/mobile-eas-preview.yml +++ b/.github/workflows/mobile-eas-preview.yml @@ -6,7 +6,7 @@ on: jobs: preview: - name: EAS Preview + name: EAS Preview (iOS) # Fork PRs deploy one reviewed commit per label through mobile-eas-preview-fork.yml. if: | github.event.pull_request.head.repo.full_name == github.repository && @@ -107,7 +107,349 @@ jobs: with: profile: preview:dev branch: pr-${{ github.event.pull_request.number }} - platform: all + # Android is the android job below: its client builds on Blacksmith. + platform: ios environment: preview working-directory: apps/mobile github-token: ${{ secrets.GITHUB_TOKEN }} + + # Android publishes its own update and comment, because the Expo action can only + # find a client that EAS built. A fingerprint with no client is built by + # android-build with `eas build --local` on Blacksmith (about 4 minutes warm, + # against about 40 on EAS) and recorded as a mobile-android-preview-build/ + # tag whose message is the install link. Clients EAS built earlier still count. + android: + name: EAS Preview (Android) + if: | + github.event.pull_request.head.repo.full_name == github.repository && + contains(github.event.pull_request.labels.*.name, '🚀 Mobile Continuous Deployment') && + (github.event.action != 'labeled' || github.event.label.name == '🚀 Mobile Continuous Deployment') + runs-on: blacksmith-8vcpu-ubuntu-2404 + concurrency: + group: mobile-eas-preview-android-${{ github.event.pull_request.number }} + cancel-in-progress: true + permissions: + contents: read + pull-requests: write + outputs: + build: ${{ steps.client.outputs.build }} + fingerprint: ${{ steps.client.outputs.fingerprint }} + env: + APP_VARIANT: preview + NODE_OPTIONS: --max-old-space-size=8192 + MOBILE_VERSION_POLICY: fingerprint + EXPO_TOKEN: ${{ secrets.EXPO_PREVIEW_TOKEN }} + GH_TOKEN: ${{ github.token }} + PR: ${{ github.event.pull_request.number }} + steps: + - id: expo-token + name: Check for EXPO_PREVIEW_TOKEN + run: | + if [ -n "$EXPO_TOKEN" ]; then + echo "present=true" >> "$GITHUB_OUTPUT" + else + echo "present=false" >> "$GITHUB_OUTPUT" + echo "EXPO_PREVIEW_TOKEN is not available; skipping the Android preview." + fi + + - name: Checkout + if: steps.expo-token.outputs.present == 'true' + uses: actions/checkout@v6 + with: + fetch-depth: 0 + + # The steps below run the PR's dependency and build scripts; they get no token. + # checkout keeps it in a file under RUNNER_TEMP that .git/config includes, so both + # go. Asking checkout not to persist it fails here: that cleanup runs `git submodule + # foreach`, which stops at the vendored gitlinks under .repos (no .gitmodules entry). + - name: Drop checkout credentials + if: steps.expo-token.outputs.present == 'true' + run: | + for dir in . .trusted; do + [ -e "$dir/.git" ] || continue + git -C "$dir" config --local --get-regexp '^includeif\..*\.path$' | while read -r key file; do + rm -f "$file" + git -C "$dir" config --local --unset-all "$key" + done + git -C "$dir" config --local --name-only --get-regexp '^http\..*\.extraheader$' | while read -r key; do + git -C "$dir" config --local --unset-all "$key" + done + if git -C "$dir" config --local --get-regexp 'extraheader|includeif'; then + echo "::error::checkout credentials are still configured in $dir" + exit 1 + fi + done + rm -f "$RUNNER_TEMP"/git-credentials-*.config + if ls "$RUNNER_TEMP"/git-credentials-*.config 2>/dev/null; then + echo "::error::a checkout credential file is still present" + exit 1 + fi + + - name: Setup Vite+ + if: steps.expo-token.outputs.present == 'true' + uses: voidzero-dev/setup-vp@v1 + with: + node-version-file: package.json + cache: true + run-install: | + args: + - --filter=@t3tools/mobile... + + - name: Expose pnpm + if: steps.expo-token.outputs.present == 'true' + run: | + pnpm_version="$(node --print "require('./package.json').packageManager.split('@').pop()")" + vp_pnpm_bin="$HOME/.vite-plus/package_manager/pnpm/$pnpm_version/pnpm/bin" + echo "$vp_pnpm_bin" >> "$GITHUB_PATH" + "$vp_pnpm_bin/pnpm" --version + + - name: Setup EAS + if: steps.expo-token.outputs.present == 'true' + uses: expo/expo-github-action@v8 + with: + eas-version: latest + token: ${{ secrets.EXPO_PREVIEW_TOKEN }} + packager: npm + + - name: Pull preview environment variables + if: steps.expo-token.outputs.present == 'true' + working-directory: apps/mobile + run: | + eas env:pull preview --non-interactive + node --input-type=module <<'NODE' + import { appendFileSync, readFileSync } from "node:fs"; + import { parseEnv } from "node:util"; + const env = parseEnv(readFileSync(".env.local", "utf8")); + const googleServicesFile = env.T3CODE_ANDROID_GOOGLE_SERVICES_FILE; + if (googleServicesFile) { + appendFileSync(process.env.GITHUB_ENV, `T3CODE_ANDROID_GOOGLE_SERVICES_FILE=${googleServicesFile}\n`); + } + NODE + + # The same command the Expo action runs, so this is the runtime version the update gets. + - id: client + name: Find the client for this fingerprint + if: steps.expo-token.outputs.present == 'true' + working-directory: apps/mobile + shell: bash + run: | + hash="$(eas env:exec --non-interactive preview "npx expo-updates fingerprint:generate --platform android" | sed -n '/^{/,$p' | jq -er '.hash | select(type == "string" and length > 0)')" + echo "fingerprint=$hash" >> "$GITHUB_OUTPUT" + tag="mobile-android-preview-build/$hash" + link="" + if git fetch --quiet origin "refs/tags/$tag:refs/tags/$tag" 2>/dev/null; then + link="$(git tag -l --format='%(contents:subject)' "$tag")" + else + # A client EAS built before this job existed; expired ones are gone. + link="$(eas build:list --platform android --build-profile preview:dev --runtime-version "$hash" --status finished --limit 5 --json --non-interactive \ + | sed -n '/^\[/,$p' \ + | jq -r --arg now "$(date -u +%Y-%m-%dT%H:%M:%SZ)" '[.[] | select((.expirationDate // "9999") > $now)][0] | select(. != null) | "https://expo.dev/accounts/pingdotgg/projects/t3-code/builds/\(.id)"')" + fi + echo "link=$link" >> "$GITHUB_OUTPUT" + if [ -z "$link" ]; then echo "build=true" >> "$GITHUB_OUTPUT"; fi + echo "android fingerprint \`$hash\`: ${link:-no client yet, building on Blacksmith}" >> "$GITHUB_STEP_SUMMARY" + + - id: update + name: Publish the update + if: steps.expo-token.outputs.present == 'true' + working-directory: apps/mobile + shell: bash + run: | + group="$(eas update --auto --branch "pr-$PR" --platform android --environment preview --non-interactive --json | sed -n '/^\[/,$p' | jq -er '.[0].group')" + echo "group=$group" >> "$GITHUB_OUTPUT" + + - name: Comment + if: steps.expo-token.outputs.present == 'true' + env: + FINGERPRINT: ${{ steps.client.outputs.fingerprint }} + CLIENT: ${{ steps.client.outputs.link }} + GROUP: ${{ steps.update.outputs.group }} + run: node .github/scripts/mobile-preview-comment.cjs + + android-build: + name: Android preview client + needs: android + if: needs.android.outputs.build == 'true' + runs-on: blacksmith-32vcpu-ubuntu-2404 + timeout-minutes: 60 + concurrency: + # One build per fingerprint; a second PR with the same one finds the tag. + group: mobile-android-preview-build-${{ needs.android.outputs.fingerprint }} + cancel-in-progress: false + permissions: + contents: write # pushes the build record tag + pull-requests: write + env: + APP_VARIANT: preview + NODE_OPTIONS: --max-old-space-size=8192 + MOBILE_VERSION_POLICY: fingerprint + EXPO_TOKEN: ${{ secrets.EXPO_PREVIEW_TOKEN }} + FINGERPRINT: ${{ needs.android.outputs.fingerprint }} + BUILD_TAG: mobile-android-preview-build/${{ needs.android.outputs.fingerprint }} + # Phones only: emulators use the development client from the native-client cache. + ORG_GRADLE_PROJECT_reactNativeArchitectures: armeabi-v7a,arm64-v8a + steps: + - name: Checkout + uses: actions/checkout@v6 + with: + fetch-depth: 0 + + - id: record + name: Check build record + run: | + if git fetch --quiet origin "refs/tags/$BUILD_TAG:refs/tags/$BUILD_TAG" 2>/dev/null; then + echo "built=true" >> "$GITHUB_OUTPUT" + echo "link=$(git tag -l --format='%(contents:subject)' "$BUILD_TAG")" >> "$GITHUB_OUTPUT" + fi + + # The steps below run the PR's dependency and build scripts; they get no token. + # checkout keeps it in a file under RUNNER_TEMP that .git/config includes, so both + # go. Asking checkout not to persist it fails here: that cleanup runs `git submodule + # foreach`, which stops at the vendored gitlinks under .repos (no .gitmodules entry). + - name: Drop checkout credentials + run: | + for dir in . .trusted; do + [ -e "$dir/.git" ] || continue + git -C "$dir" config --local --get-regexp '^includeif\..*\.path$' | while read -r key file; do + rm -f "$file" + git -C "$dir" config --local --unset-all "$key" + done + git -C "$dir" config --local --name-only --get-regexp '^http\..*\.extraheader$' | while read -r key; do + git -C "$dir" config --local --unset-all "$key" + done + if git -C "$dir" config --local --get-regexp 'extraheader|includeif'; then + echo "::error::checkout credentials are still configured in $dir" + exit 1 + fi + done + rm -f "$RUNNER_TEMP"/git-credentials-*.config + if ls "$RUNNER_TEMP"/git-credentials-*.config 2>/dev/null; then + echo "::error::a checkout credential file is still present" + exit 1 + fi + + - name: Prepare build directories + if: steps.record.outputs.built != 'true' + run: | + cache="$HOME/.cache/t3-android-preview" + { + echo "GRADLE_USER_HOME=$cache/gradle" + echo "CCACHE_DIR=$cache/ccache" + echo "CCACHE_MAXSIZE=5G" + # Gradle installs the NDK fresh each run, so key on the compiler's version, not its mtime. + echo "CCACHE_COMPILERCHECK=%compiler% -v" + echo "pnpm_config_store_dir=$cache/pnpm-store" + # eas build --local copies the project here. A fixed path keeps + # ccache keys stable, and the same disk as the store lets pnpm hardlink. + echo "EAS_LOCAL_BUILD_WORKINGDIR=$cache/eas-build" + echo "CCACHE_BASEDIR=$cache/eas-build" + echo "CMAKE_C_COMPILER_LAUNCHER=ccache" + echo "CMAKE_CXX_COMPILER_LAUNCHER=ccache" + } >> "$GITHUB_ENV" + + - name: Mount build cache + if: steps.record.outputs.built != 'true' + uses: useblacksmith/stickydisk@v1 + with: + key: ${{ github.repository }}-android-preview-build + path: ~/.cache/t3-android-preview + + - name: Configure Gradle and ccache + if: steps.record.outputs.built != 'true' + run: | + mkdir -p "$GRADLE_USER_HOME" "$CCACHE_DIR" + rm -rf "$EAS_LOCAL_BUILD_WORKINGDIR" + # No daemon: nothing should hold the sticky disk open when it is committed. + printf 'org.gradle.caching=true\norg.gradle.daemon=false\n' > "$GRADLE_USER_HOME/gradle.properties" + sudo sed -i 's|http://|https://|g' /etc/apt/blacksmith-ubuntu-mirrors.txt /etc/apt/sources.list.d/ubuntu.sources + sudo apt-get update -qq && sudo apt-get install -y -qq ccache + ccache --zero-stats + + - name: Setup Node + if: steps.record.outputs.built != 'true' + uses: actions/setup-node@v6 + with: + node-version-file: package.json + + - name: Enable corepack + if: steps.record.outputs.built != 'true' + run: | + npm install --global corepack@latest + corepack enable + pnpm --version + + - name: Install mobile dependencies + if: steps.record.outputs.built != 'true' + run: pnpm install --frozen-lockfile --prefer-offline --filter=@t3tools/mobile... + + - name: Setup Java + if: steps.record.outputs.built != 'true' + uses: actions/setup-java@v5 + with: + distribution: temurin + java-version: 17 + + - name: Setup EAS + if: steps.record.outputs.built != 'true' + uses: expo/expo-github-action@v8 + with: + eas-version: latest + token: ${{ secrets.EXPO_PREVIEW_TOKEN }} + packager: npm + + - name: Pull preview environment variables + if: steps.record.outputs.built != 'true' + working-directory: apps/mobile + run: | + eas env:pull preview --non-interactive + node --input-type=module <<'NODE' + import { appendFileSync, readFileSync } from "node:fs"; + import { parseEnv } from "node:util"; + const env = parseEnv(readFileSync(".env.local", "utf8")); + const googleServicesFile = env.T3CODE_ANDROID_GOOGLE_SERVICES_FILE; + if (googleServicesFile) { + appendFileSync(process.env.GITHUB_ENV, `T3CODE_ANDROID_GOOGLE_SERVICES_FILE=${googleServicesFile}\n`); + } + NODE + + # `eas upload` hosts the APK on EAS and returns the page testers install from. + - id: build + name: Build and upload + if: steps.record.outputs.built != 'true' + working-directory: apps/mobile + shell: bash + run: | + eas build --local --platform android --profile preview:dev --non-interactive --output "$RUNNER_TEMP/t3code-preview.apk" + eas upload --platform android --build-path "$RUNNER_TEMP/t3code-preview.apk" --fingerprint "$FINGERPRINT" --non-interactive --json | tee "$RUNNER_TEMP/upload.json" + link="$(sed -n '/^{/,$p' "$RUNNER_TEMP/upload.json" | jq -r '.url // empty')" + if [ -z "$link" ]; then + echo "::error::eas upload returned no install link" + exit 1 + fi + echo "link=$link" >> "$GITHUB_OUTPUT" + { + echo "### Android preview client for \`$FINGERPRINT\`: $link" + echo '```' + ccache --show-stats + echo '```' + } >> "$GITHUB_STEP_SUMMARY" + + # The tag is the record later runs look up; its message is the install link. + - name: Record build + if: steps.record.outputs.built != 'true' + env: + LINK: ${{ steps.build.outputs.link }} + GH_TOKEN: ${{ github.token }} + run: | + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git tag -a "$BUILD_TAG" -m "$LINK" + # Only this step has the token; the checkout's copy was removed before the build. + git push "https://x-access-token:${GH_TOKEN}@github.com/${GITHUB_REPOSITORY}.git" "refs/tags/$BUILD_TAG" + + - name: Comment + env: + GH_TOKEN: ${{ github.token }} + PR: ${{ github.event.pull_request.number }} + CLIENT: ${{ steps.build.outputs.link || steps.record.outputs.link }} + run: node .github/scripts/mobile-preview-comment.cjs diff --git a/.github/workflows/mobile-eas-production.yml b/.github/workflows/mobile-eas-production.yml index 6877f5c831a08..1bf16d8a8f65e 100644 --- a/.github/workflows/mobile-eas-production.yml +++ b/.github/workflows/mobile-eas-production.yml @@ -7,6 +7,12 @@ name: Mobile EAS Production # runner, with corepack pinning pnpm 10.24 in eas.json, local == build. # # Every merge reconciles native fingerprints and publishes matching OTAs. +# Android store builds run here with `eas build --local` on Blacksmith (the +# android job). iOS builds run on EAS unless the MOBILE_IOS_LOCAL_BUILDS +# repository variable is `true`, which moves them to the ios job the same way. +# Local builds never appear in `eas build:list`, so each finished local build +# pushes a mobile-android-build// or +# mobile-ios-build/// tag that the reconcile reads. # iOS prereleases use the testflight profile (3.0.0); App Store binaries use # production and the committed app version, built manually. Both receive OTAs # for their own major version. Android continues using production automatically. @@ -65,20 +71,29 @@ on: - pnpm-workspace.yaml - .github/workflows/mobile-eas-production.yml -# Serialize runs so OTAs publish in merge order. GitHub keeps at most one -# queued run per group, so a burst of merges collapses into one run of the -# newest commit — intermediate commits don't need their own OTA. -concurrency: - group: mobile-eas-production - cancel-in-progress: false - jobs: production: name: EAS ${{ inputs.profile || 'production' }} ${{ github.event_name == 'push' && 'auto' || inputs.mode }} if: github.repository == 'pingdotgg/t3code' runs-on: blacksmith-8vcpu-ubuntu-2404 + # Serialize runs so OTAs publish in merge order. GitHub keeps at most one + # queued job per group, so a burst of merges collapses into one run of the + # newest commit — intermediate commits don't need their own OTA. + concurrency: + group: mobile-eas-production + cancel-in-progress: false permissions: contents: read + outputs: + android_build: ${{ steps.reconcile.outputs.android_build || steps.android_manual.outputs.android_build }} + android_fingerprint: ${{ steps.reconcile.outputs.android_fingerprint || steps.android_manual.outputs.android_fingerprint }} + android_version: ${{ steps.reconcile.outputs.android_version || steps.android_manual.outputs.android_version }} + android_sha: ${{ steps.reconcile.outputs.android_sha || steps.android_manual.outputs.android_sha }} + ios_build: ${{ steps.reconcile.outputs.ios_build || steps.ios_manual.outputs.ios_build }} + ios_profile: ${{ steps.reconcile.outputs.ios_profile || steps.ios_manual.outputs.ios_profile }} + ios_fingerprint: ${{ steps.reconcile.outputs.ios_fingerprint || steps.ios_manual.outputs.ios_fingerprint }} + ios_version: ${{ steps.reconcile.outputs.ios_version || steps.ios_manual.outputs.ios_version }} + ios_sha: ${{ steps.reconcile.outputs.ios_sha || steps.ios_manual.outputs.ios_sha }} env: APP_VARIANT: production NODE_OPTIONS: --max-old-space-size=8192 @@ -215,9 +230,7 @@ jobs: git config user.name "${APP_SLUG}[bot]" git config user.email "${user_id}+${APP_SLUG}[bot]@users.noreply.github.com" git add apps/mobile/app.config.ts - git commit --no-verify \ - -m "chore(mobile): bump app version to $RELEASE_VERSION" \ - -m "Co-authored-by: codex " + git commit --no-verify -m "chore(mobile): bump app version to $RELEASE_VERSION" git push origin "HEAD:refs/heads/${GITHUB_REF_NAME}" - name: Summarize manual build version @@ -240,13 +253,51 @@ jobs: echo "> Apple closes an iOS release train after App Store approval. Before building iOS, confirm \`$version\` is newer than the approved App Store version." } >> "$GITHUB_STEP_SUMMARY" - - name: Build and submit (manual) - if: steps.expo-token.outputs.present == 'true' && github.event_name == 'workflow_dispatch' && inputs.mode == 'build' + - name: Build and submit iOS (manual) + if: steps.expo-token.outputs.present == 'true' && github.event_name == 'workflow_dispatch' && inputs.mode == 'build' && inputs.platform != 'android' && vars.MOBILE_IOS_LOCAL_BUILDS != 'true' working-directory: apps/mobile env: EXPO_TOKEN: ${{ secrets.EXPO_PRODUCTION_TOKEN }} MOBILE_BUILD_PROFILE: ${{ inputs.profile || 'production' }} - run: eas build --platform ${{ inputs.platform }} --profile "$MOBILE_BUILD_PROFILE" --auto-submit --non-interactive --no-wait + run: eas build --platform ios --profile "$MOBILE_BUILD_PROFILE" --auto-submit --non-interactive --no-wait + + - id: ios_manual + name: Queue iOS build (manual) + if: steps.expo-token.outputs.present == 'true' && github.event_name == 'workflow_dispatch' && inputs.mode == 'build' && inputs.platform != 'android' && vars.MOBILE_IOS_LOCAL_BUILDS == 'true' + working-directory: apps/mobile + env: + EXPO_TOKEN: ${{ secrets.EXPO_PRODUCTION_TOKEN }} + MOBILE_BUILD_PROFILE: ${{ inputs.profile || 'production' }} + run: | + if [ "$MOBILE_BUILD_PROFILE" = "testflight" ]; then + T3CODE_MOBILE_VERSION="$(node -p 'require("./eas.json").build.testflight.env.T3CODE_MOBILE_VERSION')" + export T3CODE_MOBILE_VERSION + fi + version="$(npx expo config --json --type public | jq -r '.version')" + hash="$(eas fingerprint:generate --platform ios --environment production --json --non-interactive | sed -n '/^{/,$p' | jq -er '.hash | select(type == "string" and length > 0)')" + { + echo "ios_build=true" + echo "ios_profile=$MOBILE_BUILD_PROFILE" + echo "ios_fingerprint=$hash" + echo "ios_version=$version" + echo "ios_sha=$(git rev-parse HEAD)" + } >> "$GITHUB_OUTPUT" + + - id: android_manual + name: Queue Android build (manual) + if: steps.expo-token.outputs.present == 'true' && github.event_name == 'workflow_dispatch' && inputs.mode == 'build' && inputs.platform != 'ios' + working-directory: apps/mobile + env: + EXPO_TOKEN: ${{ secrets.EXPO_PRODUCTION_TOKEN }} + run: | + version="$(npx expo config --json --type public | jq -r '.version')" + hash="$(eas fingerprint:generate --platform android --environment production --json --non-interactive | sed -n '/^{/,$p' | jq -er '.hash | select(type == "string" and length > 0)')" + { + echo "android_build=true" + echo "android_fingerprint=$hash" + echo "android_version=$version" + echo "android_sha=$(git rev-parse HEAD)" + } >> "$GITHUB_OUTPUT" # @expo/fingerprint swallows autolinking errors and returns no Expo module # sources, so native changes stop moving the hash and OTAs reach binaries @@ -300,6 +351,7 @@ jobs: working-directory: apps/mobile env: EXPO_TOKEN: ${{ secrets.EXPO_PRODUCTION_TOKEN }} + IOS_LOCAL_BUILDS: ${{ vars.MOBILE_IOS_LOCAL_BUILDS == 'true' }} run: | failed=0 message="$(git log -1 --pretty=%s | head -c 120) ($(git rev-parse --short=9 HEAD))" @@ -320,6 +372,22 @@ jobs: active="$(jq '[.[] | select(.status == "NEW" or .status == "IN_QUEUE" or .status == "IN_PROGRESS")] | length' <<<"$matches")" errored="$(jq '[.[] | select(.status == "ERRORED")] | length' <<<"$matches")" latest_version="$(eas build:list --platform "$platform" --build-profile "$profile" --limit 1 --json --non-interactive | jq -r '.[0].appVersion // "none"')" + # Local builds (the android and ios jobs) record each build as a tag; + # EAS records still count for builds made before that. + tag_prefix="" + if [ "$platform" = "android" ]; then + tag_prefix="mobile-android-build" + elif [ "$IOS_LOCAL_BUILDS" = "true" ]; then + tag_prefix="mobile-ios-build/$profile" + fi + if [ -n "$tag_prefix" ]; then + if [ -n "$(git tag -l "$tag_prefix/*/$hash")" ]; then + finished=$((finished + 1)) + fi + if [ -n "$(git tag -l "$tag_prefix/$version/*")" ]; then + latest_version="$version" + fi + fi if [ "$finished" -gt 0 ]; then if eas update \ @@ -363,6 +431,27 @@ jobs: fi echo "$platform: $reason — building" + if [ "$platform" = "android" ]; then + { + echo "android_build=true" + echo "android_fingerprint=$hash" + echo "android_version=$version" + echo "android_sha=$(git rev-parse HEAD)" + } >> "$GITHUB_OUTPUT" + echo ":building_construction: android: building and submitting $version in the Android store build job ($reason)" >> "$GITHUB_STEP_SUMMARY" + continue + fi + if [ "$platform" = "ios" ] && [ "$IOS_LOCAL_BUILDS" = "true" ]; then + { + echo "ios_build=true" + echo "ios_profile=$profile" + echo "ios_fingerprint=$hash" + echo "ios_version=$version" + echo "ios_sha=$(git rev-parse HEAD)" + } >> "$GITHUB_OUTPUT" + echo ":building_construction: ios/$profile: building and submitting $version in the iOS store build job ($reason)" >> "$GITHUB_STEP_SUMMARY" + continue + fi if eas build --platform "$platform" --profile "$profile" --auto-submit --non-interactive --no-wait; then echo ":building_construction: $platform: scheduled production build and submission for $version ($reason)" >> "$GITHUB_STEP_SUMMARY" else @@ -386,3 +475,368 @@ jobs: node scripts/upload-sourcemaps.mts ota "$dir" fi done + + # `eas build --local` on Blacksmith, ARM-only, without release lint (see the + # production profile in eas.json), with Gradle, ccache, and pnpm state on a + # sticky disk. Its own concurrency group keeps later OTAs from waiting on it. + android: + name: Android store build + needs: production + # Skipped when the production job failed: no store build after a failed OTA. + if: needs.production.outputs.android_build == 'true' + runs-on: blacksmith-32vcpu-ubuntu-2404 + timeout-minutes: 60 + concurrency: + group: mobile-android-store-build + cancel-in-progress: false + permissions: + contents: write # pushes the build record tag + env: + APP_VARIANT: production + NODE_OPTIONS: --max-old-space-size=8192 + EXPO_TOKEN: ${{ secrets.EXPO_PRODUCTION_TOKEN }} + BUILD_TAG: mobile-android-build/${{ needs.production.outputs.android_version }}/${{ needs.production.outputs.android_fingerprint }} + steps: + - name: Checkout + uses: actions/checkout@v6 + with: + ref: ${{ needs.production.outputs.android_sha }} + fetch-depth: 0 + + # Dependency scripts and the build run in this job; only Record build gets a token. + # persist-credentials: false would clean up via `git submodule foreach`, which fails on .repos gitlinks. + - name: Drop checkout credentials + run: | + git config --local --unset-all http.https://github.com/.extraheader || true + { git config --local --name-only --get-regexp '^includeif\.' || true; } | while read -r key; do + path=$(git config --local --get "$key") + git config --local --unset-all "$key" + case "$path" in "$RUNNER_TEMP"/*) rm -f "$path" ;; esac + done + if git config --local --get-regexp '^(http\..*\.extraheader|includeif\.)'; then + echo "::error::Checkout credentials are still configured" + exit 1 + fi + + # A queued run can follow a build of the same fingerprint; manual runs always build. + - id: record + name: Check build record + env: + FORCE: ${{ github.event_name == 'workflow_dispatch' }} + run: | + if [ "$FORCE" != "true" ] && [ -n "$(git tag -l "$BUILD_TAG")" ]; then + echo "built=true" >> "$GITHUB_OUTPUT" + echo ":white_check_mark: android: \`$BUILD_TAG\` was built by an earlier run" >> "$GITHUB_STEP_SUMMARY" + fi + + - name: Prepare build directories + if: steps.record.outputs.built != 'true' + run: | + cache="$HOME/.cache/t3-android-store" + { + echo "GRADLE_USER_HOME=$cache/gradle" + echo "CCACHE_DIR=$cache/ccache" + echo "CCACHE_MAXSIZE=5G" + # Gradle installs the NDK fresh each run, so key on the compiler's version, not its mtime. + echo "CCACHE_COMPILERCHECK=%compiler% -v" + echo "pnpm_config_store_dir=$cache/pnpm-store" + # eas build --local copies the project here. A fixed path keeps + # ccache keys stable, and the same disk as the store lets pnpm hardlink. + echo "EAS_LOCAL_BUILD_WORKINGDIR=$cache/eas-build" + echo "CCACHE_BASEDIR=$cache/eas-build" + # CMake (3.17+) reads these, and AGP passes Gradle's environment through. + echo "CMAKE_C_COMPILER_LAUNCHER=ccache" + echo "CMAKE_CXX_COMPILER_LAUNCHER=ccache" + } >> "$GITHUB_ENV" + + - name: Mount build cache + if: steps.record.outputs.built != 'true' + uses: useblacksmith/stickydisk@v1 + with: + key: ${{ github.repository }}-android-store-build + path: ~/.cache/t3-android-store + + - name: Configure Gradle and ccache + if: steps.record.outputs.built != 'true' + run: | + mkdir -p "$GRADLE_USER_HOME" "$CCACHE_DIR" + rm -rf "$EAS_LOCAL_BUILD_WORKINGDIR" + # No daemon: nothing should hold the sticky disk open when it is committed. + printf 'org.gradle.caching=true\norg.gradle.daemon=false\n' > "$GRADLE_USER_HOME/gradle.properties" + sudo sed -i 's|http://|https://|g' /etc/apt/blacksmith-ubuntu-mirrors.txt /etc/apt/sources.list.d/ubuntu.sources + sudo apt-get update -qq && sudo apt-get install -y -qq ccache + ccache --zero-stats + + - name: Setup Node + if: steps.record.outputs.built != 'true' + uses: actions/setup-node@v6 + with: + node-version-file: package.json + + - name: Enable corepack + if: steps.record.outputs.built != 'true' + run: | + npm install --global corepack@latest + corepack enable + pnpm --version + + # EAS commands load app.config.ts, whose plugins must resolve. Keep install + # scripts: some download native assets that the build's fingerprint check hashes. + - name: Install mobile dependencies + if: steps.record.outputs.built != 'true' + run: pnpm install --frozen-lockfile --prefer-offline --filter=@t3tools/mobile... + + - name: Setup Java + if: steps.record.outputs.built != 'true' + uses: actions/setup-java@v5 + with: + distribution: temurin + java-version: 17 + + - name: Setup EAS + if: steps.record.outputs.built != 'true' + uses: expo/expo-github-action@v8 + with: + eas-version: latest + token: ${{ secrets.EXPO_PRODUCTION_TOKEN }} + packager: npm + + - name: Pull production environment variables + if: steps.record.outputs.built != 'true' + working-directory: apps/mobile + run: | + eas env:pull production --non-interactive + node --input-type=module <<'NODE' + import { appendFileSync, readFileSync } from "node:fs"; + import { parseEnv } from "node:util"; + const env = parseEnv(readFileSync(".env.local", "utf8")); + const googleServicesFile = env.T3CODE_ANDROID_GOOGLE_SERVICES_FILE; + if (!googleServicesFile) { + throw new Error("T3CODE_ANDROID_GOOGLE_SERVICES_FILE is missing from the production environment"); + } + appendFileSync(process.env.GITHUB_ENV, `T3CODE_ANDROID_GOOGLE_SERVICES_FILE=${googleServicesFile}\n`); + NODE + + - name: Build + if: steps.record.outputs.built != 'true' + working-directory: apps/mobile + env: + # EAS secret variables don't reach local builds; the eas-build-on-success + # hook needs these to upload the bundle's source map. + MOBILE_SOURCEMAPS_R2: ${{ secrets.MOBILE_SOURCEMAPS_R2 }} + run: | + eas build --local --platform android --profile production --non-interactive --output "$RUNNER_TEMP/t3code.aab" + abis="$(unzip -Z1 "$RUNNER_TEMP/t3code.aab" | sed -n 's|^base/lib/\([^/]*\)/.*|\1|p' | sort -u | paste -sd, -)" + { + echo "### Android $BUILD_TAG" + echo "- ABIs: \`$abis\`" + echo '```' + ccache --show-stats + echo '```' + } >> "$GITHUB_STEP_SUMMARY" + if [ "$abis" != "arm64-v8a,armeabi-v7a" ]; then + echo "::error::Expected only ARM native libraries in the AAB, found $abis" + exit 1 + fi + + # A --path submission doesn't keep its archive on EAS; this is the only downloadable copy. + - name: Keep the AAB + if: steps.record.outputs.built != 'true' + uses: actions/upload-artifact@v7 + with: + name: android-store-aab-${{ github.run_id }}-${{ github.run_attempt }} + path: ${{ runner.temp }}/t3code.aab + retention-days: 14 + if-no-files-found: error + + - name: Submit to Google Play + if: steps.record.outputs.built != 'true' + working-directory: apps/mobile + run: eas submit --platform android --profile production --path "$RUNNER_TEMP/t3code.aab" --non-interactive + + # The tag means "delivered", so it follows the submit. If it can't be + # pushed, the next merge rebuilds with a new build number; nothing is lost. + - name: Record build + if: steps.record.outputs.built != 'true' + env: + GH_TOKEN: ${{ github.token }} + run: | + sha="$(git rev-parse HEAD)" + # GitHub occasionally fails ref writes server-side ("fatal error in commit_refs"). + for attempt in 1 2 3; do + if gh api "repos/$GITHUB_REPOSITORY/git/ref/tags/$BUILD_TAG" >/dev/null 2>&1; then + echo "$BUILD_TAG exists." + break + fi + if gh api "repos/$GITHUB_REPOSITORY/git/refs" -f ref="refs/tags/$BUILD_TAG" -f sha="$sha" >/dev/null; then + break + fi + if [ "$attempt" = 3 ]; then + echo "::error::Submitted, but could not create $BUILD_TAG; the next merge will rebuild." + exit 1 + fi + sleep $((attempt * 15)) + done + echo ":rocket: android: submitted and recorded \`$BUILD_TAG\`" >> "$GITHUB_STEP_SUMMARY" + + # Off unless the MOBILE_IOS_LOCAL_BUILDS repository variable is `true`; the + # production job only sets ios_build then. Store binaries need the final + # Xcode 27.1; MOBILE_IOS_ALLOW_XCODE_RC=true accepts the release candidate. + # No sticky disk: it mounts ext4 and only works on Linux runners. + ios: + name: iOS store build + needs: production + # Skipped when the production job failed: no store build after a failed OTA. + if: needs.production.outputs.ios_build == 'true' + runs-on: blacksmith-12vcpu-macos-27 + timeout-minutes: 60 + concurrency: + group: mobile-ios-store-build + cancel-in-progress: false + permissions: + contents: write # pushes the build record tag + env: + APP_VARIANT: production + NODE_OPTIONS: --max-old-space-size=8192 + EXPO_TOKEN: ${{ secrets.EXPO_PRODUCTION_TOKEN }} + PROFILE: ${{ needs.production.outputs.ios_profile }} + BUILD_TAG: mobile-ios-build/${{ needs.production.outputs.ios_profile }}/${{ needs.production.outputs.ios_version }}/${{ needs.production.outputs.ios_fingerprint }} + ALLOW_XCODE_RC: ${{ vars.MOBILE_IOS_ALLOW_XCODE_RC == 'true' }} + steps: + - name: Checkout + uses: actions/checkout@v6 + with: + ref: ${{ needs.production.outputs.ios_sha }} + fetch-depth: 0 + + # Dependency scripts and the build run in this job; only Record build gets a token. + # persist-credentials: false would clean up via `git submodule foreach`, which fails on .repos gitlinks. + - name: Drop checkout credentials + run: | + git config --local --unset-all http.https://github.com/.extraheader || true + { git config --local --name-only --get-regexp '^includeif\.' || true; } | while read -r key; do + path=$(git config --local --get "$key") + git config --local --unset-all "$key" + case "$path" in "$RUNNER_TEMP"/*) rm -f "$path" ;; esac + done + if git config --local --get-regexp '^(http\..*\.extraheader|includeif\.)'; then + echo "::error::Checkout credentials are still configured" + exit 1 + fi + + # A queued run can follow a build of the same fingerprint; manual runs always build. + - id: record + name: Check build record + env: + FORCE: ${{ github.event_name == 'workflow_dispatch' }} + run: | + if [ "$FORCE" != "true" ] && [ -n "$(git tag -l "$BUILD_TAG")" ]; then + echo "built=true" >> "$GITHUB_OUTPUT" + echo ":white_check_mark: ios: \`$BUILD_TAG\` was built by an earlier run" >> "$GITHUB_STEP_SUMMARY" + fi + + - name: Select Xcode 27.1 + if: steps.record.outputs.built != 'true' + run: | + xcode=/Applications/Xcode_27.1.app + if [ ! -d "$xcode" ] && [ "$ALLOW_XCODE_RC" = "true" ]; then + xcode=/Applications/Xcode_27.1_Release_Candidate.app + fi + if [ ! -d "$xcode" ]; then + ls -d /Applications/Xcode*.app + echo "::error::Xcode 27.1 (final) is not on this runner. Set the MOBILE_IOS_ALLOW_XCODE_RC repository variable to true to build store binaries with the release candidate." + exit 1 + fi + sudo xcode-select -s "$xcode/Contents/Developer" + xcodebuild -version + echo "- Xcode: \`$(basename "$xcode")\`" >> "$GITHUB_STEP_SUMMARY" + + # EAS's credentials step fails to import the distribution certificate without it. + - name: Install Apple WWDR G3 intermediate + if: steps.record.outputs.built != 'true' + run: | + curl -fsSL -o "$RUNNER_TEMP/AppleWWDRCAG3.cer" https://www.apple.com/certificateauthority/AppleWWDRCAG3.cer + sudo security add-certificates -k /Library/Keychains/System.keychain "$RUNNER_TEMP/AppleWWDRCAG3.cer" || true + security find-certificate -c "Apple Worldwide Developer Relations" /Library/Keychains/System.keychain | grep -m1 labl + + - name: Setup Node + if: steps.record.outputs.built != 'true' + uses: actions/setup-node@v6 + with: + node-version-file: package.json + + - name: Enable corepack + if: steps.record.outputs.built != 'true' + run: | + npm install --global corepack@latest + corepack enable + pnpm --version + + # EAS commands load app.config.ts, whose plugins must resolve. Keep install + # scripts: some download native assets that the build's fingerprint check hashes. + - name: Install mobile dependencies + if: steps.record.outputs.built != 'true' + run: pnpm install --frozen-lockfile --prefer-offline --filter=@t3tools/mobile... + + - name: Setup EAS + if: steps.record.outputs.built != 'true' + uses: expo/expo-github-action@v8 + with: + eas-version: latest + token: ${{ secrets.EXPO_PRODUCTION_TOKEN }} + packager: npm + + - name: Build tools + if: steps.record.outputs.built != 'true' + run: | + command -v pod || sudo gem install cocoapods --no-document + command -v fastlane || brew install fastlane + pod --version + fastlane --version | tail -1 + + - name: Pull production environment variables + if: steps.record.outputs.built != 'true' + working-directory: apps/mobile + run: eas env:pull production --non-interactive + + - name: Build + if: steps.record.outputs.built != 'true' + working-directory: apps/mobile + env: + # EAS secret variables don't reach local builds; the eas-build-on-success + # hook needs these to upload the bundle's source map. + MOBILE_SOURCEMAPS_R2: ${{ secrets.MOBILE_SOURCEMAPS_R2 }} + run: | + # A real path: /var -> /private/var symlinks break the expo-widgets bundle step. + export EAS_LOCAL_BUILD_WORKINGDIR="$RUNNER_TEMP/eas-build" + eas build --local --platform ios --profile "$PROFILE" --non-interactive --output "$RUNNER_TEMP/t3code.ipa" + + - name: Submit to App Store Connect + if: steps.record.outputs.built != 'true' + working-directory: apps/mobile + run: eas submit --platform ios --profile "$PROFILE" --path "$RUNNER_TEMP/t3code.ipa" --non-interactive + + # The tag means "delivered", so it follows the submit. If it can't be + # pushed, the next merge rebuilds with a new build number; nothing is lost. + - name: Record build + if: steps.record.outputs.built != 'true' + env: + GH_TOKEN: ${{ github.token }} + run: | + sha="$(git rev-parse HEAD)" + # GitHub occasionally fails ref writes server-side ("fatal error in commit_refs"). + for attempt in 1 2 3; do + if gh api "repos/$GITHUB_REPOSITORY/git/ref/tags/$BUILD_TAG" >/dev/null 2>&1; then + echo "$BUILD_TAG exists." + break + fi + if gh api "repos/$GITHUB_REPOSITORY/git/refs" -f ref="refs/tags/$BUILD_TAG" -f sha="$sha" >/dev/null; then + break + fi + if [ "$attempt" = 3 ]; then + echo "::error::Submitted, but could not create $BUILD_TAG; the next merge will rebuild." + exit 1 + fi + sleep $((attempt * 15)) + done + echo ":rocket: ios: submitted and recorded \`$BUILD_TAG\`" >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/mobile-showcase-screenshots.yml b/.github/workflows/mobile-showcase-screenshots.yml index c02be425c1ab7..53b0a74a1f5aa 100644 --- a/.github/workflows/mobile-showcase-screenshots.yml +++ b/.github/workflows/mobile-showcase-screenshots.yml @@ -88,7 +88,7 @@ jobs: axe --version - name: Install foldable simulator controls - run: npm install --global agent-device@0.21.25 + run: npm install --global agent-device@0.21.36 - name: Capture iOS showcase run: pnpm screenshots:mobile --platform ios --appearance "${{ inputs.appearance }}" --theme "${{ inputs.theme }}" diff --git a/.github/workflows/pr-limits.yml b/.github/workflows/pr-limits.yml new file mode 100644 index 0000000000000..75943062735cd --- /dev/null +++ b/.github/workflows/pr-limits.yml @@ -0,0 +1,72 @@ +name: PR Limits + +# Deterministic PR rules from CONTRIBUTING.md: the open PR limit, the conflict +# window, and the 72 hour close window that follows a triage notice. +# +# Nothing is labeled, commented on, or closed until the repository variable +# PR_LIMITS_ENFORCE is "true". Until then every run logs what it would do. A +# sweep reads the variable once when it starts; to stop one, cancel the run. + +on: + pull_request_target: + types: [opened, reopened] + schedule: + - cron: "23 5 * * *" + workflow_dispatch: + inputs: + dry_run: + description: Log what the sweep would do without labeling, commenting, or closing. + type: boolean + default: true + +permissions: + contents: read + issues: write + pull-requests: write + +jobs: + open-limit: + name: Open PR limit + if: github.event_name == 'pull_request_target' + runs-on: ubuntu-24.04 + steps: + # pull_request_target has a write token. Load the script from the default + # branch and never run PR code. Cone mode also checks out the files beside + # .github/scripts, which is where TRIAGE_EXEMPTIONS.td and VOUCHED.td live. + - name: Checkout trusted scripts + uses: actions/checkout@v6 + with: + ref: ${{ github.event.repository.default_branch }} + sparse-checkout: .github/scripts + + - name: Close PRs over the limit + uses: actions/github-script@v8 + env: + DRY_RUN: ${{ vars.PR_LIMITS_ENFORCE != 'true' }} + with: + script: | + const limits = require("./.github/scripts/pr-limits.cjs"); + await limits.enforceOpenLimit({ github, context, core, dryRun: process.env.DRY_RUN === "true" }); + + sweep: + name: Conflict and close windows + if: github.event_name != 'pull_request_target' + runs-on: ubuntu-24.04 + concurrency: + group: pr-limits-sweep + cancel-in-progress: false + steps: + - name: Checkout trusted scripts + uses: actions/checkout@v6 + with: + ref: ${{ github.event.repository.default_branch }} + sparse-checkout: .github/scripts + + - name: Request rebases and close what went unanswered + uses: actions/github-script@v8 + env: + DRY_RUN: ${{ vars.PR_LIMITS_ENFORCE != 'true' || (github.event_name == 'workflow_dispatch' && inputs.dry_run) }} + with: + script: | + const limits = require("./.github/scripts/pr-limits.cjs"); + await limits.sweep({ github, context, core, dryRun: process.env.DRY_RUN === "true" }); diff --git a/.github/workflows/pr-size.yml b/.github/workflows/pr-size.yml index af557dff62df8..cb0c9637fcef7 100644 --- a/.github/workflows/pr-size.yml +++ b/.github/workflows/pr-size.yml @@ -293,3 +293,55 @@ jobs: core.info( `PR #${issueNumber}: ${nonTestChangedLines} non-test lines, ${testChangedLines} test lines, ${changedLines} effective lines -> ${nextLabelName} (${classification})`, ); + # Reads the script from the base checkout above; the PR's own commits are only diffed. + - name: Report test suite growth + uses: actions/github-script@v8 + with: + script: | + const { execFileSync } = require("node:child_process"); + const { MARKER, countTestDelta, render } = require( + `${process.env.GITHUB_WORKSPACE}/.github/scripts/test-delta.cjs`, + ); + + const issueNumber = context.payload.pull_request.number; + const diff = execFileSync( + "git", + ["diff", "-U0", `${context.payload.pull_request.base.sha}...refs/remotes/pr-size/${issueNumber}`], + { encoding: "utf8", maxBuffer: 256 * 1024 * 1024 }, + ); + const delta = countTestDelta(diff); + const body = render(delta); + core.info(`PR #${issueNumber}: test cases ${JSON.stringify(delta.kinds)}, net ${delta.net}`); + + const comments = await github.paginate(github.rest.issues.listComments, { + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: issueNumber, + per_page: 100, + }); + const existing = comments.find( + (comment) => comment.user?.type === "Bot" && comment.body?.startsWith(MARKER), + ); + if (body && existing) { + if (existing.body !== body) { + await github.rest.issues.updateComment({ + owner: context.repo.owner, + repo: context.repo.repo, + comment_id: existing.id, + body, + }); + } + } else if (body) { + await github.rest.issues.createComment({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: issueNumber, + body, + }); + } else if (existing) { + await github.rest.issues.deleteComment({ + owner: context.repo.owner, + repo: context.repo.repo, + comment_id: existing.id, + }); + } diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index c56c76be7e0e7..8827876b3829f 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -300,11 +300,15 @@ jobs: # Web browser tests time out when they share a runner with the other package # suites: Chromium gets too little CPU and diff workers miss their 10s start budget. test_web: - name: Release web tests + name: Release web tests ${{ matrix.shard }} needs: [preflight] if: ${{ !failure() && !cancelled() && needs.preflight.result == 'success' }} runs-on: blacksmith-8vcpu-ubuntu-2404 timeout-minutes: 10 + strategy: + fail-fast: false + matrix: + shard: [1, 2] steps: - name: Checkout uses: actions/checkout@v6 @@ -344,10 +348,11 @@ jobs: fi - name: Test - run: vp run --filter @t3tools/web test + run: vp run --filter @t3tools/web test --shard ${{ matrix.shard }}/${{ strategy.job-total }} # Browser tests run source modules; this boots the production chunk graph. - name: Boot the built web client + if: matrix.shard == 1 run: | vp run --filter @t3tools/web build node apps/web/scripts/verify-built-boot.ts @@ -1482,7 +1487,18 @@ jobs: git add apps/server/package.json apps/desktop/package.json apps/web/package.json packages/contracts/package.json pnpm-lock.yaml git commit --no-verify -m "chore(release): prepare $RELEASE_TAG" - git push origin HEAD:main + + # main moves while this job runs, so a rejected push rebases and retries. + for attempt in 1 2 3 4 5; do + if git push origin HEAD:main; then + exit 0 + fi + echo "Push rejected (attempt $attempt). Rebasing onto latest main." + git pull --rebase origin main + done + + echo "Could not push the version bump after 5 attempts." >&2 + exit 1 announce_discord: name: Announce release on Discord diff --git a/.gitignore b/.gitignore index b6b793e020ef1..85b7c5b4bf02f 100644 --- a/.gitignore +++ b/.gitignore @@ -38,6 +38,7 @@ apps/mobile/.android-cache/ artifacts/app-store/screenshots/ artifacts/mobile-e2e/ artifacts/mobile-behavior/ +scripts/.e2e/ .github/pr-assets/ native/**/target/ node_modules/ diff --git a/.macroscope/check-run-agents/ui-consistency.md b/.macroscope/check-run-agents/ui-consistency.md index 905823d1d1a9e..4326d4e8391b7 100644 --- a/.macroscope/check-run-agents/ui-consistency.md +++ b/.macroscope/check-run-agents/ui-consistency.md @@ -44,7 +44,7 @@ The reference for how this codebase wants UI built is the composer banner system - `ComposerBanner.tsx` exports one object of small slot components (`Root`, `Row`, `Icon`, `Content`, `Actions`, `Dismiss`, `Children`, and so on). Each slot owns its own Tailwind classes, exposes a `data-slot` attribute, takes `className` and spreads the rest of its props, and uses `cn` so callers can adjust without overriding. Variants and density are props on `Root`, not ad hoc class strings at call sites. Where a slot needs a real control it renders `Button` from `ui/` rather than a styled `