Proactive and schedule-driven: one independent QA pass per PR revision found in a sweep, handled as its own unit so a failure on one never blocks the others. The only state carried forward on purpose is the landed edge-case ledger — an in-sandbox file alone doesn't survive on its own.

Step 0 — Discover and orient

bash

# Read the known-issues ledger before touching any branch — edge cases that
# have bitten before, flows that are critical, flaky tests already on file.
cat memory/qa-known-issues.md 2>/dev/null || echo "(no known issues yet)"

# Discover every PR opened or pushed to since the last sweep.
gh pr list --repo {{target_repo}} --state open \
  --json number,headRefOid,statusCheckRollup,updatedAt

Filter out any PR whose current headRefOid already carries a qa-agent check in statusCheckRollup — don't re-run against a revision you've already reported on. Work through what's left one PR at a time, each as its own independent unit: a failure on one PR is a self-contained result and never blocks or contaminates QA of the others in this sweep.

For each remaining PR, confirm before checkout:

bash

gh pr view --repo {{target_repo}} <PR_NUMBER> --json headRefOid,statusCheckRollup

Step 1 — Check out the branch clean

bash

rm -rf /workspace/pr-<PR_NUMBER>
git clone --filter=blob:none https://github.com/{{target_repo}}.git /workspace/pr-<PR_NUMBER>
cd /workspace/pr-<PR_NUMBER>
gh pr checkout <PR_NUMBER>
<install command for the stack>   # e.g. pnpm install --frozen-lockfile / npm ci / pip install -r requirements.txt

One clean clone per PR revision, keyed by PR number. This session persists across sweeps, so remove any stale checkout for this PR number first — never reuse a prior revision's working tree, and never let one PR's checkout leak into another's.

Step 2 — Run the full suite

bash

cd /workspace/pr-<PR_NUMBER>
<unit test command>        2>&1 | tee /tmp/qa-unit.log
<integration test command> 2>&1 | tee /tmp/qa-integration.log
<e2e test command>          2>&1 | tee /tmp/qa-e2e.log

Capture full failure output — stack trace, failing assertion, the exact command — not just a pass/fail count. Cross-reference qa-known-issues.md for flows that have broken before and confirm this run actually exercised them.

Step 3 — Deploy the change to the test environment

bash

<deploy command, e.g. an internal deploy script / flyctl / vercel> \
  --env test --ref <PR head SHA>

Stand up an ephemeral instance of exactly this branch. Wait for the deploy to report healthy before exercising anything against it — a QA run against a half-started deploy is a fail, not a skip.

Step 4 — Exercise the change end-to-end

Hit the new or changed behavior directly against the deployed instance — the actual endpoint, page, or flow the PR touches — not just localhost:

bash

curl -sf https://<test-deploy-host>/<changed-path> -o /tmp/qa-response.json

Then repeat the critical checks through the edge:

bash

curl -sI https://<edge-fronted-test-host>/<changed-path>   # routing, headers, caching, redirects

This is a plain, unauthenticated request against the public HTTPS host — no edge-provider credential or connector is needed for it. A route that works direct-to-origin but breaks through the edge (a caching rule, a redirect, a header transform) is a QA failure, not a pass.

Step 5 — Decide pass or fail

Result	Criteria
Pass	Suite green, deploy healthy, exercised behavior matches expectation, edge checks clean
Fail	Any suite failure, a failed or unhealthy deploy, exercised behavior wrong, or an edge-only regression
Flag as flaky	A test fails, then passes on an isolated re-run with no code change — report it as flaky with both runs' logs; don't just retry until it's green

Step 6 — Post the result

bash

gh api repos/{{target_repo}}/statuses/<PR_head_SHA> \
  -f state=<success|failure> -f context="qa-agent" -f description="<one-line result>"
gh pr comment --repo {{target_repo}} <PR_NUMBER> --body "<pass/fail summary>"

Pass: a short green summary — suite results, what was exercised, edge checks run. Fail: the failing command, the full log excerpt, and exact steps to reproduce against the test deploy. Exactly one result comment per PR revision.

Step 7 — Tear down and land the ledger update

bash

<teardown command for the ephemeral test deploy>

Tear down the ephemeral deploy for this PR so it doesn't linger — the session's own sandbox stays up for the next sweep; only the test-environment deploy is disposable.

If this run surfaced a genuinely new edge case (not already in the ledger) — a bug that would've reached staging, a flow no prior run checked — append it to memory/qa-known-issues.md with the PR link and a one-line description, then land it durably:

bash

git add memory/qa-known-issues.md
git commit -m "docs(qa): record edge case from PR #<PR_NUMBER>"

Open (and self-merge) a scoped change request for just this ledger update via the project.cr.open action — an edit that only lives in the sandbox never survives on its own; only a landed change request does. Then move to the next PR in this sweep's batch, if any, with a clean checkout (Step 1).

PR QA

Step 0 — Discover and orient

Step 1 — Check out the branch clean

Step 2 — Run the full suite

Step 3 — Deploy the change to the test environment

Step 4 — Exercise the change end-to-end

Step 5 — Decide pass or fail

Step 6 — Post the result

Step 7 — Tear down and land the ledger update

Related skills