Files
CV_Site_Nuxt/OKF_Nuxt/decisions/ai-review-setup.md
Noel Sabia c626cc9c1d
PR Check / Security scan (Snyk) (pull_request) Skipped
PR Check / Lint + typecheck (pull_request) Canceled after 45s
PR Check / Secret scan (gitleaks) (pull_request) Canceled after 0s
PR Check / Dependency audit (pull_request) Canceled after 0s
PR Check / Dockerfile lint (droast) (pull_request) Canceled after 0s
feat(ci): replace opencode review with PR-Agent
Run PR-Agent as a pinned webhook server on the Gitea Docker network and review on PR open plus slash commands. The per-PR opencode installer and verdict-posting step are removed.

The server-side .env stays outside the repository; deploy/pr-agent/ holds the compose definition and operator notes.
2026-10-30 21:00:53 +01:00

5.1 KiB

type, title, description, tags, status, generated, verified, sources
type title description tags status generated verified sources
Decision AI code review setup The review job runs opencode with a cheap medium-effort opencode-go model, env-only auth, a read-only permission set, and posts only reviews that end with a verdict line.
decision
ci
ai
opencode
security
deprecated
by at
opencode-go/glm-5.3-flash 2026-10-18T16:05:00Z
by at
human:noel not yet
id resource title
pr-checker .gitea/workflows/pr-checker.yml PR Check workflow, review job
id resource title
go-plan https://opencode.ai/v2/docs/console/go/ OpenCode Go plan documentation

Superseded by PR-Agent for Gitea reviews on 2026-10-30. The opencode job was removed from pr-checker.yml; this document remains as the record of the earlier setup and the failures it fixed.

Decision

The review job installs opencode v2 from the dedicated v2 installer, authenticates with OPENCODE_API_KEY from the environment only, runs opencode-go/claude-haiku-5-5#medium (covered by the Go plan subscription, no per-token cost), denies edit/shell/webfetch/websearch via a global permission config written outside the checkout, and posts the review as a PR comment only if the output ends with a VERDICT: line.

Why each piece

Piece Why
Provider prefix opencode-go, not opencode opencode is the pay-per-call Console gateway; the Go plan lives under its own provider id. The wrong prefix failed every run with "Insufficient account funds".
Model claude-haiku-5-5#medium Cheap per token and cheap in the usage window, with a real #medium reasoning level. deepseek-v4.1-flash only exposes low/high/max, so its reviews always paid for the maximum effort.
The v2 installer (/v2/install) The classic installer ships OpenCode 1.x (v1.19.3 on arm64), which cannot resolve model variants and failed with ProviderModelNotFoundError for #max.
Env-only auth, no opencode.json in the checkout A config file in the workspace holds the API key in plaintext where the review agent can read it; a malicious PR diff could prompt-inject the agent into quoting the key in the publicly visible review comment. Environment variables are invisible to file reads.
Deny write tools via a global config outside the checkout Headless runs cannot answer permission prompts; denials prevent both hangs and repo modification. The config file itself holds no secret, so its location leaks nothing.
Verdict line as the posting gate opencode can exit non-zero after a fully completed review (observed: benign permission denials). A completed review always ends with a verdict line; error dumps never do. The verdict, not the exit code, decides whether a comment is posted.
--print-logs Provider and auth errors otherwise die inside the captured output file, invisible in the job log.
Diff passed as a file, lockfile excluded The diff as a CLI argument exceeded MAX_ARG_STRLEN (128 KiB per argument; the diff was 190 KB, mostly pnpm-lock.yaml). The agent reads .pr-review-diff.patch with its file tools.

The review prompt

The prompt names the stack (Nuxt 4 frontend, Go backends to come), lists what the other CI jobs already catch so the model does not repeat them, and focuses on SSR/hydration pitfalls, reactivity bugs, injection and authz gaps, runtime-config leaks, and Go error/concurrency hygiene. It prescribes the output format ([BLOCKER]/[WARN]/[NIT] file:line) and derives the verdict from the findings, so the gate and the prompt agree.

Evidence

  • First real review on PR #5 (2026-10-18): produced five findings, including a genuine regression in the workflow itself (a lost chmod +x on the droast binary), verified upstream release assets and the v2 permission schema live, and returned VERDICT: REQUEST_CHANGES.
  • Fresh-container replica test (node:22-bookworm, key-only auth) reproduced the CI failure exactly, then confirmed the v2 + #max fix before it was pushed.
  • Model history: deepseek-v4-pro#max was replaced by deepseek-v4.1-flash#max after two consecutive endpoint outages, then by claude-haiku-5-5#medium (2026-10-20) to stop paying maximum reasoning effort on every review. The flash model had no #medium variant to fall back to.

Not verified

  • Whether #medium still resolves after a future CLI upgrade — the variant list comes from the model catalog, not a contract.
  • Whether the smaller model still catches the same class of findings as the first flash review. The PR #14 findings are the baseline; re-run the eval on the next few PRs.
  • Long-run behaviour of Go plan usage windows under real review frequency, now that the model is cheaper.

Revisit when

  • A PR review fails again with a new error class — check the provider prefix, the CLI version, and the usage limits first; they caused every failure so far.
  • A flash-tier model gains a #medium variant and beats the current model on the PR #14 eval set — the present pick is a cost/quality compromise, not a permanent one.
  • Go Plus or pay-as-you-go balance becomes relevant.