Files
CV_Site_Nuxt/OKF_Nuxt/decisions/ai-review-setup.md
Noel Sabia c626cc9c1d
PR Check / Security scan (Snyk) (pull_request) Skipped
PR Check / Lint + typecheck (pull_request) Canceled after 45s
PR Check / Secret scan (gitleaks) (pull_request) Canceled after 0s
PR Check / Dependency audit (pull_request) Canceled after 0s
PR Check / Dockerfile lint (droast) (pull_request) Canceled after 0s
feat(ci): replace opencode review with PR-Agent
Run PR-Agent as a pinned webhook server on the Gitea Docker network and review on PR open plus slash commands. The per-PR opencode installer and verdict-posting step are removed.

The server-side .env stays outside the repository; deploy/pr-agent/ holds the compose definition and operator notes.
2026-10-30 21:00:53 +01:00

87 lines
5.1 KiB
Markdown

---
type: Decision
title: AI code review setup
description: The review job runs opencode with a cheap medium-effort opencode-go model, env-only auth, a read-only permission set, and posts only reviews that end with a verdict line.
tags: [decision, ci, ai, opencode, security]
status: deprecated
generated: { by: opencode-go/glm-5.3-flash, at: 2026-10-18T16:05:00Z }
verified: { by: human:noel, at: not yet }
sources:
- id: pr-checker
resource: .gitea/workflows/pr-checker.yml
title: PR Check workflow, review job
- id: go-plan
resource: https://opencode.ai/v2/docs/console/go/
title: OpenCode Go plan documentation
---
> Superseded by [PR-Agent for Gitea reviews](pr-agent.md) on 2026-10-30.
> The opencode job was removed from `pr-checker.yml`; this document remains
> as the record of the earlier setup and the failures it fixed.
# Decision
The review job installs opencode v2 from the dedicated v2 installer,
authenticates with `OPENCODE_API_KEY` from the environment only, runs
`opencode-go/claude-haiku-5-5#medium` (covered by the Go plan subscription,
no per-token cost), denies `edit`/`shell`/`webfetch`/`websearch` via a
global permission config written outside the checkout, and posts the
review as a PR comment only if the output ends with a `VERDICT:` line.
# Why each piece
| Piece | Why |
|---|---|
| Provider prefix `opencode-go`, not `opencode` | `opencode` is the pay-per-call Console gateway; the Go plan lives under its own provider id. The wrong prefix failed every run with "Insufficient account funds". |
| Model `claude-haiku-5-5#medium` | Cheap per token and cheap in the usage window, with a real `#medium` reasoning level. `deepseek-v4.1-flash` only exposes low/high/max, so its reviews always paid for the maximum effort. |
| The v2 installer (`/v2/install`) | The classic installer ships OpenCode 1.x (v1.19.3 on arm64), which cannot resolve model variants and failed with `ProviderModelNotFoundError` for `#max`. |
| Env-only auth, no `opencode.json` in the checkout | A config file in the workspace holds the API key in plaintext where the review agent can read it; a malicious PR diff could prompt-inject the agent into quoting the key in the publicly visible review comment. Environment variables are invisible to file reads. |
| Deny write tools via a global config outside the checkout | Headless runs cannot answer permission prompts; denials prevent both hangs and repo modification. The config file itself holds no secret, so its location leaks nothing. |
| Verdict line as the posting gate | `opencode` can exit non-zero after a fully completed review (observed: benign permission denials). A completed review always ends with a verdict line; error dumps never do. The verdict, not the exit code, decides whether a comment is posted. |
| `--print-logs` | Provider and auth errors otherwise die inside the captured output file, invisible in the job log. |
| Diff passed as a file, lockfile excluded | The diff as a CLI argument exceeded `MAX_ARG_STRLEN` (128 KiB per argument; the diff was 190 KB, mostly `pnpm-lock.yaml`). The agent reads `.pr-review-diff.patch` with its file tools. |
# The review prompt
The prompt names the stack (Nuxt 4 frontend, Go backends to come), lists
what the other CI jobs already catch so the model does not repeat them,
and focuses on SSR/hydration pitfalls, reactivity bugs, injection and
authz gaps, runtime-config leaks, and Go error/concurrency hygiene. It
prescribes the output format (`[BLOCKER]/[WARN]/[NIT] file:line`) and
derives the verdict from the findings, so the gate and the prompt agree.
# Evidence
* First real review on PR #5 (2026-10-18): produced five findings,
including a genuine regression in the workflow itself (a lost
`chmod +x` on the droast binary), verified upstream release assets and
the v2 permission schema live, and returned `VERDICT: REQUEST_CHANGES`.
* Fresh-container replica test (node:22-bookworm, key-only auth)
reproduced the CI failure exactly, then confirmed the v2 + `#max` fix
before it was pushed.
* Model history: `deepseek-v4-pro#max` was replaced by
`deepseek-v4.1-flash#max` after two consecutive endpoint outages, then
by `claude-haiku-5-5#medium` (2026-10-20) to stop paying maximum
reasoning effort on every review. The flash model had no `#medium`
variant to fall back to.
# Not verified
* Whether `#medium` still resolves after a future CLI upgrade — the
variant list comes from the model catalog, not a contract.
* Whether the smaller model still catches the same class of findings as
the first flash review. The PR #14 findings are the baseline; re-run
the eval on the next few PRs.
* Long-run behaviour of Go plan usage windows under real review
frequency, now that the model is cheaper.
# Revisit when
* A PR review fails again with a new error class — check the provider
prefix, the CLI version, and the usage limits first; they caused every
failure so far.
* A flash-tier model gains a `#medium` variant and beats the current
model on the PR #14 eval set — the present pick is a cost/quality
compromise, not a permanent one.
* Go Plus or pay-as-you-go balance becomes relevant.