ci: use cheaper review model and cache pipeline installs
PR Check / Security scan (Snyk) (pull_request) Skipped
PR Check / Lint + typecheck (pull_request) Successful in 1m42s
PR Check / Secret scan (gitleaks) (pull_request) Successful in 6s
PR Check / Dependency audit (pull_request) Successful in 9s
PR Check / Dockerfile lint (droast) (pull_request) Successful in 5s
PR Check / AI code review (opencode) (pull_request) Successful in 45s

- review model: opencode-go/claude-haiku-5-5#medium instead of
  deepseek-v4.1-flash#max; flash has no medium variant
- merge lint and typecheck into one job with a single pnpm install
- cache the pnpm store and opencode binary via actions/cache@v4.1.0,
  keyed on a sha256 of the lockfile (hashFiles is empty on this runner)
- update the decision records and log
This commit is contained in:
Noel Sabia committed 2026-10-20 20:56:52 +02:00
1 parent 71b73aacc8
commit fa8fafb7ef
5 files changed
+110 -44

No files matched your search

+18 -6
View File
@@ -1,7 +1,7 @@
---
type: Decision
title: AI code review setup
description: The review job runs opencode with the opencode-go plan model, env-only auth, a read-only permission set, and posts only reviews that end with a verdict line.
description: The review job runs opencode with a cheap medium-effort opencode-go model, env-only auth, a read-only permission set, and posts only reviews that end with a verdict line.
tags: [decision, ci, ai, opencode, security]
status: draft
generated: { by: opencode-go/glm-5.3-flash, at: 2026-10-18T16:05:00Z }
@@ -19,7 +19,7 @@ sources:
The review job installs opencode v2 from the dedicated v2 installer,
authenticates with `OPENCODE_API_KEY` from the environment only, runs
`opencode-go/deepseek-v4-pro#max` (covered by the Go plan subscription,
`opencode-go/claude-haiku-5-5#medium` (covered by the Go plan subscription,
no per-token cost), denies `edit`/`shell`/`webfetch`/`websearch` via a
global permission config written outside the checkout, and posts the
review as a PR comment only if the output ends with a `VERDICT:` line.
@@ -29,6 +29,7 @@ review as a PR comment only if the output ends with a `VERDICT:` line.
| Piece | Why |
|---|---|
| Provider prefix `opencode-go`, not `opencode` | `opencode` is the pay-per-call Console gateway; the Go plan lives under its own provider id. The wrong prefix failed every run with "Insufficient account funds". |
| Model `claude-haiku-5-5#medium` | Cheap per token and cheap in the usage window, with a real `#medium` reasoning level. `deepseek-v4.1-flash` only exposes low/high/max, so its reviews always paid for the maximum effort. |
| The v2 installer (`/v2/install`) | The classic installer ships OpenCode 1.x (v1.19.3 on arm64), which cannot resolve model variants and failed with `ProviderModelNotFoundError` for `#max`. |
| Env-only auth, no `opencode.json` in the checkout | A config file in the workspace holds the API key in plaintext where the review agent can read it; a malicious PR diff could prompt-inject the agent into quoting the key in the publicly visible review comment. Environment variables are invisible to file reads. |
| Deny write tools via a global config outside the checkout | Headless runs cannot answer permission prompts; denials prevent both hangs and repo modification. The config file itself holds no secret, so its location leaks nothing. |
@@ -54,17 +55,28 @@ derives the verdict from the findings, so the gate and the prompt agree.
* Fresh-container replica test (node:22-bookworm, key-only auth)
reproduced the CI failure exactly, then confirmed the v2 + `#max` fix
before it was pushed.
* Model history: `deepseek-v4-pro#max` was replaced by
`deepseek-v4.1-flash#max` after two consecutive endpoint outages, then
by `claude-haiku-5-5#medium` (2026-10-20) to stop paying maximum
reasoning effort on every review. The flash model had no `#medium`
variant to fall back to.
# Not verified
* Whether `#max` still resolves after a future CLI upgrade — the variant
list comes from the model catalog, not a contract.
* Long-run behaviour of Go plan usage windows ($3 per 5 hours for
DeepSeek V4 Pro) under real review frequency.
* Whether `#medium` still resolves after a future CLI upgrade — the
variant list comes from the model catalog, not a contract.
* Whether the smaller model still catches the same class of findings as
the first flash review. The PR #14 findings are the baseline; re-run
the eval on the next few PRs.
* Long-run behaviour of Go plan usage windows under real review
frequency, now that the model is cheaper.
# Revisit when
* A PR review fails again with a new error class — check the provider
prefix, the CLI version, and the usage limits first; they caused every
failure so far.
* A flash-tier model gains a `#medium` variant and beats the current
model on the PR #14 eval set — the present pick is a cost/quality
compromise, not a permanent one.
* Go Plus or pay-as-you-go balance becomes relevant.
+22 -4
View File
@@ -22,9 +22,10 @@ Every pull request is checked by three layers, ordered by cost:
1. **Pre-commit hooks** (local, seconds, deterministic): gitleaks, file
hygiene, conventional-commit check, actionlint, eslint --fix on staged
files. A commit that fails here never exists.
2. **Gitea Actions** (per PR push, minutes): ESLint, vue-tsc typecheck,
gitleaks over full history, pnpm audit (blocking), droast Dockerfile
lint, Snyk (currently disabled).
2. **Gitea Actions** (per PR push, minutes): ESLint + vue-tsc typecheck
in one job (one install, cached pnpm store), gitleaks over full
history, pnpm audit (blocking), droast Dockerfile lint, Snyk
(currently disabled).
3. **AI code review** (per PR push): opencode with a plan-covered model,
verdict-based, posted as a PR comment.
@@ -42,6 +43,23 @@ sees intent, contracts and cross-file consequences that no linter can. The
ordering also protects the budget: the AI review only runs for code that
already passed the deterministic gates.
# Caching
Two caches, both served by the runner's own cache server (no external
service, entries stay on the home server):
* **pnpm store**, keyed on the store layout version (v11) and a `sha256sum`
of `pnpm-lock.yaml`. Installs hardlink from the store instead of
downloading tarballs again. The key is built in a shell step because
`hashFiles()` evaluates empty in this act runner.
* **opencode binary**, keyed on the pinned CLI version (`~/.opencode/bin`).
Small next to the model call, but it removes a network fetch per review.
Lint and typecheck also share one job: same image, same lockfile, same
install. Splitting them duplicated the install and made both compete for
the runner's two vCPUs. A future `pnpm build` job reuses the same store
cache key.
# What the pipeline bought us in practice
Recorded so the reasoning survives: during the first week the pipeline
@@ -57,7 +75,7 @@ exposure in the review setup — each fixed where it was cheapest.
|---|---|
| Error logs posted as review comments | Posting gate is the output contract (a `VERDICT:` line), not the exit code |
| Vendor container images without arm64 manifests | Binaries installed arch-aware in `node:22-bookworm`, never vendor CI images |
| `hashFiles()` silently empty in the act runner | File discovery via `find` in a step, not workflow expressions |
| `hashFiles()` silently empty in the act runner | File discovery via `find` and cache keys via `sha256sum`, both in shell steps - never workflow expressions |
| Silent advisory failures | `continue-on-error` removed everywhere; a failed check is red |
| Supply-chain (curl-piped binaries) | Pinned versions, checksums pinned in the workflow |
+3
View File
@@ -1,5 +1,8 @@
# Update Log
## 2026-10-20 (cheaper review model, cached installs, merged checks)
* **Update**: [pr-checker.yml](../.gitea/workflows/pr-checker.yml) - the review model is now `opencode-go/claude-haiku-5-5#medium`: cheap per token and, unlike `deepseek-v4.1-flash`, it has a real `#medium` effort level (flash exposes only low/high/max). ESLint and typecheck are merged into one `checks` job with a single `pnpm install`, and the pnpm store plus the opencode binary are restored with `actions/cache@v4.1.0` (v1 cache API, so it also works on a v1-only runner). Cache keys are built from `sha256sum pnpm-lock.yaml`, the pnpm store layout (`v11`) and the opencode version in shell steps, because `hashFiles()` evaluates empty in this act runner. See [AI review setup](decisions/ai-review-setup.md) and [PR pipeline layering](decisions/pr-pipeline-layering.md).
## 2026-10-20 (first flash review validated, two findings applied)
* **Update**: PR #14 review (flash model, `VERDICT: APPROVE`) was fact-checked finding by finding. Applied: `<NuxtRouteAnnouncer />` restored in `app.vue` (this PR's diff had dropped it with the NuxtWelcome swap), and the review job now requires `github.event_name == 'pull_request'` (manual dispatch - the new snyk trigger - would otherwise launch a paid review with no PR to post to). Rejected with evidence: the reviewer's `postcss-selector-parser` 6.1.5 override suggestion - the 6.1.5 backport exists on npm, but the blocking audit still flags it (`<7.1.6` covers 6.x), verified empirically; 7.1.6 stays. Deferred as design decisions: mobile nav, aria-current via NuxtLink, sprite-to-SVG, font fallback stacks, `borderRadius.full` token, the fall-endpoint budget in the API contract.
+1 -1
View File
@@ -17,7 +17,7 @@ generated: { by: opencode-go/glm-5.3-flash, at: 2026-10-18T12:41:52Z }
| When and how often Snyk runs once re-enabled: weekly schedule, ready-for-review gate, or per-push | re-enabling the snyk job | Stay disabled. |
| Whether the AI review job should gate merges (required status check on the verdict) or stay advisory | branch protection setup | Advisory. The review comment is information; blocking merges on an AI verdict is a policy choice, not a technical one. |
| How the opencode CLI version in CI gets upgraded: manually bump the pinned 2.0.32, or automate when a new major appears | first review failure after an opencode release | Manual bump. Pinning exists so upgrades are decisions. |
| ~~Whether the Go plan's per-model usage windows ($3 per 5 hours for DeepSeek V4 Pro) survive real review frequency, or the review model must move to `deepseek-v4.1-flash` ($80 window)~~ - **decided** (2026-10-20): review model is `opencode-go/deepseek-v4.1-flash#max`; the pro endpoint failed two consecutive review runs with `Upstream request failed: Endpoint is unavailable` (~10 min apart), so the recorded default was applied | done | — |
| ~~Whether the Go plan's per-model usage windows survive real review frequency, and which model the review should use~~ - **decided** (2026-10-20): review model is `opencode-go/claude-haiku-5-5#medium` in [pr-checker.yml](../.gitea/workflows/pr-checker.yml). Cheap per token, and it has a real `#medium` effort level; `deepseek-v4.1-flash` only exposes low/high/max, so it always paid for maximum reasoning. The earlier pro -> flash switch (two consecutive `Endpoint is unavailable` failures) is in the [log](../log.md) | done | — |
| Removing the two reviewed audit ignores when `node-forge` or `braces` ship patched releases | next dependency update round | Watch for new releases. |
# The mount trade-off, in detail