| 1 | # GitHub App Setup (Codewhale Agent reviews) |
| 2 | |
| 3 | `codewhale review --pr N` writes an advisory code review of a pull request. With |
| 4 | `--post` (or from CI) the review is published to GitHub. Published reviews can |
| 5 | appear under two identities: |
| 6 | |
| 7 | - the default token the CI job already has (`github.token`), or |
| 8 | - a dedicated **GitHub App** so the review shows as a bot — e.g. |
| 9 | `codewhale-agent[bot]` — instead of a personal account. |
| 10 | |
| 11 | The App identity is optional. Nothing below is needed to run |
| 12 | `codewhale review --pr N` locally and print the report to your terminal. |
| 13 | |
| 14 | Related docs: |
| 15 | |
| 16 | - [Automatic Workflows](AUTOMATIC_WORKFLOWS.md) — the review workflow in context |
| 17 | - [Providers](PROVIDERS.md) — the model/key used to write the review |
| 18 | - [Receipts](RECEIPTS.md) — how posted reviews are anchored to a head SHA |
| 19 | |
| 20 | ## Actions setup and model selection |
| 21 | |
| 22 | Use [the reusable GitHub Action setup](GITHUB_ACTION.md) for the workflow, |
| 23 | account machine key, exact model, release pin, limits, outcomes and retries. |
| 24 | The repository workflow is now a thin caller of that action. It uses the |
| 25 | Codewhale account relay and a checksummed release; it does not compile a PR's |
| 26 | candidate source. BYOK is an explicit option in a user's own workflow. |
| 27 | |
| 28 | ## Review evidence and precision |
| 29 | |
| 30 | The Actions-backed GitHub App and the `review` tool use the same PR review |
| 31 | contract. Findings must explain an introduced defect's trigger, source evidence, |
| 32 | impact and a useful fix. Generic requests for more tests, style preferences and |
| 33 | unsupported compiler claims do not qualify as findings. An empty findings list |
| 34 | is valid; unresolved assumptions belong in the assessment. |
| 35 | |
| 36 | When the exact PR head is available locally, each pass also receives numbered |
| 37 | source excerpts around its changed hunks and nearby module declarations. These |
| 38 | come from regular Git blobs at the pinned head, never from dirty checkout files |
| 39 | or symlink targets. Source is not executed and no additional model call is made. |
| 40 | The excerpts use only the unused portion of `CODEWHALE_REVIEW_MAX_CHARS`, capped |
| 41 | at 50000 characters and 32 files per pass; individual blobs above 128 KiB are |
| 42 | omitted. The complete diff remains intact and remains the inline-comment scope. |
| 43 | |
| 44 | The request explicitly records unavailable files and omitted context. It does |
| 45 | not inspect unchanged caller files or run builds/tests, and a completed review |
| 46 | does not establish either. These source and local-fixture guarantees do not |
| 47 | establish a model's bug-detection rate or parity with another review product. |
| 48 | |
| 49 | ## Output budget |
| 50 | |
| 51 | `CODEWHALE_REVIEW_MAX_OUTPUT_TOKENS` optionally sets the CLI's output budget |
| 52 | through `CODEWHALE_MAX_OUTPUT_TOKENS`. Without it, the CLI chooses its automatic |
| 53 | cap. The workflow rejects values below **8192** to leave room for reasoning |
| 54 | and the final review. Provider accounting and supported limits vary; an empty |
| 55 | response is not proof of any one cause. A zero-exit review with empty output |
| 56 | fails the job. |
| 57 | |
| 58 | ## One-time setup, five steps |
| 59 | |
| 60 | You need owner access to the GitHub repository once. After setup, eligible non-draft |
| 61 | same-repository pull requests can post reviews as the App. |
| 62 | |
| 63 | 1. **Create the App.** GitHub → *Settings → Developer settings → GitHub Apps → |
| 64 | New GitHub App*. Name it (e.g. `Codewhale Agent`), set a homepage URL, and |
| 65 | **uncheck Webhook → Active** — the review is pulled on PR events by Actions, |
| 66 | so no webhook is needed. |
| 67 | 2. **Grant two repository permissions.** |
| 68 | - *Pull requests* → **Read & write** (to post the review and inline comments) |
| 69 | - *Contents* → **Read-only** (to read the diff; read-only is enough — avoid |
| 70 | write unless you have another reason) |
| 71 | Choose *Only on this account*, then **Create GitHub App**. |
| 72 | 3. **Download the private key.** On the App's page, *Private keys → Generate a |
| 73 | private key*. Keep the `.pem` file secret; it is the App's credential. |
| 74 | 4. **Install the App** on your account (*Install App* on the same page) and |
| 75 | select the repositories reviews should cover. |
| 76 | 5. **Add repository settings.** GitHub → *Settings → Secrets and |
| 77 | variables → Actions*: |
| 78 | |
| 79 | | Kind | Name | Value | |
| 80 | |----------|-----------------------------|-------------------------------------| |
| 81 | | Variable | `CODEWHALE_APP_ID` | the App ID shown on the App's page | |
| 82 | | Secret | `CODEWHALE_APP_PRIVATE_KEY` | the full `.pem` file contents | |
| 83 | | Secret | `CODEWHALE_API_KEY` | a Codewhale machine key for this repository workflow | |
| 84 | | Variable | `CODEWHALE_REVIEW_MODEL` | exact account catalog `provider/model` id (required) | |
| 85 | | Variable | `CODEWHALE_REVIEW_VERSION` | exact released CLI tag; default v0.10.0 | |
| 86 | |
| 87 | App settings control identity. Model access separately requires a review |
| 88 | key and, for account mode, the catalog model. Optional budget variables are |
| 89 | `CODEWHALE_REVIEW_MAX_CHARS`, `CODEWHALE_REVIEW_MAX_PASSES`, and |
| 90 | `CODEWHALE_REVIEW_MAX_OUTPUT_TOKENS`. |
| 91 | |
| 92 | ## How the pieces connect |
| 93 | |
| 94 | [The review workflow](../.github/workflows/codewhale-review.yml) uses |
| 95 | `pull_request` and a manual `workflow_dispatch` recovery trigger. The action |
| 96 | reads the PR's exact Git objects in a fresh repository without checking them |
| 97 | out. Fork events receive no model key. Before any inference, the action |
| 98 | rejects fork, draft and closed PRs and verifies their revisions, including |
| 99 | on manual runs. |
| 100 | |
| 101 | When both `CODEWHALE_APP_ID` and `CODEWHALE_APP_PRIVATE_KEY` are present, the |
| 102 | workflow mints a short-lived installation token restricted to contents:read |
| 103 | and pull_requests:write. Otherwise it uses `github.token`. The action emits |
| 104 | one COMMENT review, never approval or a request for changes. CODEOWNERS stays |
| 105 | the human authority. Setup or provider failures fail the optional review job |
| 106 | and save a sanitized receipt; they do not post additional status comments. |
| 107 | |
| 108 | The Actions-only App setup above does not describe the managed hosted App. |
| 109 | Do not disable the webhook on an existing App that also serves hosted mentions. |
| 110 | |
| 111 | ## Running a review yourself |
| 112 | |
| 113 | ```sh |
| 114 | # print a report locally (uses your configured provider key) |
| 115 | codewhale review --pr 1234 |
| 116 | |
| 117 | # pin the route when a model is reachable through more than one provider |
| 118 | codewhale --provider deepseek --model MODEL_ID review --pr 1234 |
| 119 | |
| 120 | # account mode: check the agent, then use an exact id from the account catalog |
| 121 | codewhale --no-project-config account agent |
| 122 | codewhale --no-project-config --provider codewhale --model PROVIDER/MODEL_ID review --pr 1234 |
| 123 | |
| 124 | # explicitly increase a complete-diff input limit when needed |
| 125 | codewhale review --pr 1234 --repo OWNER/REPO --max-chars 6000000 |
| 126 | |
| 127 | # explicitly authorize at most 8 complete ordered model passes |
| 128 | codewhale review --pr 1234 --repo OWNER/REPO --max-passes 8 |
| 129 | |
| 130 | # publish it to GitHub as whichever identity GH_TOKEN carries |
| 131 | codewhale review --pr 1234 --post |
| 132 | ``` |
| 133 | |
| 134 | `GH_TOKEN` may be your `gh` CLI token (posts as you) or an App installation |
| 135 | token (posts as the App). The `--post` flag is always opt-in. |
| 136 | |
| 137 | ## Troubleshooting |
| 138 | |
| 139 | - **Review posts as you, not the bot.** The variable or the private-key secret |
| 140 | is missing/empty; the job silently falls back to `github.token`. Check both |
| 141 | names character-for-character. |
| 142 | - **No model review completed.** Read the outcome receipt and the |
| 143 | [repair guide](GITHUB_ACTION.md#outcomes-and-recovery). |
| 144 | - **Account model or provider error.** Set the provider to `codewhale` (or |
| 145 | unset it), choose the exact model from the account catalog, and check that |
| 146 | the machine key has the required scopes and the account has a configured |
| 147 | agent. A vendor key belongs in its own secret, never `CODEWHALE_API_KEY`. |
| 148 | - **Complete diff exceeds the input limit.** Inspect the reported size and |
| 149 | model context capacity before raising `CODEWHALE_REVIEW_MAX_CHARS`. An 8 MiB |
| 150 | transport-bound failure cannot be bypassed with that variable. |
| 151 | - **PR head changed or history is unavailable.** Rerun for the current |
| 152 | revision. The workflow refuses to review an unverified snapshot. |
| 153 | - **"available from configured provider route(s): ...".** Two provider keys are |
| 154 | configured and the model is reachable from both. Use an explicit provider in your own Action configuration. |
| 155 | - **Empty review.** The job fails. Inspect provider errors and output-budget |
| 156 | receipts; increasing `CODEWHALE_REVIEW_MAX_OUTPUT_TOKENS` may help when |
| 157 | reasoning exhausted the budget, but does not diagnose the cause by itself. |
| 158 | - **App token step fails.** The `.pem` was regenerated after the secret was |
| 159 | set — paste the newest key into `CODEWHALE_APP_PRIVATE_KEY` again, and |
| 160 | confirm the App is actually installed on the repository. |
| 161 | - **Name already taken.** GitHub App names are global; pick another name. The |
| 162 | bot's display login is `<slug>[bot]`, derived from the name. |
| 163 |