# CodeRabbit pilot checklist

Companion: https://aiopssre.com/coderabbit-guide-ai-code-review/
Prepared September 29, 2026. A suggested evaluation worksheet, not a validated scoring model.

## Before starting

- [ ] Select one repository and name the maintainer assessing the pilot.
- [ ] Confirm permission to process its code with the chosen service/deployment.
- [ ] Review installation permissions, caching, retention and enabled usage billing.
- [ ] Preserve required tests and human review.
- [ ] Set a bounded window, for example two weeks or 20 representative PRs.
- [ ] Choose a comparable baseline from the existing review workflow.
- [ ] Record plan, version, settings and cost limits.

## For each pull request

PR / revision:
Change type and approximate size:
Intended behavior:
Review completed / skipped / incomplete:
Files omitted and reason:
Useful findings and severity:
Rejected findings and reasons:
Evidence or test for each accepted correction:
Minutes spent checking AI feedback:
Remaining human review minutes:
Waiting time for AI review:
Retries or additional charges:
Human disposition and remaining uncertainty:

## End-of-pilot decision

- [ ] Compare similar changes and account for investigation time, waiting and cost.
- [ ] Distinguish accepted style suggestions from reproducible defects.
- [ ] Record failures and incomplete reviews, not only successes.
- [ ] Identify any important defects found by humans after the AI review.
- [ ] Choose: retain, tune the scope, extend evaluation, or remove.

Decision and reason:
What evidence could change this decision:

Do not equate an accepted suggestion with a prevented incident. Keep private code and incident details in approved storage.