Open Source Contribution to alibaba/open-code-review
alibaba/open-code-review (OCR) is an automated AI code-review tool that runs inside GitHub Actions and posts inline code comments alongside a top-level "sticky" Pull Request summary comment. To avoid spamming a PR timeline on every commit push, OCR searches existing PR comments for a hidden HTML marker (<!-- ocr-summary -->) and updates that comment in place. Furthermore, OCR uses a cross-push checkpoint system (<!-- ocr-checkpoint:v1 ... --> embedded inside the sticky summary) to record the last reviewed commit SHA so subsequent pushes only review the new delta.
Previously, the comment lookup functions (findExistingSummaryComment and findSummaryIssueComment) located the sticky summary solely by checking if the comment body contained the string <!-- ocr-summary -->. It did not verify who authored the comment during the summary update phase.
This created two critical problems:
<!-- ocr-summary -->, OCR would mistake the human's comment for its own sticky summary and overwrite the human's text on the next run.GITHUB_TOKEN (posts as github-actions[bot] via the github-actions App).<custom-app>[bot]).Because github.rest.users.getAuthenticated() (GET /user) returns 403 Resource not accessible by integration when called by a GITHUB_TOKEN or GitHub App installation token, simple login matching fails for bots, whereas strict bot-only checks fail for human PAT users. The posting path needed a unified, zero-regression authorship verification model that aligned with the repository's checkpoint trust boundary.
Instead of relying on a single fragile check, we introduced isSummaryAuthorOurs(comment, botLogin, appSlug):
botLogin is resolved (meaning GET /user succeeded, which only happens under a user PAT), and comment.user.login === botLogin, the comment is trusted as ours.isCheckpointAuthorOurs(comment, appSlug). This function immediately rejects any User-typed account (preventing a fork contributor from spoofing a marker even if they post via a third-party CLI GitHub App, where performed_via_github_app is populated). It then verifies that the author is a Bot and, if appSlug is pinned (e.g., 'github-actions'), strictly requires the comment's App slug to match.In runPostReviewComments, resolving the PAT user identity requires calling getAuthenticatedLogin(github, log).
authenticatedLogin initialized before any early exits (preventing Temporal Dead Zone errors on malformed JSON or zero-finding exits) and passed the function reference itself down into postSummary, ensureSummaryAnchor, and finalizeSummary. The API call is only awaited inside the comment loop after a comment containing <!-- ocr-summary --> is actually encountered.The checkpoint reader (readCheckpointComment) shares findSummaryIssueComment to locate the sticky summary. However, readCheckpointComment has a strict diagnostic contract tested by testCheckpointTwelveReasonsAreDistinct: if a human plants a fake summary comment, readCheckpointComment must find the comment and explicitly return { reason: "author_unverified" } rather than { reason: "no_summary_comment" }.
findSummaryIssueComment filtered out unverified authors during the checkpoint read, readCheckpointComment would think no comment existed at all. By adding an ignoreAuthor = false default parameter to findSummaryIssueComment and setting ignoreAuthor: true specifically inside readCheckpointComment, the posting path safely ignores human comments while the checkpoint reader still surfaces the exact author_unverified telemetry reason.Several unit tests in the suite pass stripped-down mock github objects that omit the rest.users namespace entirely. We hardened getAuthenticatedLogin to check for the existence of github?.rest?.users?.getAuthenticated before invoking it, returning null cleanly instead of throwing a TypeError that would trip the checkpoint resolver's catch block into a resolver_error.
This is the core runtime engine for posting reviews, managing sticky summaries, and evaluating cross-push checkpoints.
appSlug = "" to the destructured options object so the caller can pin the expected GitHub App identity ('github-actions' when running under the default token).let authenticatedLoginPromise = null;
const authenticatedLogin = () => {
if (!authenticatedLoginPromise) authenticatedLoginPromise = getAuthenticatedLogin(github, log);
return authenticatedLoginPromise;
};ensureSummaryAnchor, and finalizeSummary to pass botLogin: authenticatedLogin (as a lazy function reference) and appSlug.botLogin only when a matching marker (SUMMARY_MARKER or tag) is found, and gated the return on isSummaryAuthorOurs(comments[i], login, appSlug):async function findSummaryIssueComment({ github, owner, repo, prNumber, sticky, tag, log, botLogin, appSlug, ignoreAuthor = false }) {
const comments = await readAllPages("listIssueComments", (page, per_page) =>
github.rest.issues.listComments({ owner, repo, issue_number: prNumber, per_page, page }), log
);
for (let i = comments.length - 1; i >= 0; i--) {
const body = comments[i].body || "";
if (sticky ? body.includes(SUMMARY_MARKER) : body.includes(tag)) {
const login = typeof botLogin === "function" ? await botLogin() : botLogin;
if (ignoreAuthor || isSummaryAuthorOurs(comments[i], login, appSlug)) {
return comments[i];
}
}
}
return null;
}isSummaryAuthorOurs to bridge PAT logins and App/Bot verification, and added defensive null-checks to getAuthenticatedLogin:function isSummaryAuthorOurs(comment, botLogin, appSlug) {
if (!comment || !comment.user) return false;
if (botLogin && comment.user.login === botLogin) return true;
return isCheckpointAuthorOurs(comment, appSlug);
}This composite GitHub Action manifest defines the environment variables and inline actions/github-script execution blocks.
env: blocks do not carry over to subsequent steps. Previously, OCR_CHECKPOINT_APP_SLUG was only defined on the Resolve review range step. We added the exact same expression to the Post review comments step's env: block:OCR_CHECKPOINT_APP_SLUG: ${{ inputs.github_token == github.token && 'github-actions' || '' }}Why this expression? When the user uses the default github.token, we know with 100% certainty the writer is the github-actions App, so we pin it. If they pass a custom token (PAT or custom App), it evaluates to '' because an installation token cannot query GET /app without a JWT.
with: script: | block of Post review comments, we wired the environment variable into the function call:resolveOutdated: process.env.OCR_RESOLVE_OUTDATED, appSlug: process.env.OCR_CHECKPOINT_APP_SLUG || '',The unit and integration test suite for the review poster. Added targeted regression tests covering:
<!-- ocr-summary --> is ignored and never overwritten.[bot] account are recognized when appSlug is unpinned, while foreign bots are rejected when appSlug is pinned to 'github-actions'.testMalformedJsonWithStderrUsesAuthenticatedLogin) Verifying that early-exit error summaries properly resolve identity without throwing reference or TDZ errors.The static and behavioral contract test suite that parses action.yml without external dependencies.
action-contract.test.js only extracted run: | bash blocks and env: maps, leaving actions/github-script steps (with: script: |) invisible to contract assertions. We extended finish() inside parseSteps() to detect script: | and capture its 10-space-indented JavaScript body as step.script:let script;
const scriptMarker = rawLines.findIndex((line) => /^ script:\s*\|\s*$/.test(line));
if (scriptMarker >= 0) {
const body = [];
for (let index = scriptMarker + 1; index < rawLines.length; index += 1) {
const line = rawLines[index];
if (line.trim() === "") {
body.push("");
} else if (/^ /.test(line)) {
body.push(line.slice(10));
} else {
break;
}
}
script = body.join("\n");
}action.yml wiring so they can never drift apart in future refactors:function testPostReviewCommentsWiresAppSlug() {
const post = stepNamed("Post review comments");
assert.ok(post, "action.yml must retain the Post review comments step");
// 1. The step exposes the env value.
assert.strictEqual(
post.env.OCR_CHECKPOINT_APP_SLUG,
"${{ inputs.github_token == github.token && 'github-actions' || '' }}",
"Post review comments must resolve and expose OCR_CHECKPOINT_APP_SLUG in its env block"
);
// 2. The github-script body forwards it into the runPostReviewComments({...}) call.
assert.ok(post.script, "Post review comments must define a github-script `with: script:` body");
const call = post.script.match(/runPostReviewComments\(\{\[\s\S\]*?\n\}\);/);
assert.ok(call, "Post review comments script must call runPostReviewComments({...})");
assert.match(
call[0],
/^\s*appSlug:\s*process\.env\.OCR_CHECKPOINT_APP_SLUG\s*\|\|\s*(\['"\])\1\s*,?\s*$/m,
"runPostReviewComments must be passed the appSlug from process.env"
);
}A major highlight of this contribution was the iterative, rigorous code review process with community reviewer @Qiyuanqiii. Here is the chronological breakdown of the review rounds and how each technical challenge was resolved:
authenticatedLogin was initialized after the early exits (malformed JSON and zero-findings), causing a Temporal Dead Zone (ReferenceError) when those paths tried to post a summary.GET /user broke GitHub App and GITHUB_TOKEN runs because GET /user returns HTTP 403 for integration tokens; the summary check needed to preserve the App/Bot trust check (isCheckpointAuthorOurs).authenticatedLogin above all early exits, created isSummaryAuthorOurs to combine PAT login matching with isCheckpointAuthorOurs, and added regression tests for human, PAT, and custom-App comments.appSlug was passed to helper functions inside runPostReviewComments, but appSlug = "" was missing from runPostReviewComments's own parameter destructuring, causing ReferenceError: appSlug is not defined.OCR_CHECKPOINT_APP_SLUG was only scoped to the Resolve review range step in action.yml, not the Post review comments step.make test only runs Go tests, whereas npm run test:github-actions runs the JS suite.appSlug = "" to runPostReviewComments, added OCR_CHECKPOINT_APP_SLUG to the Post review comments step's env: block in action.yml, made botLogin lazy so testAuthenticatedLoginIsResolvedOncePerRun passed with 0 unnecessary API calls, and hardened getAuthenticatedLogin against stripped test mocks.npm run test:github-actions passed locally.OCR_CHECKPOINT_APP_SLUG in its env: block, the inline JS call runPostReviewComments(...) inside action.yml ended at resolveOutdated: process.env.OCR_RESOLVE_OUTDATED without passing appSlug: process.env.OCR_CHECKPOINT_APP_SLUG || ''.action-contract.test.js regression test asserting both the env exposure and the runPostReviewComments argument forwarding.appSlug: process.env.OCR_CHECKPOINT_APP_SLUG || '' to action.yml, upgraded parseSteps() in action-contract.test.js to parse with: script: | blocks, and added testPostReviewCommentsWiresAppSlug. Replied with a detailed breakdown showing that removing either the env line or the forwarding line caused the contract test to fail, and that all 53 action contract tests passed.After receiving @Qiyuanqiii's approval, the PR showed:
Hi! I'm Irvallen's AI assistant. Ask me anything about his projects or experience.