Code Keeps Changing, So When Do the Docs Catch Up? Manual on Release, Scheduled CI, and MR-Triggered Checks

In the last post we covered how the acquire-codebase-knowledge skill distills an unfamiliar codebase into seven documents under docs/codebase/. Once you have run the skill, a question naturally follows. The code changes every week, so when should these documents be rebuilt? Left alone, a document starts diverging from reality within days. And a stale document is worse than none, because, as the previous post argued, the whole team ends up navigating with the wrong map.

Modern Web Labs has settled on three refresh approaches, drawn from our own use and from conversations with client teams. The short answer up front: this is not a pick-one decision. The realistic combination is to start with ①, run by hand, layer on ② for scheduled automation, and use ③ not to rebuild documents but as a check that catches drift. Below we walk through them from the least automated up. The examples use GitLab, but the principles carry over to any CI system.

① Manual refresh on the release cycle

The simplest approach adds one line to the release checklist. Something like "re-run the skill and open an MR with the changes" is enough. When preparing a release, run acquire-codebase-knowledge again, eyeball the diff between the regenerated documents and the existing ones, and merge it as an MR.

The strength of this approach is that verification always passes through human eyes. A developer reviews the agent's rewrite as a diff, so nothing absurd slips in. There are two weaknesses. Drift that opens up between releases sits unaddressed until the next one. And checklist items are the first thing forgotten when things get busy. This approach fits the early adoption phase, before the team has come to trust the documents.

The one-liner that runs without a human: claude -p

To understand ② and ③ you need one building block first: claude -p. Typing !ls inside a Claude Code conversation runs a shell command within the interactive session. claude -p "..." is the reverse: it invokes Claude Code once from the shell. No interactive window opens; it asks once, prints the answer, and exits.¹ The docs call this headless mode and recommend it for scripts and CI pipelines.

To get a feel for it, try typing claude -p "Translate this sentence into French: hello world" in a terminal once. Watch how no window opens and the process simply prints the answer and ends. The two approaches below do nothing more than have CI run this one line for you.

② Scheduled CI job that regenerates the docs

GitLab's Pipeline Schedules feature runs a pipeline on a recurring basis. For example, regenerate the documents every Monday before dawn and open an MR automatically when something changed. A human only reviews the MR.

# .gitlab-ci.yml — weekly docs regeneration (cadence set in Pipeline Schedules)
docs-refresh:
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
  script:
    - claude -p --permission-mode acceptEdits "Regenerate the docs/codebase/ documents with the acquire-codebase-knowledge skill"
    - |
      if ! git diff --quiet; then
        git checkout -b "docs/refresh-$CI_PIPELINE_ID"
        git add -A
        git commit -m "docs: refresh codebase knowledge"
        git push origin "docs/refresh-$CI_PIPELINE_ID"
        glab mr create --fill --yes
      fi

Unpacking the flow: the $CI_PIPELINE_SOURCE == "schedule" rule restricts this job to scheduled runs.² claude -p runs the skill to rewrite the documents, then git diff --quiet checks whether anything actually changed. That command returns a failure code when changes exist, so if ! git diff --quiet reads as "if something changed." Only then does the job create a branch, commit, push, and open an MR with glab mr create. If nothing changed, nothing happens.

Unattended execution comes with one thing to watch. Nobody is sitting at the pipeline, so a single permission prompt stalls the run on the spot. That is why --permission-mode acceptEdits auto-approves file edits. Tools that need the shell, like running scan.py, must be opened up in advance with --allowed-tools.

③ MR-triggered drift check only

The third approach runs every time an MR is opened. Regenerating the full documentation on every MR would inflate diff noise and token cost beyond reason, so in this slot claude -p is used not to rewrite documents but as a drift check. Ask it to "point out, with file and line, anywhere this MR's diff contradicts the docs/codebase/ documents," and post the result as an MR comment.

# .gitlab-ci.yml — drift check on every MR
docs-drift-check:
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
  script:
    - |
      claude -p --permission-mode plan \
        "If any change in this MR contradicts the docs/codebase/ documents, point it out with file and line. If none, output OK only." \
        > drift.md
    - glab mr note "$CI_MERGE_REQUEST_IID" --message "$(cat drift.md)"

--permission-mode plan keeps the agent from touching files. This is a read-and-report check rather than a rewrite, so that mode is the right fit. The advantage of this approach is that drift is caught at the very moment it appears. Actually fixing the documents remains a human job through the flows of ① or ②; ③ stops at telling you "this change disagrees with the docs."

Comparing the three

Wrapping up

This is a question of sequence, not of choosing one of the three. Wire up ② and ③ from day one and all you accumulate is auto-generated MRs and comments nobody reads. Start with ①, refreshing the documents by hand and building the shared experience that these documents are useful. Around the time manual runs become a chore, automate the regeneration with ②. And once the team genuinely starts depending on the documents, add ③ to catch drift in real time. The automation level rises one step at a time, in proportion to how much the team trusts the docs.

All three approaches rest on the same one-liner, claude -p. A coding agent does not require a human sitting in an interactive session; let the CI pipeline make the call instead, and repetitive work like documentation refresh comes off people's hands. If generating the documents once with acquire-codebase-knowledge is the start, keeping them alive alongside the code is the job of these three approaches. The next post takes the same claude -p beyond documentation refresh, into work that used to need human judgment: having the pipeline review release notes.

1. Anthropic, Claude Code CLI reference, Claude Code Docs.

   ↩

2. GitLab, CI_PIPELINE_SOURCE predefined variable, GitLab Docs.

   ↩