What your best reviewer knows

Every site keeps written criteria for batch record review. The master batch record holds the steps and ranges, the specification the limits, the site checklist the reminders, and CAPA adds to them when something goes wrong. None of them holds where the reviewer everybody trusts looks harder: the night shift’s timestamps since the last two deviations on that shift, a gap in the times the morning after a storm, an equipment log that looks too perfect. She applies all three to every batch she reads.

A site’s review procedure says how review is done, including how an entry is returned to whoever made it for correction. The review criteria say what a record must show. Where-to-look knowledge is where an experienced reviewer looks harder than those criteria ask, and why. 21 CFR 211.22(d) and 211.192, the PIC/S data integrity guidance and the MHRA GXP data integrity guidance ask for written procedures and for review against them, and none asks for where-to-look knowledge in writing. Part of it can be written as a check; the rest is judgment and stays with the person. The paper closes with an inventory to run with the reviewer a site would miss most.

Three terms carry the rest of this paper.

The review procedure

The review procedure says how batch record review is done: who reviews, in what order, to which documentation conventions, and how an entry is returned to whoever made it for correction. 21 CFR 211.22(d) requires the quality unit’s procedures in writing, and 211.192 requires records to be reviewed against all of the site’s approved written procedures.

The review criteria

The review criteria say what a record must show; they’re the master, the specification, the site checklist and whatever CAPA has added, and they’re written.

Where-to-look knowledge

Where-to-look knowledge is where an experienced reviewer looks harder than the written criteria ask, and why, learned from what has gone wrong before. It attaches to a product sometimes, and as often to a shift, a supplier or a contract manufacturer.

A site’s review criteria are written down and maintained. The master batch record holds the steps and the ranges, and it changes only through change control, with the new version trained on before it’s executed. The specification holds the limits. The site checklist holds what a reviewer confirms on every record: entries complete, signatures and dates present, corrections made to the site’s documentation conventions. When something goes wrong, the deviation is investigated and CAPA adds to that list. It’s a controlled body of criteria, and review runs on it every day.

Beside it works the reviewer who knows where to look. She’s the one people mean when they say somebody knows the record. Since the last two deviations on the night shift, she reads that shift’s timestamps harder than the checklist asks. On the morning after a storm she looks for a gap in the times, and on one record she finds one, with no outage recorded anywhere. At an equipment log whose numbers look too perfect, she slows down before she could tell you why.

211.22(d) asks for the quality unit’s procedures in writing, 211.192 asks for records to be reviewed against every approved written procedure, PIC/S and MHRA ask for the review process itself to be written, and PIC/S asks for data compared against acceptance criteria. None of them asks for the list of places a reviewer looks harder. 211.25 counts experience as a qualification alongside education and training, which is as close as the texts come to naming what a reviewer brings with them.

If no inspector has asked a site where its reviewers look harder, it’s because nothing in these texts asks.

Quality system texts hold the frame for going further. ICH Q10 (FDA guidance, 2009) makes knowledge management an enabler of the pharmaceutical quality system and lists “manufacturing experience” among the sources of knowledge. The 2023 revision of ICH Q9 says subjectivity enters “in how hazards, harms and risks are perceived by different stakeholders”, and names among its controls “maximizing the use of relevant data and sources of knowledge”. Knowledge management writing has stayed close to technology transfer, and authors from Merck and TU Dublin, writing in 2020 and 2021, judged it still not a mature discipline more than a decade after ICH Q10. None of these texts mentions batch record review. Carrying the principle to where a reviewer looks harder is this paper’s extension, and it’s offered as one.

Reviewer consistency is the degree to which two qualified reviewers, applying the same procedure and the same written criteria to the same record, return the same findings. Part of why they differ is that each brings a sense of where to look, learned from what they’ve seen go wrong.

A deviation on the night shift is investigated, its cause is found and corrected, and the CAPA may add a check to the site checklist. That is the quality system working as ICH Q10 describes it, where “Corrective action is taken to prevent recurrence” and CAPA methodology “should result in product and process improvements and enhanced product and process understanding”.

The reviewer who knows the record takes a second route. For months she reads that shift’s timestamps harder, until the shift’s records have run clean for long enough. A CAPA may also set an effectiveness check, a closer look at that shift’s records for a set period, and when it does, part of her attention is written there. Hers started before the CAPA was raised and lasts after the check closes, because a corrective action removes a cause and an effectiveness check confirms it, and neither was asked to record where somebody now looks. After a minor deviation, a quality memo asking the people reviewing to keep their eyes on what slipped may write down another part. How effective CAPA is, and how it is trended, are questions outside this paper.

Where-to-look knowledge has two halves. The finding half can be written as a check by the reviewer who knows it, and once it’s written and shared, any qualified reviewer can apply it, including whoever reads the record while the senior reviewer is on leave. The judging half stays with the person, and it’s where an experienced reviewer’s years count. The senior reviewer is the author of the first half and the owner of the second.

Spotting the gap in the times on the morning after the storm is finding. It’s a comparison of the times on that morning’s records, looking for a gap longer than the step should take, with the storm as the reason to look, and it could be written as a check. Deciding whether the gap is clock drift, a power loss or a data integrity problem is judging. The equipment log that looks too perfect is judging all the way through, the sense that a page is wrong before any check says so. The night shift sits between the two. A closer look at that shift’s entries can be written as a check, and what she makes of what it turns up can’t.

The split matters because a check with a written definition can be applied the same way by a second reviewer, questioned, and used again on the next record. One held only in a reviewer’s habits can’t be compared with anyone else’s.

No published study of agreement between GMP batch record reviewers was found for this paper, and the nearest measured evidence comes from hospital medical record review, used here as an analogy. What it measured is whether a second look at the same record reproduced the first, for one question with a written definition and one without. In a study from a Dutch university hospital, published in PLoS One in 2018, a review committee re-examined fifty records it had already reviewed, blind to its first verdicts. It reproduced its finding that an adverse event had occurred, which had a written definition, far better than its judgment of whether the event was preventable, which had none. The authors suggest the definition may be why, and the study didn’t test that.

The inventory starts with the reviewer the site would miss most, one product, and its recent executed records. The places she looks harder attach to that product, and as often to a shift, a supplier or a contract manufacturer. For each one, it puts three questions to her.

• What taught you to look there?
• Is it written anywhere: the checklist, a CAPA, a quality memo, a note?
• Could it be written as a check, or is it judgment?

A place that turns out to be written in three documents, each slightly different, is an answer too. A place only she holds is the first to write, and she’s the one to write it.

One question goes to whoever owns change control. Does the part that can be written belong on the controlled checklist, and who retires an entry when the reason for it has passed?