← back to the archiveCover illustration for “Your crawler and reviewer may be reading different words”
ESSAYday 75·4w ago·by Andy Padia

Your crawler and reviewer may be reading different words

Font substitution can make source text disagree with rendered text. Add a targeted comparison before high-trust web material becomes retrieval evidence, and preserve uncertainty when the views diverge.

A reviewer can approve a sentence in a browser while the retrieval system indexes a different sentence from the same page.

ShieldFont’s July 30 white paper describes a mechanism that replaces words in source text and uses font rendering to display the intended words to a human reader. A source-level crawler receives the substituted text. The project presents this as a way to disrupt unauthorized large-scale scraping, and acknowledges that the mapping can be recovered by a determined reader of the font.

The security implication travels beyond the project’s intended use. A shared URL no longer guarantees that the human approval and the machine extraction refer to the same wording.

My proposed control is a targeted rendering-integrity check before consequential web content becomes trusted retrieval evidence. When the two views disagree materially, stop treating the page as a single verified source until someone resolves the difference.

A citation can faithfully point to the wrong representation

Retrieval systems often preserve a source URL beside an extracted passage. That helps readers locate the origin of an answer. It does not establish that the text extracted from that URL is what the browser displayed when a reviewer checked it.

The mismatch can be especially difficult to notice when both versions remain fluent. A garbled extraction attracts attention. A grammatical sentence with a changed fact can pass through ordinary language-quality checks and emerge with a credible-looking citation.

The underlying problem is representation, not merely trust in a domain. A reputable source can contain unusual rendering, and an approved page can change. The pipeline needs to record what it actually consumed instead of assuming the URL alone identifies the evidence.

A 2025 research paper on malicious font injection investigates related code-to-glyph mismatches in external resources used by language models. It reports security effects under its experimental conditions. That supports treating the representation gap as a real class of problem; it does not establish that every current assistant or crawler remains vulnerable in the same way.

Compare meaning where the stakes justify it

I would not render and manually inspect every page on the internet. The first candidates are sources that the application treats as authoritative: policy documents, product specifications, pricing terms or instructions that affect a consequential recommendation.

For those sources, preserve the extracted text and a contemporaneous rendered view obtained in an appropriately isolated environment. Compare the relevant content regions, with attention to differences that change names, quantities, conditions or instructions.

An exact character comparison will produce noise. Navigation, hidden accessibility text, formatting and dynamic elements can differ for legitimate reasons. The control should distinguish expected presentation differences from unresolved changes in meaning.

The aim is not to let a vision model declare one representation infallible. Rendering can fail, and optical character recognition can make mistakes. The useful outcome is a record of agreement, a documented benign difference or an unresolved discrepancy requiring review.

Keep the discrepancy out of the trusted index

Imagine a hypothetical assistant answering questions about a supplier’s service limits. A reviewer sees a limit in the browser, while the extracted passage contains a different quantity. I would hold that passage from the trusted index and inspect the source representations before allowing the assistant to cite either number as settled.

If the difference is an extraction bug, fix the extractor and retain the corrected evidence. If the page intentionally serves different representations, seek an authoritative alternative or record the uncertainty. Do not silently substitute whichever version fits the expected answer.

rendering diagram…

The hold applies to the disputed evidence, not necessarily the entire domain. That keeps the response proportionate and avoids turning a useful check into a blanket block on unusual typography.

Preserve enough context to reproduce the finding

A useful capture record includes the retrieval time, the extracted passage, the rendered view and the relevant resources or identifiers needed to investigate the discrepancy. Where permitted, preserving the associated font can help explain what the browser displayed. Retention should follow the organization’s rules for source material and sensitive content.

I would also identify the extraction method and version. If a later parser change alters what the system sees, we need to distinguish a source change from a processing change. Otherwise a regression can look like a publisher quietly rewriting its claims.

This is a proposed workflow, not a claim that I reproduced ShieldFont’s benchmark or tested commercial ingestion systems. The white paper’s reported training-filter results should not be turned into an estimate of how much web data any particular model has consumed incorrectly.

The narrower finding is enough to justify a practical experiment. Take a small set of high-trust sources already used in the application, compare the two representations and record the meaningful differences. The result will tell the team where rendering checks earn their cost.

Verification belongs before the answer

Once a corrupted passage has entered a trusted index, later answer generation may behave exactly as designed: retrieve the passage, summarize it accurately and cite its URL. Better prose cannot repair the evidence boundary that failed upstream.

That is why I would place the comparison at ingestion and repeat it when an authoritative source changes. The goal is to keep the human review attached to the same information the machine will use.

A verified URL is not enough; verify that the reviewer and the retrieval system received the same claim.

#rag#web-ingestion#security#provenance
← older drop
An oversubscribed round still needs a spending thesis
newer drop →
A model-routing price needs an expiry date

related drops

explore all 243 drops →
← back to the archiveday 106