Why AI cleanup should not be the extraction engine

The most misleading PromptReady output was not the obviously broken Markdown. It was the Markdown that looked cleaner after AI cleanup but had quietly lost the structure of the page.

That is a dangerous failure mode because the result feels better at a glance. Shorter paragraphs, smoother headings, and less clutter can hide the fact that commands, configs, headings, or examples disappeared.

The wrong AI boundary

The first AI boundary was too loose: give the model page source and ask for clean Markdown.

That asks the model to do extraction, cleanup, structure preservation, and editorial judgment at the same time. It may decide that a heading is redundant, that a config block can be summarized, or that a long section should be condensed.

For PromptReady, that is not acceptable. The tool is supposed to preserve source context for later use. A prettier summary is not a faithful capture.

The broken technical Markdown problem

Technical content made the failure obvious.

One class of bug looked like this:

doctor```The installer writes managed client config.

Another looked like collapsed config:

MCP Config[mcp_servers.satori]
command = "npx"
args = ["-y", "@zokizuan/satori-mcp@4.9.1"]

Those are not just spacing issues. They break the contract of a Markdown export. A command should remain a command. A TOML or JSON block should remain a fenced block. Inline prose should not be glued to a closing fence.

Offline baseline first

The fix was to build the offline Markdown first and pass it into the AI prompt as the baseline.

That changed the AI job from:

Turn this raw HTML into clean Markdown.

to:

Clean up this Markdown without summarizing it, dropping headings, breaking fences, or changing technical blocks.

The implementation also budgets the source context so the offline baseline gets priority over raw HTML. That is the correct tradeoff for this product. The local Markdown is already the best structured view of the capture. Raw HTML is secondary context, not the primary contract.

Quality gates

PromptReady now evaluates AI output against the offline baseline before accepting it.

The gate is deliberately practical. It checks for malformed fences, heading loss, heading order loss, content loss, and technical token loss. If the output fails, the extension returns the offline result instead and attaches a stable fallback warning such as ai_fallback:quality_gate_failed.

That warning matters. It means the user still gets usable Markdown, but the system is honest that AI cleanup was rejected.

The tests make that boundary concrete:

expect(result.aiOutcome).toBe("fallback_quality_gate_failed");
expect(result.fallbackCode).toBe("ai_fallback:quality_gate_failed");
expect(result.warnings).toContain("ai_quality_gate:heading_loss");
expect(result.warnings).toContain("ai_quality_gate:content_loss");

Shared repair, not AI-only repair

Another useful change was moving technical Markdown repair into shared canonicalization paths. The offline path and AI path both need protection against glued fences, whole-document Markdown wrappers, split config blocks, and malformed code fences.

If the repair only exists after AI, then offline mode becomes weaker than it needs to be. If the repair only exists offline, AI can still reintroduce broken formatting. Shared canonicalization keeps the contract in one place.

The shared normalizer is intentionally narrow:

export function normalizeTechnicalMarkdownBlocks(
  markdown: string,
  warnings?: string[],
): string {
  let result = markdown;
  result = repairCollapsedMcpConfig(result, warnings);
  result = repairFenceBoundaries(result, warnings);
  return result;
}

That is the right kind of cleanup for this product: repair structural Markdown damage without asking the model to reinterpret the source.

What this taught me

AI cleanup is useful when it is constrained. It is risky when it becomes the extraction engine.

For PromptReady, the right boundary is local-first:

1. Build the best offline Markdown document possible.
2. Ask AI to clean it without changing the source structure.
3. Verify the result against the baseline.
4. Fall back when the AI output is less faithful.

That is less exciting than accepting the nicest-looking response. It is also the only version I trust for a tool whose job is to preserve source material.

The tradeoff

Quality gates create false negatives. Sometimes an AI cleanup may be readable and still get rejected because it lost headings, commands, or technical tokens.

That is acceptable for this product. PromptReady is not trying to produce the prettiest summary. It is trying to preserve a useful source document.