Extractive vs Abstractive: Two Very Different Machines
Every automatic summarizer belongs to one of two families, and knowing which one you are using changes how much you should trust the output.
Extractive summarizers select sentences that already exist in the source. Classic algorithms like TextRank build a graph of sentences, score them by similarity to the rest of the document, and return the top-ranked ones verbatim. The result is often choppy — pronouns dangle, transitions break — but it has one enormous virtue: every sentence in the summary is guaranteed to appear in the original. An extractive system cannot invent a fact.
Abstractive summarizers, which include every LLM-based tool, generate new sentences that paraphrase and compress the source. The output reads fluently, merges points across paragraphs, and can adapt tone and length on request. The price is that generation is exactly the mechanism that produces hallucination: the model writes the most statistically plausible summary, and plausible is not the same as faithful. Research on abstractive summarization has consistently found "unfaithful" content in machine summaries — details that sound right but are not in, or are contradicted by, the source.
Practical consequence: use abstractive summaries for speed and readability, but treat every specific claim — numbers, names, dates, causal statements — as unverified until you have checked it against the original. The fluency of the summary tells you nothing about its accuracy.
Compression Ratio: Deciding What a Summary Is For
"Summarize this" is an underspecified request, and underspecified requests get generic answers. The single most useful parameter you can set is the compression ratio — how much of the original survives — because it forces you to decide what the summary is for.
A 1-sentence summary (roughly 50:1 compression on a typical article) answers "should I read this at all?" It must name the topic and the single most important finding, nothing else. A 3-5 bullet summary (about 10:1) answers "what would I have learned?" — it should carry the main claims and the strongest piece of evidence for each. A 1-2 paragraph summary (about 4:1) answers "can I discuss this without reading it?" and needs the argument structure: what was claimed, what supports it, what the caveats were. Beyond that, you are writing an abridgement, not a summary.
Two settings matter almost as much. Audience: a summary of a medical paper for a physician keeps effect sizes and confidence intervals; the same paper for a patient keeps what changed and what to ask a doctor. And extraction targets: telling the tool "always preserve all numbers, dates, and named people exactly as written" measurably reduces the most damaging class of summary errors, because it steers the model away from paraphrasing precisely the details it is most likely to garble.
Vague request:
"Summarize this article."
Specified request:
"Summarize the article below.
- Length: exactly 4 bullets, max 20 words each
- Audience: a product manager deciding whether to read it
- Preserve ALL numbers, dates, and names exactly as written
- Final line: one sentence on what the author wants readers to do
- If the article contradicts itself, say so instead of smoothing it over"Hallucination in Summaries: Where It Hides and How to Catch It
Summary hallucinations are rarely dramatic inventions. They are small, confident distortions that survive a casual read. Knowing the recurring shapes makes them much easier to catch.
Entity swaps: the study found X, but the summary attributes it to the wrong group, drug, or company — especially when the source discusses several. Number drift: 18% becomes 80%, "nearly half" becomes "most," millions become billions. Modality laundering: "may reduce risk" becomes "reduces risk"; a hypothesis in the source becomes a finding in the summary. Causal upgrades: "A was associated with B" becomes "A causes B" — the most common and most damaging distortion in summaries of research. Negation loss: "no significant difference was found" quietly becomes "a difference was found." And premature synthesis: when a document presents two opposing views, the model sometimes merges them into one blended claim that neither side made.
A five-minute verification routine catches most of these. Extract every number, date, name, and causal verb from the summary, then search the original for each one — checking the summary against the source is far faster than re-reading the source. Ask the tool to quote the exact sentence supporting each bullet; a claim it cannot ground in a quotation is the claim to distrust. And for anything you will forward to other people, re-summarize in a second pass and compare: inconsistent details between two runs are a reliable hallucination signal.
Source sentence:
"In the trial, drug A was associated with an 18% relative
reduction in events; the difference was not statistically
significant (p = 0.09)."
Hallucinated summary (3 distortions in one line):
"Drug A significantly reduces events by 18%."
1. association -> causation ("reduces")
2. "not significant" -> "significantly"
3. relative reduction presented as absolute effect
Faithful summary:
"Drug A was associated with 18% fewer events,
but the result was not statistically significant."Long Documents: Chunking, Map-Reduce, and What Gets Lost
When a document exceeds what fits comfortably in one request, summarizers fall back to a map-reduce strategy: split the text into chunks, summarize each chunk, then summarize the summaries. It works, but each stage is lossy, and the losses are not random. Cross-chunk connections die first — an objection raised in section 2 and answered in section 7 becomes two disconnected fragments, or the objection survives while the answer does not. Themes that recur weakly everywhere but peak nowhere (a running caveat, a recurring minor character) fall below every chunk's local relevance threshold and vanish entirely.
There is also a position bias worth knowing about: models weight the beginning and end of a long context more heavily than the middle — the "lost in the middle" effect documented in retrieval research. In practice, the middle third of a long report is where summary coverage is thinnest.
You can push back on all of this. Split on structural boundaries (chapters, sections) rather than fixed character counts, so no chunk starts mid-argument. Ask for chunk summaries that explicitly note open questions and forward references, so the reduce step can reconnect them. For contracts and papers, run a second targeted pass: "list every obligation/limitation/date in this document" — enumeration prompts have much better recall than free-form summaries. And when a document really matters, summarize it twice with different chunkings and merge what each run missed.
Targeted second pass (much higher recall than a summary):
"From the contract below, produce three exhaustive lists:
1. Every obligation of the Customer (quote the clause number)
2. Every fee, amount, and payment deadline
3. Every termination or renewal condition
If a clause is ambiguous, list it under 'UNCLEAR' with the
clause number. Do not omit items to save space."When Not to Summarize
A summary is a decision about what does not matter, delegated to a machine. Sometimes that delegation is wrong regardless of quality.
Do not rely on summaries of documents whose force is in exact wording: contracts, laws, medical dosing instructions, safety procedures, exam requirements. A paraphrase of "must" as "should," or the silent omission of one exception clause, changes the meaning at precisely the point where meaning has consequences. Use enumeration prompts to build a checklist, then read the flagged clauses yourself in the original.
Be cautious with texts whose point is the experience of reading them — literary writing, persuasive essays, dense philosophical arguments. A summary of these captures the "what" and destroys the "how," and the "how" was often the content. Similarly, summarizing a heated email thread strips the tone that tells you who is frustrated and why; skimming the originals is usually safer before you reply.
Finally, mind the compounding problem: summaries of summaries degrade fast. Each pass re-applies the same biases — position bias, modality laundering, loss of caveats — so by the third generation the text can be fluent, confident, and substantially disconnected from the source. Keep a link to the original attached to every summary you store or share, and when a decision hangs on a detail, let the summary tell you where to look, not what is true.