CivicSignal's playroom and experimental workbench.

Major Events already clusters coverage. This is a different question.

CivicSignal's Major Events panel groups articles that are almost certainly about the same real-world incident — shared entities, a tight date window, a minimum bar of independent publishers. That answers "did multiple outlets cover this?"

This experiment asks something else: can individual stories be linked across a longer window to see a narrative move — a demand, then a response, then a consequence — even when the entities only partially overlap from one link to the next? And separately: can we tell, without reading the text ourselves, whether two linked articles are actually the same report re-published, or a genuinely new development?

Every step here is deterministic. None of it is an LLM judgment call.

This was built, run, and re-run as plain code against content.db. Anyone can re-run the same script on the same data and get the same graph.

1. Entity extraction (already existed) — every article's PERSON/ORG entities, tagged earlier by extract_entities.py via spaCy's en_core_web_sm model. Not this experiment's work, and not an LLM — a real, offline, free NER model.
2. Document-frequency filtering — count how often each entity string appears across the article slice; drop anything above a threshold (police-force names, city names, generic wire bylines). Plain counting, nothing more.
3. Edge construction — for every pair of articles within a day-gap window, take the set intersection of their surviving entities. Three or more shared entities → an edge. That's set_a & set_b and a length check — no text is read at this step.
4. Component detection — union-find over those edges. A standard, textbook graph algorithm.
5. Duplicate-vs-development labeling — for each linked pair, compute title similarity (Python's stdlib difflib.SequenceMatcher, the Ratcliff/Obershelp algorithm — string matching, not meaning) and the share of the later article's entities that are genuinely new. High title similarity → labeled duplicate/republished. Low similarity but mostly new entities → new development. Neither → ambiguous.

The first version linked almost everything together.

Run against 1,075 real "Conflict & Security" articles from Nigeria (three weeks, Aug 2026) with loose settings — any 2 shared entities, a 10-day window, a weak frequency cutoff — one single connected component swallowed 649 of the 1,075 articles. Not a narrative graph, just a blob: everything touches everything eventually once "the Nigeria Police Force" or "Abuja" counts as a real link.

This is the same anti-chaining problem CivicSignal's Major Events detector already solves in production (detect_events.py's MAX_ENTITY_DF / MIN_SHARED_ENTITIES safeguards) — this experiment just re-discovered why those exist by hitting the same wall first.

Nigeria, Conflict & Security, Aug 21–26 2026 — found by the algorithm above, unedited.

Each dot is a real published article, positioned chronologically top to bottom. Each line is a real computed edge between two articles that shared 3+ surviving entities within 5 days of each other, colored by the duplicate/development label above.

Known limitations, not smoothed over.

• No anchor-entity check across a whole chain. A separate component (not shown above) links a report of "20 killed" in Plateau to a later report of "23 residents" killed nearby — this could be one incident's toll being revised, or two distinct attacks merged because they share generic place/entity terms. The current method only checks neighboring pairs, not whether one specific identifying detail (a named victim, a case number) persists across the entire chain.
• Duplicate-vs-development is a two-signal heuristic, not a verified label. Title similarity and new-entity ratio are cheap, transparent, and reproducible — but they're proxies. A rewritten headline with the same facts can score as "new"; a genuinely developing story with a formulaic headline can score as "duplicate."
• English-language bias. Title similarity via string matching works less reliably across languages in the same corpus (French, Arabic, Wolof coverage of the same event) than within one language — this experiment only ran on English-tagged articles.
• Parameters were hand-tuned on one slice. The thresholds that produced 157 legible components on Nigeria/Conflict & Security/Aug 2026 aren't validated on other topics, countries, or time windows — they might need to move.

Every story assigns roles. Usually silently.

The same event can cast the same actor as protector in one outlet and threat in another — "police restore order" versus "police crack down." This experiment annotates each article's actors as Hero, Threat, Victim or Neutral, then clusters articles so that stories with genuinely conflicting casts of the same actor are pushed apart ("cannot-link" constraints) rather than lumped together by topic alone. Outlet identity and any bias label are withheld from the clustering step, and only checked afterward, against the resulting clusters.

Run against 45 real climate-news articles, human-labelled in advance.

Rather than invent an example, this ran against the Narrative Frames Corpus (Frermann, Li, Khanehzar & Mikolajczak, ACL 2023) — 428 English-language climate-change articles from the NELA news archive, human-coded for Hero/Villain/Victim roles, Media Bias/Fact Check (MBFC) rating and topical frame. A balanced 45-article subset (15 right-leaning, 15 left-centre, 15 left-leaning by MBFC) was clustered with and without the role constraints:

Seven named clusters came out legible enough to read directly — real outlets, real dates, real role counts, no synthetic examples:

Numbers and cluster labels above are a specific reproduction run, not a claim of exactly replicating the original Das et al. paper's own reported results.

Limitations and Caveats

• In-sample improvement doesn't guarantee generalisation. A follow-up reproduction, using 5 random train/test splits, found that raising the role-constraint weight lifted in-sample role purity from 0.73 to 0.80 — but held-out role-prediction accuracy on unseen articles fell, from 64.4% to 53.8%. The constraint helped the model fit the labelled data more closely without helping it predict roles on new stories.
• This is not a refutation of the underlying method — it's a caution about which metric gets reported. A pipeline can look better on the number you're tracking and worse on the one you actually care about.