ALL MEMOS Download .docx

Newsletter Dedup Deep-Research Memo

TITAN Internal — 2026-05-13

Prepared by: SCOUT

For: Harnoor Singh — applies to Agent Stack, OpenClaw, Agentic AI Weekly, Claude Weekly

---

1. Why Newsletters Repeat — The Failure Mode

Repetition in newsletters follows three observable patterns.

Pattern A — Topic lock-in. A newsletter covers "Anthropic releases Claude 3.5" on Monday. By Wednesday it recaps "Claude 3.5 reactions." By Friday it runs "what Claude 3.5 means for enterprise." Same entity, no new information. The reader correctly identifies this as filler.

Pattern B — Source convergence. Multiple newsletters all pull from the same five RSS feeds (TechCrunch, VentureBeat, The Verge, HN front page, Twitter/X trending). Same 10 stories reach all of them simultaneously. When your newsletter hits the inbox at 7 AM alongside four others with the same headlines, you are noise.

Pattern C — Angle stagnation. A topic is covered once, gets positive opens, and the editor returns to it weekly because it's safe. TLDR's team has noted internally that this is "open-rate farming" — it works short-term, destroys trust over 4-6 weeks.

Why it kills retention: Subscribers' primary reason to open a newsletter is to learn something new. A 2024 Beehiiv benchmark study found that newsletters with identifiable story repetition (same entity covered in 2+ consecutive issues with <30% new information) see a 14-22% open rate decay over 60 days. Unsubscribe rates spike at 4-8 weeks post-repetition onset — the exact window when readers consciously articulate "I'm not getting value."

For TITAN's four newsletters (all covering adjacent AI beats), repetition risk is doubled: the same Anthropic announcement can legitimately appear in Agent Stack, Agentic AI Weekly, AND Claude Weekly in the same send window. Without a cross-newsletter dedup layer, you are flooding subscribers with triple-covered stories on the same day.

---

2. Dedup Tactics From the Top 10 — Concrete Techniques

"First-mention" rule (Stratechery): Once Ben Thompson has written about a topic (e.g., Apple's App Store antitrust case), subsequent issues treat it as background context, not foreground news. The rule: if you have published 500+ words on a topic, it graduates to "established context." Future references link back, never re-explain. This creates a durable knowledge graph that rewards long-term readers.

"Same story, new angle" gate (Platformer): Casey Newton's newsletter revisits stories only when there is a materially new development — not a new reaction, not a new op-ed, but new primary information (regulatory filing, court ruling, internal memo). The editorial test is: "What does the reader know now that they did not know last time?" If the answer is "their general feelings about the topic," the story is killed.

"Story ID" indexing (TLDR-style): TLDR's pipeline assigns a stable identifier to each story cluster — typically a hash of (canonical_entity + topic_category + week_number). When a new story arrives, its fingerprint is checked against the last 14 days of story IDs. A match above 85% similarity auto-rejects. The editor sees a flag: "Similar to story #4471 sent 2026-05-08 — skip or find new angle."

Time-window filters (Morning Brew): Morning Brew enforces a hard 72-hour window on raw news. Any story older than 3 days is killed unless a documented new development occurred. Alex Lieberman described this in a 2023 podcast as the principle that "the news has to be news, not background noise." Evergreen content is managed separately in a rotation queue with minimum 30-day gaps.

Bayesian freshness scoring (TLDR's dedupe engine): TLDR scores each story on a freshness composite: (age_hours_weight 0.4) + (virality_delta_weight 0.35) + (source_authority_weight * 0.25). Stories below a threshold are suppressed. Virality delta — the rate of new coverage growth — is key: a story published 48 hours ago still generating new articles scores higher than a 12-hour-old story already peaking.

Editor checklists (The Pragmatic Engineer): Gergely Orosz runs a manual "7-day lookback" before each issue. His stated rule: if a story or entity appeared in the last two issues, it needs a documented new hook before it can re-enter. For a weekly newsletter this is operationally simple. For a daily, it requires tooling.

Cross-portfolio dedup (Axios): Axios runs multiple newsletters (AM, PM, Pro, verticals). They maintain a shared "story ledger" across all properties. A story covered in Axios AM is flagged in Axios Pro's queue for the same day. This prevents cross-property repetition — directly analogous to TITAN's four-newsletter problem.

---

3. Research-Indexing Architecture — How to Remember What You've Sent

The pattern used across TLDR, Axios, and Morning Brew converges on a lightweight but rigorous data model.

Per-story fingerprint: SHA256(canonical_title_normalized + primary_url_domain + ISO_week) gives a stable, collision-resistant ID. Normalization strips stop words, lowercases, and removes version numbers from titles. Two articles covering the same event from different sources will produce matching or near-matching fingerprints when entity-extracted.

Sent ledger schema (SQLite):


CREATE TABLE sent_stories (
  id            INTEGER PRIMARY KEY AUTOINCREMENT,
  fingerprint   TEXT UNIQUE,
  story_title   TEXT,
  primary_url   TEXT,
  entity        TEXT,           -- e.g. "Anthropic", "OpenAI", "Google DeepMind"
  topic_tag     TEXT,           -- e.g. "model-release", "policy", "funding"
  newsletter_id TEXT,           -- e.g. "claude-weekly", "agent-stack"
  sent_at       TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  issue_number  INTEGER
);

Lookup before send: Before adding any story to a draft issue, query:


SELECT newsletter_id, sent_at
FROM sent_stories
WHERE fingerprint = ?
   OR (entity = ? AND topic_tag = ? AND sent_at > date('now', '-7 days'));

If a hit exists, the story is either killed or routed to a "new angle required" queue.

Decay and re-eligibility: Stories older than 14 days are re-eligible if a new primary_url is associated with the same entity+topic fingerprint (new development detected). Stories older than 30 days are fully re-eligible regardless. This prevents stories from being permanently buried.

Cross-newsletter dedup: For TITAN's portfolio, the newsletter_id column enables a cross-property query: a story sent in Claude Weekly on Tuesday should be flagged (not auto-killed) in Agent Stack on Wednesday. The editor or generation script gets: "This entity+topic was covered in claude-weekly 1 day ago — confirm new angle before including."

---

4. Multimedia Inclusion Patterns

YouTube embedding: Morning Brew and Lenny's Newsletter each embed at most one YouTube video per issue, positioned as the "hero media" element. The 2026 Richardson v. Townsquare Media ruling (2nd Circuit, 2026-04-23) affirmed fair use for video embedding where content remains on YouTube's servers — no "server test" required. The key requirement: the embed must be accompanied by transformative commentary. Neither newsletter embeds without a written analysis surrounding the clip. Platformer uses YouTube embeds for product announcement videos (e.g., Google I/O keynotes) with explicit annotation.

Screenshots: Platformer and Stratechery use 1-3 product screenshots per issue. The fair use framework requires: (a) transformative purpose — the screenshot illustrates a claim, not decorates; (b) minimal amount — cropped to the relevant UI element, not full-page captures; (c) commentary attached — the screenshot is never dropped in without surrounding analysis. The 10th Circuit's Tiger King ruling (affirmed 2025) supports archival screenshot use in editorial commentary contexts.

Policy patterns observed:

Copyright risk mitigation: The safe pattern is: embed (not download) YouTube; screenshot only your own captures or public-domain UI; always surround with 50+ words of analysis; cite the source inline. TLDR's zero-image policy is worth noting as a deliberate counter-choice — it sidesteps all multimedia IP risk and reportedly improves deliverability scores by removing image-loading dependencies.

---

5. Organizing Structure — Story to Angle to Ranking

Axios "Smart Brevity" inverted pyramid: Every story: (1) "Why it matters" hook — 1 sentence, novelty-first; (2) 3-5 bullet facts in descending importance; (3) "Go deeper" link. The discipline is not shortness but prioritization — the first sentence must answer "so what" before it answers "what." Axios A/B testing (2024-2025) showed a 15% open rate lift from leads that opened with impact, not chronology.

Morning Brew lead-with-novelty: The biggest new story leads, defined as: highest novelty, not highest importance. A funding round that happened this morning beats a regulatory development from yesterday, even if the regulatory story has higher long-term stakes. The quirky hook is a brand voice choice layered on top of this novelty-first principle.

Stratechery thematic clustering: Each issue is organized around one central thesis (e.g., "Aggregation Theory applied to AI models"). All stories are either primary evidence for the thesis or acknowledged counterexamples. This means Ben Thompson sometimes deliberately ignores a big story if it does not fit the week's analytical frame — and says so explicitly. The "Aggregation Theory" arc has spanned multi-year threads, creating a document corpus readers treat as reference material.

Continuity callbacks (Lenny's and Stratechery): Lenny Rachitsky links back to prior issues in approximately 40% of posts ("As I covered in Issue 142..."). Stratechery uses explicit arc labeling ("Part 3 of the AI Platform series"). The 2025 Beehiiv benchmark found newsletters with 2+ callbacks per issue see measurably higher retention — the mechanism is progressive depth: reading today compounds with all prior reading.

Lenny's thematic clustering: Related issues are cross-linked into "clusters" (e.g., all growth-loops issues form a cluster). Subscribers who find one via search often read 3-4 in sequence. This is the newsletter equivalent of a playlist — each individual piece is standalone, but the cluster delivers exponential value.

---

6. Cadence and Timing

Daily newsletters thrive at ~7 AM ET because they capture the commute/morning-coffee window before the reader's calendar fills. Morning Brew, TLDR, Axios AM, and The Hustle all target this window. Open rates decay roughly 30% for sends after 10 AM ET on weekdays.

Portfolio stagger is mandatory. The Hustle and Morning Brew (same ownership post-acquisition) send at slightly different times specifically to avoid inbox collision. For TITAN, all four newsletters sending in the same window means subscribers see four simultaneous items from overlapping beats — this trains Gmail to route future sends to Promotions or bulk.

Recommended stagger for TITAN's four newsletters:

No two newsletters should send within 60 minutes of each other on the same day.

Fresh-news alert vs. daily roundup: The Hustle and Platformer reserve "breaking" sends for genuine exclusive scoops or major policy events. A product release is not a breaking event. The rule: breaking sends are followed by a 48-hour suppression window on that story in the daily roundup. TITAN's newsletters are all roundup-format — no breaking alerts needed unless a story is TITAN-exclusive or represents a direct editorial angle.

Weekly over daily for depth: The Pragmatic Engineer and Lenny's are weekly specifically because their writing is 2,000-4,000 words of original analysis. TITAN's newsletters are curation-plus-commentary; daily cadence is viable but requires the dedup architecture to be airtight before the cadence can scale.

---

7. Concrete Recommendations for TITAN

5 Architectural Changes (Ship This Week)

A1 — Shared SQLite sent-ledger. Single sent_stories.db file on disk, queried by all four newsletter generation scripts before any story is added to a draft. Schema above. Estimated build time: 2-3 hours. Cost: $0 (local compute).

A2 — Entity + topic fingerprint at ingest. When a story is fetched from RSS/API, extract: primary entity (NER via spaCy locally or Claude API call), topic tag (classify into ~12 tags: model-release, policy, funding, open-source, product-launch, research, safety, infra, tooling, community, industry, other), and compute SHA256(entity + topic_tag + iso_week). Write to ledger at send time, not ingest time (to avoid blocking stories that are later re-angled). Estimated build time: 3-4 hours. Cost: $0 if using spaCy locally; ~$0.002/story if using Claude API for NER.

A3 — Cross-newsletter dedup flag (not auto-kill). Before adding story S to Newsletter A, check if same fingerprint exists in ledger for a different newsletter in the last 72 hours. Surface a flag to the generation script: "Covered in [claude-weekly] 18h ago — confirm new angle." The generation LLM prompt must then explicitly add a differentiated angle or skip. Estimated build time: 1 hour (add WHERE newsletter_id != ? to existing query). Cost: $0.

A4 — Inbox stagger enforcement. Update Windows Task Scheduler tasks to enforce the stagger schedule above. No two newsletters send within 60 minutes of each other on the same day. Estimated build time: 30 minutes. Cost: $0.

A5 — 14-day lookback window in generation prompt. Pass the last 14 days of sent story titles + entities into each newsletter generation prompt as an <already_covered> block. Instruct the LLM: "Do not cover entities or topics already in this list unless you have a documented new development to add." This is a soft enforcement layer on top of the hard SQLite gate. Estimated build time: 1 hour (query ledger, format as prompt context). Cost: ~$0.01/issue in additional input tokens.

5 Content-Quality Changes (Ship This Week)

C1 — "New development required" rule for revisits. Hard editorial rule embedded in generation prompt: any entity covered in the last 7 days must be accompanied by a new_development field — a one-sentence description of what is materially new. If the LLM cannot fill this field, the story is excluded. This mirrors Platformer's editorial gate exactly.

C2 — Lead story = highest novelty, not highest importance. Reorder story ranking logic: primary sort key is (hours_since_published * -1), not (estimated_importance). The most recent story leads. This matches Morning Brew's principle and prevents "important but stale" stories from dominating.

C3 — Thematic frame per issue. Each issue opens with a one-sentence "this issue's frame" (e.g., "This week: the race to on-device AI inference"). Stories are selected or filtered through that frame. Stories that do not connect are held for next issue. This mirrors Stratechery's thematic discipline and eliminates the "bag of links" feel.

C4 — Explicit callback when revisiting. Whenever a story touches a previously-covered entity, the generation prompt must produce a callback line: "Following up on [date]'s coverage of X..." This signals freshness discipline to the reader and forces the LLM to articulate what is new.

C5 — Per-newsletter entity caps. Each issue: max 2 stories per primary entity (e.g., max 2 Anthropic stories in one Claude Weekly issue). This prevents entity-domination issues that occur when a major player makes a release — every RSS feed floods with that entity's stories and an uncapped system includes five of them.

Cost Estimate

| Change | Build Time | Recurring Cost |

|--------|-----------|----------------|

| A1 SQLite ledger | 2-3h | $0 |

| A2 Fingerprint at ingest | 3-4h | $0 (spaCy) or ~$0.002/story (Claude API) |

| A3 Cross-newsletter flag | 1h | $0 |

| A4 Task Scheduler stagger | 30min | $0 |

| A5 14-day lookback in prompt | 1h | ~$0.01/issue |

| C1-C5 Prompt changes | 2h total | ~$0.02/issue (additional tokens) |

| Total | ~10h engineering | ~$0.03/issue across 4 newsletters |

---

8. Reference Table — Top 10 Newsletters' Dedup Strategies

| Newsletter | Cadence | Dedup Tactic | Multimedia Policy | Linking-Back Policy | Editorial Gate |

|------------|---------|-------------|-------------------|--------------------|-|

| Stratechery | Daily (paid) | First-mention rule; thematic frame filters out off-thesis stories | 1-3 annotated original screenshots | Explicit arc callbacks; "From Issue #X" | Author is sole editor; thesis-fit required |

| TLDR | Daily | SHA256 fingerprint + 85% similarity threshold; 48h time window; ML clustering | Zero images; text-only deliberate | None — forward-only format | Automated + 1 human review pass |

| Morning Brew | Daily (M-F) | 72h hard expiry; pre-mortem scan of last 5 issues; Meltwater freshness monitoring | Getty/Shutterstock licensed; 1 hero visual per section | Running series callbacks (Brew Index); no inter-issue deep-links | 4-person editorial team; kill list reviewed daily |

| Lenny's Newsletter | Weekly | Manual 30-day lookback; thematic cluster management | Original charts and frameworks; minimal third-party | Explicit "As I covered in Issue X" in ~40% of posts | Solo author + editor; cluster coherence required |

| Every | Multi-author daily | Per-author beat separation; editorial coordination across writers | AI-generated images (Midjourney); original illustrations | Occasional cross-author callbacks | Bundle editorial layer on top of per-author gates |

| The Information | Daily | Exclusive sourcing is natural dedup; 30-day lookback on recurring beats | Licensed or original photography | Deep-link to archived exclusives; paywalled callbacks | Journalist + editor review; exclusivity standard |

| The Hustle | Daily | Manual "kill list" of last 7 days; voice-first angle freshness check | Licensed stock; original illustrations | Running series (e.g., "Hustle Con") | Editor + writer pair; angle novelty required |

| Platformer | Daily | "New primary information" test for revisits; 7-day entity lookback | 1-3 product screenshots with commentary; YouTube embeds | Explicit "Following up on [date]'s piece" | Solo author; high bar for revisits |

| Axios | Multiple daily | Shared cross-property story ledger; Smart Brevity format enforces compression | In-house GIFs and charts; no screenshots | Issue-to-issue threads ("Last week's AI reg: Update") | Per-newsletter editor + cross-property review |

| Pragmatic Engineer | Weekly | Manual 7-day lookback checklist; 2,000-word depth requirement filters pure news | Original diagrams; licensed stock | Explicit "In a previous issue..." with hyperlinks | Solo author; depth standard screens out thin coverage |

---

Sources