Fragile dependency
Screenshot → exact source → useful card
A private, deleted, cropped, or poorly indexed post breaks the entire chain before the user receives value.
Product decision
Enrich the screenshot.
Stop requiring the original URL.
WeChat and Xiaohongshu screenshots rarely expose durable source identity. Search can still recover useful, grounded context, so the better product tradeoff is an improved card with carefully admitted related evidence.
What we learned
| Experiment | Result | Meaning |
|---|---|---|
| Sogou Weixin | 0 / 5 sources | Temporary result URLs and weak title identity could not establish a canonical article. |
| Self-hosted SearXNG | 0 / 5 sources | Owning the metasearch layer did not create coverage in the upstream indexes. |
| Bocha domain search | 45 candidates, 0 commits | Finding the right platform domain is not the same as finding the screenshot's source. |
| Related-evidence review | 22 / 29 useful | The same screenshot text often retrieved context worth showing, despite finding zero original URLs. |
| Blind Exa comparison | 28 / 29 useful | Exa was the strongest retrieval prototype; Bocha remained occasional corroboration. |
| Fresh product canary | 12 / 15 shown | Twelve cards earned one related resource, three correctly abstained, and no second link added enough value. |
The tradeoff
Recommended product path
Screenshot → grounded card → optional related resource
The screenshot remains useful immediately. Search adds context only when a strict quality policy admits it.
This is a fallback, not a surrender. Visible URLs, QR codes, user-supplied links, and existing stored URLs remain canonical. Source discovery can continue as a separate precision-first capability that is allowed to abstain.
How it works
One reusable capture package owns the contracts and modules. Search providers and the model judge are replaceable adapters. Deterministic code makes the final admission decision.
1Read the screenshotVersioned vision signals capture visible text, platform hints, handles, dates, and URLs.
2Retrieve bounded evidenceExa is the primary prototype; Bocha, xAI, and Serper remain independently removable providers.
3Judge the candidatesA provider-blind model evaluates directness, authority, usefulness, confidence, freshness, and risk.
4Apply fail-closed policyCode blocks unsafe claims, conflicting source guesses, duplicate links, and weak second resources.
5Render the product resultThe description improves, zero or one related resource appears, and the section disappears entirely when nothing qualifies.
Guardrails
- Related evidence never replaces the Resource source URL.
- A model recommendation cannot bypass deterministic policy.
- High-stakes claims require stronger sources, explicit caveats, repair, or abstention.
- Raw screenshots remain local; providers receive bounded, privacy-reviewed text queries.
- Production activation waits for a new labelled holdout with negative samples.
Next step
Build one production package, @snapaction/capture, and one lab tool. Develop in an isolated feature worktree from current develop, integrate agent work in supervised waves, and keep production activation separate from the first implementation PR.
