LinkDigest

Paste a Xiaohongshu link into Claude Code and the model sees nothing. Not less than usual — nothing.

Here is the whole page you get back from the share link:

<title>小红书</title>

202 KB of app-download shell. No caption, no images, no text. The same is true of Douyin, and of most TikTok links. That is the problem LinkDigest exists to solve, and every part of solving it turned out to be more awkward than expected.

yt-dlp only reads half of Xiaohongshu. Its extractor reads note.video.media.stream — that is a video note. Image notes (图文: a caption plus a stack of photos) are the majority of the platform and usually the ones worth reading, and the extractor has nothing to say about them. Those need the state blob parsed out of the page itself.

The page is different depending on who you say you are. Request a note with a mobile user agent and you get that 202 KB shell. Request the identical URL with a desktop user agent and you get 85 KB containing the real note. Both are HTTP 200. Nothing tells you that you got the wrong one except that the content is not there. This cost me an embarrassing amount of time.

Cold requests are rejected. Fetching a note URL with no prior session does not work. Fetch the /explore feed first, keep the cookie jar it hands you, then the note loads. No account, no credentials — just the same two-step a browser performs without you noticing.

Douyin does not negotiate at all. Captcha on the web page, 403 from the APIs, and yt-dlp does not support the platform. That path needs cookies from a logged-in session.

YouTube blocks your server, not your laptop. The single most common "it worked locally and broke in production" report. YouTube refuses datacenter IP ranges, so a yt-dlp fetch that is perfect on your machine fails from EC2, App Runner or anywhere else you deploy, and no amount of configuration fixes an IP-range block. The workable fallback is a model that watches the video — which costs meaningfully more, so it is worth measuring separately.

What comes back once all of that is handled: transcript with timecodes, the on-screen text, a description and OCR of every image, the caption, key points, and the metadata. Markdown for reading, JSON for pipelines.

A real measurement, from a Xiaohongshu note with 17 images: all 17 described and OCR'd, 381 fragments of on-screen text, 13 key points, 119 seconds. The words that matter on that platform live inside the pictures, and no amount of HTML parsing gets them out.

It runs as an MCP server, which is the part I care about most — the agent calls it itself when it meets a link it cannot open, so you never mention it in a prompt:

claude mcp add --transport http linkdigest https://linkdigest.dev/mcp --header "Authorization: Bearer ld_live_..."

Where it stops, because finding out after you have wired something in is worse than knowing now. Xiaohongshu, Douyin, TikTok, YouTube, X and ordinary web pages all work. Bilibili returns HTTP 412 to a datacenter address and needs a residential proxy. Instagram and Facebook need a logged-in session for most posts. Those three are listed as unsupported on the pricing page, next to the ones that do work.

Three digests free, no card, if you want to throw a link at it: linkdigest.dev

Happy to go deeper on any of the extraction problems above — the user-agent one in particular seems to catch everybody.