How Pullsy's 'Ask this video' actually works

When someone sends you a 14-minute screen recording, you don’t have 14 minutes. You have a question: “What did they change in the onboarding settings?” or “Where do they say the discount code is?”

Pullsy’s Ask this video button is the answer to that. It’s a small chat box on every share page. You type a plain-English question, and the recording answers you — grounded in what was actually said, with clickable timestamps you can jump to.

This post is the architecture diagram. What runs on the worker, what runs in the browser, what we tried and threw away, and how to bolt your own agent onto it.

What it does

Every Pullsy share page (pullsy.online/v/<id>) has a chat box. A viewer types a question like “Where did they say the password?” and gets a 1-3 sentence answer that cites the moment in the recording where it was discussed. The cited moments are clickable chips — clicking one seeks the video player to that timestamp and resumes playback. There’s no login, no signup, no chat history saved against the user.

The architecture

Five steps from question to answer. Each one is small and boring on purpose.

1. Viewer types. The chat input on pullsy.online/v/<id> is plain HTML. On Enter, the browser JS reads the input value and the recording id from the page URL — no global state, no SDK.
2. JS sends to record-api. A fetch('https://record.pullsy.online/api/chat/<id>', { method: 'POST', body: { message } }). The body is JSON; no auth header, no cookies.
3. CF Worker reads transcript from cache. The Worker’s chat.js calls getTranscript(id) against the Cloudflare Cache API. The transcript was written when the recording was uploaded (/api/transcript/:id POST) and has a 30-day TTL matching Pullsy’s free-tier retention. No database roundtrip.
4. Worker streams GPT-4o-mini response. The full transcript (capped at ~16k chars — covers the 95th-percentile Pullsy recording) is sent as a system message alongside a strict prompt: “answer ONLY from the transcript, cite approximate timestamps, say ‘That isn’t covered in this recording’ when the answer isn’t there.” OpenAI’s chat.completions endpoint is called with stream: true, and the worker pipes the response back to the browser as Server-Sent Events.
5. JS appends tokens as they arrive. The browser reads the SSE stream, appends each delta.content chunk to the message bubble as it arrives (no waiting for the full response), and runs a regex over the rendered text to wrap every [mm:ss] in a <button data-t="<seconds>"> chip. Clicking a chip sets video.currentTime and calls video.play().

That’s it. No embeddings, no vector database, no chunking, no RAG framework. A model that fits the whole transcript in its context window, told to be honest when it doesn’t know.

Why we don’t use embeddings

Three reasons, in priority order:

• Cost. Pullsy is free. Embedding every transcript + storing it + running a similarity search on every chat turn would multiply our per-question cost by ~3x. The transcript-in-context approach uses one cheap gpt-4o-mini call.
• Complexity. Embeddings introduce a second data store, an embedding pipeline, a re-rank step, a “did the question match anything?” threshold to tune. The transcript-in-context approach is one fetch. Fewer things break.
• 95th-percentile transcripts fit anyway. Pullsy recordings cap at 5 minutes on free tier. Whisper transcripts of 5-minute screen recordings land at 400-900 words (~2.5-6k tokens). gpt-4o-mini’s 128k context window eats that for breakfast. Even our 30-minute Pro recordings come in under 16k tokens — exactly where we cut the transcript off in chat.js to keep prompt cost predictable.

The honest version: embeddings would be the right call at 100k+ recordings per month, or when transcripts exceed the model’s context window. We’re nowhere near either.

Privacy

Pullsy’s chat is one of the few AI features in 2026 with a privacy story short enough to fit on a sticky note:

• No auth. No login, no signup, no API key.
• No DB write per question. The chat endpoint doesn’t persist anything about who asked what. The transcript and metadata exist; the chat history does not.
• No IP logging. The worker doesn’t log requester IPs for chat calls. (We log a hashed IP for /api/upload rate limiting only.)
• No analytics on chat content. Pullsy’s analytics engine binding records page views and play/pause events. It does not record chat message text, question topics, or response content.

The one thing we do know: the transcript content was sent to OpenAI as part of the system prompt. That’s inherent to the feature. If you record something you don’t want OpenAI to see, don’t enable AI features on that recording.

The agent story: MCP server

Pullsy runs a JSON-RPC 2.0 MCP (Model Context Protocol) server at https://record.pullsy.online/mcp so external AI agents — Claude, ChatGPT, your own bot — can search transcripts and fetch recordings by tool call instead of by hand.

The server exposes four tools: record.list, record.get, record.search, record.transcribe. Manifest at /mcp/tools. Real JSON-RPC example for searching transcripts:

POST https://record.pullsy.online/mcp
Content-Type: application/json

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "record.search",
    "arguments": {
      "query": "discount code",
      "limit": 5
    }
  }
}

Response shape (truncated):

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "query": "discount code",
    "count": 2,
    "results": [
      {
        "id": "8wjpfeum75dq",
        "title": "Q3 product walkthrough",
        "viewerUrl": "https://pullsy.online/v/8wjpfeum75dq",
        "contextSnippet": "...and the discount code for new signups is SUMMER25, valid through August..."
      }
    ]
  }
}

That snippet contains the timestamp-adjacent transcript text — your agent can either point the user at the share URL or call record.get for the full transcript and parse it itself.

How to extend this

The chat handler is ~140 lines — see workers/record-api/src/chat.js in the source tree. Three knobs to turn:

1. Swap the model. Change 'gpt-4o-mini' in streamOpenAI() to 'gpt-4o' for harder reasoning, or to any OpenAI-compatible endpoint by editing the fetch() URL. Anthropic and Google work the same way if you re-implement the streaming transformer.

2. Tighten the system prompt. SYSTEM_PROMPT at the top of chat.js controls tone, citation rules, and refusal behavior. For a “support-bot” flavor, append: “If the user asks how to do X, walk through the steps as if you were the speaker.” For a “compliance audit” flavor: “Quote the speaker verbatim wherever possible.”

3. Add a post-processor. Currently the JS regex turns [mm:ss] into chips. You can add another pass that turns code blocks into syntax-highlighted snippets, or that flags answers containing made-up timestamps (compare against the segment array).

The transcript cap (currently slice(0, 16000)) is the single most important knob. Lower it to cut cost; raise it for long recordings. Past ~32k tokens you’re better off with embeddings.

Compare to alternatives

The clickable-timestamp column is the differentiator. It’s the difference between “they mentioned it around 4 minutes in” (you scrub manually) and “4:12 — and the discount code for new signups is SUMMER25” (one click, video seeks itself).

Try it free

Open any Pullsy recording, hit the Ask this video button in the top right, and ask something the speaker might have actually said. If they said it, you’ll get a timestamped answer. If they didn’t, you’ll get “That isn’t covered in this recording.”

Record your own screen → · Try Ask this video on a sample recording →

No install, no signup, no card.