Streaming Pipeline

Most LLM APIs stream responses using Server-Sent Events (SSE). Rdakt AI intercepts these streams and deanonymizes tokens as they arrive, without buffering the entire response.

The Challenge

Streaming deanonymization is harder than it sounds:

1. HTTP chunking — SSE events may be split across HTTP chunks. A single SSE event might arrive in two or three chunks.
2. Token splitting — An anonymization token like <PERSON_1> might be split across SSE events: <PERS in one event, ON_1> in the next.
3. Gzip compression — Anthropic compresses streaming responses with gzip. The raw bytes must be decompressed before SSE parsing.
4. Provider formats — Each provider wraps text content differently in SSE data.

How It Works

HTTP chunks → [Gzip decompress] → SSE reassembly → Content extraction → Deanonymize → Re-emit

Step 1: Gzip Decompression

If the response has Content-Encoding: gzip (Anthropic), the stream is decompressed on-the-fly before any SSE parsing.

Step 2: SSE Reassembly

Raw bytes are accumulated until complete SSE lines are found (terminated by \n\n). Partial lines are buffered across chunks.

Step 3: Content Extraction

Each SSE event's data: field is parsed as JSON, and the text content is extracted based on the provider format:

Step 4: Token-Aware Buffering

The deanonymizer uses a buffer to handle tokens split across chunks:

• Text is scanned character by character
• When a < is encountered, text starts buffering
• If the buffer matches a known token prefix, buffering continues
• If the buffer completes a known token, it's replaced and flushed
• If the buffer can't possibly match any token, it's flushed as-is
• If the buffer exceeds 256 characters, it's flushed (safety valve)

This approach avoids false-positive buffering on content like x < 5 or <div>, because those don't match any token prefix in the session.

Step 5: Re-emit

The deanonymized content is placed back into the SSE event JSON, and the event is re-serialized and emitted to the SDK.

Performance

The streaming pipeline adds negligible latency:

• Regex detection: under 5ms per request
• Deanonymization: O(n) in response length, microseconds per chunk
• No full-response buffering: chunks are processed and forwarded immediately

The user perceives zero additional latency in streaming responses.