SynthID-Text: Why More Tokens Don't Always Mean More EvidenceSynthID-Text: Why More Tokens Don't Always Mean More Evidence

I want to answer a somewhat narrow infrastructure question: what happens to watermark detection when a document becomes a short chunk, gets edited, or has text duplicated on its way into a retrieval index? I'm assuming a known watermark configuration and its matching tokenizer; the public reference keys can't identify arbitrary Gemini output. Even with these prerequisites, there is quite a lot hidden behind one detector score.

Copying a marked passage is like photocopying a lottery ticket. There is more paper, sure, but the copies don't increase chances. That explains the evidence problem, but it doesn't explain how the mark is created. For that, I'll trace the sampler and run a controlled token experiment.

I follow Google DeepMind's research reference at commit addb4a1, from June 2025. That snapshot explicitly says it isn't production software. Its sampler and detector make a useful controlled experiment: both the preference and the counting depend on context. Detection provides statistical evidence under a known configuration, but doesn't establish authorship or factual truth.

Tournament in the sampling distribution

In this reference, ngram_len=5 means four preceding token IDs plus the candidate token, with each layer key producing a binary g value for that combination. The original paper's tournament starts with 2^m candidates sampled from the next-token distribution, allowing duplicates. Pair them, prefer the candidate with the higher g-value at that layer, break ties randomly, and repeat for m rounds. That's our one survivor. This happens for each emitted token, all while the model isn't asked to generate and compare whole answers.

30 layers!? That must be costly! The important detail is that update_scores calculates the resulting probabilities directly. It doesn't materialize the theoretical billion candidates.

For a single binary round, let G be the current probability mass on candidates with g=1:

G = sum(p[token] * g[token])
p_next[token] = p[token] * (1 + g[token] - G)

Let's take A and B as two possible tokens with probabilities 0.7 and 0.3, and g-values 0 and 1 respectively. With G=0.3, A becomes 0.7 × 0.7 = 0.49, while B becomes 0.3 × 1.7 = 0.51. That's also the probability that at least one of two tournament draws is B: 1 - 0.7². So you can change the favourite without creating a token outside the distribution.

The processor first applies temperature and top-k, then repeats that update across layers. If all available candidates have the same g-value at a layer, nothing changes.

Repeated contexts affect generation and detection

At first I thought repetition was just something to clean up in the retriever. The reference makes it part of the sampling contract.

During generation, the processor hashes the last four tokens and compares that hash with a rolling history. If there's a match, it uses the original temperature and top-k scores for that step. This avoids strengthening correlations by applying the same keyed preference repeatedly. The reference's default history holds 1,024 context hashes, not an unlimited record across documents or requests.

The detector's compute_context_repetition_mask reconstructs previous contexts and sets repeated ones to zero. It deduplicates contexts, not words or sentences. A familiar word in a new context can still help, but even a different next token after an already seen context is masked.

This is one of those hashing and HMAC traps where just because it's keyed doesn't mean it's an authentication mechanism. This reference snapshot cautions that its accumulate_hash provides no cryptographic security guarantee. It uses SHA-256 for the initial hash state, but the accumulator is an adapted linear congruential calculation. A high score is not a signature.

Keep samples constant but vary input to detector

I sampled 512 marked sequences once, each 256 tokens long, from a uniform vocabulary of 64 token IDs. The reference processor used four explicit layer keys, a four-token context, and 1,024 history entries. Four layers keep this synthetic CPU experiment small; these aren't the default thirty-layer results. Two disjoint sets of 2,048 unmarked sequences supplied calibration and held-out controls.

I scored the retained positions with the reference's weighted-mean detector, which gives earlier tournament layers more weight. For each length and transformation, I chose a 99th-percentile threshold from the unmarked calibration set, then counted results on the separate unmarked controls and marked sequences. Positive examples never set the threshold. Detection uses score > threshold, so ties stay negative.

Every row applies the same transformation to marked and unmarked samples. Prefixes test length; a middle crop tests the loss of surrounding context; replacement adds 17 modulo 64 to every fourth token; duplication repeats the first 32 tokens eight times. Each row uses its own threshold from identically transformed calibration controls. These are results from the synthetic CPU experiment, with thresholds rounded only for display:

The held out false positive rates range from 0.73% to 1.42%, near the calibration target of 1%. It's not a population guarantee. For comparison, 29/2,048 has an approximate Wilson interval of 95% as 0.99% to 2.03%. You can't get a rate like 1 in a million with this sample size.

The duplication row showed me the flaw in my simulated decision rule. I was using a threshold based on total token count, and so picked one from that. The regular 256-token threshold flagged 402/2,048 unmarked duplicated controls, or 19.63%. The row-specific threshold flagged 18/2,048, or 0.88%.

The mask was effective; my threshold counted repeated tokens as further evidence. There are 32 retained positions instead of the original prefix's 28, since adding copies adds four more contexts before repetition kicks in.

Replacement is different; it changes every five-token scoring window. Detection falls to 4/512. The middle crop still has internal windows and can still be detected here. There's so much choice in the sampler with a uniform vocabulary that you shouldn't extrapolate these high untouched detection rates to constrained natural language answers.

Conclusion

For this investigation, I chose the weighted mean because I could inspect every contribution and keep threshold selection separate from the positive test set. For an actual service, I'd evaluate the Transformers integration the reference points to. The first acceptance test would be the real ingestion pipeline's held-out false-positive rate, including its short chunks and duplicate passages. A watermark hit can inform a provenance investigation. It cannot decide whether the retrieved claim is true.