Matching and dials

A campaign is a protobuf document in the KV plane. Where it comes from does not matter: every source is a definition kind behind one interface, and every kind compiles to the same compiled campaign.

A campaign also carries a filter (language, section, publication window), optional floor overrides, and the namespace of articles it targets.

Campaigns are frozen into immutable, content-addressed campaign batches.

What the YS3 prototype compiles, and what it refuses:

• query_string follows Lucene's classic rules: the default operator is OR, + requires a clause, and - excludes it. A bare term or (a OR b) is scored, +"a b" is a required phrase, and prefix* is a scored prefix. AND, NOT, nesting, boosts, fuzzy, ranges, other wildcards, regexes, and optional or excluded phrases are refused with UNSUPPORTED_QUERY_FEATURE.
• Phrases are predicates, checked exactly (slop 0). They are never scored.
• Proximity is refused with PROXIMITY_UNSUPPORTED. Cells store integer features, and the proximity feature is fractional; it needs an integer definition first.
• A term is one token. A term that analyses to more or fewer tokens is refused (TERM_ANALYSIS): use a phrase instead.

Compilation is synchronous: PutCampaign refuses a campaign it cannot compile with FAILED_PRECONDITION and the typed code, never approximates it.

Every published segment must be joined with every active campaign. A new segment adds a row of the matrix; a new or changed campaign adds a column.

A cell is one segment × one campaign batch. Its key is a hash of the segment, the batch, the field fingerprints, the matcher version, the vector mode, and G_eval, the generation whose epoch it is evaluated against. A recomputed cell therefore has the same key and the same bytes.

• Lexical cells percolate each campaign over the segment's forward index.
• Vector cells use an exact threshold scan over the vector lanes, pruned by cluster radii.
• A cell stores both channels' features for every pair above either floor.

Each namespace has a planner, which holds the namespace's planner lease. It creates cells in priority order: new segments against every campaign first, then new campaigns and column recomputes on the newest segments, then everything else. Matchers lease cells and report completions.

The table writer folds cells into two views of the same data:

• by-article: for every article, its matching campaigns and their features. This is the table the ad server's lookup reads.
• by-campaign: for every campaign, its matching articles, which the dial controller ranks.

An article's by-article row is also copied into the KV plane as a hot match row m, so the ad server's lookup rarely touches S3.

The slow path takes minutes (A), so the partition's leading owner matches each new article itself: it adds the article to its in-memory fresh tail, extracts the same features a cell would, scores them under its serving epoch, keeps the pairs above the campaigns' frozen floors, and writes a provisional row n. When the table row for that article version arrives, it supersedes the provisional one. A provisional row is served only when its version equals the KV plane's current version of the article and no table row for that version exists yet.

Each campaign has a dial (α_c, τ_c). The served score fuses the two channels:

s = α_c · b̂ + (1 − α_c) · v

b̂ is the lexical score normalised by its maximum, and v is the vector channel, both in [0, 1]. There is no rank fusion: a rank depends on the other articles, so it cannot be thresholded per article.

The dial controller turns a campaign's target into τ_c:

Every computed τ_c is clamped to at least τ_min(c), the lowest cutoff at which the stored set is exact. If a target needs less, the dial reports BELOW_TAU_MIN. The specification has the planner then widen the campaign's stored set with a new column. Nodes check the clamp themselves, and never serve an inexact set silently.

A dial carries one of these statuses:

How the YS3 prototype fills the gaps in the specification:

• Default dial. A campaign with no SetDial has the target threshold = 0 and α = 1 (0 without a lexical channel). It is therefore served at τ_min(c), with status BELOW_TAU_MIN.
• A dial the controller has not written yet makes the campaign pending. It is never served at a guessed threshold.
• Widening is manual. An operator lowers the campaign's floors; the next freeze makes a new baseline, and the dial reports FLOOR_PENDING until it is complete.
• A dial change writes only the dial. SetDial writes the target and α; the next round writes the controller's output. No cell, table, or object is rewritten (verified below).

MATCH.ARTICLE <article> returns the campaigns whose score, under the serving epoch and the current dial, is at or above τ_c. A request can add PLACEMENT <id> as calibration context, and CAMPAIGNS id… to restrict the answer to the campaigns the ad server allows. The ad server, not ysearch, decides placement eligibility. The complete lookup is in architecture.

MATCH.PAIR explains one article × campaign pair: its features, its score breakdown, and τ_min, or why it is absent. Its state is one of:

ysearch.v1.Matching (api/ysearch/v1/matching.proto) is the YS3 matcher's client API. The full message list is in the gRPC reference.

Refusals are typed. A request with unknown fields anywhere is INVALID_ARGUMENT. Every matcher error carries its code as an ErrorInfo reason in the domain ysearch.match. Match on the reason, never on the message.

go run ./tools/matchcheck is the YS3 exit check. It publishes YS1 corpus articles in catalog generations, runs the matcher, and compares every article's MATCH.ARTICLE answer with a brute-force evaluation of every pair, score bits included. Then it changes dials and counts what was written.

The receipt bench/ys3/exit-100k-1k.json in the ysearch repository covers 10⁵ articles × 10³ campaigns (darwin/arm64, 24 CPUs, Go 1.27.0):

Not measured:

• The matcher's cost. The 233 s wall time covers the whole check on one host: segment builds, folds, cells, tables, and mostly the brute-force oracle. No per-stage cost is claimed from it.
• The KV plane: the check uses an in-memory sidecar fake and in-memory objects.
• The exact vector path at that scale. The YS1 corpus has no vectors, so it is checked only on a synthetic 3,000 × 200 subset.

The specification sizes the system for about 10⁶ new or updated articles per day and 10⁵ active campaigns, with 2·10⁵ stored matches per campaign on average. Every one of these is (A): assumed, not measured. YS4 runs the prototype on 3·10⁶ articles and 10⁴ campaigns and replaces them with receipts.