Why ysearch

A publisher produces a large daily volume of news articles, with corrections and takedowns. Advertising campaigns want to appear next to articles about their subject. The ad server, for every page view, asks: which campaigns match this article, and how well?

The answer has to be:

• complete: every campaign above its cutoff, not a sample;
• fresh: an article is matched within 30 seconds of being written;
• adjustable: a campaign's reach can be widened or narrowed at any time;
• stable: a daily refresh of term statistics must not rewrite everything already matched.

The owner's requirements, from the specification:

ID	Requirement
R-MATCH	Match articles to ads by content: BM25, vectors, or a mix (optionally with proximity)
R-TABLE	For every article, a lookup table of matching campaigns, updated incrementally and fully regenerable
R-REACH	Per campaign, widen or narrow the universe of matching articles
R-S3	Keep as much as possible on S3; the output is article IDs
R-EPOCH	Statistics epochs refresh daily without reading or writing much on S3
R-KV	A small, eventually consistent KV store beside the search plane for documents and per-article traffic
R-ONE	One database system, not a search engine plus a separate KV store
R-STREAM	Direct ingestion from Kafka-protocol brokers
R-API	gRPC and RESP only: no Elasticsearch-shaped HTTP API, no SQL
R-FRESH	Under 30 s from article write to matched
R-WASM	No Lua: extensions are compiled WASM apps against ysearch-defined interfaces
R-K8S	Horizontally scalable and easy to operate on Kubernetes

Store features, not scores. Classic BM25 scores depend on corpus statistics: the number of documents and each term's document frequency. If a match stored its score, every daily refresh of those statistics would stale every stored match. ysearch stores what the score is computed from and computes the score when it is read. See pinned statistics epochs.

Make reach a read-time dial. Because scores are computed at read time, a campaign's cutoff can be a value read at serving time. Changing it is one record write. See matching and dials.

Keep the bulk on S3, the hot state on SSD. Segments, match cells, match tables, and archived documents are immutable objects. Documents, traffic counters, campaigns, dials, and hot match rows change constantly, so they live in a replicated KV plane on SSD. See architecture.

Converge without coordination. The KV plane is available under partition: every record merges as a CRDT, so writes never wait for consensus. Consensus is used only where one holder must be unique: the leases of the publisher, planner, table writer, and dial controller. See the KV plane.

Determinism across architectures. A floor decided on one node and a threshold applied on another must agree at the boundary, bit for bit, on amd64 and arm64. That rules out math.Log, fused multiply-adds, and floating-point code in extensions.

• yolosearch is the code base. ysearch is a fork at 94f6b98, with full history: publication, catalog generations, segments on WavesDB, the query path, vectors, compaction, garbage collection, the operator, and the admin console.
• marekvs is the design of the KV plane: HLC envelopes, CRDT merges, HRW placement, replication, Merkle anti-entropy, and gossip membership. It is ported to Go, not linked. The Rust binary is the specification and the matcher prototype's sidecar.

WavesDB is the storage engine for both planes.

• No HTTP API and no SQL. Classic search features (query DSL, aggregations, highlighting, suggestions) are exposed through gRPC.
• No Lua. marekvs's Lua write scripts become WASM apps.
• No placement targeting. The ad server owns placement eligibility. A placement is request context for calibration, not a filter ysearch applies.
• No latency promise for search over S3. Search latency is explicitly not a requirement (R-S3). The ad server's lookup reads hot match rows from the KV plane; its latency targets are assumptions until YS12 measures them.

ysearch is pre-v1. The fork is done; infrastructure, pinned scoring, and the matcher prototype are in progress. The YS4 prototype qualification is a real go / no-go gate: if the drift recompute rate or the table sizes are far from the assumptions, the matching design changes before the KV plane is built. See Status.