Glossary

The first part defines the terms of the ysearch design. Most of what they name is specified and not built yet; Status says what exists. The second part defines the terms of the inherited search engine, which is what runs on main today.

Several of these are easy to confuse. The pairs to watch: the fencing epoch F versus the statistics epoch E(G); the catalog generation G versus the table generation T; v(d) versus v(c, d); and the floor F_c versus the forecast F(a, W).

Term	Meaning
KV plane	The replicated, convergent key-value store on SSD: documents, traffic, campaigns, dials, locators, hot match rows (KV plane)
segment plane	The S3-authoritative plane of immutable segments, catalogs, match cells, and tables (segments)
partition, pid	One of 4,096 KV partitions: pid = (crc16(userkey) % 16384) >> 2, honouring {hash tags}
H1	A partition's highest-ranked active owner under HRW placement. Not a consistency primary; it runs the indexer, the fresh tail, and provisional matching
HLC	Hybrid logical clock: 48-bit milliseconds and a 16-bit counter
v(d)	A document's current version (doc_hlc, doc_digest) in the KV plane; the authority for which indexed version is current
v(c, d)	The vector score of campaign c on article d: the cosine, clamped to [0, 1]
gossip	The SWIM-style protocol every pod runs: the source of membership, and a hint channel for capabilities, catalog and epoch heads, serving lag, and load
capability	A format or feature a binary implements; usable only once the control role has enabled it cluster-wide
meta group	The control role's cluster-wide Raft group: namespace registry, capabilities, origins, cluster settings, the maintenance lease
namespace group	The Raft group of one namespace: its publisher, planner, table-writer, and dial-controller leases and their fencing counters
fencing epoch F	A per-lease counter, Raft-committed and incremented when the holder changes. It appears in object names, and readers reject a lower one. Not the statistics epoch
generation G	An immutable catalog version of one namespace, naming its segments and their liveness bitmaps
statistics epoch E(G)	The statistics (N, field lengths, df) folded over generation G's segments (epochs)
G_chk	The newest generation the dial controller has checked for drift
checked window W	[G_chk − stats.max_serving_lag, G_chk]: the only epochs a node may serve from
serving epoch	E(min(G_node, G_chk)); a node whose serving epoch leaves W stops answering (EPOCH_LAG)
features	Statistics-free per-(campaign, article) values from which a score is computed at read time
campaign batch	An immutable, content-addressed set of campaign revisions
cell	(segment, campaign batch, analyzer, model) → match rows; the unit of matcher work
table generation T	An immutable version of the match tables
hot match row m	An article's by-article row of table generation T, copied into the KV plane
fresh tail	The H1's in-memory index of documents not yet published
provisional match row n	Features the H1 computed for a fresh-tail document; superseded by the published version's m row
floor F_c	The score below which matches of campaign c are not stored
forecast F(a, W)	Forecast impressions of article a over window W, used by impression dials
dial (α_c, τ_c)	A campaign's fusion weight and serving cutoff; changing it needs no recomputation
τ_min(c)	The lowest cutoff at which campaign c's served set is exact; dials never go below it
drift D_c	An upper bound on how much any score of campaign c can have risen since its column was computed
placement, site, publisher	The ad server's slot and the two grouping levels above it; request context and calibration hierarchy, never a filter
ad_action	One pair-level ad event (impression or click) from the ad server's topic
ctr	A pair's estimated click rate in a placement's context, in parts per 10⁹; it orders an answer, but the served set is decided on the raw score
WASM app, world	A registered, content-addressed WebAssembly component, and the ysearch WIT interface it implements (extensions)
(A)	Marks an assumed figure. Every size and cost in the specification is (A) until real volumes are known or YS4 measures it

Segment — The unit of immutability, publication, caching, placement, and compaction. One immutable WavesDB checkpoint plus its sidecars, in one object prefix. A segment is never modified after it is committed.

Commit marker — The segment.commit object. It is written only after the checkpoint, metadata, and sidecars are durable, which is what makes a partially uploaded segment invisible rather than corrupt.

Checkpoint — The WavesDB object set a segment's data lives in: a MANIFEST plus the table and blob objects of its column families.

Column family — One of the WavesDB families a segment is built from. The inherited nine are meta, terms, postings, positions, impact, filters, docvalues, vectors, and stored; format 7 adds stats, forward, and hashterms, and format 8 adds exact vector lanes. Each family has its own table files, even though all share one checkpoint and one commit.

Sidecar — A per-segment object outside WavesDB that supports routing and filtering: ids.bloom, ids.winners, ids.ordinals, terms.bloom, filters.postings.

Catalog — The authoritative record of what is visible. It holds an immutable generation lineage plus a compare-and-swap latest hint.

Generation — An immutable catalog record naming the complete active segment set. A query pins exactly one generation for its whole life.

latest — A hint object naming the current generation, updated by compare-and-swap. It is a hint, not the authority; the lineage under catalogs/generations/ is what a reader trusts.

Lineage — The parent-child chain of generations. When conflicting children of one parent appear, the lexicographically lowest generation ID wins and the losers' segments are carried onto it, which is recorded as a lineage repair rather than losing a document.

Announce — The step that makes a published segment visible, by writing a new generation that includes it.

Publication lag — The time from commit-marker durability to catalog-generation publication, measured by ysearch_publisher_publication_lag_seconds. The builder's own ysearch_builder_publication_lag_seconds measures from the first document spooled instead, so the two answer different questions.

Compaction — Physically merging several active segments into one and atomically replacing them. It writes a new segment and a new generation and retires its inputs; it never modifies a visible segment in place.

Retired — A segment no longer referenced by the active generation. Retired is not deleted: reclaiming the bytes is garbage collection's job.

GC dry run — A mark-only pass that proposes unreferenced objects against explicit retention horizons and publishes an immutable proposal record. It removes nothing.

Quarantine — Two different things. In garbage collection, the waiting period an immutable proposal must survive before a fresh mark may authorize deletion (gc.quarantine_age). In the cache, the state an entry enters when its content-addressed identity fails verification.

Logical key — The schema field marked key: true. It defines upsert identity: pushing the same key again stores a new version rather than replacing the old one.

Public ID — The deterministic 128-bit document identity used in result frames and tie ordering. Streamed as lowercase hex by search --ids.

Ordinal — A segment-local dense document number. Ordinals are how postings and docvalues address documents inside one segment; they mean nothing outside it.

Mutation version — The version number distinguishing copies of the same logical key. With query.collapse_key_versions on — the default — a query returns only the copy with the greatest mutation version.

Tombstone — The delete primitive. A key's current version is superseded by nothing, so it stops appearing in results. The bytes are not reclaimed until a later compaction merges away the segment holding the document.

Liveness bitmap — The per-generation record of which documents are the current winners, used to resolve versions and deletes at query time.

Storage mode — Where an immutable reader gets checkpoint bytes: HYDRATE_FULL, REMOTE_BLOCKS, or AUTO. It never changes query semantics or the segment format.

HYDRATE_FULL — Requires a complete, verified local copy of every selected segment before executing against it.

REMOTE_BLOCKS — Opens stable readers over immutable object-store sources. Opens read table metadata; data blocks are fetched with bounded range requests on demand.

AUTO — Decides per segment. A verified full-cache hit executes locally; otherwise predicted request count, byte count, and scan fraction are weighed against measured remote latency and throughput.

Hydration — Downloading every object a segment's checkpoint names, verifying it against the commit, and atomically admitting the completed directory.

Full-segment cache — The verified content-addressed disk cache of complete segments, bounded by cache.full_bytes and cache.full_entries.

Block cache — The persistent decoded-block disk cache, bounded by cache.block_bytes and cache.block_entries. Consulted only for remote-block execution.

Read resources — The WavesDB in-process caches: decoded blocks, table readers, and file handles, bounded by the cache.read_* settings. Distinct from the disk caches and discarded on restart.

Scan bypass — A scan or readahead read may consult existing block-cache entries but does not admit new ones, so a one-off scan cannot evict a selective hot set.

Scrubber — The background pass that re-reads cached full segments and checks them against their commits. It is the only thing that notices a cached segment rotting on disk, because verifying costs the SHA-256 of every file in the segment and doing that per request would make a many-segment query spend all its time hashing.

Pinned — A cache entry currently in use, which eviction must not take.

Role — One of eight behaviors a ysearch node process can serve. One binary; the --roles value decides what it does.

Role	Does
coordinator	Plans segment work units against the published catalog, dispatches them, and merges
worker	Executes work units against segments; owns the disposable caches
merger	A merge-tier node the coordinator delegates fan-in to
aggregator	Holds soft cache-residency and capacity state for placement
router	Admits ingest batches and routes them to builders
builder	Spools, seals, builds, and publishes segments
publisher	Holds the catalog lease and announces generations
compactor	Maintenance only: physical merges and garbage collection

Role mix — The value of the role metric label. A process serving several roles reports them joined: role="builder-publisher".

Zone — The placement zone a node advertises in (server.zone). Aggregators are zone-local.

Work unit — One segment's share of a query, dispatched by a coordinator to a worker.

Placement — Choosing which worker executes a work unit, using cache-residency and capacity hints. Placement is never a correctness dependency: losing every aggregator degrades placement to a cache-oblivious tier and nothing else.

Hierarchical merge — Combining worker results through a tree of merge nodes rather than all at once, so fan-in and buffering stay bounded as segment count rises.

Bound propagation — Passing score bounds through the merge tree so subtrees can stop early without affecting the result.

Fan-in — Children per merge node in a coordinator's plan (fleet.fan_in).

Lane — The scheduling class of a query: interactive, streaming, or background. Worker slot pools are per lane (fleet.lane_slots).

Follower — The component in every node that watches latest and installs new generations. The follower.* settings govern its polling, overlap, and retirement behavior.

Generation window — The bounded period in which a worker may resolve a generation it has not seen through latest, controlled by follower.generation_overlap. Older unseen generations are refused.

Spool — The on-disk accumulation of accepted documents before a build is sealed. Lives under ingest.dir.

Seal — Closing an open build so it can be built and published. Triggered by age, bytes, documents, an explicit flush, a schema change, or replay.

Admission — The router's decision to accept or refuse a batch, against ingest.global_queue_bytes, ingest.index_queue_bytes, and the optional rate quota.

Acknowledgement status — What a batch was told: accepted_ephemeral, throttled, rejected, or published. Only published means the documents are in a served generation.

Ephemeral acceptance — A batch accepted into the spool but not yet published. It is durable enough to be retried, not yet visible to a query.

Dialect — Which grammar a query string is parsed with: lucene or cqp.

Lowering — Turning a parsed query into the structured request the server executes. It happens client-side, so a typo lands a caret before any round trip.

BM25F — The ranking function, with per-field weights and b values declared in the schema.

Exactness — The scope of the result-ordering guarantee.

Value	Meaning
GLOBAL_EXACT	Exact ordering over the pinned generation
EXACT_WITHIN_CANDIDATES	Exact ordering within the retrieved candidate set
APPROXIMATE	Approximate ordering with a reported error bound

Exact prefix — The leading results ordered under the query's exactness class. top_k sets its requested size, from 1 up to the server limit of 100,000. A lower configured limit can apply.

Tail — The part of the result stream after the exact prefix. --tail exact requests exact ordering; --tail banded permits approximate ordering within a reported error bound.

Impact bands — Conservative per-term score-impact directories that make a bounded approximate tail possible. Persisted only when tail.impact_enabled is on; exact fallback does not require them.

Ordering error — The maximum score inversion a banded tail is allowed to contain. Zero requires exact fallback.

Collapse — Returning one hit per logical key, the copy with the greatest mutation version. On by default; --no-collapse returns every hit in rank order.

Projection — Asking for stored fields back with --fields. --keys projects only the schema key; --ids asks for no stored documents and no scores at all.

Vector field — A schema field declared to carry an embedding. An index may have several.

IVF-PQ — The inherited approximate vector index: an inverted file of coarse centroids over product-quantized codes. ysearch replaces it with ivf_rabitq in YS16.

Probe — One coarse IVF list examined during candidate generation (vector.query_default_probes, --probes).

Candidate — A vector the ANN stage produced for exact reranking (vector.query_candidate_multiplier, --candidates).

Exact rerank — Scoring the candidate set against full-precision float32 vectors, which is what makes EXACT_WITHIN_CANDIDATES exact within its candidates.

Exhaustive — Scanning every covered vector instead of generating candidates (--exhaustive).

Fusion — Combining lexical and vector scores in a hybrid query: weighted or rrf, reciprocal rank fusion.

Embedding mode — Which provider produced a vector: internal (pure-Go, in-process), external (an OpenAI-compatible HTTP endpoint), or gRPC. The mode metric label carries the schema-selected one.

Truncation — A text longer than a model's token window being cut to fit. It is a real quality loss — the vector then represents only a prefix while BM25 still indexes the whole document — so it is counted by ysearch_embedding_truncated_texts_total rather than being silent.

Schema — A protobuf message carrying ysearch.v1.document and ysearch.v1.field options. Field numbers are the field IDs the index keeps forever.

Additive change — A schema change that only adds. It mints the next version. Anything else is refused with a diff.

Indexed, stored, filterable — What the index does with a field: analyze it into the term dictionary, keep it for projection, or make it available to equality and range filters.

Setting — One entry in the configuration catalog, with a dotted key, a kind, a scope, a default, an environment variable, and usually a flag.

Scope — What may change at runtime: startup, node-runtime, or compile-time. Only node-runtime settings accept config set.

Provenance — The layer a setting's current value came from: default, file, environment, flag, or the runtime API. Reported by config get, config list, and config export.

Format version — The segment format a binary writes (format.segment_version) and the oldest it reads (format.minimum_reader_version). The Kubernetes operator gates upgrades on a declared read range and write format.

WavesDB — The immutable segment storage engine ysearch builds on.

Receipt — A durable record of what a run measured, with the revision and inputs it was measured on. Receipts record milestone evidence, benchmark results, and vendored dependency manifests.

Milestone — One numbered phase of the implementation roadmap (YS0–YS17 and YS8a; M0–M8 are yolosearch's history), with an entry gate, deliverables, and an exit gate. A milestone is not closed until its exit gate has a receipt.

• Concepts — these terms in context
• Metrics reference — where the label vocabularies are used
• Configuration reference — every setting the terms above name