Guides

Task-oriented documentation for schemas, ingest, the two query grammars, filters, vector search, the admin console, configuration, and local development.

The guides are task-oriented. Each one takes a single part of the engine — schemas, ingest, a query grammar, configuration — and explains the mechanism behind it, not only the commands that drive it. Where a page states a default or a limit, that value comes from the binary's own generated reference; see the CLI reference and the configuration reference when you need the exhaustive tables.

If you have not run it yet, start with the quick start. It gets a server, a schema, a corpus, and a served query in one sitting. Come back here when you want to know why a step behaved the way it did.

The pages

Guide	What it covers
Schemas	Writing a document schema as a protobuf message, the option defaults, canonicalization, and the rules for evolving a schema without breaking existing segments
Ingest	`push`, `delete`, the acknowledgment states, and the spool-seal-publish pipeline that turns a batch into a served generation
The Lucene grammar	The default search-bar grammar: terms, phrases, boolean occurrence, ranges, and the operators that are refused by name
The CQP grammar	The positional grammar: token constraints, sequences, quantifiers, regex values, and how a quantifier lowers to a phrase or an alternation
Filters and projection	Filterable fields, typed filter values, range lowering, and asking for stored fields back with `--fields`
Vector and hybrid search	Declaring vector fields, the four embedding modes, and running vector, hybrid, and exhaustive queries
The admin console	The separate console process, its bridge, the Ingest upload endpoint, and the operations it can and cannot perform
Configuration	One catalog of settings, five precedence layers, file discovery, reload, and what may change at runtime
Local development	The one-binary `ysearch-local` setup, the Docker Swarm fleet with Redpanda and the marekvs sidecar, and the YS1 corpus and fixture tools

A reading order

For application integration, start with:

Schemas — everything else depends on the schema. Field options decide whether a field is searchable, filterable, projectable, or all three, and most of the refusals you will meet later are schema refusals arriving one layer down.
Ingest — how documents reach a segment, and what "published" means. The distinction between an ephemeral acknowledgment and a published generation is the one that decides whether your next query sees your last write.
The Lucene grammar — the grammar a search box sends.
Filters and projection — how a query narrows and what comes back in a hit.
The CQP grammar and vectors when the lexical surface is not enough.

For someone operating a deployment, read configuration first, then the admin console, then the deployment pages: containers, Kubernetes, the object store, and observability.

What ysearch has changed so far

Two changes to the inherited engine are on main. Neither changes the commands in these guides:

Segment formats 7 and 8. The builder writes format 8 by default. Format 7 added the stats, forward, and hashterms families that pinned scoring reads (milestone YS2). Format 8 adds exact vector lanes for cosine vector fields: canonical fp16 vectors, IVF centroids with their radii, and int8 codes (YS3). Older segments stay readable; the binary reads formats 1 to 8.
A pinned scorer beside the old one. Two scoring profiles exist: bm25f-v1, where each segment scores with its own statistics, and bm25f-pinned-v1, where every segment scores under the generation's pinned statistics epoch, so scores do not depend on how documents are split into segments. query.scoring_profile sets the profile for a request that names none, and its default is bm25f-v1. The catalog-following query roles of ysearch node fold epochs into stats.dir; without it, a pinned request fails with a typed error. See pinned epochs.

What the guides assume

They assume the ysearch binary is on your path and a server is reachable at 127.0.0.1:9500, which is the default every client verb dials. Commands are written without a --server flag when that default is correct and with one when the page is talking about a topology that has more than one endpoint.

They also assume the mental model in concepts: documents become immutable segments, segments are referenced by a catalog generation (both in segments and catalog), and a server follows the catalog rather than owning mutable state. If a guide says "the next search sees it", that is the follower installing a new generation; see architecture.

When a guide and the binary disagree

Use the running binary to check its commands and settings:

zsh

ysearch manual
ysearch config list --prefix builder.

manual renders the reference from the command tree and the configuration catalog, so it cannot drift from the code it documents. The CLI reference on this site is generated from that output, as committed in docs/cli.md at a pinned ysearch revision. config list reports each setting's value, scope, and where the value came from, which is the authority when a written table and a running process disagree.