Skip to content
ysearchDocs

Reference

CLI reference

Every ysearch command, flag, and setting, generated from the binary's own manual at the pinned revision.

The manual describes the binary on ysearch main, which is still the search engine inherited from yolosearch plus the ysearch groundwork landed so far. The ysearch roles and verbs of the design are not in it yet; see Status.

ysearch serves an index, reads and changes a running server's settings, shows its statistics, and builds and publishes the immutable segments it reads. The client verbs talk to --server; the server verbs read their settings from the configuration catalog (discovered files, the environment, flags); the offline verbs work on an object store directly.

The hand-written guide with walkthroughs is guide.md.

Global flags

Every command accepts these. The catalog-backed ones have a key and an environment variable; a file may set the key too.

Flag Type Default Key Env Description
--json bool false client.json YSEARCH_CLIENT_JSON Emit JSON instead of tables. [client.json; env YSEARCH_CLIENT_JSON]
--quiet bool false client.quiet YSEARCH_CLIENT_QUIET Suppress progress output. [client.quiet; env YSEARCH_CLIENT_QUIET]
--server string 127.0.0.1:9500 client.server YSEARCH_CLIENT_SERVER Server address the client verbs dial. [client.server; env YSEARCH_CLIENT_SERVER]
--timeout duration 30s client.timeout YSEARCH_CLIENT_TIMEOUT Client request timeout. [client.timeout; env YSEARCH_CLIENT_TIMEOUT] (positive)
--config-dir string config.dir YSEARCH_CONFIG_DIR Directory whose *.yml and *.yaml files are layered in name order after the default locations and before config.file. [config.dir; env YSEARCH_CONFIG_DIR]
--config string config.file YSEARCH_CONFIG_FILE YAML configuration file layered last among files. [config.file; env YSEARCH_CONFIG_FILE]
--config-search bool true config.search YSEARCH_CONFIG_SEARCH Probe /etc/ysearch, $XDG_CONFIG_HOME/ysearch (default ~/.config/ysearch), and ./ysearch.yml at startup. [config.search; env YSEARCH_CONFIG_SEARCH]

Commands

ysearch catalog

ysearch catalog [flags]

Resolve, publish, reconcile, or dry-run garbage collection of the catalog (offline)

Subcommands:

ysearch catalog compact

ysearch catalog compact [flags]

Physically merge active immutable segments and atomically replace them

Flags:

Flag Type Default Key Env Description
--input-segments stringSlice [] active segment IDs to merge (comma-separated or repeated; at least two)
--job-id string optional 32-hex idempotency key; generated when omitted
--output-segment-id string optional 32-hex output identity; generated when omitted
--builder-version string ysearch-compactor publisher identity recorded on the output
--analysis-batch-documents int 0 builder.analysis_batch_documents YSEARCH_BUILDER_ANALYSIS_BATCH_DOCUMENTS Documents handed to the analysis workers at once; zero derives sixteen per worker. A batch is the unit that must fit in builder.analysis_memory_bytes, so a larger batch amortises the hand-off over more documents but raises the peak the analysis share has to cover. The derived shape is what the resource gates measure, so a corpus of unusually large documents is the case for lowering it rather than raising the share. [builder.analysis_batch_documents; env YSEARCH_BUILDER_ANALYSIS_BATCH_DOCUMENTS] (in [0, 1048576])
--analysis-memory-bytes bytes 0 builder.analysis_memory_bytes YSEARCH_BUILDER_ANALYSIS_MEMORY_BYTES The share of builder.sort_memory_bytes reserved for documents in flight through analysis; zero derives a thirty-second of it, clamped to between 64KiB and 32MiB and never more than a quarter of what the public-id and docvalue spools leave. It only has to hold one bounded batch, and a document larger than the whole share still runs alone under builder.max_document_working_bytes, so the derived value is deliberately small: a three-million-document run peaked at 9.5MiB against a 128MiB share. Every byte reserved here is a byte the term partitions cannot use, which raises mini-run count and write amplification directly, so raise it only against a measured analysis stall. [builder.analysis_memory_bytes; env YSEARCH_BUILDER_ANALYSIS_MEMORY_BYTES]
--analysis-workers int 0 builder.analysis_workers YSEARCH_BUILDER_ANALYSIS_WORKERS Goroutines tokenising documents inside one build; zero derives GOMAXPROCS, bounded at eight. Analysis is the one parallel stage of an otherwise serial build, so this is per build and multiplies with builder.build_concurrency: the bound exists because a dozen builds each spawning a worker per core oversubscribes the machine and the scheduler churn costs more than the parallelism returns. Output order is preserved regardless of this value. [builder.analysis_workers; env YSEARCH_BUILDER_ANALYSIS_WORKERS] (in [0, 1024])
--blob-dictionary bool true builder.blob_dictionary YSEARCH_BUILDER_BLOB_DICTIONARY Compress stored values against a dictionary trained per blob file. Denser on homogeneous documents, but the dictionary is coded on the build's hottest path, so turning it off trades index size for ingest throughput. [builder.blob_dictionary; env YSEARCH_BUILDER_BLOB_DICTIONARY]
--blob-zstd-level int 0 builder.blob_zstd_level YSEARCH_BUILDER_BLOB_ZSTD_LEVEL Zstd effort for stored values; zero keeps the storage default (7). Levels select different zstd encoders, and a build re-seeds the chosen encoder from the blob dictionary for every value it writes, so a lower level cuts build cost by more than the usual level trade-off suggests. [builder.blob_zstd_level; env YSEARCH_BUILDER_BLOB_ZSTD_LEVEL] (in [0, 11])
--build-concurrency int 0 builder.build_concurrency YSEARCH_BUILDER_BUILD_CONCURRENCY Sealed builds that may run through the seal pipeline at once; zero derives one per CPU, bounded at twelve. One build is a mostly single-threaded chain of analysis, sort, merge and publish that spends about half its time blocked in file system calls, so a single pipeline leaves a multi-core host idle under bulk ingest. The builder's memory is this number times builder.sort_memory_bytes. [builder.build_concurrency; env YSEARCH_BUILDER_BUILD_CONCURRENCY] (in [0, 256])
--builder-catalog-publication enum direct builder.catalog_publication YSEARCH_BUILDER_CATALOG_PUBLICATION Catalog writer mode: direct is the single-node compatibility path; external stops after commit markers for a lease-fenced publisher role. [builder.catalog_publication; env YSEARCH_BUILDER_CATALOG_PUBLICATION] (one of direct, external)
--dedupe-index-identities int 0 builder.dedupe_index_identities YSEARCH_BUILDER_DEDUPE_INDEX_IDENTITIES Distinct document keys one open spool's dedupe index may hold before it gives up; zero derives 1048576. A key spooled twice in one build must resolve to a single winner before analysis, or the segment's dense ordinal assignment refuses the build outright. The index resolves that as records are spooled, from identities ingest already holds, and the sealed build inherits the answer; past this bound it gives up and the build reconstructs the answer by reading its spool twice instead — slower, and exactly as correct. One entry is a 16-byte key hash, a sequence and a mutation version, so the default bounds it at tens of megabytes per open spool and is reached only by a spool of a million very small documents. Lower it to cap that memory on a host running many indexes; set it to one to force the scanning path. [builder.dedupe_index_identities; env YSEARCH_BUILDER_DEDUPE_INDEX_IDENTITIES] (in [0, 268435456])
--builder-fuse-memory-bytes bytes 0 builder.fuse_memory_bytes YSEARCH_BUILDER_FUSE_MEMORY_BYTES Optional binary fuse construction workspace during publication; zero retains Bloom, failed admission retains Bloom. [builder.fuse_memory_bytes; env YSEARCH_BUILDER_FUSE_MEMORY_BYTES] (in [0, 64MiB])
--lexical-blob-threshold bytes 0 builder.lexical_blob_threshold YSEARCH_BUILDER_LEXICAL_BLOB_THRESHOLD Size at which a posting, position or term value is stored in a blob file rather than inline in the key log; zero keeps the default of 64KiB. Separation exists to spare compaction from rewriting large values, and a segment is never compacted — while a blob value is compressed on its own, which with a trained dictionary costs an encoder reset each time, where key-log blocks are compressed in bulk. [builder.lexical_blob_threshold; env YSEARCH_BUILDER_LEXICAL_BLOB_THRESHOLD]
--lexical-block-compression string builder.lexical_block_compression YSEARCH_BUILDER_LEXICAL_BLOCK_COMPRESSION Codec the term, posting and position families compress key-log blocks with: none, snappy, lz4, lz4fast or zstd. Empty keeps the default of lz4. The storage engine's own default leaves the newest level raw because it expects compaction to rewrite it, and a segment stays where it lands. On 300k articles lz4 measured 24.8s and 1.0GB against zstd's 37.7s and 0.8GB, so zstd is the choice for an index written once and read for a long time. [builder.lexical_block_compression; env YSEARCH_BUILDER_LEXICAL_BLOCK_COMPRESSION]
--max-document-working-bytes bytes 0 builder.max_document_working_bytes YSEARCH_BUILDER_MAX_DOCUMENT_WORKING_BYTES What one document may use beyond the partition budget while it is being analyzed; zero derives 64MiB, or a quarter of builder.sort_memory_bytes when that is smaller. It is an allowance rather than a reservation: it exists so a single document larger than the whole partition budget still builds, alone, instead of failing the build. Only a corpus with documents in the hundreds of megabytes needs it raised. [builder.max_document_working_bytes; env YSEARCH_BUILDER_MAX_DOCUMENT_WORKING_BYTES]
--max-queued-builds int 0 builder.max_queued_builds YSEARCH_BUILDER_MAX_QUEUED_BUILDS Sealed builds that may wait for or run in the pipeline before ingest is throttled; zero derives it as four times builder.build_concurrency. A waiting build is a spool on disk, not memory, so this buys ingest room across a build at the cost of ingest.seal_bytes of disk per queued build. Set too low, a bulk load throttles the moment the pipeline is full and ingest runs at the speed of the builds rather than ahead of them. [builder.max_queued_builds; env YSEARCH_BUILDER_MAX_QUEUED_BUILDS] (in [0, 1024])
--max-token-bytes bytes 64KiB builder.max_token_bytes YSEARCH_BUILDER_MAX_TOKEN_BYTES Maximum analyzed token size. A build must be able to admit one token this large, so raising it raises the floor under builder.sort_memory_bytes: a budget that leaves the partitions less than one maximum term refuses to start rather than fail partway. Tokens are words after analysis, so the default is already far past any natural language; raise it only for a corpus with genuinely enormous unbroken tokens. [builder.max_token_bytes; env YSEARCH_BUILDER_MAX_TOKEN_BYTES] (positive)
--merge-fan-in int 0 builder.merge_fan_in YSEARCH_BUILDER_MERGE_FAN_IN Spilled mini-runs merged in one pass; zero derives as many as an eighth of the partition budget affords at builder.run_page_bytes plus 512 bytes of reader overhead each, capped at sixty-four and floored at two. Runs beyond the fan-in need further passes, and each pass rewrites the data, so a low fan-in on a build that spilled heavily shows up as write amplification rather than as a slow merge. The cap is a file-descriptor and seek-pattern bound, not a memory one. [builder.merge_fan_in; env YSEARCH_BUILDER_MERGE_FAN_IN] (in [0, 64])
--builder-output-format-version int 0 builder.output_format_version YSEARCH_BUILDER_OUTPUT_FORMAT_VERSION Segment format emitted by builders and compactors; zero selects this binary's current format. The Kubernetes operator pins this to compatibility.writeFormat on writer roles during staged rollouts. Format 7 adds the stats, forward and hashterms families (spec/40 §3); a compaction that includes an older input writes format 6 instead, unless 7 or 8 is set explicitly, which refuses such a merge. Format 8 adds the exact vector lanes (canonical fp16 vectors, IVF radii, int8 codes; spec/40 §3.2). [builder.output_format_version; env YSEARCH_BUILDER_OUTPUT_FORMAT_VERSION] (in [0, 8])
--builder-paged-posting-directory bool false builder.paged_posting_directory YSEARCH_BUILDER_PAGED_POSTING_DIRECTORY Experimental commit-anchored posting-directory pages under query admission and the existing persistent block quota; old commits retain canonical traversal. [builder.paged_posting_directory; env YSEARCH_BUILDER_PAGED_POSTING_DIRECTORY]
--builder-posting-partition-penalty float 4 builder.posting_partition_penalty YSEARCH_BUILDER_POSTING_PARTITION_PENALTY Metadata cost penalty for experimental variable posting blocks. [builder.posting_partition_penalty; env YSEARCH_BUILDER_POSTING_PARTITION_PENALTY] (in [0, 1e+06])
--publish-verification string checksum builder.publish_verification YSEARCH_BUILDER_PUBLISH_VERIFICATION How a published segment is confirmed: checksum, sample, or readback. checksum compares every object against the checksum the object store reported when it accepted the write — S3 computes that server-side and refuses a mismatched write, and the filesystem backend computes it over the bytes it synced — so nothing is transferred back. sample additionally opens the published copy through the store and queries it; readback additionally fetches every object and rehashes it, which costs the whole index again on every build. An object whose store reports no comparable checksum is read back in any mode. [builder.publish_verification; env YSEARCH_BUILDER_PUBLISH_VERIFICATION]
--builder-routing-fragment-bytes bytes 8MiB builder.routing_fragment_bytes YSEARCH_BUILDER_ROUTING_FRAGMENT_BYTES Optional routing fragment disk cap per built segment; zero disables, overflow keeps canonical scan fallback. Matches compaction.routing_fragment_bytes so a freshly ingested segment and a freshly compacted one carry a fragment the same way; the two stay separate keys because a background merge must be able to carry its own share independent of foreground ingestion. Only a segment built or compacted after this defaulted on carries a fragment: nothing rewrites an existing one, so a settled corpus stays without a routing table until it is backfilled or compacted. [builder.routing_fragment_bytes; env YSEARCH_BUILDER_ROUTING_FRAGMENT_BYTES] (in [0, 64MiB])
--run-page-bytes bytes 0 builder.run_page_bytes YSEARCH_BUILDER_RUN_PAGE_BYTES Read-ahead page one spilled mini-run is buffered in during the final merge; zero derives 64KiB, shrinking it when an eighth of the partition budget cannot hold sixty-four such pages. The merge reserves one page per open run for the whole build, so page size and builder.merge_fan_in trade against each other out of the same share: the derivation prefers smaller pages to a fan-in that would force extra compaction passes on a hot partition. [builder.run_page_bytes; env YSEARCH_BUILDER_RUN_PAGE_BYTES]
--sort-memory-bytes bytes 256MiB builder.sort_memory_bytes YSEARCH_BUILDER_SORT_MEMORY_BYTES Total accounted builder budget: one ledger every part of a build charges against, and the setting the other builder budgets derive from. It divides into the public-id sorter (an eighth, capped at 64MiB), the docvalue and lane spools (their own worst case), the analysis share (builder.analysis_memory_bytes), and the term partitions, which take what is left. A build that cannot hold its partitions in that remainder spills to disk instead, which is the difference between one file per build and hundreds. This is per build, so the builder's memory is builder.build_concurrency times this. [builder.sort_memory_bytes; env YSEARCH_BUILDER_SORT_MEMORY_BYTES] (positive)
--builder-spill-merge-strategy enum heap builder.spill_merge_strategy YSEARCH_BUILDER_SPILL_MERGE_STRATEGY Experimental spill merge selector; heap preserves the baseline until full ingestion benchmarks qualify an alternative. [builder.spill_merge_strategy; env YSEARCH_BUILDER_SPILL_MERGE_STRATEGY] (one of heap, replace-root, loser-tree)
--builder-startup-bundles bool false builder.startup_bundles YSEARCH_BUILDER_STARTUP_BUNDLES Asynchronously produce optional startup bundles after catalog publication. One bounded coalescing worker per publishing host; oversized sources retain canonical startup fallback. [builder.startup_bundles; env YSEARCH_BUILDER_STARTUP_BUNDLES]
--term-dictionary-block-size bytes 64KiB builder.term_dictionary_block_size YSEARCH_BUILDER_TERM_DICTIONARY_BLOCK_SIZE Key-log data-block size of the term family. A table carries one block-index entry per data block and reads the whole index before it can look up anything, so this is what a cold term probe pays to find out where to look. At the storage engine's 4KiB an 8MiB dictionary needs about two thousand entries — around 65KiB read on every open of the table and held in memory until it closes — where 64KiB needs about a sixteenth of that. A probe then reads a bigger block, which over an object store is the cheap side of the trade: measured on a 6MiB dictionary the block cost 17KiB compressed against the 48KiB of index it replaced. [builder.term_dictionary_block_size; env YSEARCH_BUILDER_TERM_DICTIONARY_BLOCK_SIZE] (positive)
--term-dictionary-bloom bool false builder.term_dictionary_bloom YSEARCH_BUILDER_TERM_DICTIONARY_BLOOM Write a Bloom filter over the term family. The storage engine writes one by default; a segment does not, because the filter is read whole — uncompressed, about 1.2 bytes per distinct term — when the table is opened, and kept in memory until it closes, to save at most one block read per probe against the single table a published segment's term family is. Measured, it was nine tenths of what a cold term probe read: 431KiB of 481KiB on a 360k-term dictionary. A table says in its own footer whether it has a filter, so segments published with one keep working unchanged. [builder.term_dictionary_bloom; env YSEARCH_BUILDER_TERM_DICTIONARY_BLOOM]
--builder-variable-posting-blocks bool false builder.variable_posting_blocks YSEARCH_BUILDER_VARIABLE_POSTING_BLOCKS Experimental bounded-window BM25F-aware posting partitioning for format5 or newer output. [builder.variable_posting_blocks; env YSEARCH_BUILDER_VARIABLE_POSTING_BLOCKS]
--cache-dir string cache.dir YSEARCH_CACHE_DIR Disposable local cache directory; symlinks in its path are resolved once at startup and the cache then anchors there without following any. It holds the two disk budgets, cache.full_bytes and cache.block_bytes, and nothing else: every other cache.* budget is process memory. A relative value in a configuration file resolves against that file's directory. [cache.dir; env YSEARCH_CACHE_DIR]
--index string index.name YSEARCH_INDEX_NAME Index name for catalog-bound serve, worker, coordinator, compactor, router, builder, and publisher roles. [index.name; env YSEARCH_INDEX_NAME]
--object-backend enum fs object.backend YSEARCH_OBJECT_BACKEND Object store backend. [object.backend; env YSEARCH_OBJECT_BACKEND] (one of fs, s3)
--object-dir string object.dir YSEARCH_OBJECT_DIR Filesystem object-store directory; required for the fs backend. A relative value in a configuration file resolves against that file's directory. [object.dir; env YSEARCH_OBJECT_DIR]
--object-root string indexes object.root YSEARCH_OBJECT_ROOT Object key root. [object.root; env YSEARCH_OBJECT_ROOT]
--s3-anonymous bool false object.s3.anonymous YSEARCH_OBJECT_S3_ANONYMOUS Sign no requests at all, for a bucket that grants public reads. This is a deliberate choice, never a fallback: an unresolved credential chain already signs anonymously, and a reader that quietly degraded to unsigned requests would report a permissions problem as an empty index. [object.s3.anonymous; env YSEARCH_OBJECT_S3_ANONYMOUS]
--s3-bucket string object.s3.bucket YSEARCH_OBJECT_S3_BUCKET S3 bucket; required for the s3 backend. [object.s3.bucket; env YSEARCH_OBJECT_S3_BUCKET]
--s3-endpoint string object.s3.endpoint YSEARCH_OBJECT_S3_ENDPOINT S3 endpoint host:port; required for the s3 backend. [object.s3.endpoint; env YSEARCH_OBJECT_S3_ENDPOINT]
--s3-path-style bool false object.s3.path_style YSEARCH_OBJECT_S3_PATH_STYLE Use path-style S3 addressing. [object.s3.path_style; env YSEARCH_OBJECT_S3_PATH_STYLE]
--s3-prefix string object.s3.prefix YSEARCH_OBJECT_S3_PREFIX S3 key prefix. [object.s3.prefix; env YSEARCH_OBJECT_S3_PREFIX]
--s3-region string object.s3.region YSEARCH_OBJECT_S3_REGION S3 region. [object.s3.region; env YSEARCH_OBJECT_S3_REGION]
--s3-use-ssl bool true object.s3.use_ssl YSEARCH_OBJECT_S3_USE_SSL Use TLS to the S3 endpoint. [object.s3.use_ssl; env YSEARCH_OBJECT_S3_USE_SSL]
--source string object.source YSEARCH_OBJECT_SOURCE Dataset URL to attach read-only, such as s3://bucket/root/ for a whole root or s3://bucket/root/indexes/name/ for one index. It derives object.backend, the bucket, prefix and root, the index when the URL names one, and server.read_only, each only where nothing more specific was set. Reading someone else's published index is the whole purpose, so attaching one implies read-only and cannot be talked out of it. [object.source; env YSEARCH_OBJECT_SOURCE]
--tail-impact-maximum-bands int 64 tail.impact_maximum_bands YSEARCH_TAIL_IMPACT_MAXIMUM_BANDS Maximum score-impact bands persisted for one term. [tail.impact_maximum_bands; env YSEARCH_TAIL_IMPACT_MAXIMUM_BANDS] (in [1, 4096])
--tail-impact-target-documents int 4096 tail.impact_target_documents YSEARCH_TAIL_IMPACT_TARGET_DOCUMENTS Target postings per persisted impact run. [tail.impact_target_documents; env YSEARCH_TAIL_IMPACT_TARGET_DOCUMENTS] (in [1, 4294967295])

ysearch catalog gc-dry-run

ysearch catalog gc-dry-run [flags]

Propose unreferenced objects against explicit retention horizons; removes nothing

Flags:

Flag Type Default Key Env Description
--retained-generations int -1 generations before the latest to keep reachable
--stream-lifetime duration -1ns longest a query stream may still read a generation
--grace duration -1ns margin beyond every other horizon
--minimum-upload-age duration -1ns age below which an object may still be an upload in flight
--index string index.name YSEARCH_INDEX_NAME Index name for catalog-bound serve, worker, coordinator, compactor, router, builder, and publisher roles. [index.name; env YSEARCH_INDEX_NAME]
--object-backend enum fs object.backend YSEARCH_OBJECT_BACKEND Object store backend. [object.backend; env YSEARCH_OBJECT_BACKEND] (one of fs, s3)
--object-dir string object.dir YSEARCH_OBJECT_DIR Filesystem object-store directory; required for the fs backend. A relative value in a configuration file resolves against that file's directory. [object.dir; env YSEARCH_OBJECT_DIR]
--object-root string indexes object.root YSEARCH_OBJECT_ROOT Object key root. [object.root; env YSEARCH_OBJECT_ROOT]
--s3-anonymous bool false object.s3.anonymous YSEARCH_OBJECT_S3_ANONYMOUS Sign no requests at all, for a bucket that grants public reads. This is a deliberate choice, never a fallback: an unresolved credential chain already signs anonymously, and a reader that quietly degraded to unsigned requests would report a permissions problem as an empty index. [object.s3.anonymous; env YSEARCH_OBJECT_S3_ANONYMOUS]
--s3-bucket string object.s3.bucket YSEARCH_OBJECT_S3_BUCKET S3 bucket; required for the s3 backend. [object.s3.bucket; env YSEARCH_OBJECT_S3_BUCKET]
--s3-endpoint string object.s3.endpoint YSEARCH_OBJECT_S3_ENDPOINT S3 endpoint host:port; required for the s3 backend. [object.s3.endpoint; env YSEARCH_OBJECT_S3_ENDPOINT]
--s3-path-style bool false object.s3.path_style YSEARCH_OBJECT_S3_PATH_STYLE Use path-style S3 addressing. [object.s3.path_style; env YSEARCH_OBJECT_S3_PATH_STYLE]
--s3-prefix string object.s3.prefix YSEARCH_OBJECT_S3_PREFIX S3 key prefix. [object.s3.prefix; env YSEARCH_OBJECT_S3_PREFIX]
--s3-region string object.s3.region YSEARCH_OBJECT_S3_REGION S3 region. [object.s3.region; env YSEARCH_OBJECT_S3_REGION]
--s3-use-ssl bool true object.s3.use_ssl YSEARCH_OBJECT_S3_USE_SSL Use TLS to the S3 endpoint. [object.s3.use_ssl; env YSEARCH_OBJECT_S3_USE_SSL]
--source string object.source YSEARCH_OBJECT_SOURCE Dataset URL to attach read-only, such as s3://bucket/root/ for a whole root or s3://bucket/root/indexes/name/ for one index. It derives object.backend, the bucket, prefix and root, the index when the URL names one, and server.read_only, each only where nothing more specific was set. Reading someone else's published index is the whole purpose, so attaching one implies read-only and cannot be talked out of it. [object.source; env YSEARCH_OBJECT_SOURCE]

ysearch catalog publish

ysearch catalog publish [flags]

Announce a published segment (also converges the catalog on everything committed)

Flags:

Flag Type Default Key Env Description
--segment-id string 32 lowercase hex characters
--expected-parent string generation this announcement expects to extend
--generation string index.generation YSEARCH_INDEX_GENERATION Exact generation to pin, or empty for the latest. [index.generation; env YSEARCH_INDEX_GENERATION]
--index string index.name YSEARCH_INDEX_NAME Index name for catalog-bound serve, worker, coordinator, compactor, router, builder, and publisher roles. [index.name; env YSEARCH_INDEX_NAME]
--object-backend enum fs object.backend YSEARCH_OBJECT_BACKEND Object store backend. [object.backend; env YSEARCH_OBJECT_BACKEND] (one of fs, s3)
--object-dir string object.dir YSEARCH_OBJECT_DIR Filesystem object-store directory; required for the fs backend. A relative value in a configuration file resolves against that file's directory. [object.dir; env YSEARCH_OBJECT_DIR]
--object-root string indexes object.root YSEARCH_OBJECT_ROOT Object key root. [object.root; env YSEARCH_OBJECT_ROOT]
--s3-anonymous bool false object.s3.anonymous YSEARCH_OBJECT_S3_ANONYMOUS Sign no requests at all, for a bucket that grants public reads. This is a deliberate choice, never a fallback: an unresolved credential chain already signs anonymously, and a reader that quietly degraded to unsigned requests would report a permissions problem as an empty index. [object.s3.anonymous; env YSEARCH_OBJECT_S3_ANONYMOUS]
--s3-bucket string object.s3.bucket YSEARCH_OBJECT_S3_BUCKET S3 bucket; required for the s3 backend. [object.s3.bucket; env YSEARCH_OBJECT_S3_BUCKET]
--s3-endpoint string object.s3.endpoint YSEARCH_OBJECT_S3_ENDPOINT S3 endpoint host:port; required for the s3 backend. [object.s3.endpoint; env YSEARCH_OBJECT_S3_ENDPOINT]
--s3-path-style bool false object.s3.path_style YSEARCH_OBJECT_S3_PATH_STYLE Use path-style S3 addressing. [object.s3.path_style; env YSEARCH_OBJECT_S3_PATH_STYLE]
--s3-prefix string object.s3.prefix YSEARCH_OBJECT_S3_PREFIX S3 key prefix. [object.s3.prefix; env YSEARCH_OBJECT_S3_PREFIX]
--s3-region string object.s3.region YSEARCH_OBJECT_S3_REGION S3 region. [object.s3.region; env YSEARCH_OBJECT_S3_REGION]
--s3-use-ssl bool true object.s3.use_ssl YSEARCH_OBJECT_S3_USE_SSL Use TLS to the S3 endpoint. [object.s3.use_ssl; env YSEARCH_OBJECT_S3_USE_SSL]
--source string object.source YSEARCH_OBJECT_SOURCE Dataset URL to attach read-only, such as s3://bucket/root/ for a whole root or s3://bucket/root/indexes/name/ for one index. It derives object.backend, the bucket, prefix and root, the index when the URL names one, and server.read_only, each only where nothing more specific was set. Reading someone else's published index is the whole purpose, so attaching one implies read-only and cannot be talked out of it. [object.source; env YSEARCH_OBJECT_SOURCE]

ysearch catalog reconcile

ysearch catalog reconcile [flags]

Repair the catalog's lineage from the committed segments without announcing anything new

Flags:

Flag Type Default Key Env Description
--generation string index.generation YSEARCH_INDEX_GENERATION Exact generation to pin, or empty for the latest. [index.generation; env YSEARCH_INDEX_GENERATION]
--index string index.name YSEARCH_INDEX_NAME Index name for catalog-bound serve, worker, coordinator, compactor, router, builder, and publisher roles. [index.name; env YSEARCH_INDEX_NAME]
--object-backend enum fs object.backend YSEARCH_OBJECT_BACKEND Object store backend. [object.backend; env YSEARCH_OBJECT_BACKEND] (one of fs, s3)
--object-dir string object.dir YSEARCH_OBJECT_DIR Filesystem object-store directory; required for the fs backend. A relative value in a configuration file resolves against that file's directory. [object.dir; env YSEARCH_OBJECT_DIR]
--object-root string indexes object.root YSEARCH_OBJECT_ROOT Object key root. [object.root; env YSEARCH_OBJECT_ROOT]
--s3-anonymous bool false object.s3.anonymous YSEARCH_OBJECT_S3_ANONYMOUS Sign no requests at all, for a bucket that grants public reads. This is a deliberate choice, never a fallback: an unresolved credential chain already signs anonymously, and a reader that quietly degraded to unsigned requests would report a permissions problem as an empty index. [object.s3.anonymous; env YSEARCH_OBJECT_S3_ANONYMOUS]
--s3-bucket string object.s3.bucket YSEARCH_OBJECT_S3_BUCKET S3 bucket; required for the s3 backend. [object.s3.bucket; env YSEARCH_OBJECT_S3_BUCKET]
--s3-endpoint string object.s3.endpoint YSEARCH_OBJECT_S3_ENDPOINT S3 endpoint host:port; required for the s3 backend. [object.s3.endpoint; env YSEARCH_OBJECT_S3_ENDPOINT]
--s3-path-style bool false object.s3.path_style YSEARCH_OBJECT_S3_PATH_STYLE Use path-style S3 addressing. [object.s3.path_style; env YSEARCH_OBJECT_S3_PATH_STYLE]
--s3-prefix string object.s3.prefix YSEARCH_OBJECT_S3_PREFIX S3 key prefix. [object.s3.prefix; env YSEARCH_OBJECT_S3_PREFIX]
--s3-region string object.s3.region YSEARCH_OBJECT_S3_REGION S3 region. [object.s3.region; env YSEARCH_OBJECT_S3_REGION]
--s3-use-ssl bool true object.s3.use_ssl YSEARCH_OBJECT_S3_USE_SSL Use TLS to the S3 endpoint. [object.s3.use_ssl; env YSEARCH_OBJECT_S3_USE_SSL]
--source string object.source YSEARCH_OBJECT_SOURCE Dataset URL to attach read-only, such as s3://bucket/root/ for a whole root or s3://bucket/root/indexes/name/ for one index. It derives object.backend, the bucket, prefix and root, the index when the URL names one, and server.read_only, each only where nothing more specific was set. Reading someone else's published index is the whole purpose, so attaching one implies read-only and cannot be talked out of it. [object.source; env YSEARCH_OBJECT_SOURCE]

ysearch catalog resolve

ysearch catalog resolve [flags]

Print the latest generation, or --generation, as the catalog records it

Flags:

Flag Type Default Key Env Description
--generation string index.generation YSEARCH_INDEX_GENERATION Exact generation to pin, or empty for the latest. [index.generation; env YSEARCH_INDEX_GENERATION]
--index string index.name YSEARCH_INDEX_NAME Index name for catalog-bound serve, worker, coordinator, compactor, router, builder, and publisher roles. [index.name; env YSEARCH_INDEX_NAME]
--object-backend enum fs object.backend YSEARCH_OBJECT_BACKEND Object store backend. [object.backend; env YSEARCH_OBJECT_BACKEND] (one of fs, s3)
--object-dir string object.dir YSEARCH_OBJECT_DIR Filesystem object-store directory; required for the fs backend. A relative value in a configuration file resolves against that file's directory. [object.dir; env YSEARCH_OBJECT_DIR]
--object-root string indexes object.root YSEARCH_OBJECT_ROOT Object key root. [object.root; env YSEARCH_OBJECT_ROOT]
--s3-anonymous bool false object.s3.anonymous YSEARCH_OBJECT_S3_ANONYMOUS Sign no requests at all, for a bucket that grants public reads. This is a deliberate choice, never a fallback: an unresolved credential chain already signs anonymously, and a reader that quietly degraded to unsigned requests would report a permissions problem as an empty index. [object.s3.anonymous; env YSEARCH_OBJECT_S3_ANONYMOUS]
--s3-bucket string object.s3.bucket YSEARCH_OBJECT_S3_BUCKET S3 bucket; required for the s3 backend. [object.s3.bucket; env YSEARCH_OBJECT_S3_BUCKET]
--s3-endpoint string object.s3.endpoint YSEARCH_OBJECT_S3_ENDPOINT S3 endpoint host:port; required for the s3 backend. [object.s3.endpoint; env YSEARCH_OBJECT_S3_ENDPOINT]
--s3-path-style bool false object.s3.path_style YSEARCH_OBJECT_S3_PATH_STYLE Use path-style S3 addressing. [object.s3.path_style; env YSEARCH_OBJECT_S3_PATH_STYLE]
--s3-prefix string object.s3.prefix YSEARCH_OBJECT_S3_PREFIX S3 key prefix. [object.s3.prefix; env YSEARCH_OBJECT_S3_PREFIX]
--s3-region string object.s3.region YSEARCH_OBJECT_S3_REGION S3 region. [object.s3.region; env YSEARCH_OBJECT_S3_REGION]
--s3-use-ssl bool true object.s3.use_ssl YSEARCH_OBJECT_S3_USE_SSL Use TLS to the S3 endpoint. [object.s3.use_ssl; env YSEARCH_OBJECT_S3_USE_SSL]
--source string object.source YSEARCH_OBJECT_SOURCE Dataset URL to attach read-only, such as s3://bucket/root/ for a whole root or s3://bucket/root/indexes/name/ for one index. It derives object.backend, the bucket, prefix and root, the index when the URL names one, and server.read_only, each only where nothing more specific was set. Reading someone else's published index is the whole purpose, so attaching one implies read-only and cannot be talked out of it. [object.source; env YSEARCH_OBJECT_SOURCE]

ysearch catalog startup-bundle

ysearch catalog startup-bundle [flags]

Build optional bounded startup metadata pages for an existing generation

Flags:

Flag Type Default Key Env Description
--generation string index.generation YSEARCH_INDEX_GENERATION Exact generation to pin, or empty for the latest. [index.generation; env YSEARCH_INDEX_GENERATION]
--index string index.name YSEARCH_INDEX_NAME Index name for catalog-bound serve, worker, coordinator, compactor, router, builder, and publisher roles. [index.name; env YSEARCH_INDEX_NAME]
--object-backend enum fs object.backend YSEARCH_OBJECT_BACKEND Object store backend. [object.backend; env YSEARCH_OBJECT_BACKEND] (one of fs, s3)
--object-dir string object.dir YSEARCH_OBJECT_DIR Filesystem object-store directory; required for the fs backend. A relative value in a configuration file resolves against that file's directory. [object.dir; env YSEARCH_OBJECT_DIR]
--object-root string indexes object.root YSEARCH_OBJECT_ROOT Object key root. [object.root; env YSEARCH_OBJECT_ROOT]
--s3-anonymous bool false object.s3.anonymous YSEARCH_OBJECT_S3_ANONYMOUS Sign no requests at all, for a bucket that grants public reads. This is a deliberate choice, never a fallback: an unresolved credential chain already signs anonymously, and a reader that quietly degraded to unsigned requests would report a permissions problem as an empty index. [object.s3.anonymous; env YSEARCH_OBJECT_S3_ANONYMOUS]
--s3-bucket string object.s3.bucket YSEARCH_OBJECT_S3_BUCKET S3 bucket; required for the s3 backend. [object.s3.bucket; env YSEARCH_OBJECT_S3_BUCKET]
--s3-endpoint string object.s3.endpoint YSEARCH_OBJECT_S3_ENDPOINT S3 endpoint host:port; required for the s3 backend. [object.s3.endpoint; env YSEARCH_OBJECT_S3_ENDPOINT]
--s3-path-style bool false object.s3.path_style YSEARCH_OBJECT_S3_PATH_STYLE Use path-style S3 addressing. [object.s3.path_style; env YSEARCH_OBJECT_S3_PATH_STYLE]
--s3-prefix string object.s3.prefix YSEARCH_OBJECT_S3_PREFIX S3 key prefix. [object.s3.prefix; env YSEARCH_OBJECT_S3_PREFIX]
--s3-region string object.s3.region YSEARCH_OBJECT_S3_REGION S3 region. [object.s3.region; env YSEARCH_OBJECT_S3_REGION]
--s3-use-ssl bool true object.s3.use_ssl YSEARCH_OBJECT_S3_USE_SSL Use TLS to the S3 endpoint. [object.s3.use_ssl; env YSEARCH_OBJECT_S3_USE_SSL]
--source string object.source YSEARCH_OBJECT_SOURCE Dataset URL to attach read-only, such as s3://bucket/root/ for a whole root or s3://bucket/root/indexes/name/ for one index. It derives object.backend, the bucket, prefix and root, the index when the URL names one, and server.read_only, each only where nothing more specific was set. Reading someone else's published index is the whole purpose, so attaching one implies read-only and cannot be talked out of it. [object.source; env YSEARCH_OBJECT_SOURCE]

ysearch config

ysearch config [flags]

config talks to the ConfigService of the server --server names. list, get, set, unset, and watch act on that node; set writes an ephemeral overlay that a restart drops and unset removes. Only node-runtime settings accept set; a startup setting answers FAILED_PRECONDITION naming its scope. export is the offline verb: it resolves this process's own layering and prints it.

Subcommands:

ysearch config export

ysearch config export [flags]

export needs no server. It layers the defaults, the discovered files (or --config / --config-dir), the environment, and the flags exactly as serve would, and prints every setting with its source. --yaml prints the values as one configuration file: secrets, the discovery keys, and compile-time settings are left out because a file may not set them.

Flags:

Flag Type Default Key Env Description
--yaml bool false print the resolved values as a configuration file

ysearch config get

ysearch config get <key> [flags]

Show one setting and where its value came from

ysearch config list

ysearch config list [flags]

List every setting with its value, scope, and source

Flags:

Flag Type Default Key Env Description
--prefix string only keys with this prefix (a trailing dot selects one section)

ysearch config set

ysearch config set <key> <value> [flags]

Override a node-runtime setting until unset or restart

Example:

ysearch config set log.level debug
ysearch config set ingest.seal_age 5s

ysearch config unset

ysearch config unset <key> [flags]

Drop a runtime override; the file, environment, or flag value returns

ysearch config watch

ysearch config watch [flags]

Print every matching setting, then one line per change, until interrupted

Flags:

Flag Type Default Key Env Description
--prefix string only keys with this prefix (a trailing dot selects one section)

ysearch delete

ysearch delete <index> <keys...|-> [flags]

delete tombstones each given key: that key's current version is superseded by nothing, and it stops appearing in results without invalidating the rest of the index. It suppresses the document — the bytes are not reclaimed until a later compaction physically merges the segment holding it away. Deleting a key that does not exist is not an error: delete is idempotent.

Keys are given directly as arguments, or read one per line from stdin (blank lines skipped) when the sole argument is -. Unless --no-flush it flushes and waits for the segment to publish, printing the sealed segment and generation, exactly as push does.

Flags:

Flag Type Default Key Env Description
--batch int 500 keys per batch
--batch-bytes int64 2097152 byte cap per batch
--no-flush bool false accept ephemerally; do not flush or wait for publication
--require-ack bool false wait for PUBLISHED on every batch

ysearch explain

ysearch explain <query> [flags]

explain parses a query and prints its AST, its stable digest, the schema it lowered against, the lowered structured query, and its filters — without touching an index. Without --index the assumed schema is used (every named field is indexed, word is every indexed field); with --index the real schema is fetched and its field names are resolved. --json prints the same as a nested object.

Flags:

Flag Type Default Key Env Description
--dialect string lucene query dialect: lucene or cqp
--index string resolve field names against this index's schema (default: the assumed schema)

ysearch index

ysearch index [flags]

index talks to the IndexService of the server --server names. list names every index in the server's object root with its schema version and active generation; describe prints one index's header and its current schema.

Subcommands:

ysearch index describe

ysearch index describe <index> [flags]

Describe one index and its current schema

ysearch index list

ysearch index list [flags]

List every index with its schema version and generation

Flags:

Flag Type Default Key Env Description
--prefix string only list indexes whose name starts with this prefix

ysearch manual

ysearch manual [flags]

manual renders the reference from the command tree and the configuration catalog, so it cannot drift from the binary. docs/cli.md is this output; go run ./tools/dev manual-check fails when it differs.

ysearch node

ysearch node [flags]

node serves any combination of the query-fleet roles plus the maintenance-only compactor role from one process. Catalog-bound roles resolve their generation before the listener opens, so a node that cannot read its catalog fails at startup rather than accepting work it cannot perform. A dedicated compactor opens no query engine or full-segment cache; it follows latest and performs single-flight copy-on-write physical merges under the compaction budgets.

Example:

ysearch node --roles aggregator --listen 127.0.0.1:9601
	  ysearch node --roles worker --listen 127.0.0.1:9602 --index demo --object-dir ./objects --cache-dir ./cache --aggregators 127.0.0.1:9601
	  ysearch node --roles compactor --listen 127.0.0.1:9701 --index demo --object-dir ./objects --ingest-dir ./scratch

Flags:

Flag Type Default Key Env Description
--analysis-batch-documents int 0 builder.analysis_batch_documents YSEARCH_BUILDER_ANALYSIS_BATCH_DOCUMENTS Documents handed to the analysis workers at once; zero derives sixteen per worker. A batch is the unit that must fit in builder.analysis_memory_bytes, so a larger batch amortises the hand-off over more documents but raises the peak the analysis share has to cover. The derived shape is what the resource gates measure, so a corpus of unusually large documents is the case for lowering it rather than raising the share. [builder.analysis_batch_documents; env YSEARCH_BUILDER_ANALYSIS_BATCH_DOCUMENTS] (in [0, 1048576])
--analysis-memory-bytes bytes 0 builder.analysis_memory_bytes YSEARCH_BUILDER_ANALYSIS_MEMORY_BYTES The share of builder.sort_memory_bytes reserved for documents in flight through analysis; zero derives a thirty-second of it, clamped to between 64KiB and 32MiB and never more than a quarter of what the public-id and docvalue spools leave. It only has to hold one bounded batch, and a document larger than the whole share still runs alone under builder.max_document_working_bytes, so the derived value is deliberately small: a three-million-document run peaked at 9.5MiB against a 128MiB share. Every byte reserved here is a byte the term partitions cannot use, which raises mini-run count and write amplification directly, so raise it only against a measured analysis stall. [builder.analysis_memory_bytes; env YSEARCH_BUILDER_ANALYSIS_MEMORY_BYTES]
--analysis-workers int 0 builder.analysis_workers YSEARCH_BUILDER_ANALYSIS_WORKERS Goroutines tokenising documents inside one build; zero derives GOMAXPROCS, bounded at eight. Analysis is the one parallel stage of an otherwise serial build, so this is per build and multiplies with builder.build_concurrency: the bound exists because a dozen builds each spawning a worker per core oversubscribes the machine and the scheduler churn costs more than the parallelism returns. Output order is preserved regardless of this value. [builder.analysis_workers; env YSEARCH_BUILDER_ANALYSIS_WORKERS] (in [0, 1024])
--blob-dictionary bool true builder.blob_dictionary YSEARCH_BUILDER_BLOB_DICTIONARY Compress stored values against a dictionary trained per blob file. Denser on homogeneous documents, but the dictionary is coded on the build's hottest path, so turning it off trades index size for ingest throughput. [builder.blob_dictionary; env YSEARCH_BUILDER_BLOB_DICTIONARY]
--blob-zstd-level int 0 builder.blob_zstd_level YSEARCH_BUILDER_BLOB_ZSTD_LEVEL Zstd effort for stored values; zero keeps the storage default (7). Levels select different zstd encoders, and a build re-seeds the chosen encoder from the blob dictionary for every value it writes, so a lower level cuts build cost by more than the usual level trade-off suggests. [builder.blob_zstd_level; env YSEARCH_BUILDER_BLOB_ZSTD_LEVEL] (in [0, 11])
--build-concurrency int 0 builder.build_concurrency YSEARCH_BUILDER_BUILD_CONCURRENCY Sealed builds that may run through the seal pipeline at once; zero derives one per CPU, bounded at twelve. One build is a mostly single-threaded chain of analysis, sort, merge and publish that spends about half its time blocked in file system calls, so a single pipeline leaves a multi-core host idle under bulk ingest. The builder's memory is this number times builder.sort_memory_bytes. [builder.build_concurrency; env YSEARCH_BUILDER_BUILD_CONCURRENCY] (in [0, 256])
--builder-catalog-publication enum direct builder.catalog_publication YSEARCH_BUILDER_CATALOG_PUBLICATION Catalog writer mode: direct is the single-node compatibility path; external stops after commit markers for a lease-fenced publisher role. [builder.catalog_publication; env YSEARCH_BUILDER_CATALOG_PUBLICATION] (one of direct, external)
--dedupe-index-identities int 0 builder.dedupe_index_identities YSEARCH_BUILDER_DEDUPE_INDEX_IDENTITIES Distinct document keys one open spool's dedupe index may hold before it gives up; zero derives 1048576. A key spooled twice in one build must resolve to a single winner before analysis, or the segment's dense ordinal assignment refuses the build outright. The index resolves that as records are spooled, from identities ingest already holds, and the sealed build inherits the answer; past this bound it gives up and the build reconstructs the answer by reading its spool twice instead — slower, and exactly as correct. One entry is a 16-byte key hash, a sequence and a mutation version, so the default bounds it at tens of megabytes per open spool and is reached only by a spool of a million very small documents. Lower it to cap that memory on a host running many indexes; set it to one to force the scanning path. [builder.dedupe_index_identities; env YSEARCH_BUILDER_DEDUPE_INDEX_IDENTITIES] (in [0, 268435456])
--builder-fuse-memory-bytes bytes 0 builder.fuse_memory_bytes YSEARCH_BUILDER_FUSE_MEMORY_BYTES Optional binary fuse construction workspace during publication; zero retains Bloom, failed admission retains Bloom. [builder.fuse_memory_bytes; env YSEARCH_BUILDER_FUSE_MEMORY_BYTES] (in [0, 64MiB])
--lexical-blob-threshold bytes 0 builder.lexical_blob_threshold YSEARCH_BUILDER_LEXICAL_BLOB_THRESHOLD Size at which a posting, position or term value is stored in a blob file rather than inline in the key log; zero keeps the default of 64KiB. Separation exists to spare compaction from rewriting large values, and a segment is never compacted — while a blob value is compressed on its own, which with a trained dictionary costs an encoder reset each time, where key-log blocks are compressed in bulk. [builder.lexical_blob_threshold; env YSEARCH_BUILDER_LEXICAL_BLOB_THRESHOLD]
--lexical-block-compression string builder.lexical_block_compression YSEARCH_BUILDER_LEXICAL_BLOCK_COMPRESSION Codec the term, posting and position families compress key-log blocks with: none, snappy, lz4, lz4fast or zstd. Empty keeps the default of lz4. The storage engine's own default leaves the newest level raw because it expects compaction to rewrite it, and a segment stays where it lands. On 300k articles lz4 measured 24.8s and 1.0GB against zstd's 37.7s and 0.8GB, so zstd is the choice for an index written once and read for a long time. [builder.lexical_block_compression; env YSEARCH_BUILDER_LEXICAL_BLOCK_COMPRESSION]
--max-document-working-bytes bytes 0 builder.max_document_working_bytes YSEARCH_BUILDER_MAX_DOCUMENT_WORKING_BYTES What one document may use beyond the partition budget while it is being analyzed; zero derives 64MiB, or a quarter of builder.sort_memory_bytes when that is smaller. It is an allowance rather than a reservation: it exists so a single document larger than the whole partition budget still builds, alone, instead of failing the build. Only a corpus with documents in the hundreds of megabytes needs it raised. [builder.max_document_working_bytes; env YSEARCH_BUILDER_MAX_DOCUMENT_WORKING_BYTES]
--max-queued-builds int 0 builder.max_queued_builds YSEARCH_BUILDER_MAX_QUEUED_BUILDS Sealed builds that may wait for or run in the pipeline before ingest is throttled; zero derives it as four times builder.build_concurrency. A waiting build is a spool on disk, not memory, so this buys ingest room across a build at the cost of ingest.seal_bytes of disk per queued build. Set too low, a bulk load throttles the moment the pipeline is full and ingest runs at the speed of the builds rather than ahead of them. [builder.max_queued_builds; env YSEARCH_BUILDER_MAX_QUEUED_BUILDS] (in [0, 1024])
--max-token-bytes bytes 64KiB builder.max_token_bytes YSEARCH_BUILDER_MAX_TOKEN_BYTES Maximum analyzed token size. A build must be able to admit one token this large, so raising it raises the floor under builder.sort_memory_bytes: a budget that leaves the partitions less than one maximum term refuses to start rather than fail partway. Tokens are words after analysis, so the default is already far past any natural language; raise it only for a corpus with genuinely enormous unbroken tokens. [builder.max_token_bytes; env YSEARCH_BUILDER_MAX_TOKEN_BYTES] (positive)
--merge-fan-in int 0 builder.merge_fan_in YSEARCH_BUILDER_MERGE_FAN_IN Spilled mini-runs merged in one pass; zero derives as many as an eighth of the partition budget affords at builder.run_page_bytes plus 512 bytes of reader overhead each, capped at sixty-four and floored at two. Runs beyond the fan-in need further passes, and each pass rewrites the data, so a low fan-in on a build that spilled heavily shows up as write amplification rather than as a slow merge. The cap is a file-descriptor and seek-pattern bound, not a memory one. [builder.merge_fan_in; env YSEARCH_BUILDER_MERGE_FAN_IN] (in [0, 64])
--builder-output-format-version int 0 builder.output_format_version YSEARCH_BUILDER_OUTPUT_FORMAT_VERSION Segment format emitted by builders and compactors; zero selects this binary's current format. The Kubernetes operator pins this to compatibility.writeFormat on writer roles during staged rollouts. Format 7 adds the stats, forward and hashterms families (spec/40 §3); a compaction that includes an older input writes format 6 instead, unless 7 or 8 is set explicitly, which refuses such a merge. Format 8 adds the exact vector lanes (canonical fp16 vectors, IVF radii, int8 codes; spec/40 §3.2). [builder.output_format_version; env YSEARCH_BUILDER_OUTPUT_FORMAT_VERSION] (in [0, 8])
--builder-paged-posting-directory bool false builder.paged_posting_directory YSEARCH_BUILDER_PAGED_POSTING_DIRECTORY Experimental commit-anchored posting-directory pages under query admission and the existing persistent block quota; old commits retain canonical traversal. [builder.paged_posting_directory; env YSEARCH_BUILDER_PAGED_POSTING_DIRECTORY]
--builder-posting-partition-penalty float 4 builder.posting_partition_penalty YSEARCH_BUILDER_POSTING_PARTITION_PENALTY Metadata cost penalty for experimental variable posting blocks. [builder.posting_partition_penalty; env YSEARCH_BUILDER_POSTING_PARTITION_PENALTY] (in [0, 1e+06])
--publish-verification string checksum builder.publish_verification YSEARCH_BUILDER_PUBLISH_VERIFICATION How a published segment is confirmed: checksum, sample, or readback. checksum compares every object against the checksum the object store reported when it accepted the write — S3 computes that server-side and refuses a mismatched write, and the filesystem backend computes it over the bytes it synced — so nothing is transferred back. sample additionally opens the published copy through the store and queries it; readback additionally fetches every object and rehashes it, which costs the whole index again on every build. An object whose store reports no comparable checksum is read back in any mode. [builder.publish_verification; env YSEARCH_BUILDER_PUBLISH_VERIFICATION]
--builder-routing-fragment-bytes bytes 8MiB builder.routing_fragment_bytes YSEARCH_BUILDER_ROUTING_FRAGMENT_BYTES Optional routing fragment disk cap per built segment; zero disables, overflow keeps canonical scan fallback. Matches compaction.routing_fragment_bytes so a freshly ingested segment and a freshly compacted one carry a fragment the same way; the two stay separate keys because a background merge must be able to carry its own share independent of foreground ingestion. Only a segment built or compacted after this defaulted on carries a fragment: nothing rewrites an existing one, so a settled corpus stays without a routing table until it is backfilled or compacted. [builder.routing_fragment_bytes; env YSEARCH_BUILDER_ROUTING_FRAGMENT_BYTES] (in [0, 64MiB])
--run-page-bytes bytes 0 builder.run_page_bytes YSEARCH_BUILDER_RUN_PAGE_BYTES Read-ahead page one spilled mini-run is buffered in during the final merge; zero derives 64KiB, shrinking it when an eighth of the partition budget cannot hold sixty-four such pages. The merge reserves one page per open run for the whole build, so page size and builder.merge_fan_in trade against each other out of the same share: the derivation prefers smaller pages to a fan-in that would force extra compaction passes on a hot partition. [builder.run_page_bytes; env YSEARCH_BUILDER_RUN_PAGE_BYTES]
--sort-memory-bytes bytes 256MiB builder.sort_memory_bytes YSEARCH_BUILDER_SORT_MEMORY_BYTES Total accounted builder budget: one ledger every part of a build charges against, and the setting the other builder budgets derive from. It divides into the public-id sorter (an eighth, capped at 64MiB), the docvalue and lane spools (their own worst case), the analysis share (builder.analysis_memory_bytes), and the term partitions, which take what is left. A build that cannot hold its partitions in that remainder spills to disk instead, which is the difference between one file per build and hundreds. This is per build, so the builder's memory is builder.build_concurrency times this. [builder.sort_memory_bytes; env YSEARCH_BUILDER_SORT_MEMORY_BYTES] (positive)
--builder-spill-merge-strategy enum heap builder.spill_merge_strategy YSEARCH_BUILDER_SPILL_MERGE_STRATEGY Experimental spill merge selector; heap preserves the baseline until full ingestion benchmarks qualify an alternative. [builder.spill_merge_strategy; env YSEARCH_BUILDER_SPILL_MERGE_STRATEGY] (one of heap, replace-root, loser-tree)
--builder-startup-bundles bool false builder.startup_bundles YSEARCH_BUILDER_STARTUP_BUNDLES Asynchronously produce optional startup bundles after catalog publication. One bounded coalescing worker per publishing host; oversized sources retain canonical startup fallback. [builder.startup_bundles; env YSEARCH_BUILDER_STARTUP_BUNDLES]
--term-dictionary-block-size bytes 64KiB builder.term_dictionary_block_size YSEARCH_BUILDER_TERM_DICTIONARY_BLOCK_SIZE Key-log data-block size of the term family. A table carries one block-index entry per data block and reads the whole index before it can look up anything, so this is what a cold term probe pays to find out where to look. At the storage engine's 4KiB an 8MiB dictionary needs about two thousand entries — around 65KiB read on every open of the table and held in memory until it closes — where 64KiB needs about a sixteenth of that. A probe then reads a bigger block, which over an object store is the cheap side of the trade: measured on a 6MiB dictionary the block cost 17KiB compressed against the 48KiB of index it replaced. [builder.term_dictionary_block_size; env YSEARCH_BUILDER_TERM_DICTIONARY_BLOCK_SIZE] (positive)
--term-dictionary-bloom bool false builder.term_dictionary_bloom YSEARCH_BUILDER_TERM_DICTIONARY_BLOOM Write a Bloom filter over the term family. The storage engine writes one by default; a segment does not, because the filter is read whole — uncompressed, about 1.2 bytes per distinct term — when the table is opened, and kept in memory until it closes, to save at most one block read per probe against the single table a published segment's term family is. Measured, it was nine tenths of what a cold term probe read: 431KiB of 481KiB on a 360k-term dictionary. A table says in its own footer whether it has a filter, so segments published with one keep working unchanged. [builder.term_dictionary_bloom; env YSEARCH_BUILDER_TERM_DICTIONARY_BLOOM]
--builder-variable-posting-blocks bool false builder.variable_posting_blocks YSEARCH_BUILDER_VARIABLE_POSTING_BLOCKS Experimental bounded-window BM25F-aware posting partitioning for format5 or newer output. [builder.variable_posting_blocks; env YSEARCH_BUILDER_VARIABLE_POSTING_BLOCKS]
--cache-block-bytes bytes 2GiB cache.block_bytes YSEARCH_CACHE_BLOCK_BYTES Disk, not memory: bytes of verified decoded blocks this node may keep under cache.dir. This is the cheap half of block caching - it converts a remote read into a local one and costs no resident memory - so size it above one query's working set and leave cache.read_block_bytes to decide what stays in RAM. Measured on 294 segments: one single-term query touched about 363MB of term dictionary and postings and one scored two-term query about 544MB, so the default holds several such working sets. [cache.block_bytes; env YSEARCH_CACHE_BLOCK_BYTES] (positive)
--cache-block-entries int 1048576 cache.block_entries YSEARCH_CACHE_BLOCK_ENTRIES Disk, not memory: how many decoded blocks cache.block_bytes may be spread over. Whichever is reached first bounds the cache; at the measured ~237KiB per block the byte budget is reached first by a wide margin. [cache.block_entries; env YSEARCH_CACHE_BLOCK_ENTRIES] (positive)
--cache-dir string cache.dir YSEARCH_CACHE_DIR Disposable local cache directory; symlinks in its path are resolved once at startup and the cache then anchors there without following any. It holds the two disk budgets, cache.full_bytes and cache.block_bytes, and nothing else: every other cache.* budget is process memory. A relative value in a configuration file resolves against that file's directory. [cache.dir; env YSEARCH_CACHE_DIR]
--cache-full-bytes bytes 8GiB cache.full_bytes YSEARCH_CACHE_FULL_BYTES Disk, not memory: bytes of whole hydrated segments this node may keep under cache.dir. With cache.block_bytes it is the whole of what the cache directory holds, so a volume smaller than their sum is an eviction loop the node cannot see. [cache.full_bytes; env YSEARCH_CACHE_FULL_BYTES] (positive)
--cache-full-entries int 1024 cache.full_entries YSEARCH_CACHE_FULL_ENTRIES Disk, not memory: how many whole segments cache.full_bytes may be spread over. Whichever of the two is reached first bounds the cache, and at the measured ~340MB per segment of a 101GB corpus the byte budget is reached long before this one. [cache.full_entries; env YSEARCH_CACHE_FULL_ENTRIES] (positive)
--cache-lazy-readers bool false cache.lazy_readers YSEARCH_CACHE_LAZY_READERS Experimental: acquire immutable segment readers and optional filter payloads only for admitted query work. It is also what routing.presence_bytes zero means: eager readers read that zero as unbounded and hold every segment's presence filter, lazy readers read it as none. [cache.lazy_readers; env YSEARCH_CACHE_LAZY_READERS]
--cache-ordinal-bytes bytes 64MiB cache.ordinal_bytes YSEARCH_CACHE_ORDINAL_BYTES Memory, not disk: process-wide verified ordinal-ID pages, including pinned pages and in-flight page workspace, shared across indexes and generations. [cache.ordinal_bytes; env YSEARCH_CACHE_ORDINAL_BYTES] (at least 327936)
--cache-read-block-bytes bytes 512MiB cache.read_block_bytes YSEARCH_CACHE_READ_BLOCK_BYTES Memory, not disk: process-wide decoded WavesDB blocks held resident and shared by immutable segment readers. This is what decides whether a repeated query is warm. Below one query's working set nothing survives to the next query and every query pays the cold price forever: measured on 294 segments, a 24MiB budget re-read all 214MB of term dictionary byte for byte while about 495MiB answered the repeat from 57,421 bytes. Size it from the corpus, not the machine, and from the layout its segments were built with. A term probe costs the same whatever the term, but what it costs depends on whether the segment carries a term-family Bloom filter: about 711KiB per segment on one built before builder.term_dictionary_bloom defaulted to false, and about 21KiB on one built since. The term dictionary alone needs segments times that figure, and the postings of the shape you serve come on top. A mixed corpus pays the older rate for the segments it has not rebuilt. [cache.read_block_bytes; env YSEARCH_CACHE_READ_BLOCK_BYTES] (positive)
--cache-read-open-files int 1024 cache.read_open_files YSEARCH_CACHE_READ_OPEN_FILES Process-wide open WavesDB table-file limit shared by immutable segment readers. It spends file descriptors, and nothing here raises RLIMIT_NOFILE, so the default sits at the 1024 a Linux process is commonly given with the listeners, connections and cache files sharing it: raise the limit before raising this. [cache.read_open_files; env YSEARCH_CACHE_READ_OPEN_FILES] (positive)
--cache-read-open-readers int 4098 cache.read_open_readers YSEARCH_CACHE_READ_OPEN_READERS Process-wide decoded WavesDB table-reader count limit. Each retained reader also spends cache.read_reader_bytes, and whichever of the two is reached first evicts. Unlike cache.read_open_files this spends memory rather than file descriptors, so size it from the corpus: a segment holds several table families, and a count that cannot hold them all evicts on every query. At 1024, which this defaulted to, a 253-segment corpus needing 1,321 readers held 77% of them and evicted 3,022 times while its byte budget sat 89% empty. That is dearer than it sounds, because a re-opened table re-reads its Bloom filter and block index through a path that bypasses the block cache, so the eviction becomes object reads that repeat for the life of the node: the same query read 62.8MiB in 275 range GETs every time it ran, and none after the count was raised. Measured resident cost is about 760KiB per reader. [cache.read_open_readers; env YSEARCH_CACHE_READ_OPEN_READERS] (positive)
--cache-read-reader-bytes bytes 2GiB cache.read_reader_bytes YSEARCH_CACHE_READ_READER_BYTES Memory, not disk: process-wide WavesDB table index and Bloom-filter bytes held resident. It is the only cache budget that scales with the corpus rather than with the query, at a measured ~14MiB per segment, so the default holds roughly 145 segments and a larger corpus re-reads a segment's index after evicting it. [cache.read_reader_bytes; env YSEARCH_CACHE_READ_READER_BYTES] (positive)
--cache-result-enabled bool true cache.result_enabled YSEARCH_CACHE_RESULT_ENABLED Cache generation and segment ranked results. Disable for execution benchmarks; reader, block, plan and stored-field caches remain active. [cache.result_enabled; env YSEARCH_CACHE_RESULT_ENABLED]
--cache-scrub-entries int 4 cache.scrub_entries YSEARCH_CACHE_SCRUB_ENTRIES How many cached full segments one scrubber pass re-reads. The scrubber walks the cache in a rotating cursor, so this and cache.scrub_interval together set how long a full sweep takes and how much disk read it costs; a segment currently in use is skipped and picked up on a later pass. [cache.scrub_entries; env YSEARCH_CACHE_SCRUB_ENTRIES] (in [0, 4096])
--cache-scrub-interval duration 5m cache.scrub_interval YSEARCH_CACHE_SCRUB_INTERVAL How often the background scrubber re-reads cached full segments and checks them against their commits. This is the only thing that notices a cached segment rotting on disk: a query trusts an entry once it has been verified, because verifying costs the SHA-256 of every file in the segment and doing that per request made a many-segment query spend all its time hashing. Zero disables the scrubber, which leaves local corruption undetected until the entry is evicted. [cache.scrub_interval; env YSEARCH_CACHE_SCRUB_INTERVAL] (in [0s, 24h0m0s])
--cache-sidecar-bytes bytes 8MiB cache.sidecar_bytes YSEARCH_CACHE_SIDECAR_BYTES Memory, not disk: process-wide optional exact-filter decoder and lease budget shared across indexes and generations. [cache.sidecar_bytes; env YSEARCH_CACHE_SIDECAR_BYTES] (positive)
--cache-warm-prefetch-bytes bytes 0 cache.warm_prefetch_bytes YSEARCH_CACHE_WARM_PREFETCH_BYTES Optional routing-page warmup byte limit after the first served query; zero disables. It spends object reads, not a budget of its own: warmed pages land in the process-wide routing page cache and are charged there, so warming past that cache only evicts what it just read. Shares foreground cache admission. [cache.warm_prefetch_bytes; env YSEARCH_CACHE_WARM_PREFETCH_BYTES] (in [0, 64MiB])
--cache-warm-prefetch-concurrency int 1 cache.warm_prefetch_concurrency YSEARCH_CACHE_WARM_PREFETCH_CONCURRENCY Maximum concurrent optional warmup reads; foreground queries stop new prefetch work. [cache.warm_prefetch_concurrency; env YSEARCH_CACHE_WARM_PREFETCH_CONCURRENCY] (in [0, 8])
--compaction-cluster-field string compaction.cluster_field YSEARCH_COMPACTION_CLUSTER_FIELD Existing scalar filterable string used by cluster layout; an absent field preserves order. [compaction.cluster_field; env YSEARCH_COMPACTION_CLUSTER_FIELD]
--compaction-document-layout enum preserve compaction.document_layout YSEARCH_COMPACTION_DOCUMENT_LAYOUT Experimental survivor layout for new compaction outputs; preserve retains input order, cluster groups scalar categories, bisection uses a bounded sampled term graph. [compaction.document_layout; env YSEARCH_COMPACTION_DOCUMENT_LAYOUT] (one of preserve, cluster, bisection)
--compaction-enabled bool false compaction.enabled YSEARCH_COMPACTION_ENABLED Run single-flight size-tiered compaction inside all-in-one serve. The explicit node compactor role always runs; this switch stays off there and by default so production can isolate maintenance from query and ingest. [compaction.enabled; env YSEARCH_COMPACTION_ENABLED]
--compaction-hydration-bytes bytes 8GiB compaction.hydration_bytes YSEARCH_COMPACTION_HYDRATION_BYTES Maximum downloaded checkpoint payload retained per compaction job across all input hydrations. Borrowed resident inputs are not copied or charged. Filesystem metadata, output database and spill space have separate accounting. [compaction.hydration_bytes; env YSEARCH_COMPACTION_HYDRATION_BYTES] (positive)
--compaction-hydration-workers int 2 compaction.hydration_workers YSEARCH_COMPACTION_HYDRATION_WORKERS Maximum concurrent input hydrations per compaction job. [compaction.hydration_workers; env YSEARCH_COMPACTION_HYDRATION_WORKERS] (in [1, 8])
--compaction-interval duration 1m compaction.interval YSEARCH_COMPACTION_INTERVAL Interval between background compaction eligibility checks. The all-in-one server waits for sustained ingest idleness; use the isolated compactor role when fan-out must be reduced during continuous ingest. [compaction.interval; env YSEARCH_COMPACTION_INTERVAL] (in [1s, 24h0m0s])
--compaction-max-active-segments int 16 compaction.max_active_segments YSEARCH_COMPACTION_MAX_ACTIVE_SEGMENTS Active searchable segment fan-out above which the scheduler may compact. [compaction.max_active_segments; env YSEARCH_COMPACTION_MAX_ACTIVE_SEGMENTS] (positive)
--compaction-max-concurrent-jobs int 1 compaction.max_concurrent_jobs YSEARCH_COMPACTION_MAX_CONCURRENT_JOBS Shared background compaction job slots across hosted indexes; physical merges are serial and hydration has a separate worker cap. [compaction.max_concurrent_jobs; env YSEARCH_COMPACTION_MAX_CONCURRENT_JOBS] (in [1, 8])
--compaction-max-inputs int 8 compaction.max_inputs YSEARCH_COMPACTION_MAX_INPUTS Maximum inputs in one compaction job; jobs are always single-flight. [compaction.max_inputs; env YSEARCH_COMPACTION_MAX_INPUTS] (in [2, 64])
--compaction-max-output-bytes bytes 4GiB compaction.max_output_bytes YSEARCH_COMPACTION_MAX_OUTPUT_BYTES Maximum summed input bytes admitted to one compaction proposal. [compaction.max_output_bytes; env YSEARCH_COMPACTION_MAX_OUTPUT_BYTES] (positive)
--compaction-max-size-ratio int 2 compaction.max_size_ratio YSEARCH_COMPACTION_MAX_SIZE_RATIO Largest size ratio allowed between inputs in one tier. [compaction.max_size_ratio; env YSEARCH_COMPACTION_MAX_SIZE_RATIO] (in [1, 1024])
--compaction-min-inputs int 4 compaction.min_inputs YSEARCH_COMPACTION_MIN_INPUTS Minimum adjacent similarly-sized inputs in one compaction job. [compaction.min_inputs; env YSEARCH_COMPACTION_MIN_INPUTS] (in [2, 64])
--compaction-read-inflight-bytes bytes 64MiB compaction.read_inflight_bytes YSEARCH_COMPACTION_READ_INFLIGHT_BYTES Shared compaction read payload bytes in flight across jobs. Open reads hold credits until Close; an individual object or range larger than this limit is rejected. [compaction.read_inflight_bytes; env YSEARCH_COMPACTION_READ_INFLIGHT_BYTES] (at least 64KiB)
--compaction-recent-delta int 4 compaction.recent_delta YSEARCH_COMPACTION_RECENT_DELTA Newest active segments left uncompacted as an ingest delta tier. [compaction.recent_delta; env YSEARCH_COMPACTION_RECENT_DELTA] (in [0, 1024])
--compaction-reorder-leaf-documents int 128 compaction.reorder_leaf_documents YSEARCH_COMPACTION_REORDER_LEAF_DOCUMENTS Maximum stable leaf size of experimental balanced graph bisection; each document samples at most 32 terms. [compaction.reorder_leaf_documents; env YSEARCH_COMPACTION_REORDER_LEAF_DOCUMENTS] (in [1, 4096])
--compaction-reorder-temp-bytes bytes 1GiB compaction.reorder_temp_bytes YSEARCH_COMPACTION_REORDER_TEMP_BYTES Temporary spill cap for reordered outputs, including permutation, sampled graph and posting resort; the smaller compaction.temporary_bytes cap also applies. [compaction.reorder_temp_bytes; env YSEARCH_COMPACTION_REORDER_TEMP_BYTES] (positive)
--compaction-routing-fragment-bytes bytes 8MiB compaction.routing_fragment_bytes YSEARCH_COMPACTION_ROUTING_FRAGMENT_BYTES Optional routing fragment disk cap per compaction output segment; overrides builder.routing_fragment_bytes for compaction's own merges so background maintenance can carry its own share independent of foreground ingestion. Zero disables fragment writing for compaction outputs, which also stops this generation's routing descriptor from ever completing until every one of its segments gets a fragment some other way. [compaction.routing_fragment_bytes; env YSEARCH_COMPACTION_ROUTING_FRAGMENT_BYTES] (in [0, 64MiB])
--compaction-routing-memory-bytes bytes 64MiB compaction.routing_memory_bytes YSEARCH_COMPACTION_ROUTING_MEMORY_BYTES Memory budget for assembling one generation's routing descriptor from its member segments' own fragments after a successful compaction. Separate from compaction.temporary_bytes and the segment-merge budgets: descriptor assembly reads small fragment objects, not segment checkpoints. [compaction.routing_memory_bytes; env YSEARCH_COMPACTION_ROUTING_MEMORY_BYTES] (at least 64MiB)
--compaction-routing-temp-bytes bytes 1GiB compaction.routing_temp_bytes YSEARCH_COMPACTION_ROUTING_TEMP_BYTES Spill and page-file budget for the same post-compaction descriptor assembly. Bounded well below compaction.temporary_bytes: a table assembled from fragments alone is the right-sized presence table (internal/routing/value.go), not the segments' own content. [compaction.routing_temp_bytes; env YSEARCH_COMPACTION_ROUTING_TEMP_BYTES] (positive)
--compaction-target-bytes bytes 512MiB compaction.target_bytes YSEARCH_COMPACTION_TARGET_BYTES Approximate input-byte target for one size-tiered compaction group. [compaction.target_bytes; env YSEARCH_COMPACTION_TARGET_BYTES] (positive)
--compaction-temporary-bytes bytes 8GiB compaction.temporary_bytes YSEARCH_COMPACTION_TEMPORARY_BYTES Shared accounted spill-byte limit within one physical compaction; hydration and final database storage are separate from spill space. [compaction.temporary_bytes; env YSEARCH_COMPACTION_TEMPORARY_BYTES] (positive)
--compaction-write-inflight-bytes bytes 64MiB compaction.write_inflight_bytes YSEARCH_COMPACTION_WRITE_INFLIGHT_BYTES Shared compaction write payload bytes in flight across jobs. An individual object larger than this limit is rejected before upload. [compaction.write_inflight_bytes; env YSEARCH_COMPACTION_WRITE_INFLIGHT_BYTES] (at least 64KiB)
--config-watch bool true config.watch YSEARCH_CONFIG_WATCH Reload the discovered configuration files when any of them changes. [config.watch; env YSEARCH_CONFIG_WATCH]
--pprof-listen string debug.pprof_listen YSEARCH_DEBUG_PPROF_LISTEN host:port serving net/http/pprof profiles; empty disables it. It binds before the catalog is opened, so a slow start can be profiled, and an address with no host (:6060) binds loopback. Profiles expose heap contents and goroutine stacks, so keep it on loopback and never on a public interface. [debug.pprof_listen; env YSEARCH_DEBUG_PPROF_LISTEN]
--embedding-external-allow-http bool false embedding.external_allow_http YSEARCH_EMBEDDING_EXTERNAL_ALLOW_HTTP Allow plain HTTP external embedding endpoints for trusted development networks. [embedding.external_allow_http; env YSEARCH_EMBEDDING_EXTERNAL_ALLOW_HTTP]
--embedding-external-batch-size int 128 embedding.external_batch_size YSEARCH_EMBEDDING_EXTERNAL_BATCH_SIZE Maximum texts in one external embedding request. [embedding.external_batch_size; env YSEARCH_EMBEDDING_EXTERNAL_BATCH_SIZE] (in [1, 4096])
--embedding-external-concurrency int 8 embedding.external_concurrency YSEARCH_EMBEDDING_EXTERNAL_CONCURRENCY Concurrent external embedding requests per process. [embedding.external_concurrency; env YSEARCH_EMBEDDING_EXTERNAL_CONCURRENCY] (in [1, 1024])
--embedding-external-max-retry-after duration 2s embedding.external_max_retry_after YSEARCH_EMBEDDING_EXTERNAL_MAX_RETRY_AFTER Largest Retry-After delay accepted from an external embedding endpoint. [embedding.external_max_retry_after; env YSEARCH_EMBEDDING_EXTERNAL_MAX_RETRY_AFTER] (positive)
--embedding-external-request-bytes bytes 8MiB embedding.external_request_bytes YSEARCH_EMBEDDING_EXTERNAL_REQUEST_BYTES Maximum encoded external embedding request bytes. [embedding.external_request_bytes; env YSEARCH_EMBEDDING_EXTERNAL_REQUEST_BYTES] (positive)
--embedding-external-response-bytes bytes 64MiB embedding.external_response_bytes YSEARCH_EMBEDDING_EXTERNAL_RESPONSE_BYTES Maximum external embedding response bytes read before JSON decoding. [embedding.external_response_bytes; env YSEARCH_EMBEDDING_EXTERNAL_RESPONSE_BYTES] (positive)
--embedding-external-retries int 2 embedding.external_retries YSEARCH_EMBEDDING_EXTERNAL_RETRIES Retry attempts after the first retryable external embedding failure. [embedding.external_retries; env YSEARCH_EMBEDDING_EXTERNAL_RETRIES] (in [0, 16])
--embedding-external-retry-base duration 50ms embedding.external_retry_base YSEARCH_EMBEDDING_EXTERNAL_RETRY_BASE Base exponential backoff for external embedding retries. [embedding.external_retry_base; env YSEARCH_EMBEDDING_EXTERNAL_RETRY_BASE] (positive)
--embedding-external-timeout duration 30s embedding.external_timeout YSEARCH_EMBEDDING_EXTERNAL_TIMEOUT Deadline for one external embedding HTTP attempt sequence. [embedding.external_timeout; env YSEARCH_EMBEDDING_EXTERNAL_TIMEOUT] (positive)
--embedding-grpc-allow-insecure bool false embedding.grpc_allow_insecure YSEARCH_EMBEDDING_GRPC_ALLOW_INSECURE Allow plaintext gRPC embedding endpoints on trusted internal networks. [embedding.grpc_allow_insecure; env YSEARCH_EMBEDDING_GRPC_ALLOW_INSECURE]
--embedding-grpc-batch-size int 128 embedding.grpc_batch_size YSEARCH_EMBEDDING_GRPC_BATCH_SIZE Maximum texts in one gRPC embedding request. [embedding.grpc_batch_size; env YSEARCH_EMBEDDING_GRPC_BATCH_SIZE] (in [1, 4096])
--embedding-grpc-concurrency int 8 embedding.grpc_concurrency YSEARCH_EMBEDDING_GRPC_CONCURRENCY Concurrent gRPC embedding requests per process. [embedding.grpc_concurrency; env YSEARCH_EMBEDDING_GRPC_CONCURRENCY] (in [1, 1024])
--embedding-grpc-request-bytes bytes 8MiB embedding.grpc_request_bytes YSEARCH_EMBEDDING_GRPC_REQUEST_BYTES Maximum encoded gRPC embedding request bytes. [embedding.grpc_request_bytes; env YSEARCH_EMBEDDING_GRPC_REQUEST_BYTES] (positive)
--embedding-grpc-response-bytes bytes 64MiB embedding.grpc_response_bytes YSEARCH_EMBEDDING_GRPC_RESPONSE_BYTES Maximum encoded gRPC embedding response bytes. [embedding.grpc_response_bytes; env YSEARCH_EMBEDDING_GRPC_RESPONSE_BYTES] (positive)
--embedding-grpc-timeout duration 30s embedding.grpc_timeout YSEARCH_EMBEDDING_GRPC_TIMEOUT Deadline for one gRPC embedding request. [embedding.grpc_timeout; env YSEARCH_EMBEDDING_GRPC_TIMEOUT] (positive)
--embedding-grpc-work-class enum auto embedding.grpc_work_class YSEARCH_EMBEDDING_GRPC_WORK_CLASS gRPC embedding scheduler class; auto maps queries to interactive and indexing to bulk. [embedding.grpc_work_class; env YSEARCH_EMBEDDING_GRPC_WORK_CLASS] (one of auto, interactive, bulk)
--embedding-internal-batch-size int 32 embedding.internal_batch_size YSEARCH_EMBEDDING_INTERNAL_BATCH_SIZE Maximum texts in one internal embedding batch. [embedding.internal_batch_size; env YSEARCH_EMBEDDING_INTERNAL_BATCH_SIZE] (in [1, 4096])
--embedding-internal-concurrency int 2 embedding.internal_concurrency YSEARCH_EMBEDDING_INTERNAL_CONCURRENCY Concurrent pure-Go internal embedding batches. [embedding.internal_concurrency; env YSEARCH_EMBEDDING_INTERNAL_CONCURRENCY] (in [1, 256])
--embedding-internal-enabled bool false embedding.internal_enabled YSEARCH_EMBEDDING_INTERNAL_ENABLED Enable the pure-Go internal embedding runtime. Models remain lazy and checksum-pinned by schema profiles. [embedding.internal_enabled; env YSEARCH_EMBEDDING_INTERNAL_ENABLED]
--embedding-model-cache-bytes bytes 10GiB embedding.model_cache_bytes YSEARCH_EMBEDDING_MODEL_CACHE_BYTES Maximum installed bytes in the internal-model cache. [embedding.model_cache_bytes; env YSEARCH_EMBEDDING_MODEL_CACHE_BYTES] (positive)
--embedding-model-cache-dir string embedding.model_cache_dir YSEARCH_EMBEDDING_MODEL_CACHE_DIR Checksum-addressed internal-model cache directory. Empty disables internal embedding; keep it distinct from cache.dir. A relative value in a configuration file resolves against that file's directory. [embedding.model_cache_dir; env YSEARCH_EMBEDDING_MODEL_CACHE_DIR]
--embedding-model-cache-packages int 8 embedding.model_cache_packages YSEARCH_EMBEDDING_MODEL_CACHE_PACKAGES Maximum installed internal-model packages. [embedding.model_cache_packages; env YSEARCH_EMBEDDING_MODEL_CACHE_PACKAGES] (positive)
--embedding-model-download-allow-http bool false embedding.model_download_allow_http YSEARCH_EMBEDDING_MODEL_DOWNLOAD_ALLOW_HTTP Allow plain HTTP model manifests and artifacts for trusted development networks. [embedding.model_download_allow_http; env YSEARCH_EMBEDDING_MODEL_DOWNLOAD_ALLOW_HTTP]
--embedding-model-download-concurrency int 2 embedding.model_download_concurrency YSEARCH_EMBEDDING_MODEL_DOWNLOAD_CONCURRENCY Concurrent verified internal-model artifact downloads. [embedding.model_download_concurrency; env YSEARCH_EMBEDDING_MODEL_DOWNLOAD_CONCURRENCY] (in [1, 64])
--embedding-model-download-timeout duration 10m embedding.model_download_timeout YSEARCH_EMBEDDING_MODEL_DOWNLOAD_TIMEOUT Deadline for one internal-model package installation. [embedding.model_download_timeout; env YSEARCH_EMBEDDING_MODEL_DOWNLOAD_TIMEOUT] (positive)
--embedding-model-manifest-bytes bytes 1MiB embedding.model_manifest_bytes YSEARCH_EMBEDDING_MODEL_MANIFEST_BYTES Maximum bytes in one internal-model manifest. [embedding.model_manifest_bytes; env YSEARCH_EMBEDDING_MODEL_MANIFEST_BYTES] (positive)
--embedding-model-package-bytes bytes 4GiB embedding.model_package_bytes YSEARCH_EMBEDDING_MODEL_PACKAGE_BYTES Maximum bytes in one downloaded internal-model package. [embedding.model_package_bytes; env YSEARCH_EMBEDDING_MODEL_PACKAGE_BYTES] (positive)
--embedding-query-cache-bytes bytes 64MiB embedding.query_cache_bytes YSEARCH_EMBEDDING_QUERY_CACHE_BYTES Maximum float-vector bytes retained by the text-query embedding cache. [embedding.query_cache_bytes; env YSEARCH_EMBEDDING_QUERY_CACHE_BYTES] (positive)
--embedding-query-cache-entries int 4096 embedding.query_cache_entries YSEARCH_EMBEDDING_QUERY_CACHE_ENTRIES Maximum completed text-query embeddings retained per process. [embedding.query_cache_entries; env YSEARCH_EMBEDDING_QUERY_CACHE_ENTRIES] (positive)
--embedding-query-cache-ttl duration 10m embedding.query_cache_ttl YSEARCH_EMBEDDING_QUERY_CACHE_TTL Lifetime of a completed text-query embedding cache entry. [embedding.query_cache_ttl; env YSEARCH_EMBEDDING_QUERY_CACHE_TTL] (positive)
--admin-fleet list fleet.admin_fleet YSEARCH_FLEET_ADMIN_FLEET Node host:port endpoints StreamFleetStats fans in; empty refuses the fleet stream. [fleet.admin_fleet; env YSEARCH_FLEET_ADMIN_FLEET]
--aggregators list fleet.aggregators YSEARCH_FLEET_AGGREGATORS Aggregator host:port endpoints. [fleet.aggregators; env YSEARCH_FLEET_AGGREGATORS]
--builders list fleet.builders YSEARCH_FLEET_BUILDERS Builder host:port endpoints the router polls and rendezvous-routes batches across. [fleet.builders; env YSEARCH_FLEET_BUILDERS]
--fallback-workers list fleet.fallback_workers YSEARCH_FLEET_FALLBACK_WORKERS Worker host:port endpoints: the dispatch universe and the cache-oblivious fallback. [fleet.fallback_workers; env YSEARCH_FLEET_FALLBACK_WORKERS]
--fan-in int 0 fleet.fan_in YSEARCH_FLEET_FAN_IN Children per merge node in coordinator plans; zero means the planner's default, otherwise at least 2. [fleet.fan_in; env YSEARCH_FLEET_FAN_IN] (in [0, 1024])
--lane-slots string interactive=4,streaming=2 fleet.lane_slots YSEARCH_FLEET_LANE_SLOTS Per-lane worker slot pools, lane=count. [fleet.lane_slots; env YSEARCH_FLEET_LANE_SLOTS]
--mergers list fleet.mergers YSEARCH_FLEET_MERGERS Merge host:port endpoints: the coordinator's merge tier, a merger's delegation peers. [fleet.mergers; env YSEARCH_FLEET_MERGERS]
--follower-discovery-interval duration 30s follower.discovery_interval YSEARCH_FOLLOWER_DISCOVERY_INTERVAL How often a root attachment looks for indexes that appeared or disappeared under it. Separate from follower.poll_interval because discovery lists prefixes while a poll reads one pointer, and listing is the more expensive and more rate-limited of the two. [follower.discovery_interval; env YSEARCH_FOLLOWER_DISCOVERY_INTERVAL] (positive)
--follower-generation-overlap duration 30s follower.generation_overlap YSEARCH_FOLLOWER_GENERATION_OVERLAP Maximum age of an unseen catalog generation a worker may resolve on demand; in-flight references may keep its engine beyond this window. [follower.generation_overlap; env YSEARCH_FOLLOWER_GENERATION_OVERLAP] (positive)
--follower-max-staleness duration 5m follower.max_staleness YSEARCH_FOLLOWER_MAX_STALENESS How long a follower may go without confirming its installed generation is still current before it refuses new queries. A reader that cannot reach its source keeps answering from cached data indefinitely otherwise, which is worse than an error: the results look fine and are silently frozen. Status stays readable past this point so an operator can see why. [follower.max_staleness; env YSEARCH_FOLLOWER_MAX_STALENESS] (positive)
--follower-poll-interval duration 2s follower.poll_interval YSEARCH_FOLLOWER_POLL_INTERVAL How often a node checks the catalog's latest pointer. [follower.poll_interval; env YSEARCH_FOLLOWER_POLL_INTERVAL] (positive)
--follower-retired-generation-grace duration 30s follower.retired_generation_grace YSEARCH_FOLLOWER_RETIRED_GENERATION_GRACE How long a superseded generation's engine stays open after its last query. [follower.retired_generation_grace; env YSEARCH_FOLLOWER_RETIRED_GENERATION_GRACE] (positive)
--gc-grace duration 1h gc.grace YSEARCH_GC_GRACE Additional safety margin added to every scheduled GC reachability horizon. [gc.grace; env YSEARCH_GC_GRACE] (in [0s, 168h0m0s])
--gc-interval duration 1h gc.interval YSEARCH_GC_INTERVAL Cadence at which the dedicated compactor role writes a GC proposal and considers a quarantined older proposal. [gc.interval; env YSEARCH_GC_INTERVAL] (in [1s, 168h0m0s])
--gc-maximum-stream-lifetime duration 1h gc.maximum_stream_lifetime YSEARCH_GC_MAXIMUM_STREAM_LIFETIME Longest query-stream lifetime protected when marking superseded generations for scheduled GC. [gc.maximum_stream_lifetime; env YSEARCH_GC_MAXIMUM_STREAM_LIFETIME] (in [0s, 168h0m0s])
--gc-minimum-upload-age duration 24h gc.minimum_upload_age YSEARCH_GC_MINIMUM_UPLOAD_AGE Minimum age before an unreferenced object can enter a scheduled GC proposal. [gc.minimum_upload_age; env YSEARCH_GC_MINIMUM_UPLOAD_AGE] (in [0s, 720h0m0s])
--gc-quarantine-age duration 24h gc.quarantine_age YSEARCH_GC_QUARANTINE_AGE Minimum age of an immutable GC proposal before a fresh mark may authorize deleting its still-unreachable exact keys. [gc.quarantine_age; env YSEARCH_GC_QUARANTINE_AGE] (in [1s, 720h0m0s])
--gc-retained-generations int 2 gc.retained_generations YSEARCH_GC_RETAINED_GENERATIONS Historical catalog generations retained in addition to latest during scheduled GC. [gc.retained_generations; env YSEARCH_GC_RETAINED_GENERATIONS] (in [0, 1024])
--gc-sweep-enabled bool false gc.sweep_enabled YSEARCH_GC_SWEEP_ENABLED Allow the dedicated compactor role to delete freshly re-proven orphan objects after an immutable dry-run proposal passes quarantine. Disabled by default. [gc.sweep_enabled; env YSEARCH_GC_SWEEP_ENABLED]
--generation string index.generation YSEARCH_INDEX_GENERATION Exact generation to pin, or empty for the latest. [index.generation; env YSEARCH_INDEX_GENERATION]
--index string index.name YSEARCH_INDEX_NAME Index name for catalog-bound serve, worker, coordinator, compactor, router, builder, and publisher roles. [index.name; env YSEARCH_INDEX_NAME]
--ingest-burst-documents int 100000 ingest.burst_documents YSEARCH_INGEST_BURST_DOCUMENTS Per-index document tokens available for an ingest burst when the rate quota is enabled. [ingest.burst_documents; env YSEARCH_INGEST_BURST_DOCUMENTS] (positive)
--ingest-dir string ingest.dir YSEARCH_INGEST_DIR Ingest spool/build directory for builders and scratch root for the dedicated compactor role. A relative value in a configuration file resolves against that file's directory. [ingest.dir; env YSEARCH_INGEST_DIR]
--ingest-documents-per-second int 0 ingest.documents_per_second YSEARCH_INGEST_DOCUMENTS_PER_SECOND Per-router, per-index document admission rate; zero disables the rate quota. [ingest.documents_per_second; env YSEARCH_INGEST_DOCUMENTS_PER_SECOND] (in [0, 1099511627776])
--ingest-global-queue-bytes bytes 256MiB ingest.global_queue_bytes YSEARCH_INGEST_GLOBAL_QUEUE_BYTES Encoded ingest bytes all router sessions may hold while waiting for builders. [ingest.global_queue_bytes; env YSEARCH_INGEST_GLOBAL_QUEUE_BYTES] (positive)
--ingest-index-queue-bytes bytes 64MiB ingest.index_queue_bytes YSEARCH_INGEST_INDEX_QUEUE_BYTES Encoded ingest bytes one index may hold while waiting for builders. [ingest.index_queue_bytes; env YSEARCH_INGEST_INDEX_QUEUE_BYTES] (positive)
--ingest-max-batch-bytes bytes 4MiB ingest.max_batch_bytes YSEARCH_INGEST_MAX_BATCH_BYTES Largest accepted ingest batch. [ingest.max_batch_bytes; env YSEARCH_INGEST_MAX_BATCH_BYTES] (positive)
--ingest-max-document-bytes bytes 16MiB ingest.max_document_bytes YSEARCH_INGEST_MAX_DOCUMENT_BYTES Largest accepted document. [ingest.max_document_bytes; env YSEARCH_INGEST_MAX_DOCUMENT_BYTES] (positive)
--ingest-seal-age duration 30s ingest.seal_age YSEARCH_INGEST_SEAL_AGE Age at which an open build seals. [ingest.seal_age; env YSEARCH_INGEST_SEAL_AGE] (positive)
--ingest-seal-bytes bytes 0 ingest.seal_bytes YSEARCH_INGEST_SEAL_BYTES Spooled bytes at which an open build seals; zero derives it as a quarter of builder.sort_memory_bytes. A build's postings are about twice its spooled bytes and the partitions get roughly half the sort budget, so a spool larger than that quarter cannot be sorted in memory and the build spills instead — which is the difference between one file per build and hundreds, and measured 2.5x on the build itself. [ingest.seal_bytes; env YSEARCH_INGEST_SEAL_BYTES] (in [0, 1TiB])
--ingest-seal-documents int 100000 ingest.seal_documents YSEARCH_INGEST_SEAL_DOCUMENTS Spooled documents at which an open build seals. [ingest.seal_documents; env YSEARCH_INGEST_SEAL_DOCUMENTS] (positive)
--log-format enum text log.format YSEARCH_LOG_FORMAT Log line format. [log.format; env YSEARCH_LOG_FORMAT] (one of text, json)
--log-level enum info log.level YSEARCH_LOG_LEVEL Minimum level written to stderr. [log.level; env YSEARCH_LOG_LEVEL] (one of error, warn, info, debug)
--object-backend enum fs object.backend YSEARCH_OBJECT_BACKEND Object store backend. [object.backend; env YSEARCH_OBJECT_BACKEND] (one of fs, s3)
--object-dir string object.dir YSEARCH_OBJECT_DIR Filesystem object-store directory; required for the fs backend. A relative value in a configuration file resolves against that file's directory. [object.dir; env YSEARCH_OBJECT_DIR]
--object-root string indexes object.root YSEARCH_OBJECT_ROOT Object key root. [object.root; env YSEARCH_OBJECT_ROOT]
--s3-anonymous bool false object.s3.anonymous YSEARCH_OBJECT_S3_ANONYMOUS Sign no requests at all, for a bucket that grants public reads. This is a deliberate choice, never a fallback: an unresolved credential chain already signs anonymously, and a reader that quietly degraded to unsigned requests would report a permissions problem as an empty index. [object.s3.anonymous; env YSEARCH_OBJECT_S3_ANONYMOUS]
--s3-bucket string object.s3.bucket YSEARCH_OBJECT_S3_BUCKET S3 bucket; required for the s3 backend. [object.s3.bucket; env YSEARCH_OBJECT_S3_BUCKET]
--s3-endpoint string object.s3.endpoint YSEARCH_OBJECT_S3_ENDPOINT S3 endpoint host:port; required for the s3 backend. [object.s3.endpoint; env YSEARCH_OBJECT_S3_ENDPOINT]
--s3-path-style bool false object.s3.path_style YSEARCH_OBJECT_S3_PATH_STYLE Use path-style S3 addressing. [object.s3.path_style; env YSEARCH_OBJECT_S3_PATH_STYLE]
--s3-prefix string object.s3.prefix YSEARCH_OBJECT_S3_PREFIX S3 key prefix. [object.s3.prefix; env YSEARCH_OBJECT_S3_PREFIX]
--s3-region string object.s3.region YSEARCH_OBJECT_S3_REGION S3 region. [object.s3.region; env YSEARCH_OBJECT_S3_REGION]
--s3-use-ssl bool true object.s3.use_ssl YSEARCH_OBJECT_S3_USE_SSL Use TLS to the S3 endpoint. [object.s3.use_ssl; env YSEARCH_OBJECT_S3_USE_SSL]
--source string object.source YSEARCH_OBJECT_SOURCE Dataset URL to attach read-only, such as s3://bucket/root/ for a whole root or s3://bucket/root/indexes/name/ for one index. It derives object.backend, the bucket, prefix and root, the index when the URL names one, and server.read_only, each only where nothing more specific was set. Reading someone else's published index is the whole purpose, so attaching one implies read-only and cannot be talked out of it. [object.source; env YSEARCH_OBJECT_SOURCE]
--metrics-listen string :9550 observability.metrics_listen YSEARCH_OBSERVABILITY_METRICS_LISTEN host:port serving Prometheus exposition at /metrics; empty disables it. Unlike debug.pprof_listen this is on by default and binds every interface, because a metric carries no document text, no query, and no key material, and a scrape target that has to be switched on is one nobody switches on. Every series already carries the role label, so the scrape target only has to supply instance and pod. [observability.metrics_listen; env YSEARCH_OBSERVABILITY_METRICS_LISTEN]
--publisher-announce-interval duration 15s publisher.announce_interval YSEARCH_PUBLISHER_ANNOUNCE_INTERVAL Requested commit-marker publication cadence; the daemon floors it at follower.poll_interval and 15 seconds. [publisher.announce_interval; env YSEARCH_PUBLISHER_ANNOUNCE_INTERVAL] (in [1s, 1h0m0s])
--publisher-lease-ttl duration 30s publisher.lease_ttl YSEARCH_PUBLISHER_LEASE_TTL Startup-fixed object-store lease lifetime for the catalog publisher; renewal runs at one third of this value. [publisher.lease_ttl; env YSEARCH_PUBLISHER_LEASE_TTL] (in [5s, 10m0s])
--collapse-key-versions bool true query.collapse_key_versions YSEARCH_QUERY_COLLAPSE_KEY_VERSIONS Return one hit per document key, the copy with the greatest mutation version. A re-pushed document is stored as a new version rather than replacing the old one, so without this a key appears once per copy. Collapsing removes candidates after selection, so a request may return fewer than top_k. [query.collapse_key_versions; env YSEARCH_QUERY_COLLAPSE_KEY_VERSIONS]
--query-compiled-cache bool false query.compiled_cache YSEARCH_QUERY_COMPILED_CACHE Experimental leased compiled-query cache within the shared metadata allowance. Also reuses bounded dictionary plans for eager generation-owned readers; lazy reader plans stay request-owned. Cache pressure falls back to request-owned compilation or preparation. [query.compiled_cache; env YSEARCH_QUERY_COMPILED_CACHE]
--query-default-top-k int 10 query.default_top_k YSEARCH_QUERY_DEFAULT_TOP_K top_k when a request names none. [query.default_top_k; env YSEARCH_QUERY_DEFAULT_TOP_K] (in [1, 10000])
--query-defer-public-ids bool true query.defer_public_ids YSEARCH_QUERY_DEFER_PUBLIC_IDS Resolve a segment survivor's public ID only when the global merge returns it or must break a score tie with it, instead of for every survivor of every segment. Results are identical either way; false restores eager resolution as a same-code control. [query.defer_public_ids; env YSEARCH_QUERY_DEFER_PUBLIC_IDS]
--query-lexical-memory-bytes bytes 384MiB query.lexical_memory_bytes YSEARCH_QUERY_LEXICAL_MEMORY_BYTES Process-wide lexical request, candidate, queue and merge memory admission budget. Storage blocks and ordinal pages have separate cache budgets; busy requests fail with resource exhaustion. Sized to admit one request merging 64 segments at the product's 100,000-result top_k ceiling (lexical.MaxResultLimit; estimateLexicalFanoutMemory(64, 100000) = 182,632,832 bytes) with room for a second concurrent one that size. [query.lexical_memory_bytes; env YSEARCH_QUERY_LEXICAL_MEMORY_BYTES] (positive)
--query-lexical-strategy enum block-max query.lexical_strategy YSEARCH_QUERY_LEXICAL_STRATEGY Experimental exact lexical traversal selection; unsupported query shapes conservatively fall back to block-max. [query.lexical_strategy; env YSEARCH_QUERY_LEXICAL_STRATEGY] (one of block-max, auto, single-term-block-max, conjunction, block-max-maxscore)
--query-logical-bound-entries int 0 query.logical_bound_entries YSEARCH_QUERY_LOGICAL_BOUND_ENTRIES Experimental score-bound subblock size within decoded postings; zero uses physical blocks. Does not reduce physical read size. [query.logical_bound_entries; env YSEARCH_QUERY_LOGICAL_BOUND_ENTRIES] (in [0, 256])
--query-max-ast-depth int 32 query.max_ast_depth YSEARCH_QUERY_MAX_AST_DEPTH Deepest query nesting either grammar accepts. [query.max_ast_depth; env YSEARCH_QUERY_MAX_AST_DEPTH] (in [1, 1024])
--query-max-results int 0 query.max_results YSEARCH_QUERY_MAX_RESULTS Ceiling on maximum_results per request; zero keeps the engine default. [query.max_results; env YSEARCH_QUERY_MAX_RESULTS]
--query-ordinal-mode enum pages query.ordinal_mode YSEARCH_QUERY_ORDINAL_MODE Experimental ID residency policy under the shared ordinal budget. Whole tables are capped at 8 MiB and fall back to pages; adaptive promotion requires repeated broad page coverage. [query.ordinal_mode; env YSEARCH_QUERY_ORDINAL_MODE] (one of pages, whole, adaptive)
--query-packed-simd bool false query.packed_simd YSEARCH_QUERY_PACKED_SIMD Experimental native packed-posting decoding on supported CPUs; preserves scalar fallback and semantic validation. [query.packed_simd; env YSEARCH_QUERY_PACKED_SIMD]
--query-phrase-gap-expansion-limit int 64 query.phrase_gap_expansion_limit YSEARCH_QUERY_PHRASE_GAP_EXPANSION_LIMIT Most exact gap variants a {m,n} quantifier may expand to. [query.phrase_gap_expansion_limit; env YSEARCH_QUERY_PHRASE_GAP_EXPANSION_LIMIT] (in [1, 4096])
--query-regex-max-expansions int 256 query.regex_max_expansions YSEARCH_QUERY_REGEX_MAX_EXPANSIONS Most dictionary terms a regex or prefix may expand to before the query is refused. [query.regex_max_expansions; env YSEARCH_QUERY_REGEX_MAX_EXPANSIONS] (in [1, 65536])
--query-scoring-profile enum bm25f-v1 query.scoring_profile YSEARCH_QUERY_SCORING_PROFILE Scoring profile for a request that names none. bm25f-v1 scores each segment with its own statistics; bm25f-pinned-v1 (spec/50 §2) scores every segment under the generation's pinned statistics epoch, so scores do not depend on segment layout. It needs stats.dir; without it a pinned request fails with a typed error, never with segment statistics. [query.scoring_profile; env YSEARCH_QUERY_SCORING_PROFILE] (one of bm25f-v1, bm25f-pinned-v1)
--query-set-simd bool false query.set_simd YSEARCH_QUERY_SET_SIMD Experimental native intersection for balanced sparse filter arrays; skewed arrays retain scalar galloping. [query.set_simd; env YSEARCH_QUERY_SET_SIMD]
--routing-presence-bytes bytes 0 routing.presence_bytes YSEARCH_ROUTING_PRESENCE_BYTES Memory, not disk: resident budget for per-segment term-presence filters, used only when the generation has no routing table. A generation that has one prunes through it instead and holds no filters at all, whatever this says, because the table is read through a small page cache rather than held. Zero means none under cache.lazy_readers and unbounded without it, which at 294 segments was 1.4GiB of resident filters. That promotion for eager readers stays on for now: builder.routing_fragment_bytes and compaction.routing_fragment_bytes only default on from 2026-09-23, nothing rewrites a segment built before that, and no existing corpus has been rebuilt or fully compacted since - so a generation with an unfragmented member still depends on this filter to prune anything, including the published demo image, which does not set this flag. Filters are loaded in catalog order until the budget is reached; segments without one stay unknown and are still executed, so results never change. Budget against the decoded size, not the transfer: 680MiB of admitted filters measured 1.4GiB resident, about 2.1 times what this charges them. [routing.presence_bytes; env YSEARCH_ROUTING_PRESENCE_BYTES] (in [0, 8GiB])
--data-dir string server.data_dir YSEARCH_SERVER_DATA_DIR Developer shortcut: derives object.backend=fs, object.dir, cache.dir, ingest.dir, and config.file beneath one directory where nothing more specific is set. A relative value in a configuration file resolves against that file's directory. [server.data_dir; env YSEARCH_SERVER_DATA_DIR]
--grace-period duration 5s server.grace_period YSEARCH_SERVER_GRACE_PERIOD Graceful shutdown period. [server.grace_period; env YSEARCH_SERVER_GRACE_PERIOD] (positive)
--listen string 127.0.0.1:9500 server.listen YSEARCH_SERVER_LISTEN gRPC listen host:port serve and node bind; the default is the address the client verbs dial (client.server). [server.listen; env YSEARCH_SERVER_LISTEN]
--max-receive-bytes bytes 4MiB server.max_receive_bytes YSEARCH_SERVER_MAX_RECEIVE_BYTES Maximum gRPC request bytes (at least 64KiB). [server.max_receive_bytes; env YSEARCH_SERVER_MAX_RECEIVE_BYTES] (at least 64KiB)
--max-send-bytes bytes 4MiB server.max_send_bytes YSEARCH_SERVER_MAX_SEND_BYTES Maximum gRPC response bytes (at least 64KiB, one result frame). [server.max_send_bytes; env YSEARCH_SERVER_MAX_SEND_BYTES] (at least 64KiB)
--node-id string server.node_id YSEARCH_SERVER_NODE_ID Node identity; empty defaults to the bound listen address. [server.node_id; env YSEARCH_SERVER_NODE_ID]
--server-query-receive-memory-bytes bytes 320MiB server.query_receive_memory_bytes YSEARCH_SERVER_QUERY_RECEIVE_MEMORY_BYTES Shared request-memory allowance for Search, worker Execute, Merge, ValidateQuery, CacheState and FetchStored. Holds decoded requests until RPC completion. Receive workspace waits at most 5ms in a 64-call queue; decoded admission refuses immediately when full. Must fit three receive buffers (each at least 1MiB) plus decoded requests. Unary wire reception/decompression and HTTP/2 queues precede this admission. Sized for at least 24 concurrent full server.max_receive_bytes receives at its default (24 * 13,238,272 bytes = 303MiB, plus headroom): fewer than that admits only a handful of concurrent requests before refusing the rest with ResourceExhausted. [server.query_receive_memory_bytes; env YSEARCH_SERVER_QUERY_RECEIVE_MEMORY_BYTES] (positive)
--read-only bool false server.read_only YSEARCH_SERVER_READ_ONLY Refuse every object-store write for this process's lifetime. Enforced at the single store every writer resolves through, so background work that never touches an RPC - garbage collection, and the GC dry run, which writes a proposal object before any delete decision - fails closed too. Startup-fixed on purpose: a running process cannot be talked into writing by a config reload. [server.read_only; env YSEARCH_SERVER_READ_ONLY]
--roles list server.roles YSEARCH_SERVER_ROLES Comma-separated roles for node: coordinator, worker, merger, aggregator, compactor, router, builder, publisher. [server.roles; env YSEARCH_SERVER_ROLES]
--server-source-management enum loopback server.source_management YSEARCH_SERVER_SOURCE_MANAGEMENT Which peers may call SourceService, the RPC that changes which datasets this process reads. loopback answers only a caller that reached this process without crossing a network, so binding server.listen to every interface still does not expose it. any serves it to anything that can reach the port: the service has no authentication of its own, so choose it only where something in front of the address authenticates. [server.source_management; env YSEARCH_SERVER_SOURCE_MANAGEMENT] (one of loopback, any)
--zone string default server.zone YSEARCH_SERVER_ZONE Placement zone this node advertises in. [server.zone; env YSEARCH_SERVER_ZONE]
--stats-dir string stats.dir YSEARCH_STATS_DIR Directory of this node's statistics databases, one per index (spec/50 §3.3). Every served generation is folded into it and its epoch pinned for bm25f-pinned-v1. Unlike cache.dir it is not disposable: a node that loses it refolds from the catalog. Empty disables epochs. A relative value in a configuration file resolves against that file's directory. [stats.dir; env YSEARCH_STATS_DIR]
--stats-history-generations int 6000 stats.history_generations YSEARCH_STATS_HISTORY_GENERATIONS Generations of per-generation statistics deltas kept to reconstruct an earlier epoch (spec/50 §3.3); about two days at one generation per 30 s. [stats.history_generations; env YSEARCH_STATS_HISTORY_GENERATIONS] (in [1, 16777216])
--hydrate-throughput-floor bytes 128MiB storage.hydrate_throughput_floor YSEARCH_STORAGE_HYDRATE_THROUGHPUT_FLOOR Lowest believable hydration throughput in bytes per second. [storage.hydrate_throughput_floor; env YSEARCH_STORAGE_HYDRATE_THROUGHPUT_FLOOR] (positive)
--hysteresis-denominator int 4 storage.hysteresis_denominator YSEARCH_STORAGE_HYSTERESIS_DENOMINATOR Hysteresis denominator; must be below the numerator. [storage.hysteresis_denominator; env YSEARCH_STORAGE_HYSTERESIS_DENOMINATOR] (in [1, 4294967295])
--hysteresis-numerator int 5 storage.hysteresis_numerator YSEARCH_STORAGE_HYSTERESIS_NUMERATOR Hysteresis numerator; must exceed the denominator. [storage.hysteresis_numerator; env YSEARCH_STORAGE_HYSTERESIS_NUMERATOR] (in [1, 4294967295])
--mode enum AUTO storage.mode YSEARCH_STORAGE_MODE Storage mode: HYDRATE_FULL, REMOTE_BLOCKS, or AUTO (AUTO is refused by the checkpoint-bound serve). [storage.mode; env YSEARCH_STORAGE_MODE] (one of HYDRATE_FULL, REMOTE_BLOCKS, AUTO)
--remote-latency-floor duration 2ms storage.remote_latency_floor YSEARCH_STORAGE_REMOTE_LATENCY_FLOOR Lowest believable remote request latency. [storage.remote_latency_floor; env YSEARCH_STORAGE_REMOTE_LATENCY_FLOOR] (positive)
--remote-throughput-floor bytes 64MiB storage.remote_throughput_floor YSEARCH_STORAGE_REMOTE_THROUGHPUT_FLOOR Lowest believable remote throughput in bytes per second. [storage.remote_throughput_floor; env YSEARCH_STORAGE_REMOTE_THROUGHPUT_FLOOR] (positive)
--scan-threshold-permille int 200 storage.scan_threshold_permille YSEARCH_STORAGE_SCAN_THRESHOLD_PERMILLE Scan fraction above which AUTO hydrates. [storage.scan_threshold_permille; env YSEARCH_STORAGE_SCAN_THRESHOLD_PERMILLE] (in [0, 1000])
--tail-impact-enabled bool false tail.impact_enabled YSEARCH_TAIL_IMPACT_ENABLED Persist score-banded approximate-tail data. Disabled avoids two additional records per unique term when exact tails are sufficient. [tail.impact_enabled; env YSEARCH_TAIL_IMPACT_ENABLED]
--tail-impact-maximum-bands int 64 tail.impact_maximum_bands YSEARCH_TAIL_IMPACT_MAXIMUM_BANDS Maximum score-impact bands persisted for one term. [tail.impact_maximum_bands; env YSEARCH_TAIL_IMPACT_MAXIMUM_BANDS] (in [1, 4096])
--tail-impact-target-documents int 4096 tail.impact_target_documents YSEARCH_TAIL_IMPACT_TARGET_DOCUMENTS Target postings per persisted impact run. [tail.impact_target_documents; env YSEARCH_TAIL_IMPACT_TARGET_DOCUMENTS] (in [1, 4294967295])
--tail-maximum-ordering-error float 0 tail.maximum_ordering_error YSEARCH_TAIL_MAXIMUM_ORDERING_ERROR Server ceiling on accepted approximate-tail score-ordering error; zero requires exact fallback. [tail.maximum_ordering_error; env YSEARCH_TAIL_MAXIMUM_ORDERING_ERROR] (in [0, 1.7976931348623157e+308])
--vector-bits-per-code int 8 vector.bits_per_code YSEARCH_VECTOR_BITS_PER_CODE Bits in each PQ subquantizer code. [vector.bits_per_code; env YSEARCH_VECTOR_BITS_PER_CODE] (in [1, 8])
--vector-build-memory-bytes bytes 0 vector.build_memory_bytes YSEARCH_VECTOR_BUILD_MEMORY_BYTES Per-build vector training and assignment memory; zero derives a bounded share of builder.sort_memory_bytes. [vector.build_memory_bytes; env YSEARCH_VECTOR_BUILD_MEMORY_BYTES] (at least 0)
--vector-centroids int 0 vector.centroids YSEARCH_VECTOR_CENTROIDS IVF coarse centroid count; zero derives it deterministically from population. [vector.centroids; env YSEARCH_VECTOR_CENTROIDS] (in [0, 4294967295])
--vector-flat-threshold int 10000 vector.flat_threshold YSEARCH_VECTOR_FLAT_THRESHOLD Per-segment vector population at or below which candidate generation stays exhaustive. [vector.flat_threshold; env YSEARCH_VECTOR_FLAT_THRESHOLD] (in [1, 4294967295])
--vector-list-block-documents int 0 vector.list_block_documents YSEARCH_VECTOR_LIST_BLOCK_DOCUMENTS Target vector entries per IVF-list block; zero follows the document-value block geometry. [vector.list_block_documents; env YSEARCH_VECTOR_LIST_BLOCK_DOCUMENTS] (in [0, 4294967295])
--vector-maximum-training-vectors int 4096 vector.maximum_training_vectors YSEARCH_VECTOR_MAXIMUM_TRAINING_VECTORS Maximum deterministic training samples retained per vector field. [vector.maximum_training_vectors; env YSEARCH_VECTOR_MAXIMUM_TRAINING_VECTORS] (in [1, 4294967295])
--vector-query-candidate-multiplier int 10 vector.query_candidate_multiplier YSEARCH_VECTOR_QUERY_CANDIDATE_MULTIPLIER Default ANN candidates as a multiple of requested top_k. [vector.query_candidate_multiplier; env YSEARCH_VECTOR_QUERY_CANDIDATE_MULTIPLIER] (in [1, 10000])
--vector-query-default-probes int 8 vector.query_default_probes YSEARCH_VECTOR_QUERY_DEFAULT_PROBES IVF coarse lists probed when a vector request does not specify probes. [vector.query_default_probes; env YSEARCH_VECTOR_QUERY_DEFAULT_PROBES] (in [1, 4294967295])
--vector-query-maximum-candidates int 100000 vector.query_maximum_candidates YSEARCH_VECTOR_QUERY_MAXIMUM_CANDIDATES Hard per-request candidate generation and exact-rerank cap. [vector.query_maximum_candidates; env YSEARCH_VECTOR_QUERY_MAXIMUM_CANDIDATES] (in [1, 4294967295])
--vector-query-maximum-probes int 1024 vector.query_maximum_probes YSEARCH_VECTOR_QUERY_MAXIMUM_PROBES Hard per-request IVF probe cap. [vector.query_maximum_probes; env YSEARCH_VECTOR_QUERY_MAXIMUM_PROBES] (in [1, 4294967295])
--vector-query-memory-bytes bytes 256MiB vector.query_memory_bytes YSEARCH_VECTOR_QUERY_MEMORY_BYTES Per-process admission budget for decoded vector-query working sets. [vector.query_memory_bytes; env YSEARCH_VECTOR_QUERY_MEMORY_BYTES] (positive)
--vector-subquantizers int 0 vector.subquantizers YSEARCH_VECTOR_SUBQUANTIZERS PQ subquantizer count; zero derives a geometry compatible with each vector field's dimension. [vector.subquantizers; env YSEARCH_VECTOR_SUBQUANTIZERS] (in [0, 4294967295])
--vector-training-iterations int 20 vector.training_iterations YSEARCH_VECTOR_TRAINING_ITERATIONS Maximum deterministic k-means iterations during IVF-PQ training. [vector.training_iterations; env YSEARCH_VECTOR_TRAINING_ITERATIONS] (in [1, 1000])

ysearch push

ysearch push <index> <files...|-> [flags]

push reads JSONL, a JSON array, or a single JSON object (auto-detected), validates each document against the index's schema, converts it to a typed document, and streams the batches to the server. Unless --no-flush it flushes and waits for the segment to publish, printing the sealed segment and generation.

A file holding one pretty-printed object is one document; a file of one-line objects is one per line. To load compressed files or archives without uncompressing them first, use the admin console's Ingest section.

Flags:

Flag Type Default Key Env Description
--batch int 500 documents per batch
--batch-bytes int64 2097152 byte cap per batch
--format string auto input format: auto, jsonl, or json
--no-flush bool false accept ephemerally; do not flush or wait for publication
--require-ack bool false wait for PUBLISHED on every batch

ysearch routing

ysearch routing [flags]

Build optional query routing artifacts from existing segments (offline)

Subcommands:

ysearch routing build

ysearch routing build [flags]

Build and publish exact presence routing for a catalog generation

Flags:

Flag Type Default Key Env Description
--generation string index.generation YSEARCH_INDEX_GENERATION Exact generation to pin, or empty for the latest. [index.generation; env YSEARCH_INDEX_GENERATION]
--index string index.name YSEARCH_INDEX_NAME Index name for catalog-bound serve, worker, coordinator, compactor, router, builder, and publisher roles. [index.name; env YSEARCH_INDEX_NAME]
--legacy-value-encoding bool false publish the pre-compaction format, whose every presence value is a full 4096-slot bitmap, for a reader too old to open the compact one
--object-backend enum fs object.backend YSEARCH_OBJECT_BACKEND Object store backend. [object.backend; env YSEARCH_OBJECT_BACKEND] (one of fs, s3)
--object-dir string object.dir YSEARCH_OBJECT_DIR Filesystem object-store directory; required for the fs backend. A relative value in a configuration file resolves against that file's directory. [object.dir; env YSEARCH_OBJECT_DIR]
--object-root string indexes object.root YSEARCH_OBJECT_ROOT Object key root. [object.root; env YSEARCH_OBJECT_ROOT]
--routing-memory-bytes int64 67108864 budget for routing buffers and shared source caches
--routing-temp-bytes int64 4294967296 maximum bytes of routing spill and page files
--s3-anonymous bool false object.s3.anonymous YSEARCH_OBJECT_S3_ANONYMOUS Sign no requests at all, for a bucket that grants public reads. This is a deliberate choice, never a fallback: an unresolved credential chain already signs anonymously, and a reader that quietly degraded to unsigned requests would report a permissions problem as an empty index. [object.s3.anonymous; env YSEARCH_OBJECT_S3_ANONYMOUS]
--s3-bucket string object.s3.bucket YSEARCH_OBJECT_S3_BUCKET S3 bucket; required for the s3 backend. [object.s3.bucket; env YSEARCH_OBJECT_S3_BUCKET]
--s3-endpoint string object.s3.endpoint YSEARCH_OBJECT_S3_ENDPOINT S3 endpoint host:port; required for the s3 backend. [object.s3.endpoint; env YSEARCH_OBJECT_S3_ENDPOINT]
--s3-path-style bool false object.s3.path_style YSEARCH_OBJECT_S3_PATH_STYLE Use path-style S3 addressing. [object.s3.path_style; env YSEARCH_OBJECT_S3_PATH_STYLE]
--s3-prefix string object.s3.prefix YSEARCH_OBJECT_S3_PREFIX S3 key prefix. [object.s3.prefix; env YSEARCH_OBJECT_S3_PREFIX]
--s3-region string object.s3.region YSEARCH_OBJECT_S3_REGION S3 region. [object.s3.region; env YSEARCH_OBJECT_S3_REGION]
--s3-use-ssl bool true object.s3.use_ssl YSEARCH_OBJECT_S3_USE_SSL Use TLS to the S3 endpoint. [object.s3.use_ssl; env YSEARCH_OBJECT_S3_USE_SSL]
--score-bounds bool false include query-dependent scoring statistics in a separate descriptor
--scratch string parent directory for temporary routing files
--source string object.source YSEARCH_OBJECT_SOURCE Dataset URL to attach read-only, such as s3://bucket/root/ for a whole root or s3://bucket/root/indexes/name/ for one index. It derives object.backend, the bucket, prefix and root, the index when the URL names one, and server.read_only, each only where nothing more specific was set. Reading someone else's published index is the whole purpose, so attaching one implies read-only and cannot be talked out of it. [object.source; env YSEARCH_OBJECT_SOURCE]

ysearch schema

ysearch schema [flags]

schema talks to the IndexService of the server --server names. apply sends a .proto, a descriptor set, or an authored schema; changes are additive and auto-versioned, so re-applying an unchanged schema is a no-op and adding a field mints the next version. show prints one version, diff previews a change without writing it, history lists every version, and options prints the annotation .proto a document schema imports.

Subcommands:

ysearch schema apply

ysearch schema apply [file] [flags]

apply sends a schema to the server's IndexService: a .proto compiled in-process, a descriptor set (.binpb), or an authored schema (.yaml, .json, .textproto), named positionally or with --proto/--schema. The index is the one the .proto's (ysearch.v1.document) option declares, or --index. An index that does not exist is created at version 1; re-applying an unchanged schema is a no-op; an additive change mints the next version; anything else is refused with the diff.

Example:

ysearch schema apply articles.proto
ysearch schema apply --proto articles.proto --index articles --dry-run
ysearch schema apply --schema articles.yaml --index articles

Flags:

Flag Type Default Key Env Description
--dry-run bool false show the diff and the would-be version without writing
--index string the index to apply to
--message string the document message when the file declares more than one
--proto string a .proto file compiled in-process
--schema string a descriptor set (.binpb) or authored schema (.yaml/.json/.textproto)

ysearch schema diff

ysearch schema diff [file] [flags]

diff renders the change apply would make - the same table the server puts in a refusal - without writing anything, and exits 1 when the change is not additive. The file and index are named as for apply.

Example:

ysearch schema diff articles.proto
ysearch schema diff --schema articles.yaml --index articles

Flags:

Flag Type Default Key Env Description
--index string the index to diff against
--message string the document message when the file declares more than one
--proto string a .proto file compiled in-process
--schema string a descriptor set (.binpb) or authored schema

ysearch schema history

ysearch schema history [flags]

List every version of an index's schema

Flags:

Flag Type Default Key Env Description
--index string the index whose history to list

ysearch schema options

ysearch schema options [flags]

Print the annotation .proto a document schema imports

ysearch schema propose

ysearch schema propose <directory> [flags]

Recursively analyze JSON objects, JSON arrays, JSONL, XML, and UTF-8 text without connecting to a server. Save the schema and normalized JSONL, then print commands to create the index and ingest the converted documents. Review the proposal before applying it. Existing output files are never overwritten. Limits: 8 MiB per file, 32 MiB total input, 10000 documents. Normalized output is capped at 64 MiB total and less than 16 MiB per record. Unsupported file extensions and symlinks are skipped. Identical document content shares a generated key.

Flags:

Flag Type Default Key Env Description
--index string index namespace to propose
--output string new schema .json file (also writes FILE.documents.jsonl)

ysearch schema show

ysearch schema show [flags]

Print one version of an index's schema

Flags:

Flag Type Default Key Env Description
--index string the index to show
--version uint32 0 a specific schema version (default: latest) 
ysearch search <index> <query> [flags]

search parses a Lucene or CQP query, lowers it to the structured request the server executes, and prints the hits — collapsed by key by default, or every hit in rank order with --no-collapse. --keys and --ids are line-oriented streaming modes for large result sets: --keys projects only the schema key, while --ids requests no stored documents or scores. Lowering happens client-side, so a typo lands a caret before any round trip. --request reads a protojson SearchRequest instead of a query, with output flags still overriding its result shape.

Flags:

Flag Type Default Key Env Description
--candidates uint64 0 ANN candidates to exact-rerank (zero uses the server default)
--dialect string lucene query dialect: lucene or cqp
--exhaustive bool false scan every covered vector instead of IVF-PQ candidate generation
--fields string comma-separated projected fields, or * for all stored
--filter stringArray [] an equality filter field=value (repeatable)
--fusion string weighted hybrid fusion: weighted or rrf
--ids bool false stream 128-bit public document IDs as lowercase hex, one per line
--keys bool false stream logical document keys, one per line
--lexical-weight float64 1 lexical score weight for hybrid search; an explicit 0 scores every candidate 0
--maximum-ordering-error float64 0 maximum accepted score inversion in a banded tail
--no-collapse bool false print every hit in rank order without collapsing by key
--probes uint32 0 IVF lists to probe (zero uses the server default)
--request string read a protojson SearchRequest from FILE instead of a query
--score-ranges bool false include conservative score ranges for an approximate tail
--tail string exact tail ordering: exact or banded
--top-k int 0 number of results (default 10)
--vector-field string vector field used for vector or hybrid search
--vector-file string read a JSON float array from FILE (or - for stdin)
--vector-text string embed this text with the selected vector field's schema profile
--vector-weight float64 1 vector score weight; an explicit 0 drops the vector contribution

ysearch segment

ysearch segment [flags]

Build and publish immutable segments (offline)

Subcommands:

ysearch segment build

ysearch segment build [flags]

build analyzes documents (one JSON object per line), sorts and merges the postings within the builder limits, writes the segment beneath --work-dir, and checkpoints it under --checkpoint in the fs object directory, which it creates. The receipt on stdout carries the segment_id that publish requires. The builder limits are the builder.* settings: flags here, or a file, or the environment.

Accepted input (the M1 JSON forms, still accepted here and by push):

A schema file (--schema) is one JSON object:

{"version":1,"fields":[
  {"id":1,"name":"title","indexed":true,"weight":2,"b":0.6},
  {"id":2,"name":"body","indexed":true,"weight":1,"b":0.75}
]}

An input file (--input) is one JSON document per line; the members of "fields" are the schema's field names:

{"id":{"high":1,"low":1},"fields":{"title":"First","body":"alpha beta"}}
{"id":{"high":1,"low":2},"fields":{"title":"Second","body":"alpha gamma"}}

A schema authored from a .proto (the schema apply <file.proto> form) and a document keyed by a string field are the modern forms; see the Getting started section of the README and schema apply --help.

Example:

ysearch segment build --schema schema.json --input docs.jsonl --work-dir ./work --object-dir ./objects --checkpoint segments/demo

Flags:

Flag Type Default Key Env Description
--schema string schema: .proto, .binpb, .yaml, .json (M1 or IndexSchema), or .textproto
--input string document JSONL path
--work-dir string build and local segment directory
--checkpoint string checkpoint key prefix
--message string document message when the .proto declares several
--node-id string mint a mutation_version for documents without one, tagged with this node identity; empty leaves them absent
--max-line-bytes int 16777216 maximum JSONL line size
--analysis-batch-documents int 0 builder.analysis_batch_documents YSEARCH_BUILDER_ANALYSIS_BATCH_DOCUMENTS Documents handed to the analysis workers at once; zero derives sixteen per worker. A batch is the unit that must fit in builder.analysis_memory_bytes, so a larger batch amortises the hand-off over more documents but raises the peak the analysis share has to cover. The derived shape is what the resource gates measure, so a corpus of unusually large documents is the case for lowering it rather than raising the share. [builder.analysis_batch_documents; env YSEARCH_BUILDER_ANALYSIS_BATCH_DOCUMENTS] (in [0, 1048576])
--analysis-memory-bytes bytes 0 builder.analysis_memory_bytes YSEARCH_BUILDER_ANALYSIS_MEMORY_BYTES The share of builder.sort_memory_bytes reserved for documents in flight through analysis; zero derives a thirty-second of it, clamped to between 64KiB and 32MiB and never more than a quarter of what the public-id and docvalue spools leave. It only has to hold one bounded batch, and a document larger than the whole share still runs alone under builder.max_document_working_bytes, so the derived value is deliberately small: a three-million-document run peaked at 9.5MiB against a 128MiB share. Every byte reserved here is a byte the term partitions cannot use, which raises mini-run count and write amplification directly, so raise it only against a measured analysis stall. [builder.analysis_memory_bytes; env YSEARCH_BUILDER_ANALYSIS_MEMORY_BYTES]
--analysis-workers int 0 builder.analysis_workers YSEARCH_BUILDER_ANALYSIS_WORKERS Goroutines tokenising documents inside one build; zero derives GOMAXPROCS, bounded at eight. Analysis is the one parallel stage of an otherwise serial build, so this is per build and multiplies with builder.build_concurrency: the bound exists because a dozen builds each spawning a worker per core oversubscribes the machine and the scheduler churn costs more than the parallelism returns. Output order is preserved regardless of this value. [builder.analysis_workers; env YSEARCH_BUILDER_ANALYSIS_WORKERS] (in [0, 1024])
--blob-dictionary bool true builder.blob_dictionary YSEARCH_BUILDER_BLOB_DICTIONARY Compress stored values against a dictionary trained per blob file. Denser on homogeneous documents, but the dictionary is coded on the build's hottest path, so turning it off trades index size for ingest throughput. [builder.blob_dictionary; env YSEARCH_BUILDER_BLOB_DICTIONARY]
--blob-zstd-level int 0 builder.blob_zstd_level YSEARCH_BUILDER_BLOB_ZSTD_LEVEL Zstd effort for stored values; zero keeps the storage default (7). Levels select different zstd encoders, and a build re-seeds the chosen encoder from the blob dictionary for every value it writes, so a lower level cuts build cost by more than the usual level trade-off suggests. [builder.blob_zstd_level; env YSEARCH_BUILDER_BLOB_ZSTD_LEVEL] (in [0, 11])
--build-concurrency int 0 builder.build_concurrency YSEARCH_BUILDER_BUILD_CONCURRENCY Sealed builds that may run through the seal pipeline at once; zero derives one per CPU, bounded at twelve. One build is a mostly single-threaded chain of analysis, sort, merge and publish that spends about half its time blocked in file system calls, so a single pipeline leaves a multi-core host idle under bulk ingest. The builder's memory is this number times builder.sort_memory_bytes. [builder.build_concurrency; env YSEARCH_BUILDER_BUILD_CONCURRENCY] (in [0, 256])
--builder-catalog-publication enum direct builder.catalog_publication YSEARCH_BUILDER_CATALOG_PUBLICATION Catalog writer mode: direct is the single-node compatibility path; external stops after commit markers for a lease-fenced publisher role. [builder.catalog_publication; env YSEARCH_BUILDER_CATALOG_PUBLICATION] (one of direct, external)
--dedupe-index-identities int 0 builder.dedupe_index_identities YSEARCH_BUILDER_DEDUPE_INDEX_IDENTITIES Distinct document keys one open spool's dedupe index may hold before it gives up; zero derives 1048576. A key spooled twice in one build must resolve to a single winner before analysis, or the segment's dense ordinal assignment refuses the build outright. The index resolves that as records are spooled, from identities ingest already holds, and the sealed build inherits the answer; past this bound it gives up and the build reconstructs the answer by reading its spool twice instead — slower, and exactly as correct. One entry is a 16-byte key hash, a sequence and a mutation version, so the default bounds it at tens of megabytes per open spool and is reached only by a spool of a million very small documents. Lower it to cap that memory on a host running many indexes; set it to one to force the scanning path. [builder.dedupe_index_identities; env YSEARCH_BUILDER_DEDUPE_INDEX_IDENTITIES] (in [0, 268435456])
--builder-fuse-memory-bytes bytes 0 builder.fuse_memory_bytes YSEARCH_BUILDER_FUSE_MEMORY_BYTES Optional binary fuse construction workspace during publication; zero retains Bloom, failed admission retains Bloom. [builder.fuse_memory_bytes; env YSEARCH_BUILDER_FUSE_MEMORY_BYTES] (in [0, 64MiB])
--lexical-blob-threshold bytes 0 builder.lexical_blob_threshold YSEARCH_BUILDER_LEXICAL_BLOB_THRESHOLD Size at which a posting, position or term value is stored in a blob file rather than inline in the key log; zero keeps the default of 64KiB. Separation exists to spare compaction from rewriting large values, and a segment is never compacted — while a blob value is compressed on its own, which with a trained dictionary costs an encoder reset each time, where key-log blocks are compressed in bulk. [builder.lexical_blob_threshold; env YSEARCH_BUILDER_LEXICAL_BLOB_THRESHOLD]
--lexical-block-compression string builder.lexical_block_compression YSEARCH_BUILDER_LEXICAL_BLOCK_COMPRESSION Codec the term, posting and position families compress key-log blocks with: none, snappy, lz4, lz4fast or zstd. Empty keeps the default of lz4. The storage engine's own default leaves the newest level raw because it expects compaction to rewrite it, and a segment stays where it lands. On 300k articles lz4 measured 24.8s and 1.0GB against zstd's 37.7s and 0.8GB, so zstd is the choice for an index written once and read for a long time. [builder.lexical_block_compression; env YSEARCH_BUILDER_LEXICAL_BLOCK_COMPRESSION]
--max-document-working-bytes bytes 0 builder.max_document_working_bytes YSEARCH_BUILDER_MAX_DOCUMENT_WORKING_BYTES What one document may use beyond the partition budget while it is being analyzed; zero derives 64MiB, or a quarter of builder.sort_memory_bytes when that is smaller. It is an allowance rather than a reservation: it exists so a single document larger than the whole partition budget still builds, alone, instead of failing the build. Only a corpus with documents in the hundreds of megabytes needs it raised. [builder.max_document_working_bytes; env YSEARCH_BUILDER_MAX_DOCUMENT_WORKING_BYTES]
--max-queued-builds int 0 builder.max_queued_builds YSEARCH_BUILDER_MAX_QUEUED_BUILDS Sealed builds that may wait for or run in the pipeline before ingest is throttled; zero derives it as four times builder.build_concurrency. A waiting build is a spool on disk, not memory, so this buys ingest room across a build at the cost of ingest.seal_bytes of disk per queued build. Set too low, a bulk load throttles the moment the pipeline is full and ingest runs at the speed of the builds rather than ahead of them. [builder.max_queued_builds; env YSEARCH_BUILDER_MAX_QUEUED_BUILDS] (in [0, 1024])
--max-token-bytes bytes 64KiB builder.max_token_bytes YSEARCH_BUILDER_MAX_TOKEN_BYTES Maximum analyzed token size. A build must be able to admit one token this large, so raising it raises the floor under builder.sort_memory_bytes: a budget that leaves the partitions less than one maximum term refuses to start rather than fail partway. Tokens are words after analysis, so the default is already far past any natural language; raise it only for a corpus with genuinely enormous unbroken tokens. [builder.max_token_bytes; env YSEARCH_BUILDER_MAX_TOKEN_BYTES] (positive)
--merge-fan-in int 0 builder.merge_fan_in YSEARCH_BUILDER_MERGE_FAN_IN Spilled mini-runs merged in one pass; zero derives as many as an eighth of the partition budget affords at builder.run_page_bytes plus 512 bytes of reader overhead each, capped at sixty-four and floored at two. Runs beyond the fan-in need further passes, and each pass rewrites the data, so a low fan-in on a build that spilled heavily shows up as write amplification rather than as a slow merge. The cap is a file-descriptor and seek-pattern bound, not a memory one. [builder.merge_fan_in; env YSEARCH_BUILDER_MERGE_FAN_IN] (in [0, 64])
--builder-output-format-version int 0 builder.output_format_version YSEARCH_BUILDER_OUTPUT_FORMAT_VERSION Segment format emitted by builders and compactors; zero selects this binary's current format. The Kubernetes operator pins this to compatibility.writeFormat on writer roles during staged rollouts. Format 7 adds the stats, forward and hashterms families (spec/40 §3); a compaction that includes an older input writes format 6 instead, unless 7 or 8 is set explicitly, which refuses such a merge. Format 8 adds the exact vector lanes (canonical fp16 vectors, IVF radii, int8 codes; spec/40 §3.2). [builder.output_format_version; env YSEARCH_BUILDER_OUTPUT_FORMAT_VERSION] (in [0, 8])
--builder-paged-posting-directory bool false builder.paged_posting_directory YSEARCH_BUILDER_PAGED_POSTING_DIRECTORY Experimental commit-anchored posting-directory pages under query admission and the existing persistent block quota; old commits retain canonical traversal. [builder.paged_posting_directory; env YSEARCH_BUILDER_PAGED_POSTING_DIRECTORY]
--builder-posting-partition-penalty float 4 builder.posting_partition_penalty YSEARCH_BUILDER_POSTING_PARTITION_PENALTY Metadata cost penalty for experimental variable posting blocks. [builder.posting_partition_penalty; env YSEARCH_BUILDER_POSTING_PARTITION_PENALTY] (in [0, 1e+06])
--publish-verification string checksum builder.publish_verification YSEARCH_BUILDER_PUBLISH_VERIFICATION How a published segment is confirmed: checksum, sample, or readback. checksum compares every object against the checksum the object store reported when it accepted the write — S3 computes that server-side and refuses a mismatched write, and the filesystem backend computes it over the bytes it synced — so nothing is transferred back. sample additionally opens the published copy through the store and queries it; readback additionally fetches every object and rehashes it, which costs the whole index again on every build. An object whose store reports no comparable checksum is read back in any mode. [builder.publish_verification; env YSEARCH_BUILDER_PUBLISH_VERIFICATION]
--builder-routing-fragment-bytes bytes 8MiB builder.routing_fragment_bytes YSEARCH_BUILDER_ROUTING_FRAGMENT_BYTES Optional routing fragment disk cap per built segment; zero disables, overflow keeps canonical scan fallback. Matches compaction.routing_fragment_bytes so a freshly ingested segment and a freshly compacted one carry a fragment the same way; the two stay separate keys because a background merge must be able to carry its own share independent of foreground ingestion. Only a segment built or compacted after this defaulted on carries a fragment: nothing rewrites an existing one, so a settled corpus stays without a routing table until it is backfilled or compacted. [builder.routing_fragment_bytes; env YSEARCH_BUILDER_ROUTING_FRAGMENT_BYTES] (in [0, 64MiB])
--run-page-bytes bytes 0 builder.run_page_bytes YSEARCH_BUILDER_RUN_PAGE_BYTES Read-ahead page one spilled mini-run is buffered in during the final merge; zero derives 64KiB, shrinking it when an eighth of the partition budget cannot hold sixty-four such pages. The merge reserves one page per open run for the whole build, so page size and builder.merge_fan_in trade against each other out of the same share: the derivation prefers smaller pages to a fan-in that would force extra compaction passes on a hot partition. [builder.run_page_bytes; env YSEARCH_BUILDER_RUN_PAGE_BYTES]
--sort-memory-bytes bytes 256MiB builder.sort_memory_bytes YSEARCH_BUILDER_SORT_MEMORY_BYTES Total accounted builder budget: one ledger every part of a build charges against, and the setting the other builder budgets derive from. It divides into the public-id sorter (an eighth, capped at 64MiB), the docvalue and lane spools (their own worst case), the analysis share (builder.analysis_memory_bytes), and the term partitions, which take what is left. A build that cannot hold its partitions in that remainder spills to disk instead, which is the difference between one file per build and hundreds. This is per build, so the builder's memory is builder.build_concurrency times this. [builder.sort_memory_bytes; env YSEARCH_BUILDER_SORT_MEMORY_BYTES] (positive)
--builder-spill-merge-strategy enum heap builder.spill_merge_strategy YSEARCH_BUILDER_SPILL_MERGE_STRATEGY Experimental spill merge selector; heap preserves the baseline until full ingestion benchmarks qualify an alternative. [builder.spill_merge_strategy; env YSEARCH_BUILDER_SPILL_MERGE_STRATEGY] (one of heap, replace-root, loser-tree)
--builder-startup-bundles bool false builder.startup_bundles YSEARCH_BUILDER_STARTUP_BUNDLES Asynchronously produce optional startup bundles after catalog publication. One bounded coalescing worker per publishing host; oversized sources retain canonical startup fallback. [builder.startup_bundles; env YSEARCH_BUILDER_STARTUP_BUNDLES]
--term-dictionary-block-size bytes 64KiB builder.term_dictionary_block_size YSEARCH_BUILDER_TERM_DICTIONARY_BLOCK_SIZE Key-log data-block size of the term family. A table carries one block-index entry per data block and reads the whole index before it can look up anything, so this is what a cold term probe pays to find out where to look. At the storage engine's 4KiB an 8MiB dictionary needs about two thousand entries — around 65KiB read on every open of the table and held in memory until it closes — where 64KiB needs about a sixteenth of that. A probe then reads a bigger block, which over an object store is the cheap side of the trade: measured on a 6MiB dictionary the block cost 17KiB compressed against the 48KiB of index it replaced. [builder.term_dictionary_block_size; env YSEARCH_BUILDER_TERM_DICTIONARY_BLOCK_SIZE] (positive)
--term-dictionary-bloom bool false builder.term_dictionary_bloom YSEARCH_BUILDER_TERM_DICTIONARY_BLOOM Write a Bloom filter over the term family. The storage engine writes one by default; a segment does not, because the filter is read whole — uncompressed, about 1.2 bytes per distinct term — when the table is opened, and kept in memory until it closes, to save at most one block read per probe against the single table a published segment's term family is. Measured, it was nine tenths of what a cold term probe read: 431KiB of 481KiB on a 360k-term dictionary. A table says in its own footer whether it has a filter, so segments published with one keep working unchanged. [builder.term_dictionary_bloom; env YSEARCH_BUILDER_TERM_DICTIONARY_BLOOM]
--builder-variable-posting-blocks bool false builder.variable_posting_blocks YSEARCH_BUILDER_VARIABLE_POSTING_BLOCKS Experimental bounded-window BM25F-aware posting partitioning for format5 or newer output. [builder.variable_posting_blocks; env YSEARCH_BUILDER_VARIABLE_POSTING_BLOCKS]
--index string index.name YSEARCH_INDEX_NAME Index name for catalog-bound serve, worker, coordinator, compactor, router, builder, and publisher roles. [index.name; env YSEARCH_INDEX_NAME]
--object-backend enum fs object.backend YSEARCH_OBJECT_BACKEND Object store backend. [object.backend; env YSEARCH_OBJECT_BACKEND] (one of fs, s3)
--object-dir string object.dir YSEARCH_OBJECT_DIR Filesystem object-store directory; required for the fs backend. A relative value in a configuration file resolves against that file's directory. [object.dir; env YSEARCH_OBJECT_DIR]
--object-root string indexes object.root YSEARCH_OBJECT_ROOT Object key root. [object.root; env YSEARCH_OBJECT_ROOT]
--s3-anonymous bool false object.s3.anonymous YSEARCH_OBJECT_S3_ANONYMOUS Sign no requests at all, for a bucket that grants public reads. This is a deliberate choice, never a fallback: an unresolved credential chain already signs anonymously, and a reader that quietly degraded to unsigned requests would report a permissions problem as an empty index. [object.s3.anonymous; env YSEARCH_OBJECT_S3_ANONYMOUS]
--s3-bucket string object.s3.bucket YSEARCH_OBJECT_S3_BUCKET S3 bucket; required for the s3 backend. [object.s3.bucket; env YSEARCH_OBJECT_S3_BUCKET]
--s3-endpoint string object.s3.endpoint YSEARCH_OBJECT_S3_ENDPOINT S3 endpoint host:port; required for the s3 backend. [object.s3.endpoint; env YSEARCH_OBJECT_S3_ENDPOINT]
--s3-path-style bool false object.s3.path_style YSEARCH_OBJECT_S3_PATH_STYLE Use path-style S3 addressing. [object.s3.path_style; env YSEARCH_OBJECT_S3_PATH_STYLE]
--s3-prefix string object.s3.prefix YSEARCH_OBJECT_S3_PREFIX S3 key prefix. [object.s3.prefix; env YSEARCH_OBJECT_S3_PREFIX]
--s3-region string object.s3.region YSEARCH_OBJECT_S3_REGION S3 region. [object.s3.region; env YSEARCH_OBJECT_S3_REGION]
--s3-use-ssl bool true object.s3.use_ssl YSEARCH_OBJECT_S3_USE_SSL Use TLS to the S3 endpoint. [object.s3.use_ssl; env YSEARCH_OBJECT_S3_USE_SSL]
--source string object.source YSEARCH_OBJECT_SOURCE Dataset URL to attach read-only, such as s3://bucket/root/ for a whole root or s3://bucket/root/indexes/name/ for one index. It derives object.backend, the bucket, prefix and root, the index when the URL names one, and server.read_only, each only where nothing more specific was set. Reading someone else's published index is the whole purpose, so attaching one implies read-only and cannot be talked out of it. [object.source; env YSEARCH_OBJECT_SOURCE]
--tail-impact-maximum-bands int 64 tail.impact_maximum_bands YSEARCH_TAIL_IMPACT_MAXIMUM_BANDS Maximum score-impact bands persisted for one term. [tail.impact_maximum_bands; env YSEARCH_TAIL_IMPACT_MAXIMUM_BANDS] (in [1, 4096])
--tail-impact-target-documents int 4096 tail.impact_target_documents YSEARCH_TAIL_IMPACT_TARGET_DOCUMENTS Target postings per persisted impact run. [tail.impact_target_documents; env YSEARCH_TAIL_IMPACT_TARGET_DOCUMENTS] (in [1, 4294967295])

ysearch segment publish

ysearch segment publish [flags]

publish uploads a locally built segment's objects, reads them back, writes the commit marker that makes them visible, and announces the segment to the catalog. Every step is idempotent, so the command can be run again after any failure. --segment-id must be the built segment's own identity from the build receipt.

Flags:

Flag Type Default Key Env Description
--segment-id string 32 lowercase hex characters
--segment-dir string locally built segment directory
--builder-version string builder identity recorded in the commit
--expected-parent string generation this announcement expects to extend
--announce bool true announce the segment to the catalog after publishing
--generation string index.generation YSEARCH_INDEX_GENERATION Exact generation to pin, or empty for the latest. [index.generation; env YSEARCH_INDEX_GENERATION]
--index string index.name YSEARCH_INDEX_NAME Index name for catalog-bound serve, worker, coordinator, compactor, router, builder, and publisher roles. [index.name; env YSEARCH_INDEX_NAME]
--object-backend enum fs object.backend YSEARCH_OBJECT_BACKEND Object store backend. [object.backend; env YSEARCH_OBJECT_BACKEND] (one of fs, s3)
--object-dir string object.dir YSEARCH_OBJECT_DIR Filesystem object-store directory; required for the fs backend. A relative value in a configuration file resolves against that file's directory. [object.dir; env YSEARCH_OBJECT_DIR]
--object-root string indexes object.root YSEARCH_OBJECT_ROOT Object key root. [object.root; env YSEARCH_OBJECT_ROOT]
--s3-anonymous bool false object.s3.anonymous YSEARCH_OBJECT_S3_ANONYMOUS Sign no requests at all, for a bucket that grants public reads. This is a deliberate choice, never a fallback: an unresolved credential chain already signs anonymously, and a reader that quietly degraded to unsigned requests would report a permissions problem as an empty index. [object.s3.anonymous; env YSEARCH_OBJECT_S3_ANONYMOUS]
--s3-bucket string object.s3.bucket YSEARCH_OBJECT_S3_BUCKET S3 bucket; required for the s3 backend. [object.s3.bucket; env YSEARCH_OBJECT_S3_BUCKET]
--s3-endpoint string object.s3.endpoint YSEARCH_OBJECT_S3_ENDPOINT S3 endpoint host:port; required for the s3 backend. [object.s3.endpoint; env YSEARCH_OBJECT_S3_ENDPOINT]
--s3-path-style bool false object.s3.path_style YSEARCH_OBJECT_S3_PATH_STYLE Use path-style S3 addressing. [object.s3.path_style; env YSEARCH_OBJECT_S3_PATH_STYLE]
--s3-prefix string object.s3.prefix YSEARCH_OBJECT_S3_PREFIX S3 key prefix. [object.s3.prefix; env YSEARCH_OBJECT_S3_PREFIX]
--s3-region string object.s3.region YSEARCH_OBJECT_S3_REGION S3 region. [object.s3.region; env YSEARCH_OBJECT_S3_REGION]
--s3-use-ssl bool true object.s3.use_ssl YSEARCH_OBJECT_S3_USE_SSL Use TLS to the S3 endpoint. [object.s3.use_ssl; env YSEARCH_OBJECT_S3_USE_SSL]
--source string object.source YSEARCH_OBJECT_SOURCE Dataset URL to attach read-only, such as s3://bucket/root/ for a whole root or s3://bucket/root/indexes/name/ for one index. It derives object.backend, the bucket, prefix and root, the index when the URL names one, and server.read_only, each only where nothing more specific was set. Reading someone else's published index is the whole purpose, so attaching one implies read-only and cannot be talked out of it. [object.source; env YSEARCH_OBJECT_SOURCE]

ysearch serve

ysearch serve [SOURCE_URL] [flags]

serve hosts every index under the object root - or exactly one with --index - and answers SearchService, IngestService, IndexService, CatalogService, AdminService, and ConfigService for them. An empty object root is a valid start: schema apply creates an index while serving, push fills it, and each index follows the catalog so a published generation is served without a restart. With --checkpoint it is instead the Milestone 1 form serving one checkpoint (SearchService and ConfigService only). Settings come from the catalog - defaults, then the discovered files, the environment, and the flags - and the listening= line on stdout says where it serves once every hosted index is resolved and the socket is bound.

Example:

ysearch serve --data-dir ./ys --listen 127.0.0.1:9500
ysearch serve --index demo --listen 127.0.0.1:9500 --cache-dir ./cache --ingest-dir ./ingest --object-dir ./objects
ysearch --config ys.yml serve
ysearch serve s3://search-data/production/indexes/articles/

Flags:

Flag Type Default Key Env Description
--analysis-batch-documents int 0 builder.analysis_batch_documents YSEARCH_BUILDER_ANALYSIS_BATCH_DOCUMENTS Documents handed to the analysis workers at once; zero derives sixteen per worker. A batch is the unit that must fit in builder.analysis_memory_bytes, so a larger batch amortises the hand-off over more documents but raises the peak the analysis share has to cover. The derived shape is what the resource gates measure, so a corpus of unusually large documents is the case for lowering it rather than raising the share. [builder.analysis_batch_documents; env YSEARCH_BUILDER_ANALYSIS_BATCH_DOCUMENTS] (in [0, 1048576])
--analysis-memory-bytes bytes 0 builder.analysis_memory_bytes YSEARCH_BUILDER_ANALYSIS_MEMORY_BYTES The share of builder.sort_memory_bytes reserved for documents in flight through analysis; zero derives a thirty-second of it, clamped to between 64KiB and 32MiB and never more than a quarter of what the public-id and docvalue spools leave. It only has to hold one bounded batch, and a document larger than the whole share still runs alone under builder.max_document_working_bytes, so the derived value is deliberately small: a three-million-document run peaked at 9.5MiB against a 128MiB share. Every byte reserved here is a byte the term partitions cannot use, which raises mini-run count and write amplification directly, so raise it only against a measured analysis stall. [builder.analysis_memory_bytes; env YSEARCH_BUILDER_ANALYSIS_MEMORY_BYTES]
--analysis-workers int 0 builder.analysis_workers YSEARCH_BUILDER_ANALYSIS_WORKERS Goroutines tokenising documents inside one build; zero derives GOMAXPROCS, bounded at eight. Analysis is the one parallel stage of an otherwise serial build, so this is per build and multiplies with builder.build_concurrency: the bound exists because a dozen builds each spawning a worker per core oversubscribes the machine and the scheduler churn costs more than the parallelism returns. Output order is preserved regardless of this value. [builder.analysis_workers; env YSEARCH_BUILDER_ANALYSIS_WORKERS] (in [0, 1024])
--blob-dictionary bool true builder.blob_dictionary YSEARCH_BUILDER_BLOB_DICTIONARY Compress stored values against a dictionary trained per blob file. Denser on homogeneous documents, but the dictionary is coded on the build's hottest path, so turning it off trades index size for ingest throughput. [builder.blob_dictionary; env YSEARCH_BUILDER_BLOB_DICTIONARY]
--blob-zstd-level int 0 builder.blob_zstd_level YSEARCH_BUILDER_BLOB_ZSTD_LEVEL Zstd effort for stored values; zero keeps the storage default (7). Levels select different zstd encoders, and a build re-seeds the chosen encoder from the blob dictionary for every value it writes, so a lower level cuts build cost by more than the usual level trade-off suggests. [builder.blob_zstd_level; env YSEARCH_BUILDER_BLOB_ZSTD_LEVEL] (in [0, 11])
--build-concurrency int 0 builder.build_concurrency YSEARCH_BUILDER_BUILD_CONCURRENCY Sealed builds that may run through the seal pipeline at once; zero derives one per CPU, bounded at twelve. One build is a mostly single-threaded chain of analysis, sort, merge and publish that spends about half its time blocked in file system calls, so a single pipeline leaves a multi-core host idle under bulk ingest. The builder's memory is this number times builder.sort_memory_bytes. [builder.build_concurrency; env YSEARCH_BUILDER_BUILD_CONCURRENCY] (in [0, 256])
--builder-catalog-publication enum direct builder.catalog_publication YSEARCH_BUILDER_CATALOG_PUBLICATION Catalog writer mode: direct is the single-node compatibility path; external stops after commit markers for a lease-fenced publisher role. [builder.catalog_publication; env YSEARCH_BUILDER_CATALOG_PUBLICATION] (one of direct, external)
--dedupe-index-identities int 0 builder.dedupe_index_identities YSEARCH_BUILDER_DEDUPE_INDEX_IDENTITIES Distinct document keys one open spool's dedupe index may hold before it gives up; zero derives 1048576. A key spooled twice in one build must resolve to a single winner before analysis, or the segment's dense ordinal assignment refuses the build outright. The index resolves that as records are spooled, from identities ingest already holds, and the sealed build inherits the answer; past this bound it gives up and the build reconstructs the answer by reading its spool twice instead — slower, and exactly as correct. One entry is a 16-byte key hash, a sequence and a mutation version, so the default bounds it at tens of megabytes per open spool and is reached only by a spool of a million very small documents. Lower it to cap that memory on a host running many indexes; set it to one to force the scanning path. [builder.dedupe_index_identities; env YSEARCH_BUILDER_DEDUPE_INDEX_IDENTITIES] (in [0, 268435456])
--builder-fuse-memory-bytes bytes 0 builder.fuse_memory_bytes YSEARCH_BUILDER_FUSE_MEMORY_BYTES Optional binary fuse construction workspace during publication; zero retains Bloom, failed admission retains Bloom. [builder.fuse_memory_bytes; env YSEARCH_BUILDER_FUSE_MEMORY_BYTES] (in [0, 64MiB])
--lexical-blob-threshold bytes 0 builder.lexical_blob_threshold YSEARCH_BUILDER_LEXICAL_BLOB_THRESHOLD Size at which a posting, position or term value is stored in a blob file rather than inline in the key log; zero keeps the default of 64KiB. Separation exists to spare compaction from rewriting large values, and a segment is never compacted — while a blob value is compressed on its own, which with a trained dictionary costs an encoder reset each time, where key-log blocks are compressed in bulk. [builder.lexical_blob_threshold; env YSEARCH_BUILDER_LEXICAL_BLOB_THRESHOLD]
--lexical-block-compression string builder.lexical_block_compression YSEARCH_BUILDER_LEXICAL_BLOCK_COMPRESSION Codec the term, posting and position families compress key-log blocks with: none, snappy, lz4, lz4fast or zstd. Empty keeps the default of lz4. The storage engine's own default leaves the newest level raw because it expects compaction to rewrite it, and a segment stays where it lands. On 300k articles lz4 measured 24.8s and 1.0GB against zstd's 37.7s and 0.8GB, so zstd is the choice for an index written once and read for a long time. [builder.lexical_block_compression; env YSEARCH_BUILDER_LEXICAL_BLOCK_COMPRESSION]
--max-document-working-bytes bytes 0 builder.max_document_working_bytes YSEARCH_BUILDER_MAX_DOCUMENT_WORKING_BYTES What one document may use beyond the partition budget while it is being analyzed; zero derives 64MiB, or a quarter of builder.sort_memory_bytes when that is smaller. It is an allowance rather than a reservation: it exists so a single document larger than the whole partition budget still builds, alone, instead of failing the build. Only a corpus with documents in the hundreds of megabytes needs it raised. [builder.max_document_working_bytes; env YSEARCH_BUILDER_MAX_DOCUMENT_WORKING_BYTES]
--max-queued-builds int 0 builder.max_queued_builds YSEARCH_BUILDER_MAX_QUEUED_BUILDS Sealed builds that may wait for or run in the pipeline before ingest is throttled; zero derives it as four times builder.build_concurrency. A waiting build is a spool on disk, not memory, so this buys ingest room across a build at the cost of ingest.seal_bytes of disk per queued build. Set too low, a bulk load throttles the moment the pipeline is full and ingest runs at the speed of the builds rather than ahead of them. [builder.max_queued_builds; env YSEARCH_BUILDER_MAX_QUEUED_BUILDS] (in [0, 1024])
--max-token-bytes bytes 64KiB builder.max_token_bytes YSEARCH_BUILDER_MAX_TOKEN_BYTES Maximum analyzed token size. A build must be able to admit one token this large, so raising it raises the floor under builder.sort_memory_bytes: a budget that leaves the partitions less than one maximum term refuses to start rather than fail partway. Tokens are words after analysis, so the default is already far past any natural language; raise it only for a corpus with genuinely enormous unbroken tokens. [builder.max_token_bytes; env YSEARCH_BUILDER_MAX_TOKEN_BYTES] (positive)
--merge-fan-in int 0 builder.merge_fan_in YSEARCH_BUILDER_MERGE_FAN_IN Spilled mini-runs merged in one pass; zero derives as many as an eighth of the partition budget affords at builder.run_page_bytes plus 512 bytes of reader overhead each, capped at sixty-four and floored at two. Runs beyond the fan-in need further passes, and each pass rewrites the data, so a low fan-in on a build that spilled heavily shows up as write amplification rather than as a slow merge. The cap is a file-descriptor and seek-pattern bound, not a memory one. [builder.merge_fan_in; env YSEARCH_BUILDER_MERGE_FAN_IN] (in [0, 64])
--builder-output-format-version int 0 builder.output_format_version YSEARCH_BUILDER_OUTPUT_FORMAT_VERSION Segment format emitted by builders and compactors; zero selects this binary's current format. The Kubernetes operator pins this to compatibility.writeFormat on writer roles during staged rollouts. Format 7 adds the stats, forward and hashterms families (spec/40 §3); a compaction that includes an older input writes format 6 instead, unless 7 or 8 is set explicitly, which refuses such a merge. Format 8 adds the exact vector lanes (canonical fp16 vectors, IVF radii, int8 codes; spec/40 §3.2). [builder.output_format_version; env YSEARCH_BUILDER_OUTPUT_FORMAT_VERSION] (in [0, 8])
--builder-paged-posting-directory bool false builder.paged_posting_directory YSEARCH_BUILDER_PAGED_POSTING_DIRECTORY Experimental commit-anchored posting-directory pages under query admission and the existing persistent block quota; old commits retain canonical traversal. [builder.paged_posting_directory; env YSEARCH_BUILDER_PAGED_POSTING_DIRECTORY]
--builder-posting-partition-penalty float 4 builder.posting_partition_penalty YSEARCH_BUILDER_POSTING_PARTITION_PENALTY Metadata cost penalty for experimental variable posting blocks. [builder.posting_partition_penalty; env YSEARCH_BUILDER_POSTING_PARTITION_PENALTY] (in [0, 1e+06])
--publish-verification string checksum builder.publish_verification YSEARCH_BUILDER_PUBLISH_VERIFICATION How a published segment is confirmed: checksum, sample, or readback. checksum compares every object against the checksum the object store reported when it accepted the write — S3 computes that server-side and refuses a mismatched write, and the filesystem backend computes it over the bytes it synced — so nothing is transferred back. sample additionally opens the published copy through the store and queries it; readback additionally fetches every object and rehashes it, which costs the whole index again on every build. An object whose store reports no comparable checksum is read back in any mode. [builder.publish_verification; env YSEARCH_BUILDER_PUBLISH_VERIFICATION]
--builder-routing-fragment-bytes bytes 8MiB builder.routing_fragment_bytes YSEARCH_BUILDER_ROUTING_FRAGMENT_BYTES Optional routing fragment disk cap per built segment; zero disables, overflow keeps canonical scan fallback. Matches compaction.routing_fragment_bytes so a freshly ingested segment and a freshly compacted one carry a fragment the same way; the two stay separate keys because a background merge must be able to carry its own share independent of foreground ingestion. Only a segment built or compacted after this defaulted on carries a fragment: nothing rewrites an existing one, so a settled corpus stays without a routing table until it is backfilled or compacted. [builder.routing_fragment_bytes; env YSEARCH_BUILDER_ROUTING_FRAGMENT_BYTES] (in [0, 64MiB])
--run-page-bytes bytes 0 builder.run_page_bytes YSEARCH_BUILDER_RUN_PAGE_BYTES Read-ahead page one spilled mini-run is buffered in during the final merge; zero derives 64KiB, shrinking it when an eighth of the partition budget cannot hold sixty-four such pages. The merge reserves one page per open run for the whole build, so page size and builder.merge_fan_in trade against each other out of the same share: the derivation prefers smaller pages to a fan-in that would force extra compaction passes on a hot partition. [builder.run_page_bytes; env YSEARCH_BUILDER_RUN_PAGE_BYTES]
--sort-memory-bytes bytes 256MiB builder.sort_memory_bytes YSEARCH_BUILDER_SORT_MEMORY_BYTES Total accounted builder budget: one ledger every part of a build charges against, and the setting the other builder budgets derive from. It divides into the public-id sorter (an eighth, capped at 64MiB), the docvalue and lane spools (their own worst case), the analysis share (builder.analysis_memory_bytes), and the term partitions, which take what is left. A build that cannot hold its partitions in that remainder spills to disk instead, which is the difference between one file per build and hundreds. This is per build, so the builder's memory is builder.build_concurrency times this. [builder.sort_memory_bytes; env YSEARCH_BUILDER_SORT_MEMORY_BYTES] (positive)
--builder-spill-merge-strategy enum heap builder.spill_merge_strategy YSEARCH_BUILDER_SPILL_MERGE_STRATEGY Experimental spill merge selector; heap preserves the baseline until full ingestion benchmarks qualify an alternative. [builder.spill_merge_strategy; env YSEARCH_BUILDER_SPILL_MERGE_STRATEGY] (one of heap, replace-root, loser-tree)
--builder-startup-bundles bool false builder.startup_bundles YSEARCH_BUILDER_STARTUP_BUNDLES Asynchronously produce optional startup bundles after catalog publication. One bounded coalescing worker per publishing host; oversized sources retain canonical startup fallback. [builder.startup_bundles; env YSEARCH_BUILDER_STARTUP_BUNDLES]
--term-dictionary-block-size bytes 64KiB builder.term_dictionary_block_size YSEARCH_BUILDER_TERM_DICTIONARY_BLOCK_SIZE Key-log data-block size of the term family. A table carries one block-index entry per data block and reads the whole index before it can look up anything, so this is what a cold term probe pays to find out where to look. At the storage engine's 4KiB an 8MiB dictionary needs about two thousand entries — around 65KiB read on every open of the table and held in memory until it closes — where 64KiB needs about a sixteenth of that. A probe then reads a bigger block, which over an object store is the cheap side of the trade: measured on a 6MiB dictionary the block cost 17KiB compressed against the 48KiB of index it replaced. [builder.term_dictionary_block_size; env YSEARCH_BUILDER_TERM_DICTIONARY_BLOCK_SIZE] (positive)
--term-dictionary-bloom bool false builder.term_dictionary_bloom YSEARCH_BUILDER_TERM_DICTIONARY_BLOOM Write a Bloom filter over the term family. The storage engine writes one by default; a segment does not, because the filter is read whole — uncompressed, about 1.2 bytes per distinct term — when the table is opened, and kept in memory until it closes, to save at most one block read per probe against the single table a published segment's term family is. Measured, it was nine tenths of what a cold term probe read: 431KiB of 481KiB on a 360k-term dictionary. A table says in its own footer whether it has a filter, so segments published with one keep working unchanged. [builder.term_dictionary_bloom; env YSEARCH_BUILDER_TERM_DICTIONARY_BLOOM]
--builder-variable-posting-blocks bool false builder.variable_posting_blocks YSEARCH_BUILDER_VARIABLE_POSTING_BLOCKS Experimental bounded-window BM25F-aware posting partitioning for format5 or newer output. [builder.variable_posting_blocks; env YSEARCH_BUILDER_VARIABLE_POSTING_BLOCKS]
--cache-block-bytes bytes 2GiB cache.block_bytes YSEARCH_CACHE_BLOCK_BYTES Disk, not memory: bytes of verified decoded blocks this node may keep under cache.dir. This is the cheap half of block caching - it converts a remote read into a local one and costs no resident memory - so size it above one query's working set and leave cache.read_block_bytes to decide what stays in RAM. Measured on 294 segments: one single-term query touched about 363MB of term dictionary and postings and one scored two-term query about 544MB, so the default holds several such working sets. [cache.block_bytes; env YSEARCH_CACHE_BLOCK_BYTES] (positive)
--cache-block-entries int 1048576 cache.block_entries YSEARCH_CACHE_BLOCK_ENTRIES Disk, not memory: how many decoded blocks cache.block_bytes may be spread over. Whichever is reached first bounds the cache; at the measured ~237KiB per block the byte budget is reached first by a wide margin. [cache.block_entries; env YSEARCH_CACHE_BLOCK_ENTRIES] (positive)
--cache-dir string cache.dir YSEARCH_CACHE_DIR Disposable local cache directory; symlinks in its path are resolved once at startup and the cache then anchors there without following any. It holds the two disk budgets, cache.full_bytes and cache.block_bytes, and nothing else: every other cache.* budget is process memory. A relative value in a configuration file resolves against that file's directory. [cache.dir; env YSEARCH_CACHE_DIR]
--cache-full-bytes bytes 8GiB cache.full_bytes YSEARCH_CACHE_FULL_BYTES Disk, not memory: bytes of whole hydrated segments this node may keep under cache.dir. With cache.block_bytes it is the whole of what the cache directory holds, so a volume smaller than their sum is an eviction loop the node cannot see. [cache.full_bytes; env YSEARCH_CACHE_FULL_BYTES] (positive)
--cache-full-entries int 1024 cache.full_entries YSEARCH_CACHE_FULL_ENTRIES Disk, not memory: how many whole segments cache.full_bytes may be spread over. Whichever of the two is reached first bounds the cache, and at the measured ~340MB per segment of a 101GB corpus the byte budget is reached long before this one. [cache.full_entries; env YSEARCH_CACHE_FULL_ENTRIES] (positive)
--cache-lazy-readers bool false cache.lazy_readers YSEARCH_CACHE_LAZY_READERS Experimental: acquire immutable segment readers and optional filter payloads only for admitted query work. It is also what routing.presence_bytes zero means: eager readers read that zero as unbounded and hold every segment's presence filter, lazy readers read it as none. [cache.lazy_readers; env YSEARCH_CACHE_LAZY_READERS]
--cache-ordinal-bytes bytes 64MiB cache.ordinal_bytes YSEARCH_CACHE_ORDINAL_BYTES Memory, not disk: process-wide verified ordinal-ID pages, including pinned pages and in-flight page workspace, shared across indexes and generations. [cache.ordinal_bytes; env YSEARCH_CACHE_ORDINAL_BYTES] (at least 327936)
--cache-read-block-bytes bytes 512MiB cache.read_block_bytes YSEARCH_CACHE_READ_BLOCK_BYTES Memory, not disk: process-wide decoded WavesDB blocks held resident and shared by immutable segment readers. This is what decides whether a repeated query is warm. Below one query's working set nothing survives to the next query and every query pays the cold price forever: measured on 294 segments, a 24MiB budget re-read all 214MB of term dictionary byte for byte while about 495MiB answered the repeat from 57,421 bytes. Size it from the corpus, not the machine, and from the layout its segments were built with. A term probe costs the same whatever the term, but what it costs depends on whether the segment carries a term-family Bloom filter: about 711KiB per segment on one built before builder.term_dictionary_bloom defaulted to false, and about 21KiB on one built since. The term dictionary alone needs segments times that figure, and the postings of the shape you serve come on top. A mixed corpus pays the older rate for the segments it has not rebuilt. [cache.read_block_bytes; env YSEARCH_CACHE_READ_BLOCK_BYTES] (positive)
--cache-read-open-files int 1024 cache.read_open_files YSEARCH_CACHE_READ_OPEN_FILES Process-wide open WavesDB table-file limit shared by immutable segment readers. It spends file descriptors, and nothing here raises RLIMIT_NOFILE, so the default sits at the 1024 a Linux process is commonly given with the listeners, connections and cache files sharing it: raise the limit before raising this. [cache.read_open_files; env YSEARCH_CACHE_READ_OPEN_FILES] (positive)
--cache-read-open-readers int 4098 cache.read_open_readers YSEARCH_CACHE_READ_OPEN_READERS Process-wide decoded WavesDB table-reader count limit. Each retained reader also spends cache.read_reader_bytes, and whichever of the two is reached first evicts. Unlike cache.read_open_files this spends memory rather than file descriptors, so size it from the corpus: a segment holds several table families, and a count that cannot hold them all evicts on every query. At 1024, which this defaulted to, a 253-segment corpus needing 1,321 readers held 77% of them and evicted 3,022 times while its byte budget sat 89% empty. That is dearer than it sounds, because a re-opened table re-reads its Bloom filter and block index through a path that bypasses the block cache, so the eviction becomes object reads that repeat for the life of the node: the same query read 62.8MiB in 275 range GETs every time it ran, and none after the count was raised. Measured resident cost is about 760KiB per reader. [cache.read_open_readers; env YSEARCH_CACHE_READ_OPEN_READERS] (positive)
--cache-read-reader-bytes bytes 2GiB cache.read_reader_bytes YSEARCH_CACHE_READ_READER_BYTES Memory, not disk: process-wide WavesDB table index and Bloom-filter bytes held resident. It is the only cache budget that scales with the corpus rather than with the query, at a measured ~14MiB per segment, so the default holds roughly 145 segments and a larger corpus re-reads a segment's index after evicting it. [cache.read_reader_bytes; env YSEARCH_CACHE_READ_READER_BYTES] (positive)
--cache-result-enabled bool true cache.result_enabled YSEARCH_CACHE_RESULT_ENABLED Cache generation and segment ranked results. Disable for execution benchmarks; reader, block, plan and stored-field caches remain active. [cache.result_enabled; env YSEARCH_CACHE_RESULT_ENABLED]
--cache-scrub-entries int 4 cache.scrub_entries YSEARCH_CACHE_SCRUB_ENTRIES How many cached full segments one scrubber pass re-reads. The scrubber walks the cache in a rotating cursor, so this and cache.scrub_interval together set how long a full sweep takes and how much disk read it costs; a segment currently in use is skipped and picked up on a later pass. [cache.scrub_entries; env YSEARCH_CACHE_SCRUB_ENTRIES] (in [0, 4096])
--cache-scrub-interval duration 5m cache.scrub_interval YSEARCH_CACHE_SCRUB_INTERVAL How often the background scrubber re-reads cached full segments and checks them against their commits. This is the only thing that notices a cached segment rotting on disk: a query trusts an entry once it has been verified, because verifying costs the SHA-256 of every file in the segment and doing that per request made a many-segment query spend all its time hashing. Zero disables the scrubber, which leaves local corruption undetected until the entry is evicted. [cache.scrub_interval; env YSEARCH_CACHE_SCRUB_INTERVAL] (in [0s, 24h0m0s])
--cache-sidecar-bytes bytes 8MiB cache.sidecar_bytes YSEARCH_CACHE_SIDECAR_BYTES Memory, not disk: process-wide optional exact-filter decoder and lease budget shared across indexes and generations. [cache.sidecar_bytes; env YSEARCH_CACHE_SIDECAR_BYTES] (positive)
--cache-warm-prefetch-bytes bytes 0 cache.warm_prefetch_bytes YSEARCH_CACHE_WARM_PREFETCH_BYTES Optional routing-page warmup byte limit after the first served query; zero disables. It spends object reads, not a budget of its own: warmed pages land in the process-wide routing page cache and are charged there, so warming past that cache only evicts what it just read. Shares foreground cache admission. [cache.warm_prefetch_bytes; env YSEARCH_CACHE_WARM_PREFETCH_BYTES] (in [0, 64MiB])
--cache-warm-prefetch-concurrency int 1 cache.warm_prefetch_concurrency YSEARCH_CACHE_WARM_PREFETCH_CONCURRENCY Maximum concurrent optional warmup reads; foreground queries stop new prefetch work. [cache.warm_prefetch_concurrency; env YSEARCH_CACHE_WARM_PREFETCH_CONCURRENCY] (in [0, 8])
--compaction-cluster-field string compaction.cluster_field YSEARCH_COMPACTION_CLUSTER_FIELD Existing scalar filterable string used by cluster layout; an absent field preserves order. [compaction.cluster_field; env YSEARCH_COMPACTION_CLUSTER_FIELD]
--compaction-document-layout enum preserve compaction.document_layout YSEARCH_COMPACTION_DOCUMENT_LAYOUT Experimental survivor layout for new compaction outputs; preserve retains input order, cluster groups scalar categories, bisection uses a bounded sampled term graph. [compaction.document_layout; env YSEARCH_COMPACTION_DOCUMENT_LAYOUT] (one of preserve, cluster, bisection)
--compaction-enabled bool false compaction.enabled YSEARCH_COMPACTION_ENABLED Run single-flight size-tiered compaction inside all-in-one serve. The explicit node compactor role always runs; this switch stays off there and by default so production can isolate maintenance from query and ingest. [compaction.enabled; env YSEARCH_COMPACTION_ENABLED]
--compaction-hydration-bytes bytes 8GiB compaction.hydration_bytes YSEARCH_COMPACTION_HYDRATION_BYTES Maximum downloaded checkpoint payload retained per compaction job across all input hydrations. Borrowed resident inputs are not copied or charged. Filesystem metadata, output database and spill space have separate accounting. [compaction.hydration_bytes; env YSEARCH_COMPACTION_HYDRATION_BYTES] (positive)
--compaction-hydration-workers int 2 compaction.hydration_workers YSEARCH_COMPACTION_HYDRATION_WORKERS Maximum concurrent input hydrations per compaction job. [compaction.hydration_workers; env YSEARCH_COMPACTION_HYDRATION_WORKERS] (in [1, 8])
--compaction-interval duration 1m compaction.interval YSEARCH_COMPACTION_INTERVAL Interval between background compaction eligibility checks. The all-in-one server waits for sustained ingest idleness; use the isolated compactor role when fan-out must be reduced during continuous ingest. [compaction.interval; env YSEARCH_COMPACTION_INTERVAL] (in [1s, 24h0m0s])
--compaction-max-active-segments int 16 compaction.max_active_segments YSEARCH_COMPACTION_MAX_ACTIVE_SEGMENTS Active searchable segment fan-out above which the scheduler may compact. [compaction.max_active_segments; env YSEARCH_COMPACTION_MAX_ACTIVE_SEGMENTS] (positive)
--compaction-max-concurrent-jobs int 1 compaction.max_concurrent_jobs YSEARCH_COMPACTION_MAX_CONCURRENT_JOBS Shared background compaction job slots across hosted indexes; physical merges are serial and hydration has a separate worker cap. [compaction.max_concurrent_jobs; env YSEARCH_COMPACTION_MAX_CONCURRENT_JOBS] (in [1, 8])
--compaction-max-inputs int 8 compaction.max_inputs YSEARCH_COMPACTION_MAX_INPUTS Maximum inputs in one compaction job; jobs are always single-flight. [compaction.max_inputs; env YSEARCH_COMPACTION_MAX_INPUTS] (in [2, 64])
--compaction-max-output-bytes bytes 4GiB compaction.max_output_bytes YSEARCH_COMPACTION_MAX_OUTPUT_BYTES Maximum summed input bytes admitted to one compaction proposal. [compaction.max_output_bytes; env YSEARCH_COMPACTION_MAX_OUTPUT_BYTES] (positive)
--compaction-max-size-ratio int 2 compaction.max_size_ratio YSEARCH_COMPACTION_MAX_SIZE_RATIO Largest size ratio allowed between inputs in one tier. [compaction.max_size_ratio; env YSEARCH_COMPACTION_MAX_SIZE_RATIO] (in [1, 1024])
--compaction-min-inputs int 4 compaction.min_inputs YSEARCH_COMPACTION_MIN_INPUTS Minimum adjacent similarly-sized inputs in one compaction job. [compaction.min_inputs; env YSEARCH_COMPACTION_MIN_INPUTS] (in [2, 64])
--compaction-read-inflight-bytes bytes 64MiB compaction.read_inflight_bytes YSEARCH_COMPACTION_READ_INFLIGHT_BYTES Shared compaction read payload bytes in flight across jobs. Open reads hold credits until Close; an individual object or range larger than this limit is rejected. [compaction.read_inflight_bytes; env YSEARCH_COMPACTION_READ_INFLIGHT_BYTES] (at least 64KiB)
--compaction-recent-delta int 4 compaction.recent_delta YSEARCH_COMPACTION_RECENT_DELTA Newest active segments left uncompacted as an ingest delta tier. [compaction.recent_delta; env YSEARCH_COMPACTION_RECENT_DELTA] (in [0, 1024])
--compaction-reorder-leaf-documents int 128 compaction.reorder_leaf_documents YSEARCH_COMPACTION_REORDER_LEAF_DOCUMENTS Maximum stable leaf size of experimental balanced graph bisection; each document samples at most 32 terms. [compaction.reorder_leaf_documents; env YSEARCH_COMPACTION_REORDER_LEAF_DOCUMENTS] (in [1, 4096])
--compaction-reorder-temp-bytes bytes 1GiB compaction.reorder_temp_bytes YSEARCH_COMPACTION_REORDER_TEMP_BYTES Temporary spill cap for reordered outputs, including permutation, sampled graph and posting resort; the smaller compaction.temporary_bytes cap also applies. [compaction.reorder_temp_bytes; env YSEARCH_COMPACTION_REORDER_TEMP_BYTES] (positive)
--compaction-routing-fragment-bytes bytes 8MiB compaction.routing_fragment_bytes YSEARCH_COMPACTION_ROUTING_FRAGMENT_BYTES Optional routing fragment disk cap per compaction output segment; overrides builder.routing_fragment_bytes for compaction's own merges so background maintenance can carry its own share independent of foreground ingestion. Zero disables fragment writing for compaction outputs, which also stops this generation's routing descriptor from ever completing until every one of its segments gets a fragment some other way. [compaction.routing_fragment_bytes; env YSEARCH_COMPACTION_ROUTING_FRAGMENT_BYTES] (in [0, 64MiB])
--compaction-routing-memory-bytes bytes 64MiB compaction.routing_memory_bytes YSEARCH_COMPACTION_ROUTING_MEMORY_BYTES Memory budget for assembling one generation's routing descriptor from its member segments' own fragments after a successful compaction. Separate from compaction.temporary_bytes and the segment-merge budgets: descriptor assembly reads small fragment objects, not segment checkpoints. [compaction.routing_memory_bytes; env YSEARCH_COMPACTION_ROUTING_MEMORY_BYTES] (at least 64MiB)
--compaction-routing-temp-bytes bytes 1GiB compaction.routing_temp_bytes YSEARCH_COMPACTION_ROUTING_TEMP_BYTES Spill and page-file budget for the same post-compaction descriptor assembly. Bounded well below compaction.temporary_bytes: a table assembled from fragments alone is the right-sized presence table (internal/routing/value.go), not the segments' own content. [compaction.routing_temp_bytes; env YSEARCH_COMPACTION_ROUTING_TEMP_BYTES] (positive)
--compaction-target-bytes bytes 512MiB compaction.target_bytes YSEARCH_COMPACTION_TARGET_BYTES Approximate input-byte target for one size-tiered compaction group. [compaction.target_bytes; env YSEARCH_COMPACTION_TARGET_BYTES] (positive)
--compaction-temporary-bytes bytes 8GiB compaction.temporary_bytes YSEARCH_COMPACTION_TEMPORARY_BYTES Shared accounted spill-byte limit within one physical compaction; hydration and final database storage are separate from spill space. [compaction.temporary_bytes; env YSEARCH_COMPACTION_TEMPORARY_BYTES] (positive)
--compaction-write-inflight-bytes bytes 64MiB compaction.write_inflight_bytes YSEARCH_COMPACTION_WRITE_INFLIGHT_BYTES Shared compaction write payload bytes in flight across jobs. An individual object larger than this limit is rejected before upload. [compaction.write_inflight_bytes; env YSEARCH_COMPACTION_WRITE_INFLIGHT_BYTES] (at least 64KiB)
--config-watch bool true config.watch YSEARCH_CONFIG_WATCH Reload the discovered configuration files when any of them changes. [config.watch; env YSEARCH_CONFIG_WATCH]
--pprof-listen string debug.pprof_listen YSEARCH_DEBUG_PPROF_LISTEN host:port serving net/http/pprof profiles; empty disables it. It binds before the catalog is opened, so a slow start can be profiled, and an address with no host (:6060) binds loopback. Profiles expose heap contents and goroutine stacks, so keep it on loopback and never on a public interface. [debug.pprof_listen; env YSEARCH_DEBUG_PPROF_LISTEN]
--embedding-external-allow-http bool false embedding.external_allow_http YSEARCH_EMBEDDING_EXTERNAL_ALLOW_HTTP Allow plain HTTP external embedding endpoints for trusted development networks. [embedding.external_allow_http; env YSEARCH_EMBEDDING_EXTERNAL_ALLOW_HTTP]
--embedding-external-batch-size int 128 embedding.external_batch_size YSEARCH_EMBEDDING_EXTERNAL_BATCH_SIZE Maximum texts in one external embedding request. [embedding.external_batch_size; env YSEARCH_EMBEDDING_EXTERNAL_BATCH_SIZE] (in [1, 4096])
--embedding-external-concurrency int 8 embedding.external_concurrency YSEARCH_EMBEDDING_EXTERNAL_CONCURRENCY Concurrent external embedding requests per process. [embedding.external_concurrency; env YSEARCH_EMBEDDING_EXTERNAL_CONCURRENCY] (in [1, 1024])
--embedding-external-max-retry-after duration 2s embedding.external_max_retry_after YSEARCH_EMBEDDING_EXTERNAL_MAX_RETRY_AFTER Largest Retry-After delay accepted from an external embedding endpoint. [embedding.external_max_retry_after; env YSEARCH_EMBEDDING_EXTERNAL_MAX_RETRY_AFTER] (positive)
--embedding-external-request-bytes bytes 8MiB embedding.external_request_bytes YSEARCH_EMBEDDING_EXTERNAL_REQUEST_BYTES Maximum encoded external embedding request bytes. [embedding.external_request_bytes; env YSEARCH_EMBEDDING_EXTERNAL_REQUEST_BYTES] (positive)
--embedding-external-response-bytes bytes 64MiB embedding.external_response_bytes YSEARCH_EMBEDDING_EXTERNAL_RESPONSE_BYTES Maximum external embedding response bytes read before JSON decoding. [embedding.external_response_bytes; env YSEARCH_EMBEDDING_EXTERNAL_RESPONSE_BYTES] (positive)
--embedding-external-retries int 2 embedding.external_retries YSEARCH_EMBEDDING_EXTERNAL_RETRIES Retry attempts after the first retryable external embedding failure. [embedding.external_retries; env YSEARCH_EMBEDDING_EXTERNAL_RETRIES] (in [0, 16])
--embedding-external-retry-base duration 50ms embedding.external_retry_base YSEARCH_EMBEDDING_EXTERNAL_RETRY_BASE Base exponential backoff for external embedding retries. [embedding.external_retry_base; env YSEARCH_EMBEDDING_EXTERNAL_RETRY_BASE] (positive)
--embedding-external-timeout duration 30s embedding.external_timeout YSEARCH_EMBEDDING_EXTERNAL_TIMEOUT Deadline for one external embedding HTTP attempt sequence. [embedding.external_timeout; env YSEARCH_EMBEDDING_EXTERNAL_TIMEOUT] (positive)
--embedding-grpc-allow-insecure bool false embedding.grpc_allow_insecure YSEARCH_EMBEDDING_GRPC_ALLOW_INSECURE Allow plaintext gRPC embedding endpoints on trusted internal networks. [embedding.grpc_allow_insecure; env YSEARCH_EMBEDDING_GRPC_ALLOW_INSECURE]
--embedding-grpc-batch-size int 128 embedding.grpc_batch_size YSEARCH_EMBEDDING_GRPC_BATCH_SIZE Maximum texts in one gRPC embedding request. [embedding.grpc_batch_size; env YSEARCH_EMBEDDING_GRPC_BATCH_SIZE] (in [1, 4096])
--embedding-grpc-concurrency int 8 embedding.grpc_concurrency YSEARCH_EMBEDDING_GRPC_CONCURRENCY Concurrent gRPC embedding requests per process. [embedding.grpc_concurrency; env YSEARCH_EMBEDDING_GRPC_CONCURRENCY] (in [1, 1024])
--embedding-grpc-request-bytes bytes 8MiB embedding.grpc_request_bytes YSEARCH_EMBEDDING_GRPC_REQUEST_BYTES Maximum encoded gRPC embedding request bytes. [embedding.grpc_request_bytes; env YSEARCH_EMBEDDING_GRPC_REQUEST_BYTES] (positive)
--embedding-grpc-response-bytes bytes 64MiB embedding.grpc_response_bytes YSEARCH_EMBEDDING_GRPC_RESPONSE_BYTES Maximum encoded gRPC embedding response bytes. [embedding.grpc_response_bytes; env YSEARCH_EMBEDDING_GRPC_RESPONSE_BYTES] (positive)
--embedding-grpc-timeout duration 30s embedding.grpc_timeout YSEARCH_EMBEDDING_GRPC_TIMEOUT Deadline for one gRPC embedding request. [embedding.grpc_timeout; env YSEARCH_EMBEDDING_GRPC_TIMEOUT] (positive)
--embedding-grpc-work-class enum auto embedding.grpc_work_class YSEARCH_EMBEDDING_GRPC_WORK_CLASS gRPC embedding scheduler class; auto maps queries to interactive and indexing to bulk. [embedding.grpc_work_class; env YSEARCH_EMBEDDING_GRPC_WORK_CLASS] (one of auto, interactive, bulk)
--embedding-internal-batch-size int 32 embedding.internal_batch_size YSEARCH_EMBEDDING_INTERNAL_BATCH_SIZE Maximum texts in one internal embedding batch. [embedding.internal_batch_size; env YSEARCH_EMBEDDING_INTERNAL_BATCH_SIZE] (in [1, 4096])
--embedding-internal-concurrency int 2 embedding.internal_concurrency YSEARCH_EMBEDDING_INTERNAL_CONCURRENCY Concurrent pure-Go internal embedding batches. [embedding.internal_concurrency; env YSEARCH_EMBEDDING_INTERNAL_CONCURRENCY] (in [1, 256])
--embedding-internal-enabled bool false embedding.internal_enabled YSEARCH_EMBEDDING_INTERNAL_ENABLED Enable the pure-Go internal embedding runtime. Models remain lazy and checksum-pinned by schema profiles. [embedding.internal_enabled; env YSEARCH_EMBEDDING_INTERNAL_ENABLED]
--embedding-model-cache-bytes bytes 10GiB embedding.model_cache_bytes YSEARCH_EMBEDDING_MODEL_CACHE_BYTES Maximum installed bytes in the internal-model cache. [embedding.model_cache_bytes; env YSEARCH_EMBEDDING_MODEL_CACHE_BYTES] (positive)
--embedding-model-cache-dir string embedding.model_cache_dir YSEARCH_EMBEDDING_MODEL_CACHE_DIR Checksum-addressed internal-model cache directory. Empty disables internal embedding; keep it distinct from cache.dir. A relative value in a configuration file resolves against that file's directory. [embedding.model_cache_dir; env YSEARCH_EMBEDDING_MODEL_CACHE_DIR]
--embedding-model-cache-packages int 8 embedding.model_cache_packages YSEARCH_EMBEDDING_MODEL_CACHE_PACKAGES Maximum installed internal-model packages. [embedding.model_cache_packages; env YSEARCH_EMBEDDING_MODEL_CACHE_PACKAGES] (positive)
--embedding-model-download-allow-http bool false embedding.model_download_allow_http YSEARCH_EMBEDDING_MODEL_DOWNLOAD_ALLOW_HTTP Allow plain HTTP model manifests and artifacts for trusted development networks. [embedding.model_download_allow_http; env YSEARCH_EMBEDDING_MODEL_DOWNLOAD_ALLOW_HTTP]
--embedding-model-download-concurrency int 2 embedding.model_download_concurrency YSEARCH_EMBEDDING_MODEL_DOWNLOAD_CONCURRENCY Concurrent verified internal-model artifact downloads. [embedding.model_download_concurrency; env YSEARCH_EMBEDDING_MODEL_DOWNLOAD_CONCURRENCY] (in [1, 64])
--embedding-model-download-timeout duration 10m embedding.model_download_timeout YSEARCH_EMBEDDING_MODEL_DOWNLOAD_TIMEOUT Deadline for one internal-model package installation. [embedding.model_download_timeout; env YSEARCH_EMBEDDING_MODEL_DOWNLOAD_TIMEOUT] (positive)
--embedding-model-manifest-bytes bytes 1MiB embedding.model_manifest_bytes YSEARCH_EMBEDDING_MODEL_MANIFEST_BYTES Maximum bytes in one internal-model manifest. [embedding.model_manifest_bytes; env YSEARCH_EMBEDDING_MODEL_MANIFEST_BYTES] (positive)
--embedding-model-package-bytes bytes 4GiB embedding.model_package_bytes YSEARCH_EMBEDDING_MODEL_PACKAGE_BYTES Maximum bytes in one downloaded internal-model package. [embedding.model_package_bytes; env YSEARCH_EMBEDDING_MODEL_PACKAGE_BYTES] (positive)
--embedding-query-cache-bytes bytes 64MiB embedding.query_cache_bytes YSEARCH_EMBEDDING_QUERY_CACHE_BYTES Maximum float-vector bytes retained by the text-query embedding cache. [embedding.query_cache_bytes; env YSEARCH_EMBEDDING_QUERY_CACHE_BYTES] (positive)
--embedding-query-cache-entries int 4096 embedding.query_cache_entries YSEARCH_EMBEDDING_QUERY_CACHE_ENTRIES Maximum completed text-query embeddings retained per process. [embedding.query_cache_entries; env YSEARCH_EMBEDDING_QUERY_CACHE_ENTRIES] (positive)
--embedding-query-cache-ttl duration 10m embedding.query_cache_ttl YSEARCH_EMBEDDING_QUERY_CACHE_TTL Lifetime of a completed text-query embedding cache entry. [embedding.query_cache_ttl; env YSEARCH_EMBEDDING_QUERY_CACHE_TTL] (positive)
--admin-fleet list fleet.admin_fleet YSEARCH_FLEET_ADMIN_FLEET Node host:port endpoints StreamFleetStats fans in; empty refuses the fleet stream. [fleet.admin_fleet; env YSEARCH_FLEET_ADMIN_FLEET]
--aggregators list fleet.aggregators YSEARCH_FLEET_AGGREGATORS Aggregator host:port endpoints. [fleet.aggregators; env YSEARCH_FLEET_AGGREGATORS]
--builders list fleet.builders YSEARCH_FLEET_BUILDERS Builder host:port endpoints the router polls and rendezvous-routes batches across. [fleet.builders; env YSEARCH_FLEET_BUILDERS]
--fallback-workers list fleet.fallback_workers YSEARCH_FLEET_FALLBACK_WORKERS Worker host:port endpoints: the dispatch universe and the cache-oblivious fallback. [fleet.fallback_workers; env YSEARCH_FLEET_FALLBACK_WORKERS]
--fan-in int 0 fleet.fan_in YSEARCH_FLEET_FAN_IN Children per merge node in coordinator plans; zero means the planner's default, otherwise at least 2. [fleet.fan_in; env YSEARCH_FLEET_FAN_IN] (in [0, 1024])
--lane-slots string interactive=4,streaming=2 fleet.lane_slots YSEARCH_FLEET_LANE_SLOTS Per-lane worker slot pools, lane=count. [fleet.lane_slots; env YSEARCH_FLEET_LANE_SLOTS]
--mergers list fleet.mergers YSEARCH_FLEET_MERGERS Merge host:port endpoints: the coordinator's merge tier, a merger's delegation peers. [fleet.mergers; env YSEARCH_FLEET_MERGERS]
--follower-discovery-interval duration 30s follower.discovery_interval YSEARCH_FOLLOWER_DISCOVERY_INTERVAL How often a root attachment looks for indexes that appeared or disappeared under it. Separate from follower.poll_interval because discovery lists prefixes while a poll reads one pointer, and listing is the more expensive and more rate-limited of the two. [follower.discovery_interval; env YSEARCH_FOLLOWER_DISCOVERY_INTERVAL] (positive)
--follower-generation-overlap duration 30s follower.generation_overlap YSEARCH_FOLLOWER_GENERATION_OVERLAP Maximum age of an unseen catalog generation a worker may resolve on demand; in-flight references may keep its engine beyond this window. [follower.generation_overlap; env YSEARCH_FOLLOWER_GENERATION_OVERLAP] (positive)
--follower-max-staleness duration 5m follower.max_staleness YSEARCH_FOLLOWER_MAX_STALENESS How long a follower may go without confirming its installed generation is still current before it refuses new queries. A reader that cannot reach its source keeps answering from cached data indefinitely otherwise, which is worse than an error: the results look fine and are silently frozen. Status stays readable past this point so an operator can see why. [follower.max_staleness; env YSEARCH_FOLLOWER_MAX_STALENESS] (positive)
--follower-poll-interval duration 2s follower.poll_interval YSEARCH_FOLLOWER_POLL_INTERVAL How often a node checks the catalog's latest pointer. [follower.poll_interval; env YSEARCH_FOLLOWER_POLL_INTERVAL] (positive)
--follower-retired-generation-grace duration 30s follower.retired_generation_grace YSEARCH_FOLLOWER_RETIRED_GENERATION_GRACE How long a superseded generation's engine stays open after its last query. [follower.retired_generation_grace; env YSEARCH_FOLLOWER_RETIRED_GENERATION_GRACE] (positive)
--gc-grace duration 1h gc.grace YSEARCH_GC_GRACE Additional safety margin added to every scheduled GC reachability horizon. [gc.grace; env YSEARCH_GC_GRACE] (in [0s, 168h0m0s])
--gc-interval duration 1h gc.interval YSEARCH_GC_INTERVAL Cadence at which the dedicated compactor role writes a GC proposal and considers a quarantined older proposal. [gc.interval; env YSEARCH_GC_INTERVAL] (in [1s, 168h0m0s])
--gc-maximum-stream-lifetime duration 1h gc.maximum_stream_lifetime YSEARCH_GC_MAXIMUM_STREAM_LIFETIME Longest query-stream lifetime protected when marking superseded generations for scheduled GC. [gc.maximum_stream_lifetime; env YSEARCH_GC_MAXIMUM_STREAM_LIFETIME] (in [0s, 168h0m0s])
--gc-minimum-upload-age duration 24h gc.minimum_upload_age YSEARCH_GC_MINIMUM_UPLOAD_AGE Minimum age before an unreferenced object can enter a scheduled GC proposal. [gc.minimum_upload_age; env YSEARCH_GC_MINIMUM_UPLOAD_AGE] (in [0s, 720h0m0s])
--gc-quarantine-age duration 24h gc.quarantine_age YSEARCH_GC_QUARANTINE_AGE Minimum age of an immutable GC proposal before a fresh mark may authorize deleting its still-unreachable exact keys. [gc.quarantine_age; env YSEARCH_GC_QUARANTINE_AGE] (in [1s, 720h0m0s])
--gc-retained-generations int 2 gc.retained_generations YSEARCH_GC_RETAINED_GENERATIONS Historical catalog generations retained in addition to latest during scheduled GC. [gc.retained_generations; env YSEARCH_GC_RETAINED_GENERATIONS] (in [0, 1024])
--gc-sweep-enabled bool false gc.sweep_enabled YSEARCH_GC_SWEEP_ENABLED Allow the dedicated compactor role to delete freshly re-proven orphan objects after an immutable dry-run proposal passes quarantine. Disabled by default. [gc.sweep_enabled; env YSEARCH_GC_SWEEP_ENABLED]
--generation string index.generation YSEARCH_INDEX_GENERATION Exact generation to pin, or empty for the latest. [index.generation; env YSEARCH_INDEX_GENERATION]
--index string index.name YSEARCH_INDEX_NAME Index name for catalog-bound serve, worker, coordinator, compactor, router, builder, and publisher roles. [index.name; env YSEARCH_INDEX_NAME]
--ingest-burst-documents int 100000 ingest.burst_documents YSEARCH_INGEST_BURST_DOCUMENTS Per-index document tokens available for an ingest burst when the rate quota is enabled. [ingest.burst_documents; env YSEARCH_INGEST_BURST_DOCUMENTS] (positive)
--ingest-dir string ingest.dir YSEARCH_INGEST_DIR Ingest spool/build directory for builders and scratch root for the dedicated compactor role. A relative value in a configuration file resolves against that file's directory. [ingest.dir; env YSEARCH_INGEST_DIR]
--ingest-documents-per-second int 0 ingest.documents_per_second YSEARCH_INGEST_DOCUMENTS_PER_SECOND Per-router, per-index document admission rate; zero disables the rate quota. [ingest.documents_per_second; env YSEARCH_INGEST_DOCUMENTS_PER_SECOND] (in [0, 1099511627776])
--ingest-global-queue-bytes bytes 256MiB ingest.global_queue_bytes YSEARCH_INGEST_GLOBAL_QUEUE_BYTES Encoded ingest bytes all router sessions may hold while waiting for builders. [ingest.global_queue_bytes; env YSEARCH_INGEST_GLOBAL_QUEUE_BYTES] (positive)
--ingest-index-queue-bytes bytes 64MiB ingest.index_queue_bytes YSEARCH_INGEST_INDEX_QUEUE_BYTES Encoded ingest bytes one index may hold while waiting for builders. [ingest.index_queue_bytes; env YSEARCH_INGEST_INDEX_QUEUE_BYTES] (positive)
--ingest-max-batch-bytes bytes 4MiB ingest.max_batch_bytes YSEARCH_INGEST_MAX_BATCH_BYTES Largest accepted ingest batch. [ingest.max_batch_bytes; env YSEARCH_INGEST_MAX_BATCH_BYTES] (positive)
--ingest-max-document-bytes bytes 16MiB ingest.max_document_bytes YSEARCH_INGEST_MAX_DOCUMENT_BYTES Largest accepted document. [ingest.max_document_bytes; env YSEARCH_INGEST_MAX_DOCUMENT_BYTES] (positive)
--ingest-seal-age duration 30s ingest.seal_age YSEARCH_INGEST_SEAL_AGE Age at which an open build seals. [ingest.seal_age; env YSEARCH_INGEST_SEAL_AGE] (positive)
--ingest-seal-bytes bytes 0 ingest.seal_bytes YSEARCH_INGEST_SEAL_BYTES Spooled bytes at which an open build seals; zero derives it as a quarter of builder.sort_memory_bytes. A build's postings are about twice its spooled bytes and the partitions get roughly half the sort budget, so a spool larger than that quarter cannot be sorted in memory and the build spills instead — which is the difference between one file per build and hundreds, and measured 2.5x on the build itself. [ingest.seal_bytes; env YSEARCH_INGEST_SEAL_BYTES] (in [0, 1TiB])
--ingest-seal-documents int 100000 ingest.seal_documents YSEARCH_INGEST_SEAL_DOCUMENTS Spooled documents at which an open build seals. [ingest.seal_documents; env YSEARCH_INGEST_SEAL_DOCUMENTS] (positive)
--log-format enum text log.format YSEARCH_LOG_FORMAT Log line format. [log.format; env YSEARCH_LOG_FORMAT] (one of text, json)
--log-level enum info log.level YSEARCH_LOG_LEVEL Minimum level written to stderr. [log.level; env YSEARCH_LOG_LEVEL] (one of error, warn, info, debug)
--object-backend enum fs object.backend YSEARCH_OBJECT_BACKEND Object store backend. [object.backend; env YSEARCH_OBJECT_BACKEND] (one of fs, s3)
--object-dir string object.dir YSEARCH_OBJECT_DIR Filesystem object-store directory; required for the fs backend. A relative value in a configuration file resolves against that file's directory. [object.dir; env YSEARCH_OBJECT_DIR]
--object-root string indexes object.root YSEARCH_OBJECT_ROOT Object key root. [object.root; env YSEARCH_OBJECT_ROOT]
--s3-anonymous bool false object.s3.anonymous YSEARCH_OBJECT_S3_ANONYMOUS Sign no requests at all, for a bucket that grants public reads. This is a deliberate choice, never a fallback: an unresolved credential chain already signs anonymously, and a reader that quietly degraded to unsigned requests would report a permissions problem as an empty index. [object.s3.anonymous; env YSEARCH_OBJECT_S3_ANONYMOUS]
--s3-bucket string object.s3.bucket YSEARCH_OBJECT_S3_BUCKET S3 bucket; required for the s3 backend. [object.s3.bucket; env YSEARCH_OBJECT_S3_BUCKET]
--s3-endpoint string object.s3.endpoint YSEARCH_OBJECT_S3_ENDPOINT S3 endpoint host:port; required for the s3 backend. [object.s3.endpoint; env YSEARCH_OBJECT_S3_ENDPOINT]
--s3-path-style bool false object.s3.path_style YSEARCH_OBJECT_S3_PATH_STYLE Use path-style S3 addressing. [object.s3.path_style; env YSEARCH_OBJECT_S3_PATH_STYLE]
--s3-prefix string object.s3.prefix YSEARCH_OBJECT_S3_PREFIX S3 key prefix. [object.s3.prefix; env YSEARCH_OBJECT_S3_PREFIX]
--s3-region string object.s3.region YSEARCH_OBJECT_S3_REGION S3 region. [object.s3.region; env YSEARCH_OBJECT_S3_REGION]
--s3-use-ssl bool true object.s3.use_ssl YSEARCH_OBJECT_S3_USE_SSL Use TLS to the S3 endpoint. [object.s3.use_ssl; env YSEARCH_OBJECT_S3_USE_SSL]
--source string object.source YSEARCH_OBJECT_SOURCE Dataset URL to attach read-only, such as s3://bucket/root/ for a whole root or s3://bucket/root/indexes/name/ for one index. It derives object.backend, the bucket, prefix and root, the index when the URL names one, and server.read_only, each only where nothing more specific was set. Reading someone else's published index is the whole purpose, so attaching one implies read-only and cannot be talked out of it. [object.source; env YSEARCH_OBJECT_SOURCE]
--metrics-listen string :9550 observability.metrics_listen YSEARCH_OBSERVABILITY_METRICS_LISTEN host:port serving Prometheus exposition at /metrics; empty disables it. Unlike debug.pprof_listen this is on by default and binds every interface, because a metric carries no document text, no query, and no key material, and a scrape target that has to be switched on is one nobody switches on. Every series already carries the role label, so the scrape target only has to supply instance and pod. [observability.metrics_listen; env YSEARCH_OBSERVABILITY_METRICS_LISTEN]
--publisher-announce-interval duration 15s publisher.announce_interval YSEARCH_PUBLISHER_ANNOUNCE_INTERVAL Requested commit-marker publication cadence; the daemon floors it at follower.poll_interval and 15 seconds. [publisher.announce_interval; env YSEARCH_PUBLISHER_ANNOUNCE_INTERVAL] (in [1s, 1h0m0s])
--publisher-lease-ttl duration 30s publisher.lease_ttl YSEARCH_PUBLISHER_LEASE_TTL Startup-fixed object-store lease lifetime for the catalog publisher; renewal runs at one third of this value. [publisher.lease_ttl; env YSEARCH_PUBLISHER_LEASE_TTL] (in [5s, 10m0s])
--collapse-key-versions bool true query.collapse_key_versions YSEARCH_QUERY_COLLAPSE_KEY_VERSIONS Return one hit per document key, the copy with the greatest mutation version. A re-pushed document is stored as a new version rather than replacing the old one, so without this a key appears once per copy. Collapsing removes candidates after selection, so a request may return fewer than top_k. [query.collapse_key_versions; env YSEARCH_QUERY_COLLAPSE_KEY_VERSIONS]
--query-compiled-cache bool false query.compiled_cache YSEARCH_QUERY_COMPILED_CACHE Experimental leased compiled-query cache within the shared metadata allowance. Also reuses bounded dictionary plans for eager generation-owned readers; lazy reader plans stay request-owned. Cache pressure falls back to request-owned compilation or preparation. [query.compiled_cache; env YSEARCH_QUERY_COMPILED_CACHE]
--query-default-top-k int 10 query.default_top_k YSEARCH_QUERY_DEFAULT_TOP_K top_k when a request names none. [query.default_top_k; env YSEARCH_QUERY_DEFAULT_TOP_K] (in [1, 10000])
--query-defer-public-ids bool true query.defer_public_ids YSEARCH_QUERY_DEFER_PUBLIC_IDS Resolve a segment survivor's public ID only when the global merge returns it or must break a score tie with it, instead of for every survivor of every segment. Results are identical either way; false restores eager resolution as a same-code control. [query.defer_public_ids; env YSEARCH_QUERY_DEFER_PUBLIC_IDS]
--query-lexical-memory-bytes bytes 384MiB query.lexical_memory_bytes YSEARCH_QUERY_LEXICAL_MEMORY_BYTES Process-wide lexical request, candidate, queue and merge memory admission budget. Storage blocks and ordinal pages have separate cache budgets; busy requests fail with resource exhaustion. Sized to admit one request merging 64 segments at the product's 100,000-result top_k ceiling (lexical.MaxResultLimit; estimateLexicalFanoutMemory(64, 100000) = 182,632,832 bytes) with room for a second concurrent one that size. [query.lexical_memory_bytes; env YSEARCH_QUERY_LEXICAL_MEMORY_BYTES] (positive)
--query-lexical-strategy enum block-max query.lexical_strategy YSEARCH_QUERY_LEXICAL_STRATEGY Experimental exact lexical traversal selection; unsupported query shapes conservatively fall back to block-max. [query.lexical_strategy; env YSEARCH_QUERY_LEXICAL_STRATEGY] (one of block-max, auto, single-term-block-max, conjunction, block-max-maxscore)
--query-logical-bound-entries int 0 query.logical_bound_entries YSEARCH_QUERY_LOGICAL_BOUND_ENTRIES Experimental score-bound subblock size within decoded postings; zero uses physical blocks. Does not reduce physical read size. [query.logical_bound_entries; env YSEARCH_QUERY_LOGICAL_BOUND_ENTRIES] (in [0, 256])
--query-max-ast-depth int 32 query.max_ast_depth YSEARCH_QUERY_MAX_AST_DEPTH Deepest query nesting either grammar accepts. [query.max_ast_depth; env YSEARCH_QUERY_MAX_AST_DEPTH] (in [1, 1024])
--query-max-results int 0 query.max_results YSEARCH_QUERY_MAX_RESULTS Ceiling on maximum_results per request; zero keeps the engine default. [query.max_results; env YSEARCH_QUERY_MAX_RESULTS]
--query-ordinal-mode enum pages query.ordinal_mode YSEARCH_QUERY_ORDINAL_MODE Experimental ID residency policy under the shared ordinal budget. Whole tables are capped at 8 MiB and fall back to pages; adaptive promotion requires repeated broad page coverage. [query.ordinal_mode; env YSEARCH_QUERY_ORDINAL_MODE] (one of pages, whole, adaptive)
--query-packed-simd bool false query.packed_simd YSEARCH_QUERY_PACKED_SIMD Experimental native packed-posting decoding on supported CPUs; preserves scalar fallback and semantic validation. [query.packed_simd; env YSEARCH_QUERY_PACKED_SIMD]
--query-phrase-gap-expansion-limit int 64 query.phrase_gap_expansion_limit YSEARCH_QUERY_PHRASE_GAP_EXPANSION_LIMIT Most exact gap variants a {m,n} quantifier may expand to. [query.phrase_gap_expansion_limit; env YSEARCH_QUERY_PHRASE_GAP_EXPANSION_LIMIT] (in [1, 4096])
--query-regex-max-expansions int 256 query.regex_max_expansions YSEARCH_QUERY_REGEX_MAX_EXPANSIONS Most dictionary terms a regex or prefix may expand to before the query is refused. [query.regex_max_expansions; env YSEARCH_QUERY_REGEX_MAX_EXPANSIONS] (in [1, 65536])
--query-scoring-profile enum bm25f-v1 query.scoring_profile YSEARCH_QUERY_SCORING_PROFILE Scoring profile for a request that names none. bm25f-v1 scores each segment with its own statistics; bm25f-pinned-v1 (spec/50 §2) scores every segment under the generation's pinned statistics epoch, so scores do not depend on segment layout. It needs stats.dir; without it a pinned request fails with a typed error, never with segment statistics. [query.scoring_profile; env YSEARCH_QUERY_SCORING_PROFILE] (one of bm25f-v1, bm25f-pinned-v1)
--query-set-simd bool false query.set_simd YSEARCH_QUERY_SET_SIMD Experimental native intersection for balanced sparse filter arrays; skewed arrays retain scalar galloping. [query.set_simd; env YSEARCH_QUERY_SET_SIMD]
--routing-presence-bytes bytes 0 routing.presence_bytes YSEARCH_ROUTING_PRESENCE_BYTES Memory, not disk: resident budget for per-segment term-presence filters, used only when the generation has no routing table. A generation that has one prunes through it instead and holds no filters at all, whatever this says, because the table is read through a small page cache rather than held. Zero means none under cache.lazy_readers and unbounded without it, which at 294 segments was 1.4GiB of resident filters. That promotion for eager readers stays on for now: builder.routing_fragment_bytes and compaction.routing_fragment_bytes only default on from 2026-09-23, nothing rewrites a segment built before that, and no existing corpus has been rebuilt or fully compacted since - so a generation with an unfragmented member still depends on this filter to prune anything, including the published demo image, which does not set this flag. Filters are loaded in catalog order until the budget is reached; segments without one stay unknown and are still executed, so results never change. Budget against the decoded size, not the transfer: 680MiB of admitted filters measured 1.4GiB resident, about 2.1 times what this charges them. [routing.presence_bytes; env YSEARCH_ROUTING_PRESENCE_BYTES] (in [0, 8GiB])
--data-dir string server.data_dir YSEARCH_SERVER_DATA_DIR Developer shortcut: derives object.backend=fs, object.dir, cache.dir, ingest.dir, and config.file beneath one directory where nothing more specific is set. A relative value in a configuration file resolves against that file's directory. [server.data_dir; env YSEARCH_SERVER_DATA_DIR]
--grace-period duration 5s server.grace_period YSEARCH_SERVER_GRACE_PERIOD Graceful shutdown period. [server.grace_period; env YSEARCH_SERVER_GRACE_PERIOD] (positive)
--listen string 127.0.0.1:9500 server.listen YSEARCH_SERVER_LISTEN gRPC listen host:port serve and node bind; the default is the address the client verbs dial (client.server). [server.listen; env YSEARCH_SERVER_LISTEN]
--max-receive-bytes bytes 4MiB server.max_receive_bytes YSEARCH_SERVER_MAX_RECEIVE_BYTES Maximum gRPC request bytes (at least 64KiB). [server.max_receive_bytes; env YSEARCH_SERVER_MAX_RECEIVE_BYTES] (at least 64KiB)
--max-send-bytes bytes 4MiB server.max_send_bytes YSEARCH_SERVER_MAX_SEND_BYTES Maximum gRPC response bytes (at least 64KiB, one result frame). [server.max_send_bytes; env YSEARCH_SERVER_MAX_SEND_BYTES] (at least 64KiB)
--node-id string server.node_id YSEARCH_SERVER_NODE_ID Node identity; empty defaults to the bound listen address. [server.node_id; env YSEARCH_SERVER_NODE_ID]
--server-query-receive-memory-bytes bytes 320MiB server.query_receive_memory_bytes YSEARCH_SERVER_QUERY_RECEIVE_MEMORY_BYTES Shared request-memory allowance for Search, worker Execute, Merge, ValidateQuery, CacheState and FetchStored. Holds decoded requests until RPC completion. Receive workspace waits at most 5ms in a 64-call queue; decoded admission refuses immediately when full. Must fit three receive buffers (each at least 1MiB) plus decoded requests. Unary wire reception/decompression and HTTP/2 queues precede this admission. Sized for at least 24 concurrent full server.max_receive_bytes receives at its default (24 * 13,238,272 bytes = 303MiB, plus headroom): fewer than that admits only a handful of concurrent requests before refusing the rest with ResourceExhausted. [server.query_receive_memory_bytes; env YSEARCH_SERVER_QUERY_RECEIVE_MEMORY_BYTES] (positive)
--read-only bool false server.read_only YSEARCH_SERVER_READ_ONLY Refuse every object-store write for this process's lifetime. Enforced at the single store every writer resolves through, so background work that never touches an RPC - garbage collection, and the GC dry run, which writes a proposal object before any delete decision - fails closed too. Startup-fixed on purpose: a running process cannot be talked into writing by a config reload. [server.read_only; env YSEARCH_SERVER_READ_ONLY]
--roles list server.roles YSEARCH_SERVER_ROLES Comma-separated roles for node: coordinator, worker, merger, aggregator, compactor, router, builder, publisher. [server.roles; env YSEARCH_SERVER_ROLES]
--server-source-management enum loopback server.source_management YSEARCH_SERVER_SOURCE_MANAGEMENT Which peers may call SourceService, the RPC that changes which datasets this process reads. loopback answers only a caller that reached this process without crossing a network, so binding server.listen to every interface still does not expose it. any serves it to anything that can reach the port: the service has no authentication of its own, so choose it only where something in front of the address authenticates. [server.source_management; env YSEARCH_SERVER_SOURCE_MANAGEMENT] (one of loopback, any)
--zone string default server.zone YSEARCH_SERVER_ZONE Placement zone this node advertises in. [server.zone; env YSEARCH_SERVER_ZONE]
--stats-dir string stats.dir YSEARCH_STATS_DIR Directory of this node's statistics databases, one per index (spec/50 §3.3). Every served generation is folded into it and its epoch pinned for bm25f-pinned-v1. Unlike cache.dir it is not disposable: a node that loses it refolds from the catalog. Empty disables epochs. A relative value in a configuration file resolves against that file's directory. [stats.dir; env YSEARCH_STATS_DIR]
--stats-history-generations int 6000 stats.history_generations YSEARCH_STATS_HISTORY_GENERATIONS Generations of per-generation statistics deltas kept to reconstruct an earlier epoch (spec/50 §3.3); about two days at one generation per 30 s. [stats.history_generations; env YSEARCH_STATS_HISTORY_GENERATIONS] (in [1, 16777216])
--hydrate-throughput-floor bytes 128MiB storage.hydrate_throughput_floor YSEARCH_STORAGE_HYDRATE_THROUGHPUT_FLOOR Lowest believable hydration throughput in bytes per second. [storage.hydrate_throughput_floor; env YSEARCH_STORAGE_HYDRATE_THROUGHPUT_FLOOR] (positive)
--hysteresis-denominator int 4 storage.hysteresis_denominator YSEARCH_STORAGE_HYSTERESIS_DENOMINATOR Hysteresis denominator; must be below the numerator. [storage.hysteresis_denominator; env YSEARCH_STORAGE_HYSTERESIS_DENOMINATOR] (in [1, 4294967295])
--hysteresis-numerator int 5 storage.hysteresis_numerator YSEARCH_STORAGE_HYSTERESIS_NUMERATOR Hysteresis numerator; must exceed the denominator. [storage.hysteresis_numerator; env YSEARCH_STORAGE_HYSTERESIS_NUMERATOR] (in [1, 4294967295])
--mode enum AUTO storage.mode YSEARCH_STORAGE_MODE Storage mode: HYDRATE_FULL, REMOTE_BLOCKS, or AUTO (AUTO is refused by the checkpoint-bound serve). [storage.mode; env YSEARCH_STORAGE_MODE] (one of HYDRATE_FULL, REMOTE_BLOCKS, AUTO)
--remote-latency-floor duration 2ms storage.remote_latency_floor YSEARCH_STORAGE_REMOTE_LATENCY_FLOOR Lowest believable remote request latency. [storage.remote_latency_floor; env YSEARCH_STORAGE_REMOTE_LATENCY_FLOOR] (positive)
--remote-throughput-floor bytes 64MiB storage.remote_throughput_floor YSEARCH_STORAGE_REMOTE_THROUGHPUT_FLOOR Lowest believable remote throughput in bytes per second. [storage.remote_throughput_floor; env YSEARCH_STORAGE_REMOTE_THROUGHPUT_FLOOR] (positive)
--scan-threshold-permille int 200 storage.scan_threshold_permille YSEARCH_STORAGE_SCAN_THRESHOLD_PERMILLE Scan fraction above which AUTO hydrates. [storage.scan_threshold_permille; env YSEARCH_STORAGE_SCAN_THRESHOLD_PERMILLE] (in [0, 1000])
--tail-impact-enabled bool false tail.impact_enabled YSEARCH_TAIL_IMPACT_ENABLED Persist score-banded approximate-tail data. Disabled avoids two additional records per unique term when exact tails are sufficient. [tail.impact_enabled; env YSEARCH_TAIL_IMPACT_ENABLED]
--tail-impact-maximum-bands int 64 tail.impact_maximum_bands YSEARCH_TAIL_IMPACT_MAXIMUM_BANDS Maximum score-impact bands persisted for one term. [tail.impact_maximum_bands; env YSEARCH_TAIL_IMPACT_MAXIMUM_BANDS] (in [1, 4096])
--tail-impact-target-documents int 4096 tail.impact_target_documents YSEARCH_TAIL_IMPACT_TARGET_DOCUMENTS Target postings per persisted impact run. [tail.impact_target_documents; env YSEARCH_TAIL_IMPACT_TARGET_DOCUMENTS] (in [1, 4294967295])
--tail-maximum-ordering-error float 0 tail.maximum_ordering_error YSEARCH_TAIL_MAXIMUM_ORDERING_ERROR Server ceiling on accepted approximate-tail score-ordering error; zero requires exact fallback. [tail.maximum_ordering_error; env YSEARCH_TAIL_MAXIMUM_ORDERING_ERROR] (in [0, 1.7976931348623157e+308])
--vector-bits-per-code int 8 vector.bits_per_code YSEARCH_VECTOR_BITS_PER_CODE Bits in each PQ subquantizer code. [vector.bits_per_code; env YSEARCH_VECTOR_BITS_PER_CODE] (in [1, 8])
--vector-build-memory-bytes bytes 0 vector.build_memory_bytes YSEARCH_VECTOR_BUILD_MEMORY_BYTES Per-build vector training and assignment memory; zero derives a bounded share of builder.sort_memory_bytes. [vector.build_memory_bytes; env YSEARCH_VECTOR_BUILD_MEMORY_BYTES] (at least 0)
--vector-centroids int 0 vector.centroids YSEARCH_VECTOR_CENTROIDS IVF coarse centroid count; zero derives it deterministically from population. [vector.centroids; env YSEARCH_VECTOR_CENTROIDS] (in [0, 4294967295])
--vector-flat-threshold int 10000 vector.flat_threshold YSEARCH_VECTOR_FLAT_THRESHOLD Per-segment vector population at or below which candidate generation stays exhaustive. [vector.flat_threshold; env YSEARCH_VECTOR_FLAT_THRESHOLD] (in [1, 4294967295])
--vector-list-block-documents int 0 vector.list_block_documents YSEARCH_VECTOR_LIST_BLOCK_DOCUMENTS Target vector entries per IVF-list block; zero follows the document-value block geometry. [vector.list_block_documents; env YSEARCH_VECTOR_LIST_BLOCK_DOCUMENTS] (in [0, 4294967295])
--vector-maximum-training-vectors int 4096 vector.maximum_training_vectors YSEARCH_VECTOR_MAXIMUM_TRAINING_VECTORS Maximum deterministic training samples retained per vector field. [vector.maximum_training_vectors; env YSEARCH_VECTOR_MAXIMUM_TRAINING_VECTORS] (in [1, 4294967295])
--vector-query-candidate-multiplier int 10 vector.query_candidate_multiplier YSEARCH_VECTOR_QUERY_CANDIDATE_MULTIPLIER Default ANN candidates as a multiple of requested top_k. [vector.query_candidate_multiplier; env YSEARCH_VECTOR_QUERY_CANDIDATE_MULTIPLIER] (in [1, 10000])
--vector-query-default-probes int 8 vector.query_default_probes YSEARCH_VECTOR_QUERY_DEFAULT_PROBES IVF coarse lists probed when a vector request does not specify probes. [vector.query_default_probes; env YSEARCH_VECTOR_QUERY_DEFAULT_PROBES] (in [1, 4294967295])
--vector-query-maximum-candidates int 100000 vector.query_maximum_candidates YSEARCH_VECTOR_QUERY_MAXIMUM_CANDIDATES Hard per-request candidate generation and exact-rerank cap. [vector.query_maximum_candidates; env YSEARCH_VECTOR_QUERY_MAXIMUM_CANDIDATES] (in [1, 4294967295])
--vector-query-maximum-probes int 1024 vector.query_maximum_probes YSEARCH_VECTOR_QUERY_MAXIMUM_PROBES Hard per-request IVF probe cap. [vector.query_maximum_probes; env YSEARCH_VECTOR_QUERY_MAXIMUM_PROBES] (in [1, 4294967295])
--vector-query-memory-bytes bytes 256MiB vector.query_memory_bytes YSEARCH_VECTOR_QUERY_MEMORY_BYTES Per-process admission budget for decoded vector-query working sets. [vector.query_memory_bytes; env YSEARCH_VECTOR_QUERY_MEMORY_BYTES] (positive)
--vector-subquantizers int 0 vector.subquantizers YSEARCH_VECTOR_SUBQUANTIZERS PQ subquantizer count; zero derives a geometry compatible with each vector field's dimension. [vector.subquantizers; env YSEARCH_VECTOR_SUBQUANTIZERS] (in [0, 4294967295])
--vector-training-iterations int 20 vector.training_iterations YSEARCH_VECTOR_TRAINING_ITERATIONS Maximum deterministic k-means iterations during IVF-PQ training. [vector.training_iterations; env YSEARCH_VECTOR_TRAINING_ITERATIONS] (in [1, 1000])
--checkpoint string checkpoint key prefix: serve one checkpoint directly (the Milestone 1 form) instead of the catalog

ysearch sources

ysearch sources [flags]

sources talks to the SourceService of the server --server names. A source is one read-only S3 dataset the server attaches: list and get read the registry, add and replace declare one, remove withdraws it, and watch streams changes.

Only locally managed sources can be changed here. A source declared in a configuration file is reapplied on every reload, so changing it through the API would be silently reverted; those answer FAILED_PRECONDITION naming their owner.

Every change is a compare-and-swap against the registry revision. By default each verb reads the current revision and pins itself to it, so a change racing another one is refused with ABORTED rather than clobbering it; --revision pins explicitly. Source management answers only a local caller unless the server sets server.source_management=any.

Subcommands:

ysearch sources add

ysearch sources add <source-id> <url> [flags]

add declares a source that does not exist yet. Use replace to change one that does: keeping them apart means a typo in the id adds a second source instead of silently repointing the first.

Flags:

Flag Type Default Key Env Description
--anonymous bool false sign no requests, for a bucket that grants public reads
--endpoint string S3 endpoint host:port for this source
--path-style bool false use path-style addressing for this source
--persist bool false keep this source across a restart; needs an installation directory
--profile string shared-credentials profile for this source alone
--region string S3 region for this source
--revision uint64 0 registry revision to pin the change to (default: the revision read now)
--use-ssl bool true use TLS to this source's endpoint

ysearch sources get

ysearch sources get <source-id> [flags]

Show one declared source in full

ysearch sources list

ysearch sources list [flags]

List every declared source with its state

ysearch sources remove

ysearch sources remove <source-id> [flags]

Withdraw a source and drain its in-flight reads

Flags:

Flag Type Default Key Env Description
--revision uint64 0 registry revision to pin the removal to (default: the revision read now)

ysearch sources replace

ysearch sources replace <source-id> <url> [flags]

replace supplies the whole declaration, not a patch: every flag left unset takes its default rather than the stored value. A source that is serving keeps serving until the replacement is ready.

Flags:

Flag Type Default Key Env Description
--anonymous bool false sign no requests, for a bucket that grants public reads
--endpoint string S3 endpoint host:port for this source
--path-style bool false use path-style addressing for this source
--persist bool false keep this source across a restart; needs an installation directory
--profile string shared-credentials profile for this source alone
--region string S3 region for this source
--revision uint64 0 registry revision to pin the change to (default: the revision read now)
--use-ssl bool true use TLS to this source's endpoint

ysearch sources watch

ysearch sources watch [flags]

Stream the registry: a snapshot, then every change

ysearch stats

ysearch stats [<index>] [flags]

stats with no argument lists every index the server's object root holds with the totals of its active generation; an index the node does not serve is named as such rather than shown empty. stats <index> prints the detail and the head of the generation lineage. --node prints one frame of the node's own metrics; --fleet one assembled fleet frame with its completeness block (unreachable nodes are named, never zero-filled). --watch keeps the view live until q or ctrl-c.

Example:

ysearch stats
ysearch stats articles --json
ysearch stats --node --watch

Flags:

Flag Type Default Key Env Description
--fleet bool false the fleet's assembled metrics and completeness (one StreamFleetStats frame)
--generation string describe this catalog generation instead of the latest (with <index>)
--node bool false this node's metrics (one StreamNodeStats frame)
--watch bool false redraw on every frame until q or ctrl-c

ysearch version

ysearch version [flags]

Print the build identity (the same line as --version)

Settings

Every setting, from the catalog. Precedence: default < file < environment < flag < runtime API (config set). Scope says what may change at runtime: only node-runtime keys accept config set. A secret is read from the environment only. A compile-time setting is display-only.

Key Kind Scope Default Env Flag Constraint Description
builder.analysis_batch_documents int64 node-runtime 0 YSEARCH_BUILDER_ANALYSIS_BATCH_DOCUMENTS --analysis-batch-documents (--builder-analysis-batch-documents) in [0, 1.048576e+06] Documents handed to the analysis workers at once; zero derives sixteen per worker. A batch is the unit that must fit in builder.analysis_memory_bytes, so a larger batch amortises the hand-off over more documents but raises the peak the analysis share has to cover. The derived shape is what the resource gates measure, so a corpus of unusually large documents is the case for lowering it rather than raising the share.
builder.analysis_memory_bytes bytes size node-runtime 0 YSEARCH_BUILDER_ANALYSIS_MEMORY_BYTES --analysis-memory-bytes (--builder-analysis-memory-bytes) The share of builder.sort_memory_bytes reserved for documents in flight through analysis; zero derives a thirty-second of it, clamped to between 64KiB and 32MiB and never more than a quarter of what the public-id and docvalue spools leave. It only has to hold one bounded batch, and a document larger than the whole share still runs alone under builder.max_document_working_bytes, so the derived value is deliberately small: a three-million-document run peaked at 9.5MiB against a 128MiB share. Every byte reserved here is a byte the term partitions cannot use, which raises mini-run count and write amplification directly, so raise it only against a measured analysis stall.
builder.analysis_workers int64 node-runtime 0 YSEARCH_BUILDER_ANALYSIS_WORKERS --analysis-workers (--builder-analysis-workers) in [0, 1024] Goroutines tokenising documents inside one build; zero derives GOMAXPROCS, bounded at eight. Analysis is the one parallel stage of an otherwise serial build, so this is per build and multiplies with builder.build_concurrency: the bound exists because a dozen builds each spawning a worker per core oversubscribes the machine and the scheduler churn costs more than the parallelism returns. Output order is preserved regardless of this value.
builder.blob_dictionary bool node-runtime true YSEARCH_BUILDER_BLOB_DICTIONARY --blob-dictionary (--builder-blob-dictionary) Compress stored values against a dictionary trained per blob file. Denser on homogeneous documents, but the dictionary is coded on the build's hottest path, so turning it off trades index size for ingest throughput.
builder.blob_zstd_level int64 node-runtime 0 YSEARCH_BUILDER_BLOB_ZSTD_LEVEL --blob-zstd-level (--builder-blob-zstd-level) in [0, 11] Zstd effort for stored values; zero keeps the storage default (7). Levels select different zstd encoders, and a build re-seeds the chosen encoder from the blob dictionary for every value it writes, so a lower level cuts build cost by more than the usual level trade-off suggests.
builder.build_concurrency int64 node-runtime 0 YSEARCH_BUILDER_BUILD_CONCURRENCY --build-concurrency (--builder-build-concurrency) in [0, 256] Sealed builds that may run through the seal pipeline at once; zero derives one per CPU, bounded at twelve. One build is a mostly single-threaded chain of analysis, sort, merge and publish that spends about half its time blocked in file system calls, so a single pipeline leaves a multi-core host idle under bulk ingest. The builder's memory is this number times builder.sort_memory_bytes.
builder.catalog_publication enum startup direct YSEARCH_BUILDER_CATALOG_PUBLICATION builder-catalog-publication one of direct, external Catalog writer mode: direct is the single-node compatibility path; external stops after commit markers for a lease-fenced publisher role.
builder.dedupe_index_identities int64 node-runtime 0 YSEARCH_BUILDER_DEDUPE_INDEX_IDENTITIES --dedupe-index-identities (--builder-dedupe-index-identities) in [0, 2.68435456e+08] Distinct document keys one open spool's dedupe index may hold before it gives up; zero derives 1048576. A key spooled twice in one build must resolve to a single winner before analysis, or the segment's dense ordinal assignment refuses the build outright. The index resolves that as records are spooled, from identities ingest already holds, and the sealed build inherits the answer; past this bound it gives up and the build reconstructs the answer by reading its spool twice instead — slower, and exactly as correct. One entry is a 16-byte key hash, a sequence and a mutation version, so the default bounds it at tens of megabytes per open spool and is reached only by a spool of a million very small documents. Lower it to cap that memory on a host running many indexes; set it to one to force the scanning path.
builder.fuse_memory_bytes bytes size startup 0 YSEARCH_BUILDER_FUSE_MEMORY_BYTES builder-fuse-memory-bytes in [0, 6.7108864e+07] Optional binary fuse construction workspace during publication; zero retains Bloom, failed admission retains Bloom.
builder.lexical_blob_threshold bytes size node-runtime 0 YSEARCH_BUILDER_LEXICAL_BLOB_THRESHOLD --lexical-blob-threshold (--builder-lexical-blob-threshold) Size at which a posting, position or term value is stored in a blob file rather than inline in the key log; zero keeps the default of 64KiB. Separation exists to spare compaction from rewriting large values, and a segment is never compacted — while a blob value is compressed on its own, which with a trained dictionary costs an encoder reset each time, where key-log blocks are compressed in bulk.
builder.lexical_block_compression string node-runtime YSEARCH_BUILDER_LEXICAL_BLOCK_COMPRESSION --lexical-block-compression (--builder-lexical-block-compression) Codec the term, posting and position families compress key-log blocks with: none, snappy, lz4, lz4fast or zstd. Empty keeps the default of lz4. The storage engine's own default leaves the newest level raw because it expects compaction to rewrite it, and a segment stays where it lands. On 300k articles lz4 measured 24.8s and 1.0GB against zstd's 37.7s and 0.8GB, so zstd is the choice for an index written once and read for a long time.
builder.max_document_working_bytes bytes size node-runtime 0 YSEARCH_BUILDER_MAX_DOCUMENT_WORKING_BYTES --max-document-working-bytes (--builder-max-document-working-bytes) What one document may use beyond the partition budget while it is being analyzed; zero derives 64MiB, or a quarter of builder.sort_memory_bytes when that is smaller. It is an allowance rather than a reservation: it exists so a single document larger than the whole partition budget still builds, alone, instead of failing the build. Only a corpus with documents in the hundreds of megabytes needs it raised.
builder.max_queued_builds int64 node-runtime 0 YSEARCH_BUILDER_MAX_QUEUED_BUILDS --max-queued-builds (--builder-max-queued-builds) in [0, 1024] Sealed builds that may wait for or run in the pipeline before ingest is throttled; zero derives it as four times builder.build_concurrency. A waiting build is a spool on disk, not memory, so this buys ingest room across a build at the cost of ingest.seal_bytes of disk per queued build. Set too low, a bulk load throttles the moment the pipeline is full and ingest runs at the speed of the builds rather than ahead of them.
builder.max_token_bytes bytes size node-runtime 64KiB YSEARCH_BUILDER_MAX_TOKEN_BYTES --max-token-bytes (--builder-max-token-bytes) positive Maximum analyzed token size. A build must be able to admit one token this large, so raising it raises the floor under builder.sort_memory_bytes: a budget that leaves the partitions less than one maximum term refuses to start rather than fail partway. Tokens are words after analysis, so the default is already far past any natural language; raise it only for a corpus with genuinely enormous unbroken tokens.
builder.merge_fan_in int64 node-runtime 0 YSEARCH_BUILDER_MERGE_FAN_IN --merge-fan-in (--builder-merge-fan-in) in [0, 64] Spilled mini-runs merged in one pass; zero derives as many as an eighth of the partition budget affords at builder.run_page_bytes plus 512 bytes of reader overhead each, capped at sixty-four and floored at two. Runs beyond the fan-in need further passes, and each pass rewrites the data, so a low fan-in on a build that spilled heavily shows up as write amplification rather than as a slow merge. The cap is a file-descriptor and seek-pattern bound, not a memory one.
builder.output_format_version uint64 startup 0 YSEARCH_BUILDER_OUTPUT_FORMAT_VERSION builder-output-format-version in [0, 8] Segment format emitted by builders and compactors; zero selects this binary's current format. The Kubernetes operator pins this to compatibility.writeFormat on writer roles during staged rollouts. Format 7 adds the stats, forward and hashterms families (spec/40 §3); a compaction that includes an older input writes format 6 instead, unless 7 or 8 is set explicitly, which refuses such a merge. Format 8 adds the exact vector lanes (canonical fp16 vectors, IVF radii, int8 codes; spec/40 §3.2).
builder.paged_posting_directory bool startup false YSEARCH_BUILDER_PAGED_POSTING_DIRECTORY builder-paged-posting-directory Experimental commit-anchored posting-directory pages under query admission and the existing persistent block quota; old commits retain canonical traversal.
builder.posting_partition_penalty float64 node-runtime 4 YSEARCH_BUILDER_POSTING_PARTITION_PENALTY builder-posting-partition-penalty in [0, 1e+06] Metadata cost penalty for experimental variable posting blocks.
builder.publish_verification string node-runtime checksum YSEARCH_BUILDER_PUBLISH_VERIFICATION --publish-verification (--builder-publish-verification) How a published segment is confirmed: checksum, sample, or readback. checksum compares every object against the checksum the object store reported when it accepted the write — S3 computes that server-side and refuses a mismatched write, and the filesystem backend computes it over the bytes it synced — so nothing is transferred back. sample additionally opens the published copy through the store and queries it; readback additionally fetches every object and rehashes it, which costs the whole index again on every build. An object whose store reports no comparable checksum is read back in any mode.
builder.routing_fragment_bytes bytes size node-runtime 8MiB YSEARCH_BUILDER_ROUTING_FRAGMENT_BYTES builder-routing-fragment-bytes in [0, 6.7108864e+07] Optional routing fragment disk cap per built segment; zero disables, overflow keeps canonical scan fallback. Matches compaction.routing_fragment_bytes so a freshly ingested segment and a freshly compacted one carry a fragment the same way; the two stay separate keys because a background merge must be able to carry its own share independent of foreground ingestion. Only a segment built or compacted after this defaulted on carries a fragment: nothing rewrites an existing one, so a settled corpus stays without a routing table until it is backfilled or compacted.
builder.run_page_bytes bytes size node-runtime 0 YSEARCH_BUILDER_RUN_PAGE_BYTES --run-page-bytes (--builder-run-page-bytes) Read-ahead page one spilled mini-run is buffered in during the final merge; zero derives 64KiB, shrinking it when an eighth of the partition budget cannot hold sixty-four such pages. The merge reserves one page per open run for the whole build, so page size and builder.merge_fan_in trade against each other out of the same share: the derivation prefers smaller pages to a fan-in that would force extra compaction passes on a hot partition.
builder.sort_memory_bytes bytes size node-runtime 256MiB YSEARCH_BUILDER_SORT_MEMORY_BYTES --sort-memory-bytes (--builder-sort-memory-bytes) positive Total accounted builder budget: one ledger every part of a build charges against, and the setting the other builder budgets derive from. It divides into the public-id sorter (an eighth, capped at 64MiB), the docvalue and lane spools (their own worst case), the analysis share (builder.analysis_memory_bytes), and the term partitions, which take what is left. A build that cannot hold its partitions in that remainder spills to disk instead, which is the difference between one file per build and hundreds. This is per build, so the builder's memory is builder.build_concurrency times this.
builder.spill_merge_strategy enum node-runtime heap YSEARCH_BUILDER_SPILL_MERGE_STRATEGY builder-spill-merge-strategy one of heap, replace-root, loser-tree Experimental spill merge selector; heap preserves the baseline until full ingestion benchmarks qualify an alternative.
builder.startup_bundles bool startup false YSEARCH_BUILDER_STARTUP_BUNDLES builder-startup-bundles Asynchronously produce optional startup bundles after catalog publication. One bounded coalescing worker per publishing host; oversized sources retain canonical startup fallback.
builder.term_dictionary_block_size bytes size node-runtime 64KiB YSEARCH_BUILDER_TERM_DICTIONARY_BLOCK_SIZE --term-dictionary-block-size (--builder-term-dictionary-block-size) positive Key-log data-block size of the term family. A table carries one block-index entry per data block and reads the whole index before it can look up anything, so this is what a cold term probe pays to find out where to look. At the storage engine's 4KiB an 8MiB dictionary needs about two thousand entries — around 65KiB read on every open of the table and held in memory until it closes — where 64KiB needs about a sixteenth of that. A probe then reads a bigger block, which over an object store is the cheap side of the trade: measured on a 6MiB dictionary the block cost 17KiB compressed against the 48KiB of index it replaced.
builder.term_dictionary_bloom bool node-runtime false YSEARCH_BUILDER_TERM_DICTIONARY_BLOOM --term-dictionary-bloom (--builder-term-dictionary-bloom) Write a Bloom filter over the term family. The storage engine writes one by default; a segment does not, because the filter is read whole — uncompressed, about 1.2 bytes per distinct term — when the table is opened, and kept in memory until it closes, to save at most one block read per probe against the single table a published segment's term family is. Measured, it was nine tenths of what a cold term probe read: 431KiB of 481KiB on a 360k-term dictionary. A table says in its own footer whether it has a filter, so segments published with one keep working unchanged.
builder.variable_posting_blocks bool node-runtime false YSEARCH_BUILDER_VARIABLE_POSTING_BLOCKS builder-variable-posting-blocks Experimental bounded-window BM25F-aware posting partitioning for format5 or newer output.
cache.block_bytes bytes size startup 2GiB YSEARCH_CACHE_BLOCK_BYTES cache-block-bytes positive Disk, not memory: bytes of verified decoded blocks this node may keep under cache.dir. This is the cheap half of block caching - it converts a remote read into a local one and costs no resident memory - so size it above one query's working set and leave cache.read_block_bytes to decide what stays in RAM. Measured on 294 segments: one single-term query touched about 363MB of term dictionary and postings and one scored two-term query about 544MB, so the default holds several such working sets.
cache.block_entries int64 startup 1048576 YSEARCH_CACHE_BLOCK_ENTRIES cache-block-entries positive Disk, not memory: how many decoded blocks cache.block_bytes may be spread over. Whichever is reached first bounds the cache; at the measured ~237KiB per block the byte budget is reached first by a wide margin.
cache.dir string startup YSEARCH_CACHE_DIR cache-dir Disposable local cache directory; symlinks in its path are resolved once at startup and the cache then anchors there without following any. It holds the two disk budgets, cache.full_bytes and cache.block_bytes, and nothing else: every other cache.* budget is process memory. A relative value in a configuration file resolves against that file's directory.
cache.full_bytes bytes size startup 8GiB YSEARCH_CACHE_FULL_BYTES cache-full-bytes positive Disk, not memory: bytes of whole hydrated segments this node may keep under cache.dir. With cache.block_bytes it is the whole of what the cache directory holds, so a volume smaller than their sum is an eviction loop the node cannot see.
cache.full_entries int64 startup 1024 YSEARCH_CACHE_FULL_ENTRIES cache-full-entries positive Disk, not memory: how many whole segments cache.full_bytes may be spread over. Whichever of the two is reached first bounds the cache, and at the measured ~340MB per segment of a 101GB corpus the byte budget is reached long before this one.
cache.lazy_readers bool startup false YSEARCH_CACHE_LAZY_READERS cache-lazy-readers Experimental: acquire immutable segment readers and optional filter payloads only for admitted query work. It is also what routing.presence_bytes zero means: eager readers read that zero as unbounded and hold every segment's presence filter, lazy readers read it as none.
cache.ordinal_bytes bytes size startup 64MiB YSEARCH_CACHE_ORDINAL_BYTES cache-ordinal-bytes in [327936, 9.223372036854776e+18] Memory, not disk: process-wide verified ordinal-ID pages, including pinned pages and in-flight page workspace, shared across indexes and generations.
cache.read_block_bytes bytes size startup 512MiB YSEARCH_CACHE_READ_BLOCK_BYTES cache-read-block-bytes positive Memory, not disk: process-wide decoded WavesDB blocks held resident and shared by immutable segment readers. This is what decides whether a repeated query is warm. Below one query's working set nothing survives to the next query and every query pays the cold price forever: measured on 294 segments, a 24MiB budget re-read all 214MB of term dictionary byte for byte while about 495MiB answered the repeat from 57,421 bytes. Size it from the corpus, not the machine, and from the layout its segments were built with. A term probe costs the same whatever the term, but what it costs depends on whether the segment carries a term-family Bloom filter: about 711KiB per segment on one built before builder.term_dictionary_bloom defaulted to false, and about 21KiB on one built since. The term dictionary alone needs segments times that figure, and the postings of the shape you serve come on top. A mixed corpus pays the older rate for the segments it has not rebuilt.
cache.read_open_files int64 startup 1024 YSEARCH_CACHE_READ_OPEN_FILES cache-read-open-files positive Process-wide open WavesDB table-file limit shared by immutable segment readers. It spends file descriptors, and nothing here raises RLIMIT_NOFILE, so the default sits at the 1024 a Linux process is commonly given with the listeners, connections and cache files sharing it: raise the limit before raising this.
cache.read_open_readers int64 startup 4098 YSEARCH_CACHE_READ_OPEN_READERS cache-read-open-readers positive Process-wide decoded WavesDB table-reader count limit. Each retained reader also spends cache.read_reader_bytes, and whichever of the two is reached first evicts. Unlike cache.read_open_files this spends memory rather than file descriptors, so size it from the corpus: a segment holds several table families, and a count that cannot hold them all evicts on every query. At 1024, which this defaulted to, a 253-segment corpus needing 1,321 readers held 77% of them and evicted 3,022 times while its byte budget sat 89% empty. That is dearer than it sounds, because a re-opened table re-reads its Bloom filter and block index through a path that bypasses the block cache, so the eviction becomes object reads that repeat for the life of the node: the same query read 62.8MiB in 275 range GETs every time it ran, and none after the count was raised. Measured resident cost is about 760KiB per reader.
cache.read_reader_bytes bytes size startup 2GiB YSEARCH_CACHE_READ_READER_BYTES cache-read-reader-bytes positive Memory, not disk: process-wide WavesDB table index and Bloom-filter bytes held resident. It is the only cache budget that scales with the corpus rather than with the query, at a measured ~14MiB per segment, so the default holds roughly 145 segments and a larger corpus re-reads a segment's index after evicting it.
cache.result_enabled bool startup true YSEARCH_CACHE_RESULT_ENABLED cache-result-enabled Cache generation and segment ranked results. Disable for execution benchmarks; reader, block, plan and stored-field caches remain active.
cache.scrub_entries int64 node-runtime 4 YSEARCH_CACHE_SCRUB_ENTRIES cache-scrub-entries in [0, 4096] How many cached full segments one scrubber pass re-reads. The scrubber walks the cache in a rotating cursor, so this and cache.scrub_interval together set how long a full sweep takes and how much disk read it costs; a segment currently in use is skipped and picked up on a later pass.
cache.scrub_interval duration node-runtime 5m YSEARCH_CACHE_SCRUB_INTERVAL cache-scrub-interval in [0, 8.64e+13] How often the background scrubber re-reads cached full segments and checks them against their commits. This is the only thing that notices a cached segment rotting on disk: a query trusts an entry once it has been verified, because verifying costs the SHA-256 of every file in the segment and doing that per request made a many-segment query spend all its time hashing. Zero disables the scrubber, which leaves local corruption undetected until the entry is evicted.
cache.sidecar_bytes bytes size startup 8MiB YSEARCH_CACHE_SIDECAR_BYTES cache-sidecar-bytes positive Memory, not disk: process-wide optional exact-filter decoder and lease budget shared across indexes and generations.
cache.warm_prefetch_bytes bytes size startup 0 YSEARCH_CACHE_WARM_PREFETCH_BYTES cache-warm-prefetch-bytes in [0, 6.7108864e+07] Optional routing-page warmup byte limit after the first served query; zero disables. It spends object reads, not a budget of its own: warmed pages land in the process-wide routing page cache and are charged there, so warming past that cache only evicts what it just read. Shares foreground cache admission.
cache.warm_prefetch_concurrency int64 startup 1 YSEARCH_CACHE_WARM_PREFETCH_CONCURRENCY cache-warm-prefetch-concurrency in [0, 8] Maximum concurrent optional warmup reads; foreground queries stop new prefetch work.
client.json bool startup false YSEARCH_CLIENT_JSON --json (--client-json) Emit JSON instead of tables.
client.quiet bool startup false YSEARCH_CLIENT_QUIET --quiet (--client-quiet) Suppress progress output.
client.server string startup 127.0.0.1:9500 YSEARCH_CLIENT_SERVER --server (--client-server) Server address the client verbs dial.
client.timeout duration startup 30s YSEARCH_CLIENT_TIMEOUT --timeout (--client-timeout) positive Client request timeout.
compaction.cluster_field string node-runtime YSEARCH_COMPACTION_CLUSTER_FIELD compaction-cluster-field Existing scalar filterable string used by cluster layout; an absent field preserves order.
compaction.document_layout enum node-runtime preserve YSEARCH_COMPACTION_DOCUMENT_LAYOUT compaction-document-layout one of preserve, cluster, bisection Experimental survivor layout for new compaction outputs; preserve retains input order, cluster groups scalar categories, bisection uses a bounded sampled term graph.
compaction.enabled bool node-runtime false YSEARCH_COMPACTION_ENABLED compaction-enabled Run single-flight size-tiered compaction inside all-in-one serve. The explicit node compactor role always runs; this switch stays off there and by default so production can isolate maintenance from query and ingest.
compaction.hydration_bytes bytes size node-runtime 8GiB YSEARCH_COMPACTION_HYDRATION_BYTES compaction-hydration-bytes positive Maximum downloaded checkpoint payload retained per compaction job across all input hydrations. Borrowed resident inputs are not copied or charged. Filesystem metadata, output database and spill space have separate accounting.
compaction.hydration_workers int64 node-runtime 2 YSEARCH_COMPACTION_HYDRATION_WORKERS compaction-hydration-workers in [1, 8] Maximum concurrent input hydrations per compaction job.
compaction.interval duration node-runtime 1m YSEARCH_COMPACTION_INTERVAL compaction-interval in [1e+09, 8.64e+13] Interval between background compaction eligibility checks. The all-in-one server waits for sustained ingest idleness; use the isolated compactor role when fan-out must be reduced during continuous ingest.
compaction.max_active_segments int64 node-runtime 16 YSEARCH_COMPACTION_MAX_ACTIVE_SEGMENTS compaction-max-active-segments positive Active searchable segment fan-out above which the scheduler may compact.
compaction.max_concurrent_jobs int64 startup 1 YSEARCH_COMPACTION_MAX_CONCURRENT_JOBS compaction-max-concurrent-jobs in [1, 8] Shared background compaction job slots across hosted indexes; physical merges are serial and hydration has a separate worker cap.
compaction.max_inputs int64 node-runtime 8 YSEARCH_COMPACTION_MAX_INPUTS compaction-max-inputs in [2, 64] Maximum inputs in one compaction job; jobs are always single-flight.
compaction.max_output_bytes bytes size node-runtime 4GiB YSEARCH_COMPACTION_MAX_OUTPUT_BYTES compaction-max-output-bytes positive Maximum summed input bytes admitted to one compaction proposal.
compaction.max_size_ratio uint64 node-runtime 2 YSEARCH_COMPACTION_MAX_SIZE_RATIO compaction-max-size-ratio in [1, 1024] Largest size ratio allowed between inputs in one tier.
compaction.min_inputs int64 node-runtime 4 YSEARCH_COMPACTION_MIN_INPUTS compaction-min-inputs in [2, 64] Minimum adjacent similarly-sized inputs in one compaction job.
compaction.read_inflight_bytes bytes size startup 64MiB YSEARCH_COMPACTION_READ_INFLIGHT_BYTES compaction-read-inflight-bytes in [65536, 9.223372036854776e+18] Shared compaction read payload bytes in flight across jobs. Open reads hold credits until Close; an individual object or range larger than this limit is rejected.
compaction.recent_delta int64 node-runtime 4 YSEARCH_COMPACTION_RECENT_DELTA compaction-recent-delta in [0, 1024] Newest active segments left uncompacted as an ingest delta tier.
compaction.reorder_leaf_documents uint64 node-runtime 128 YSEARCH_COMPACTION_REORDER_LEAF_DOCUMENTS compaction-reorder-leaf-documents in [1, 4096] Maximum stable leaf size of experimental balanced graph bisection; each document samples at most 32 terms.
compaction.reorder_temp_bytes bytes size node-runtime 1GiB YSEARCH_COMPACTION_REORDER_TEMP_BYTES compaction-reorder-temp-bytes positive Temporary spill cap for reordered outputs, including permutation, sampled graph and posting resort; the smaller compaction.temporary_bytes cap also applies.
compaction.routing_fragment_bytes bytes size node-runtime 8MiB YSEARCH_COMPACTION_ROUTING_FRAGMENT_BYTES compaction-routing-fragment-bytes in [0, 6.7108864e+07] Optional routing fragment disk cap per compaction output segment; overrides builder.routing_fragment_bytes for compaction's own merges so background maintenance can carry its own share independent of foreground ingestion. Zero disables fragment writing for compaction outputs, which also stops this generation's routing descriptor from ever completing until every one of its segments gets a fragment some other way.
compaction.routing_memory_bytes bytes size node-runtime 64MiB YSEARCH_COMPACTION_ROUTING_MEMORY_BYTES compaction-routing-memory-bytes in [6.7108864e+07, 9.223372036854776e+18] Memory budget for assembling one generation's routing descriptor from its member segments' own fragments after a successful compaction. Separate from compaction.temporary_bytes and the segment-merge budgets: descriptor assembly reads small fragment objects, not segment checkpoints.
compaction.routing_temp_bytes bytes size node-runtime 1GiB YSEARCH_COMPACTION_ROUTING_TEMP_BYTES compaction-routing-temp-bytes positive Spill and page-file budget for the same post-compaction descriptor assembly. Bounded well below compaction.temporary_bytes: a table assembled from fragments alone is the right-sized presence table (internal/routing/value.go), not the segments' own content.
compaction.target_bytes bytes size node-runtime 512MiB YSEARCH_COMPACTION_TARGET_BYTES compaction-target-bytes positive Approximate input-byte target for one size-tiered compaction group.
compaction.temporary_bytes bytes size node-runtime 8GiB YSEARCH_COMPACTION_TEMPORARY_BYTES compaction-temporary-bytes positive Shared accounted spill-byte limit within one physical compaction; hydration and final database storage are separate from spill space.
compaction.write_inflight_bytes bytes size startup 64MiB YSEARCH_COMPACTION_WRITE_INFLIGHT_BYTES compaction-write-inflight-bytes in [65536, 9.223372036854776e+18] Shared compaction write payload bytes in flight across jobs. An individual object larger than this limit is rejected before upload.
config.dir string startup YSEARCH_CONFIG_DIR config-dir Directory whose *.yml and *.yaml files are layered in name order after the default locations and before config.file.
config.file string startup YSEARCH_CONFIG_FILE --config (--config-file) YAML configuration file layered last among files.
config.search bool startup true YSEARCH_CONFIG_SEARCH config-search Probe /etc/ysearch, $XDG_CONFIG_HOME/ysearch (default ~/.config/ysearch), and ./ysearch.yml at startup.
config.watch bool startup true YSEARCH_CONFIG_WATCH config-watch Reload the discovered configuration files when any of them changes.
debug.pprof_listen string startup YSEARCH_DEBUG_PPROF_LISTEN --pprof-listen (--debug-pprof-listen) host:port serving net/http/pprof profiles; empty disables it. It binds before the catalog is opened, so a slow start can be profiled, and an address with no host (:6060) binds loopback. Profiles expose heap contents and goroutine stacks, so keep it on loopback and never on a public interface.
embedding.credential string startup YSEARCH_EMBEDDING_CREDENTIAL env only Optional external embedding bearer token. Environment only; reference YSEARCH_EMBEDDING_CREDENTIAL from an embedding profile.
embedding.external_allow_http bool startup false YSEARCH_EMBEDDING_EXTERNAL_ALLOW_HTTP embedding-external-allow-http Allow plain HTTP external embedding endpoints for trusted development networks.
embedding.external_batch_size int64 startup 128 YSEARCH_EMBEDDING_EXTERNAL_BATCH_SIZE embedding-external-batch-size in [1, 4096] Maximum texts in one external embedding request.
embedding.external_concurrency int64 startup 8 YSEARCH_EMBEDDING_EXTERNAL_CONCURRENCY embedding-external-concurrency in [1, 1024] Concurrent external embedding requests per process.
embedding.external_max_retry_after duration startup 2s YSEARCH_EMBEDDING_EXTERNAL_MAX_RETRY_AFTER embedding-external-max-retry-after positive Largest Retry-After delay accepted from an external embedding endpoint.
embedding.external_request_bytes bytes size startup 8MiB YSEARCH_EMBEDDING_EXTERNAL_REQUEST_BYTES embedding-external-request-bytes positive Maximum encoded external embedding request bytes.
embedding.external_response_bytes bytes size startup 64MiB YSEARCH_EMBEDDING_EXTERNAL_RESPONSE_BYTES embedding-external-response-bytes positive Maximum external embedding response bytes read before JSON decoding.
embedding.external_retries int64 startup 2 YSEARCH_EMBEDDING_EXTERNAL_RETRIES embedding-external-retries in [0, 16] Retry attempts after the first retryable external embedding failure.
embedding.external_retry_base duration startup 50ms YSEARCH_EMBEDDING_EXTERNAL_RETRY_BASE embedding-external-retry-base positive Base exponential backoff for external embedding retries.
embedding.external_timeout duration startup 30s YSEARCH_EMBEDDING_EXTERNAL_TIMEOUT embedding-external-timeout positive Deadline for one external embedding HTTP attempt sequence.
embedding.grpc_allow_insecure bool startup false YSEARCH_EMBEDDING_GRPC_ALLOW_INSECURE embedding-grpc-allow-insecure Allow plaintext gRPC embedding endpoints on trusted internal networks.
embedding.grpc_batch_size int64 startup 128 YSEARCH_EMBEDDING_GRPC_BATCH_SIZE embedding-grpc-batch-size in [1, 4096] Maximum texts in one gRPC embedding request.
embedding.grpc_concurrency int64 startup 8 YSEARCH_EMBEDDING_GRPC_CONCURRENCY embedding-grpc-concurrency in [1, 1024] Concurrent gRPC embedding requests per process.
embedding.grpc_request_bytes bytes size startup 8MiB YSEARCH_EMBEDDING_GRPC_REQUEST_BYTES embedding-grpc-request-bytes positive Maximum encoded gRPC embedding request bytes.
embedding.grpc_response_bytes bytes size startup 64MiB YSEARCH_EMBEDDING_GRPC_RESPONSE_BYTES embedding-grpc-response-bytes positive Maximum encoded gRPC embedding response bytes.
embedding.grpc_timeout duration startup 30s YSEARCH_EMBEDDING_GRPC_TIMEOUT embedding-grpc-timeout positive Deadline for one gRPC embedding request.
embedding.grpc_work_class enum startup auto YSEARCH_EMBEDDING_GRPC_WORK_CLASS embedding-grpc-work-class one of auto, interactive, bulk gRPC embedding scheduler class; auto maps queries to interactive and indexing to bulk.
embedding.internal_batch_size int64 startup 32 YSEARCH_EMBEDDING_INTERNAL_BATCH_SIZE embedding-internal-batch-size in [1, 4096] Maximum texts in one internal embedding batch.
embedding.internal_concurrency int64 startup 2 YSEARCH_EMBEDDING_INTERNAL_CONCURRENCY embedding-internal-concurrency in [1, 256] Concurrent pure-Go internal embedding batches.
embedding.internal_enabled bool startup false YSEARCH_EMBEDDING_INTERNAL_ENABLED embedding-internal-enabled Enable the pure-Go internal embedding runtime. Models remain lazy and checksum-pinned by schema profiles.
embedding.model_cache_bytes bytes size startup 10GiB YSEARCH_EMBEDDING_MODEL_CACHE_BYTES embedding-model-cache-bytes positive Maximum installed bytes in the internal-model cache.
embedding.model_cache_dir string startup YSEARCH_EMBEDDING_MODEL_CACHE_DIR embedding-model-cache-dir Checksum-addressed internal-model cache directory. Empty disables internal embedding; keep it distinct from cache.dir. A relative value in a configuration file resolves against that file's directory.
embedding.model_cache_packages int64 startup 8 YSEARCH_EMBEDDING_MODEL_CACHE_PACKAGES embedding-model-cache-packages positive Maximum installed internal-model packages.
embedding.model_download_allow_http bool startup false YSEARCH_EMBEDDING_MODEL_DOWNLOAD_ALLOW_HTTP embedding-model-download-allow-http Allow plain HTTP model manifests and artifacts for trusted development networks.
embedding.model_download_concurrency int64 startup 2 YSEARCH_EMBEDDING_MODEL_DOWNLOAD_CONCURRENCY embedding-model-download-concurrency in [1, 64] Concurrent verified internal-model artifact downloads.
embedding.model_download_timeout duration startup 10m YSEARCH_EMBEDDING_MODEL_DOWNLOAD_TIMEOUT embedding-model-download-timeout positive Deadline for one internal-model package installation.
embedding.model_manifest_bytes bytes size startup 1MiB YSEARCH_EMBEDDING_MODEL_MANIFEST_BYTES embedding-model-manifest-bytes positive Maximum bytes in one internal-model manifest.
embedding.model_package_bytes bytes size startup 4GiB YSEARCH_EMBEDDING_MODEL_PACKAGE_BYTES embedding-model-package-bytes positive Maximum bytes in one downloaded internal-model package.
embedding.query_cache_bytes bytes size startup 64MiB YSEARCH_EMBEDDING_QUERY_CACHE_BYTES embedding-query-cache-bytes positive Maximum float-vector bytes retained by the text-query embedding cache.
embedding.query_cache_entries int64 startup 4096 YSEARCH_EMBEDDING_QUERY_CACHE_ENTRIES embedding-query-cache-entries positive Maximum completed text-query embeddings retained per process.
embedding.query_cache_ttl duration startup 10m YSEARCH_EMBEDDING_QUERY_CACHE_TTL embedding-query-cache-ttl positive Lifetime of a completed text-query embedding cache entry.
fleet.admin_fleet string list startup YSEARCH_FLEET_ADMIN_FLEET --admin-fleet (--fleet-admin-fleet) Node host:port endpoints StreamFleetStats fans in; empty refuses the fleet stream.
fleet.aggregators string list startup YSEARCH_FLEET_AGGREGATORS --aggregators (--fleet-aggregators) Aggregator host:port endpoints.
fleet.builders string list startup YSEARCH_FLEET_BUILDERS --builders (--fleet-builders) Builder host:port endpoints the router polls and rendezvous-routes batches across.
fleet.fallback_workers string list startup YSEARCH_FLEET_FALLBACK_WORKERS --fallback-workers (--fleet-fallback-workers) Worker host:port endpoints: the dispatch universe and the cache-oblivious fallback.
fleet.fan_in int64 startup 0 YSEARCH_FLEET_FAN_IN --fan-in (--fleet-fan-in) in [0, 1024] Children per merge node in coordinator plans; zero means the planner's default, otherwise at least 2.
fleet.lane_slots string startup interactive=4,streaming=2 YSEARCH_FLEET_LANE_SLOTS --lane-slots (--fleet-lane-slots) Per-lane worker slot pools, lane=count.
fleet.mergers string list startup YSEARCH_FLEET_MERGERS --mergers (--fleet-mergers) Merge host:port endpoints: the coordinator's merge tier, a merger's delegation peers.
follower.discovery_interval duration node-runtime 30s YSEARCH_FOLLOWER_DISCOVERY_INTERVAL follower-discovery-interval positive How often a root attachment looks for indexes that appeared or disappeared under it. Separate from follower.poll_interval because discovery lists prefixes while a poll reads one pointer, and listing is the more expensive and more rate-limited of the two.
follower.generation_overlap duration node-runtime 30s YSEARCH_FOLLOWER_GENERATION_OVERLAP follower-generation-overlap positive Maximum age of an unseen catalog generation a worker may resolve on demand; in-flight references may keep its engine beyond this window.
follower.max_staleness duration node-runtime 5m YSEARCH_FOLLOWER_MAX_STALENESS follower-max-staleness positive How long a follower may go without confirming its installed generation is still current before it refuses new queries. A reader that cannot reach its source keeps answering from cached data indefinitely otherwise, which is worse than an error: the results look fine and are silently frozen. Status stays readable past this point so an operator can see why.
follower.poll_interval duration node-runtime 2s YSEARCH_FOLLOWER_POLL_INTERVAL follower-poll-interval positive How often a node checks the catalog's latest pointer.
follower.retired_generation_grace duration node-runtime 30s YSEARCH_FOLLOWER_RETIRED_GENERATION_GRACE follower-retired-generation-grace positive How long a superseded generation's engine stays open after its last query.
format.minimum_reader_version uint64 compile-time 1 Oldest segment format this binary reads.
format.segment_version uint64 compile-time 8 Segment format this binary writes.
gc.grace duration node-runtime 1h YSEARCH_GC_GRACE gc-grace in [0, 6.048e+14] Additional safety margin added to every scheduled GC reachability horizon.
gc.interval duration node-runtime 1h YSEARCH_GC_INTERVAL gc-interval in [1e+09, 6.048e+14] Cadence at which the dedicated compactor role writes a GC proposal and considers a quarantined older proposal.
gc.maximum_stream_lifetime duration node-runtime 1h YSEARCH_GC_MAXIMUM_STREAM_LIFETIME gc-maximum-stream-lifetime in [0, 6.048e+14] Longest query-stream lifetime protected when marking superseded generations for scheduled GC.
gc.minimum_upload_age duration node-runtime 24h YSEARCH_GC_MINIMUM_UPLOAD_AGE gc-minimum-upload-age in [0, 2.592e+15] Minimum age before an unreferenced object can enter a scheduled GC proposal.
gc.quarantine_age duration node-runtime 24h YSEARCH_GC_QUARANTINE_AGE gc-quarantine-age in [1e+09, 2.592e+15] Minimum age of an immutable GC proposal before a fresh mark may authorize deleting its still-unreachable exact keys.
gc.retained_generations int64 node-runtime 2 YSEARCH_GC_RETAINED_GENERATIONS gc-retained-generations in [0, 1024] Historical catalog generations retained in addition to latest during scheduled GC.
gc.sweep_enabled bool node-runtime false YSEARCH_GC_SWEEP_ENABLED gc-sweep-enabled Allow the dedicated compactor role to delete freshly re-proven orphan objects after an immutable dry-run proposal passes quarantine. Disabled by default.
index.generation string startup YSEARCH_INDEX_GENERATION --generation (--index-generation) Exact generation to pin, or empty for the latest.
index.name string startup YSEARCH_INDEX_NAME --index (--index-name) Index name for catalog-bound serve, worker, coordinator, compactor, router, builder, and publisher roles.
ingest.burst_documents int64 node-runtime 100000 YSEARCH_INGEST_BURST_DOCUMENTS ingest-burst-documents positive Per-index document tokens available for an ingest burst when the rate quota is enabled.
ingest.dir string startup YSEARCH_INGEST_DIR ingest-dir Ingest spool/build directory for builders and scratch root for the dedicated compactor role. A relative value in a configuration file resolves against that file's directory.
ingest.documents_per_second int64 node-runtime 0 YSEARCH_INGEST_DOCUMENTS_PER_SECOND ingest-documents-per-second in [0, 1.099511627776e+12] Per-router, per-index document admission rate; zero disables the rate quota.
ingest.global_queue_bytes bytes size node-runtime 256MiB YSEARCH_INGEST_GLOBAL_QUEUE_BYTES ingest-global-queue-bytes positive Encoded ingest bytes all router sessions may hold while waiting for builders.
ingest.index_queue_bytes bytes size node-runtime 64MiB YSEARCH_INGEST_INDEX_QUEUE_BYTES ingest-index-queue-bytes positive Encoded ingest bytes one index may hold while waiting for builders.
ingest.max_batch_bytes bytes size node-runtime 4MiB YSEARCH_INGEST_MAX_BATCH_BYTES ingest-max-batch-bytes positive Largest accepted ingest batch.
ingest.max_document_bytes bytes size node-runtime 16MiB YSEARCH_INGEST_MAX_DOCUMENT_BYTES ingest-max-document-bytes positive Largest accepted document.
ingest.seal_age duration node-runtime 30s YSEARCH_INGEST_SEAL_AGE ingest-seal-age positive Age at which an open build seals.
ingest.seal_bytes bytes size node-runtime 0 YSEARCH_INGEST_SEAL_BYTES ingest-seal-bytes in [0, 1.099511627776e+12] Spooled bytes at which an open build seals; zero derives it as a quarter of builder.sort_memory_bytes. A build's postings are about twice its spooled bytes and the partitions get roughly half the sort budget, so a spool larger than that quarter cannot be sorted in memory and the build spills instead — which is the difference between one file per build and hundreds, and measured 2.5x on the build itself.
ingest.seal_documents int64 node-runtime 100000 YSEARCH_INGEST_SEAL_DOCUMENTS ingest-seal-documents positive Spooled documents at which an open build seals.
log.format enum startup text YSEARCH_LOG_FORMAT log-format one of text, json Log line format.
log.level enum node-runtime info YSEARCH_LOG_LEVEL log-level one of error, warn, info, debug Minimum level written to stderr.
object.backend enum startup fs YSEARCH_OBJECT_BACKEND object-backend one of fs, s3 Object store backend.
object.dir string startup YSEARCH_OBJECT_DIR object-dir Filesystem object-store directory; required for the fs backend. A relative value in a configuration file resolves against that file's directory.
object.root string startup indexes YSEARCH_OBJECT_ROOT object-root Object key root.
object.s3.access_key string startup YSEARCH_OBJECT_S3_ACCESS_KEY env only S3 access key. Environment only (YSEARCH_OBJECT_S3_ACCESS_KEY, or the legacy YSEARCH_S3_ACCESS_KEY); never a flag, never in a file.
object.s3.anonymous bool startup false YSEARCH_OBJECT_S3_ANONYMOUS --s3-anonymous (--object-s3-anonymous) Sign no requests at all, for a bucket that grants public reads. This is a deliberate choice, never a fallback: an unresolved credential chain already signs anonymously, and a reader that quietly degraded to unsigned requests would report a permissions problem as an empty index.
object.s3.bucket string startup YSEARCH_OBJECT_S3_BUCKET --s3-bucket (--object-s3-bucket) S3 bucket; required for the s3 backend.
object.s3.endpoint string startup YSEARCH_OBJECT_S3_ENDPOINT --s3-endpoint (--object-s3-endpoint) S3 endpoint host:port; required for the s3 backend.
object.s3.path_style bool startup false YSEARCH_OBJECT_S3_PATH_STYLE --s3-path-style (--object-s3-path-style) Use path-style S3 addressing.
object.s3.prefix string startup YSEARCH_OBJECT_S3_PREFIX --s3-prefix (--object-s3-prefix) S3 key prefix.
object.s3.region string startup YSEARCH_OBJECT_S3_REGION --s3-region (--object-s3-region) S3 region.
object.s3.secret_key string startup YSEARCH_OBJECT_S3_SECRET_KEY env only S3 secret key. Environment only (YSEARCH_OBJECT_S3_SECRET_KEY, or the legacy YSEARCH_S3_SECRET_KEY).
object.s3.session_token string startup YSEARCH_OBJECT_S3_SESSION_TOKEN env only S3 session token accompanying a temporary access/secret pair, as issued by SSO, an assumed role, or a web identity. Environment only. Supplying an expiring credential without it is rejected by the provider; leave every credential setting empty instead to let the ambient chain resolve and refresh one.
object.s3.use_ssl bool startup true YSEARCH_OBJECT_S3_USE_SSL --s3-use-ssl (--object-s3-use-ssl) Use TLS to the S3 endpoint.
object.source string startup YSEARCH_OBJECT_SOURCE --source (--object-source) Dataset URL to attach read-only, such as s3://bucket/root/ for a whole root or s3://bucket/root/indexes/name/ for one index. It derives object.backend, the bucket, prefix and root, the index when the URL names one, and server.read_only, each only where nothing more specific was set. Reading someone else's published index is the whole purpose, so attaching one implies read-only and cannot be talked out of it.
observability.metrics_listen string startup :9550 YSEARCH_OBSERVABILITY_METRICS_LISTEN --metrics-listen (--observability-metrics-listen) host:port serving Prometheus exposition at /metrics; empty disables it. Unlike debug.pprof_listen this is on by default and binds every interface, because a metric carries no document text, no query, and no key material, and a scrape target that has to be switched on is one nobody switches on. Every series already carries the role label, so the scrape target only has to supply instance and pod.
prototype.adapter.batch_documents int64 startup 500 YSEARCH_PROTOTYPE_ADAPTER_BATCH_DOCUMENTS prototype-adapter-batch-documents in [1, 65536] Documents per ingest request.
prototype.adapter.brokers string list startup 127.0.0.1:9092 YSEARCH_PROTOTYPE_ADAPTER_BROKERS prototype-adapter-brokers Comma-separated Kafka-protocol seed brokers of the article topics.
prototype.adapter.group string startup ysearch-prototype-articles YSEARCH_PROTOTYPE_ADAPTER_GROUP prototype-adapter-group Consumer group id. Offsets are committed manually, only up to the last contiguously acknowledged record (spec/28 §4).
prototype.adapter.index string startup articles YSEARCH_PROTOTYPE_ADAPTER_INDEX prototype-adapter-index Index the adapter feeds through the ingest path. Its schema declares any subset of the article fields; its key field receives the document key.
prototype.adapter.ingest_server string startup 127.0.0.1:9500 YSEARCH_PROTOTYPE_ADAPTER_INGEST_SERVER prototype-adapter-ingest-server gRPC host:port of the ysearch server whose IngestService the adapter streams to.
prototype.adapter.namespace string startup articles YSEARCH_PROTOTYPE_ADAPTER_NAMESPACE prototype-adapter-namespace Document key prefix: an article's key is <namespace>:<article id> in the sidecar and in the index.
prototype.adapter.partitions int64 startup 0 YSEARCH_PROTOTYPE_ADAPTER_PARTITIONS prototype-adapter-partitions in [0, 65536] Partition count of every article topic. 0 uses the topics as they are; a positive count creates a missing topic with it and refuses one with another count, since a count change needs a source epoch the prototype does not have (spec/28 §3.3).
prototype.adapter.poll_records int64 startup 2000 YSEARCH_PROTOTYPE_ADAPTER_POLL_RECORDS prototype-adapter-poll-records in [1, 1.048576e+06] Records per poll. One poll is one dual-write round: sidecar first, then ingest, then the offset commit.
prototype.adapter.start enum startup earliest YSEARCH_PROTOTYPE_ADAPTER_START prototype-adapter-start one of earliest, latest Where a group without committed offsets starts. Committed offsets always win, and an out-of-range committed offset stops its partition instead of jumping.
prototype.adapter.tombstones_delete bool startup false YSEARCH_PROTOTYPE_ADAPTER_TOMBSTONES_DELETE prototype-adapter-tombstones-delete A null-value record deletes its article. Off, a tombstone is poison and stops its partition (spec/28 §2, §6).
prototype.adapter.topics string list startup articles YSEARCH_PROTOTYPE_ADAPTER_TOPICS prototype-adapter-topics Comma-separated article topics: ysearch.fixture.v1.Article protobufs, as tools/corpus replays them, keyed by article id (spec/28 §3.1). Topic names are deployment configuration.
prototype.sidecar.address string startup 127.0.0.1:6379 YSEARCH_PROTOTYPE_SIDECAR_ADDRESS prototype-sidecar-address RESP host:port of a marekvs 5df6ff5 node, the prototype's throwaway sidecar KV (docs/ysearch/05 §5). Any node serves any key.
prototype.sidecar.password string startup YSEARCH_PROTOTYPE_SIDECAR_PASSWORD env only The sidecar's MAREKVS_REQUIREPASS. Environment only (YSEARCH_PROTOTYPE_SIDECAR_PASSWORD); never a flag, never in a file.
prototype.sidecar.timeout duration startup 5s YSEARCH_PROTOTYPE_SIDECAR_TIMEOUT prototype-sidecar-timeout positive Dial, read and write timeout of every sidecar call.
publisher.announce_interval duration node-runtime 15s YSEARCH_PUBLISHER_ANNOUNCE_INTERVAL publisher-announce-interval in [1e+09, 3.6e+12] Requested commit-marker publication cadence; the daemon floors it at follower.poll_interval and 15 seconds.
publisher.lease_ttl duration startup 30s YSEARCH_PUBLISHER_LEASE_TTL publisher-lease-ttl in [5e+09, 6e+11] Startup-fixed object-store lease lifetime for the catalog publisher; renewal runs at one third of this value.
query.collapse_key_versions bool startup true YSEARCH_QUERY_COLLAPSE_KEY_VERSIONS --collapse-key-versions (--query-collapse-key-versions) Return one hit per document key, the copy with the greatest mutation version. A re-pushed document is stored as a new version rather than replacing the old one, so without this a key appears once per copy. Collapsing removes candidates after selection, so a request may return fewer than top_k.
query.compiled_cache bool node-runtime false YSEARCH_QUERY_COMPILED_CACHE query-compiled-cache Experimental leased compiled-query cache within the shared metadata allowance. Also reuses bounded dictionary plans for eager generation-owned readers; lazy reader plans stay request-owned. Cache pressure falls back to request-owned compilation or preparation.
query.default_top_k uint64 node-runtime 10 YSEARCH_QUERY_DEFAULT_TOP_K query-default-top-k in [1, 10000] top_k when a request names none.
query.defer_public_ids bool startup true YSEARCH_QUERY_DEFER_PUBLIC_IDS query-defer-public-ids Resolve a segment survivor's public ID only when the global merge returns it or must break a score tie with it, instead of for every survivor of every segment. Results are identical either way; false restores eager resolution as a same-code control.
query.lexical_memory_bytes bytes size startup 384MiB YSEARCH_QUERY_LEXICAL_MEMORY_BYTES query-lexical-memory-bytes positive Process-wide lexical request, candidate, queue and merge memory admission budget. Storage blocks and ordinal pages have separate cache budgets; busy requests fail with resource exhaustion. Sized to admit one request merging 64 segments at the product's 100,000-result top_k ceiling (lexical.MaxResultLimit; estimateLexicalFanoutMemory(64, 100000) = 182,632,832 bytes) with room for a second concurrent one that size.
query.lexical_strategy enum startup block-max YSEARCH_QUERY_LEXICAL_STRATEGY query-lexical-strategy one of block-max, auto, single-term-block-max, conjunction, block-max-maxscore Experimental exact lexical traversal selection; unsupported query shapes conservatively fall back to block-max.
query.logical_bound_entries uint64 startup 0 YSEARCH_QUERY_LOGICAL_BOUND_ENTRIES query-logical-bound-entries in [0, 256] Experimental score-bound subblock size within decoded postings; zero uses physical blocks. Does not reduce physical read size.
query.max_ast_depth uint64 node-runtime 32 YSEARCH_QUERY_MAX_AST_DEPTH query-max-ast-depth in [1, 1024] Deepest query nesting either grammar accepts.
query.max_results uint64 node-runtime 0 YSEARCH_QUERY_MAX_RESULTS query-max-results Ceiling on maximum_results per request; zero keeps the engine default.
query.ordinal_mode enum node-runtime pages YSEARCH_QUERY_ORDINAL_MODE query-ordinal-mode one of pages, whole, adaptive Experimental ID residency policy under the shared ordinal budget. Whole tables are capped at 8 MiB and fall back to pages; adaptive promotion requires repeated broad page coverage.
query.packed_simd bool startup false YSEARCH_QUERY_PACKED_SIMD query-packed-simd Experimental native packed-posting decoding on supported CPUs; preserves scalar fallback and semantic validation.
query.phrase_gap_expansion_limit uint64 node-runtime 64 YSEARCH_QUERY_PHRASE_GAP_EXPANSION_LIMIT query-phrase-gap-expansion-limit in [1, 4096] Most exact gap variants a {m,n} quantifier may expand to.
query.regex_max_expansions uint64 node-runtime 256 YSEARCH_QUERY_REGEX_MAX_EXPANSIONS query-regex-max-expansions in [1, 65536] Most dictionary terms a regex or prefix may expand to before the query is refused.
query.scoring_profile enum startup bm25f-v1 YSEARCH_QUERY_SCORING_PROFILE query-scoring-profile one of bm25f-v1, bm25f-pinned-v1 Scoring profile for a request that names none. bm25f-v1 scores each segment with its own statistics; bm25f-pinned-v1 (spec/50 §2) scores every segment under the generation's pinned statistics epoch, so scores do not depend on segment layout. It needs stats.dir; without it a pinned request fails with a typed error, never with segment statistics.
query.set_simd bool startup false YSEARCH_QUERY_SET_SIMD query-set-simd Experimental native intersection for balanced sparse filter arrays; skewed arrays retain scalar galloping.
routing.presence_bytes bytes size startup 0 YSEARCH_ROUTING_PRESENCE_BYTES routing-presence-bytes in [0, 8.589934592e+09] Memory, not disk: resident budget for per-segment term-presence filters, used only when the generation has no routing table. A generation that has one prunes through it instead and holds no filters at all, whatever this says, because the table is read through a small page cache rather than held. Zero means none under cache.lazy_readers and unbounded without it, which at 294 segments was 1.4GiB of resident filters. That promotion for eager readers stays on for now: builder.routing_fragment_bytes and compaction.routing_fragment_bytes only default on from 2026-09-23, nothing rewrites a segment built before that, and no existing corpus has been rebuilt or fully compacted since - so a generation with an unfragmented member still depends on this filter to prune anything, including the published demo image, which does not set this flag. Filters are loaded in catalog order until the budget is reached; segments without one stay unknown and are still executed, so results never change. Budget against the decoded size, not the transfer: 680MiB of admitted filters measured 1.4GiB resident, about 2.1 times what this charges them.
server.data_dir string startup YSEARCH_SERVER_DATA_DIR --data-dir (--server-data-dir) Developer shortcut: derives object.backend=fs, object.dir, cache.dir, ingest.dir, and config.file beneath one directory where nothing more specific is set. A relative value in a configuration file resolves against that file's directory.
server.grace_period duration startup 5s YSEARCH_SERVER_GRACE_PERIOD --grace-period (--server-grace-period) positive Graceful shutdown period.
server.listen string startup 127.0.0.1:9500 YSEARCH_SERVER_LISTEN --listen (--server-listen) gRPC listen host:port serve and node bind; the default is the address the client verbs dial (client.server).
server.max_receive_bytes bytes size startup 4MiB YSEARCH_SERVER_MAX_RECEIVE_BYTES --max-receive-bytes (--server-max-receive-bytes) in [65536, 9.223372036854776e+18] Maximum gRPC request bytes (at least 64KiB).
server.max_send_bytes bytes size startup 4MiB YSEARCH_SERVER_MAX_SEND_BYTES --max-send-bytes (--server-max-send-bytes) in [65536, 9.223372036854776e+18] Maximum gRPC response bytes (at least 64KiB, one result frame).
server.node_id string startup YSEARCH_SERVER_NODE_ID --node-id (--server-node-id) Node identity; empty defaults to the bound listen address.
server.query_receive_memory_bytes bytes size startup 320MiB YSEARCH_SERVER_QUERY_RECEIVE_MEMORY_BYTES server-query-receive-memory-bytes positive Shared request-memory allowance for Search, worker Execute, Merge, ValidateQuery, CacheState and FetchStored. Holds decoded requests until RPC completion. Receive workspace waits at most 5ms in a 64-call queue; decoded admission refuses immediately when full. Must fit three receive buffers (each at least 1MiB) plus decoded requests. Unary wire reception/decompression and HTTP/2 queues precede this admission. Sized for at least 24 concurrent full server.max_receive_bytes receives at its default (24 * 13,238,272 bytes = 303MiB, plus headroom): fewer than that admits only a handful of concurrent requests before refusing the rest with ResourceExhausted.
server.read_only bool startup false YSEARCH_SERVER_READ_ONLY --read-only (--server-read-only) Refuse every object-store write for this process's lifetime. Enforced at the single store every writer resolves through, so background work that never touches an RPC - garbage collection, and the GC dry run, which writes a proposal object before any delete decision - fails closed too. Startup-fixed on purpose: a running process cannot be talked into writing by a config reload.
server.roles string list startup YSEARCH_SERVER_ROLES --roles (--server-roles) Comma-separated roles for node: coordinator, worker, merger, aggregator, compactor, router, builder, publisher.
server.source_management enum startup loopback YSEARCH_SERVER_SOURCE_MANAGEMENT server-source-management one of loopback, any Which peers may call SourceService, the RPC that changes which datasets this process reads. loopback answers only a caller that reached this process without crossing a network, so binding server.listen to every interface still does not expose it. any serves it to anything that can reach the port: the service has no authentication of its own, so choose it only where something in front of the address authenticates.
server.zone string startup default YSEARCH_SERVER_ZONE --zone (--server-zone) Placement zone this node advertises in.
stats.dir string startup YSEARCH_STATS_DIR stats-dir Directory of this node's statistics databases, one per index (spec/50 §3.3). Every served generation is folded into it and its epoch pinned for bm25f-pinned-v1. Unlike cache.dir it is not disposable: a node that loses it refolds from the catalog. Empty disables epochs. A relative value in a configuration file resolves against that file's directory.
stats.history_generations uint64 startup 6000 YSEARCH_STATS_HISTORY_GENERATIONS stats-history-generations in [1, 1.6777216e+07] Generations of per-generation statistics deltas kept to reconstruct an earlier epoch (spec/50 §3.3); about two days at one generation per 30 s.
storage.hydrate_throughput_floor bytes size node-runtime 128MiB YSEARCH_STORAGE_HYDRATE_THROUGHPUT_FLOOR --hydrate-throughput-floor (--storage-hydrate-throughput-floor) positive Lowest believable hydration throughput in bytes per second.
storage.hysteresis_denominator uint64 node-runtime 4 YSEARCH_STORAGE_HYSTERESIS_DENOMINATOR --hysteresis-denominator (--storage-hysteresis-denominator) in [1, 4.294967295e+09] Hysteresis denominator; must be below the numerator.
storage.hysteresis_numerator uint64 node-runtime 5 YSEARCH_STORAGE_HYSTERESIS_NUMERATOR --hysteresis-numerator (--storage-hysteresis-numerator) in [1, 4.294967295e+09] Hysteresis numerator; must exceed the denominator.
storage.mode enum startup AUTO YSEARCH_STORAGE_MODE --mode (--storage-mode) one of HYDRATE_FULL, REMOTE_BLOCKS, AUTO Storage mode: HYDRATE_FULL, REMOTE_BLOCKS, or AUTO (AUTO is refused by the checkpoint-bound serve).
storage.remote_latency_floor duration node-runtime 2ms YSEARCH_STORAGE_REMOTE_LATENCY_FLOOR --remote-latency-floor (--storage-remote-latency-floor) positive Lowest believable remote request latency.
storage.remote_throughput_floor bytes size node-runtime 64MiB YSEARCH_STORAGE_REMOTE_THROUGHPUT_FLOOR --remote-throughput-floor (--storage-remote-throughput-floor) positive Lowest believable remote throughput in bytes per second.
storage.scan_threshold_permille uint64 node-runtime 200 YSEARCH_STORAGE_SCAN_THRESHOLD_PERMILLE --scan-threshold-permille (--storage-scan-threshold-permille) in [0, 1000] Scan fraction above which AUTO hydrates.
tail.impact_enabled bool node-runtime false YSEARCH_TAIL_IMPACT_ENABLED tail-impact-enabled Persist score-banded approximate-tail data. Disabled avoids two additional records per unique term when exact tails are sufficient.
tail.impact_maximum_bands uint64 node-runtime 64 YSEARCH_TAIL_IMPACT_MAXIMUM_BANDS tail-impact-maximum-bands in [1, 4096] Maximum score-impact bands persisted for one term.
tail.impact_target_documents uint64 node-runtime 4096 YSEARCH_TAIL_IMPACT_TARGET_DOCUMENTS tail-impact-target-documents in [1, 4.294967295e+09] Target postings per persisted impact run.
tail.maximum_ordering_error float64 node-runtime 0 YSEARCH_TAIL_MAXIMUM_ORDERING_ERROR tail-maximum-ordering-error in [0, 1.7976931348623157e+308] Server ceiling on accepted approximate-tail score-ordering error; zero requires exact fallback.
vector.bits_per_code uint64 node-runtime 8 YSEARCH_VECTOR_BITS_PER_CODE vector-bits-per-code in [1, 8] Bits in each PQ subquantizer code.
vector.build_memory_bytes bytes size node-runtime 0 YSEARCH_VECTOR_BUILD_MEMORY_BYTES vector-build-memory-bytes in [0, 9.223372036854776e+18] Per-build vector training and assignment memory; zero derives a bounded share of builder.sort_memory_bytes.
vector.centroids uint64 node-runtime 0 YSEARCH_VECTOR_CENTROIDS vector-centroids in [0, 4.294967295e+09] IVF coarse centroid count; zero derives it deterministically from population.
vector.flat_threshold uint64 node-runtime 10000 YSEARCH_VECTOR_FLAT_THRESHOLD vector-flat-threshold in [1, 4.294967295e+09] Per-segment vector population at or below which candidate generation stays exhaustive.
vector.list_block_documents uint64 node-runtime 0 YSEARCH_VECTOR_LIST_BLOCK_DOCUMENTS vector-list-block-documents in [0, 4.294967295e+09] Target vector entries per IVF-list block; zero follows the document-value block geometry.
vector.maximum_training_vectors uint64 node-runtime 4096 YSEARCH_VECTOR_MAXIMUM_TRAINING_VECTORS vector-maximum-training-vectors in [1, 4.294967295e+09] Maximum deterministic training samples retained per vector field.
vector.query_candidate_multiplier uint64 node-runtime 10 YSEARCH_VECTOR_QUERY_CANDIDATE_MULTIPLIER vector-query-candidate-multiplier in [1, 10000] Default ANN candidates as a multiple of requested top_k.
vector.query_default_probes uint64 node-runtime 8 YSEARCH_VECTOR_QUERY_DEFAULT_PROBES vector-query-default-probes in [1, 4.294967295e+09] IVF coarse lists probed when a vector request does not specify probes.
vector.query_maximum_candidates uint64 node-runtime 100000 YSEARCH_VECTOR_QUERY_MAXIMUM_CANDIDATES vector-query-maximum-candidates in [1, 4.294967295e+09] Hard per-request candidate generation and exact-rerank cap.
vector.query_maximum_probes uint64 node-runtime 1024 YSEARCH_VECTOR_QUERY_MAXIMUM_PROBES vector-query-maximum-probes in [1, 4.294967295e+09] Hard per-request IVF probe cap.
vector.query_memory_bytes bytes size node-runtime 256MiB YSEARCH_VECTOR_QUERY_MEMORY_BYTES vector-query-memory-bytes positive Per-process admission budget for decoded vector-query working sets.
vector.subquantizers uint64 node-runtime 0 YSEARCH_VECTOR_SUBQUANTIZERS vector-subquantizers in [0, 4.294967295e+09] PQ subquantizer count; zero derives a geometry compatible with each vector field's dimension.
vector.training_iterations uint64 node-runtime 20 YSEARCH_VECTOR_TRAINING_ITERATIONS vector-training-iterations in [1, 1000] Maximum deterministic k-means iterations during IVF-PQ training.

Exit codes

  • 0 — success; --help anywhere.
  • 1 — the command ran and failed (a refused request, an unreachable server, a bad file); the message names the cause and, for a server failure, its gRPC code and a remedy.
  • 2 — the invocation was wrong (unknown command, missing subcommand, unknown flag, wrong argument count); the usage follows the message.

Where settings come from

Files are discovered in a fixed order and later files override earlier ones: /etc/ysearch/ysearch.yml, then /etc/ysearch/conf.d/*.yml; $XDG_CONFIG_HOME/ysearch/ysearch.yml (default ~/.config/ysearch/), then its conf.d/*.yml; ./ysearch.yml; <data-dir>/config.yaml when --data-dir is set; --config-dir DIR's *.yml; --config FILE. .yaml is accepted wherever .yml is. --config-search=false (YSEARCH_CONFIG_SEARCH=false) skips the default locations. Then the environment (YSEARCH_<KEY> with dots and underscores as _), then the flags, then the runtime API. Every value carries its provenance; config get, config list, and config export show it.