Start

Status and roadmap

Where ysearch stands today, milestone by milestone, from the fork (YS0) to scale qualification and the classic-feature track.

ysearch is pre-v1. Status as of 2026-09-25. The milestone names and goals come from the roadmap at the pinned revision; the status column is maintained in this site's upstream/status.json.

Milestone	Status	Where it stands
YS0 Fork and rename	Done	Closed 2026-09-25. The repository is the fork of yolosearch at 94f6b98, renamed; the gate is green on Forgejo and amd64 and arm64 images are published (docs/milestones/ys0-evidence.md).
YS1 Infrastructure	In progress	The local part is done on main: the swarm stack with Redpanda and the marekvs sidecar, container test fixtures, the 3·10⁶-article corpus tooling with synthetic campaigns and beacons, the prototype's sidecar client and Redpanda adapter, and the tools/dev infra-check verb. Staging, CI runners and bench hosts wait for the owner. No evidence file yet.
YS2 Pinned scoring (prototype track T2)	Exit met, not closed	The exit criteria are met on main: segment format 7 (stats, forward, hashterms), the statistics-epoch fold with history, the bm25f-pinned-v1 scorer beside bm25f-v1, internal/ysmath and the determinism check. The milestone stays open until its evidence file is written.
YS3 Matcher prototype (T3, T4)	Exit met, not closed	The exit check passed: on 10^5 YS1 articles by 10^3 campaigns the served sets equal brute force, with 0 mismatches over 227.5·10^6 pair evaluations and bit-exact scores, and a dial change wrote only dial records (bench/ys3/exit-100k-1k.json, go run ./tools/matchcheck). internal/match covers compilation (keywords, query_string, vectors), cells, fenced table generations, read-time scoring under G_chk and threshold and count dials. The Matching gRPC service exists but is not yet mounted in any server binary or role. Not measured: matcher cost; the vector path is checked only on a synthetic 3,000 by 200 subset. No evidence file yet.
YS4 Prototype qualification (T5): go / no-go	Planned	The go / no-go gate. Every (A) figure in the specs stays an assumption until it runs.
YS5 KV plane and the control Raft groups (P1)	Groundwork landed	Pure groundwork is on main: internal/kv hlc, envelope, ikey, counter, merge and record, with merge-law property tests; the element join is a true lattice with no live-dot cap, a deliberate fix of two marekvs merge defects. The differential harness against marekvs is deferred by the owner. No storage, replication, gossip, RESP server or Raft yet.
YS6 Documents and the changelog indexer (P2, P11.1, first half of P4)	Planned	Not started.
YS7 Stream ingestion and traffic (P3, P3a)	Planned	Not started. Only the YS3 prototype adapter reads from Kafka today.
YS8 Segment plane completion (rest of P4)	Planned	Not started.
YS8a yexpr and the WASM app engine core (P15a)	Planned	Not started.
YS9 Matching in production shape (P5, P6, P7)	Planned	Not started.
YS10 Archiving and system snapshots (P8)	Planned	Not started.
YS11 Operator and operations (P9)	Planned	Not started. The operator on main is yolosearch's, renamed.
YS12 Scale qualification: ready for the ad-matching workload (P10)	Planned	Not started.
YS13 mappings, analysis, pipelines (P11)	Planned	Not started.
YS14 query DSL (P12)	Planned	Not started. The Lucene and CQP grammars on main are inherited.
YS15 aggregations and presentation (P13)	Planned	Not started.
YS16 vector store (P14)	Planned	Not started. The inherited IVF-PQ vector search is still on main.
YS17 app worlds, jobs, APIs, security (P15b, P16)	Planned	Not started.

The sequence

YS0 fork ─► YS1 infrastructure ─► YS2 pinned scoring ─► YS3 matcher prototype ─► YS4 prototype qualification
                                                                                    │ go / no-go
               ┌────────────────────────────────────────────────────────────────────┘
               ▼
YS5 KV plane + control Raft ─► YS6 documents + indexer ─► YS7 streams + traffic ─► YS8 segment plane ─► YS9 matching
     │                                                                                                    │
     └─► YS8a yexpr + WASM app engine core                                                                │
YS10 archiving ─► YS11 operator + operations ─► YS12 scale qualification (ad-matching ready) ◄────────────┘

Classic-feature track, from YS8 and YS8a on, in parallel:
YS13 mappings + analysis + pipelines ─► YS14 query DSL ─► YS15 aggregations + presentation
                                     └► YS16 vector store        └► YS17 app worlds, jobs, APIs, security

YS0 — Fork and rename

Status: Done. Closed 2026-09-25. The repository is the fork of yolosearch at 94f6b98, renamed; the gate is green on Forgejo and amd64 and arm64 images are published (docs/milestones/ys0-evidence.md).

Goal: a renamed fork that is exactly as green as yolosearch.

YS1 — Infrastructure

Status: In progress. The local part is done on main: the swarm stack with Redpanda and the marekvs sidecar, container test fixtures, the 3·10⁶-article corpus tooling with synthetic campaigns and beacons, the prototype's sidecar client and Redpanda adapter, and the tools/dev infra-check verb. Staging, CI runners and bench hosts wait for the owner. No evidence file yet.

Goal: everything later milestones need to run, test and measure, before any feature work.

YS2 — Pinned scoring (prototype track T2)

Status: Exit met, not closed. The exit criteria are met on main: segment format 7 (stats, forward, hashterms), the statistics-epoch fold with history, the bm25f-pinned-v1 scorer beside bm25f-v1, internal/ysmath and the determinism check. The milestone stays open until its evidence file is written.

Goal: scores that do not depend on segment layout.

YS3 — Matcher prototype (T3, T4)

Status: Exit met, not closed. The exit check passed: on 10^5 YS1 articles by 10^3 campaigns the served sets equal brute force, with 0 mismatches over 227.5·10^6 pair evaluations and bit-exact scores, and a dial change wrote only dial records (bench/ys3/exit-100k-1k.json, go run ./tools/matchcheck). internal/match covers compilation (keywords, query_string, vectors), cells, fenced table generations, read-time scoring under G_chk and threshold and count dials. The Matching gRPC service exists but is not yet mounted in any server binary or role. Not measured: matcher cost; the vector path is checked only on a synthetic 3,000 by 200 subset. No evidence file yet.

Goal: the match matrix end to end on the existing ingest path.

YS4 — Prototype qualification (T5): go / no-go

Status: Planned. The go / no-go gate. Every (A) figure in the specs stays an assumption until it runs.

Goal: measure what the specs only assume, before building the rest.

YS5 — KV plane and the control Raft groups (P1)

Status: Groundwork landed. Pure groundwork is on main: internal/kv hlc, envelope, ikey, counter, merge and record, with merge-law property tests; the element join is a true lattice with no live-dot cap, a deliberate fix of two marekvs merge defects. The differential harness against marekvs is deferred by the owner. No storage, replication, gossip, RESP server or Raft yet.

Goal: marekvs's design, in Go, on WavesDB, and the control role's Raft groups that its capability gates and leases need.