The calculus of search

Scry’s corpora arrive as relations: hackernews.items, reddit.posts, openalex.works, the historical Twitter archive’s reply and quote graphs. A query over relations yields a relation, so every result is a legitimate input to the next query — a CTE, a subquery, a named set in a fixpoint program. Relational closure is the foundational property: the unit of work is the construction of new relations from old ones, and each construction is immediately raw material for the next. Below, storytellers is a relation that exists only because a query defined it, queried in the same statement like any base table (1.1 s over 90M rows on the live engine).

WITH storytellers AS (      -- a relation defined by a query
  SELECT original_author AS author
  FROM hackernews.items
  WHERE hn_type = 'story' AND upvotes > 500
  GROUP BY author HAVING count() >= 10
)
SELECT i.title, i.upvotes     -- queried like any base relation
FROM hackernews.items AS i
JOIN storytellers AS s ON i.original_author = s.author
WHERE i.hn_type = 'story'
ORDER BY i.upvotes DESC LIMIT 5

A conjunctive query names every x for which a pattern of facts holds, with intermediate entities that are existential — they must exist, they need not be returned. In SQL the conjunction is a join; in a fixpoint program it is a body, where every atom (rel, edge, filter, in) further constrains one walk. This is the workhorse layer of structured search — relational pattern matching over the corpus, well past keyword retrieval, and the layer the engine optimizes best. The query below runs the pattern as written: every author whose post or comment drew a reply from Hacker News’s moderator, in ~240 ms.

Q(a) ← post(p, a) ∧ reply(r, p) ∧ author(r, 'dang')

SELECT DISTINCT p.original_author
FROM hackernews.items AS p
JOIN hackernews.items AS r ON r.parent_hn_id = p.hn_id
WHERE r.original_author = 'dang' AND p.original_author != ''
LIMIT 12

Real concepts rarely have one structural definition, so the calculus gives a relation several bodies: each body a conjunctive clause, the relation their union. Below, context is defined twice — the papers ResNet cites, and the papers citing it — and the engine returns the union: 131 works, both branches contributing, two metered statements, empty truncations. Each branch stays precise while the union names the concept, which is a different philosophy from embedding similarity, where a concept is whatever lands nearby: a relation defined as a union of explicit patterns is inspectable — read its definition, delete a branch, and know exactly what changed.

{"program": {"relations": {
  "seed":    {"bodies": [[{"ids": ["https://openalex.org/W2194775991"]}]]},
  "context": {"bodies": [
      [{"rel": "seed"}, {"edge": "references"}],
      [{"rel": "seed"}, {"edge": "cited_by"}]
  ]}
}, "out": [], "depth": 1}}

Many of the sharpest questions are contrastive — who writes about a field yet sits outside the population that obviously writes about it. SQL carries EXCEPT and NOT EXISTS; programs carry not_in, and its semantics are stratified: the subtracted relation belongs to a lower stratum and is evaluated to completion before the walk that subtracts it begins, so at each frontier an excluded node is dropped before expansion and never billed. That ordering is what keeps negation coherent under recursion: a definition can say “candidates not already established” and can never say “good is whatever is not good”.

novel(x) ← candidate(x) ∧ ¬established(x)   -- established evaluated first

{"bodies": [[{"rel": "candidate"}, {"not_in": "established"}]]}

A relation can be defined in terms of itself: reachable is the seeds, plus everything one edge past what is already reachable. The least fixed point μ is the meaning of that definition — start from the seeds, apply the rule, keep what is new, stop at the round that discovers nothing — so the evaluation terminates at fixpoint rather than an arbitrary manual unrolling. WITH RECURSIVE runs μ over computed sequences; a fixpoint program runs it over the graph corpora semi-naively, expanding only each round’s frontier. On the wire, recursion is a body whose rel atom names the relation it belongs to: add [{"rel": "context"}, {"edge": "cited_by"}] as a third body of context above, raise the envelope’s depth (the round cap, at most 8), and the same program walks the citation closure instead of one hop. The envelope’s truncations list is μ’s honest signature: it names every resource bound that fired, and an empty list means the relation you hold is the least fixed point of your rules over the registered edges.

reachable = μX. seed ∪ step(X)

round 1:  seed
round 2:  seed ∪ step(seed)
round 3:  seed ∪ step(seed) ∪ step²(seed)
…until a round adds nothing new

A walked row returns as {id, parent, depth}: one witness to how the walk reached it, the node that discovered it and the round at which it was discovered. Each row carries exactly one witness, not the set of all derivations that would admit it; a node reachable through several branches reports a single discovering parent at the shallowest round the walk reached it. That is enough to change what an agent can do with results: inspect the path that admitted a row, notice a branch doing the wrong work, subtract it with not_in, re-run — the witness is the debugging surface for the query itself. Aggregate provenance rides the same envelope: per-depth counts for every output relation, so when the shape of a population is the answer, it arrives whole at zero row egress.

Semantic judgment enters the calculus as ordinary relations. An embedding handle minted at /v1/scry/embed becomes an ann atom — a semantic neighborhood as a seed set — and rank orders a derived set by exact cosine distance to a handle; on the SQL plane the registered vector helpers do the same inside a SELECT. The division of labor is the design: models propose the fuzzy predicates only they can express, and the relational engine composes them exactly — joins, unions, negation, fixpoints — over the corpus. Fuzzy semantics at the leaves, crisp composition above them, and the whole construction stays inspectable.

Each symbol added to the calculus buys a qualitatively new class of questions, and every Scry query is a point on this ladder. Traditional search asks an index a question; a query in this calculus constructs a computation over the index. The operational contract — measured latencies, the thirty-one registered edges, and the deadline, budget, and depth bounds you set — is on the Turing-complete search page.