Vector search and compositional embeddings

embeddings.chunks is the universal chunk-embedding relation: every embedded corpus under one schema keyed by (source, target_key, chunk_index), with a readable embedding column, model_name, and embedding_dim. The per-corpus views — embeddings.forum_posts, reddit_comments, hackernews_items, stackexchange_posts, wikipedia_articles, pubmed_papers, arxiv_papers, openalex_works, crawl_pages, mailing_list_messages, tweets, bluesky_posts, and the rest listed by /v1/scry/schema — serve the same rows under corpus-native key columns; embeddings.sources lists which (source, family) pairs currently hold rows. Voyage 4 nano and lite share one embedding space, so distances compare across them.

The live helper catalog includes cosine similarity, norm, unit-vector, scaling, projection, debiasing, contrast-axis, pairwise-matrix, and centroid operations. Inspect /v1/scry/schema for the current catalog before composing.

SELECT scry_cosine_similarity(@interp, @safety) AS similarity,
       scry_debias_removed_fraction(@interp, @safety) AS removed_fraction
LIMIT 1;

Create the named vector before using @interp in SQL. This request needs a key with scry plus embed or write scope; a console session token also works.

curl -X POST https://api.scry.io/v1/scry/embed \
  -H "Authorization: Bearer $SCRY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text": "alignment via debate", "name": "interp"}'

scry_vector_topk_distance ranks one embeddings relation by cosine distance to a @handle through the ANN lane. It must appear as an aliased projection, ordered ascending on that alias, with LIMIT at most 100. Do not filter on model_name (nano and lite share the space); any WHERE clause post-filters a window of roughly 400 nearest candidates, so a selective filter can return fewer than LIMIT rows — narrow with row-level vector math on embeddings.chunks when the filter is the point.

SELECT post_key, chunk_index,
       scry_vector_topk_distance(embedding_voyage4, @interp) AS dist
FROM embeddings.forum_posts
ORDER BY dist ASC
LIMIT 10;

Row-level vector math (scry_cosine_similarity and friends) needs a readable vector column, which embeddings.chunks has: filter source first (it is the partition key; an unfiltered scan reads the whole family), then score. The per-corpus views expose their vectors only through the ANN lane. Use only helpers listed by /v1/scry/schema, and read each relation's query_guidance there — it is the contract the docs summarize.