CRYSTAL documentation

Overview

CRYSTAL is a self-curating AI memory system. Its core is a memory that sits between your application and any upstream language model:

Your app ──► CRYSTAL memory ──► any upstream LLM
                (crystals: clustered, keyed, reviewed)   (Anthropic, OpenAI, Vertex)

Your existing SDKs and tools work unchanged. CRYSTAL injects relevant memory into each request, captures what's worth keeping, and organizes it into structured knowledge that improves over time — noticing its own gaps, researching them, and surfacing what it learns.

CRYSTAL ships with two companion products on the same codebase: CRYS (an agent that works on the memory) and the Inspector (a web console). Standing up CRYSTAL gives you all three.

How it works

There are two ways to use CRYSTAL:

1. As a transparent proxy

Point your OpenAI-compatible client at CRYSTAL's /v1/chat/completions endpoint. CRYSTAL retrieves relevant memory, injects it into the request, forwards to your chosen model, and captures anything worth remembering from the exchange — all without code changes on your side.

2. As a direct memory API

Manage memory explicitly: POST /v1/store to add a piece of knowledge, POST /v1/retrieve to pull relevant context (without a model call) and drop it into your own prompt. This is the SDK mode — for when you know exactly what to add and how you want to use it.

Crystals

A crystal is a unit of curated knowledge — not a raw text chunk. Knowledge entering CRYSTAL is chunked, typed, reviewed, and crystallized into a retrievable form that carries structure: what it's about, how it was learned, how vetted it is, and how it relates to the rest of your knowledge.

That structure is what separates CRYSTAL from a plain vector store. Because crystals are typed and keyed, retrieval can be precise, and the system can reason about its knowledge — deduplicating it, resolving contradictions between crystals, and promoting well-supported ones.

Sparse keys

When you store knowledge, CRYSTAL derives a sparse key — a wide-to-specific path built from the key and the value together. Folding the value into the key yields a deeper, more specific path, which improves how retrieval matches related queries to the right crystal.

The practical upshot: a query worded quite differently from the stored knowledge can still match, because the sparse key captures the underlying subject path rather than just surface words. You saw this if you ran the quickstart — storing "the team database → PostgreSQL 16" and retrieving it with "what database do we run in prod?" resolves to a perfect match.

Self-curation

This is what makes CRYSTAL a memory rather than a database. Beyond storing and retrieving, the system continuously works on its own knowledge:

• Gap discovery — it identifies subjects where its knowledge is thin or missing.
• Convergence — during idle time it looks for contradictions and duplicates across crystals and resolves them.
• Tier promotion — well-supported, well-cited knowledge is promoted; stale or unsupported knowledge decays.
• Reflection — the system learns from successes and failures, generating knowledge from what worked and what didn't.

These run as background workers and are individually configurable (and off by default where they spend model budget — see the config reference).

The compounding ratchet

The reason CRYSTAL improves with use. When a query can't be answered well, that miss isn't discarded — it's captured as a gap. Gaps get filled, and filled knowledge surfaces the next time a similar query appears. So the cost of not-knowing (the "loop tax" of re-deriving the same thing) decays over repeated query-classes instead of resetting each session.

Epistemic tiers

Not all knowledge is equal. Every fact carries a quality tier that moves with evidence — how many times it's been cited, how old it is, whether it's been contradicted. Retrieval surfaces this to the model, so a downstream reasoner can weigh a well-established fact differently from a provisional one. Confidence, not just relevance, becomes part of what CRYSTAL communicates.

Cognition & information barriers

CRYSTAL's heavier reasoning runs through a three-tier cognition workflow. Its defining property is information barriers: the worker doing a task never sees the acceptance criteria, and the validator judging the result never sees the plan. The system, by construction, can't grade its own homework — which keeps self-generated knowledge honest.

This same architecture powers CRYS, the agent, which can spin up sub-agents that inherit these barriers.

Components

A CRYSTAL deployment is a small number of moving parts:

The request path

When a request hits the chat proxy:

1. The caller is authenticated (a per-customer key resolves the tenant).
2. CRYSTAL retrieves relevant crystals for the request and composes an injection.
3. The enriched request is forwarded to the customer's configured upstream model.
4. The response returns to the caller; the exchange is captured for possible crystallization.
5. Every model call is recorded in the cost ledger — per customer, per session.

Model routing

CRYSTAL's own curation uses a three-tier model scheme so cheap work runs cheap:

The store-and-retrieve path itself runs keyless — memory works with no model configured at all. Models are only needed for the LLM-backed curation and the chat proxy.

Self-hosting: quickstart

CRYSTAL is a Docker image you run wherever you like. The fastest path:

git clone https://github.com/EraHQ/CRYSTAL.git
cd CRYSTAL
cp .env.example .env
# generate the one required secret (never leaves your machine)
echo "CC_TOKEN_ENCRYPTION_KEY=$(openssl rand -hex 32)" >> .env
docker compose up -d

That brings up Postgres, the API on http://localhost:8000, background workers, and a bundled zero-key web search provider. No accounts and no provider API keys are needed to boot — the memory store/retrieve path runs keyless.

Required configuration

Almost everything is optional and has a sensible default. The essentials:

Config reference

The configuration surface is large because nearly every behavior is tunable. The most useful groups:

Models & inference

Self-curation (opt-in where it spends budget)

Budgets & safety

Storage & search

Production checklist

Running CRYSTAL exposed to a network (not just localhost) tightens several requirements. When CC_ENVIRONMENT=production, the boot guard refuses to start unless the security-critical secrets are set:

Also recommended

• Use managed Postgres with backups rather than the bundled dev database.
• Put the API behind TLS (a load balancer or reverse proxy).
• Set explicit budgets (CC_DAILY_TEAM_BUDGET_MICRO_USD) if you enable model-spending curation.
• Enable rate limiting (CC_ENABLE_RATE_LIMITING).

API — authentication

Callers authenticate with a per-customer key as a bearer token. Create a customer to get one:

curl -X POST http://localhost:8000/v1/customers \
  -H "content-type: application/json" \
  -d '{"provider":"anthropic","model_id":"claude-sonnet-5","api_key_ref":"YOUR_UPSTREAM_KEY"}'

# returns { "id": "cus_...", "api_key": "cc_sk_...", ... }

Use the returned cc_sk_... as Authorization: Bearer cc_sk_... on subsequent calls. Customer minting is part of the admin surface — in production it requires the platform admin key.

API — memory

curl -X POST http://localhost:8000/v1/store \
  -H "content-type: application/json" \
  -H "Authorization: Bearer cc_sk_..." \
  -d '{"key":"the team database","value":"We use PostgreSQL 16 in production."}'

# returns { "crystal_id": "crys_...", "fact_id": "fact_...", "sparse_key": "..." }

curl -X POST http://localhost:8000/v1/retrieve \
  -H "content-type: application/json" \
  -H "Authorization: Bearer cc_sk_..." \
  -d '{"query":"what database do we run in prod?"}'

# returns { "injection": "...", "routing": "perfect",
#           "matched_crystal_ids": [...], "score": 0.66 }

The routing field reports match quality: perfect, spread, low_confidence, or no_match.

API — chat proxy

API — cost & crystals

Read-side endpoints for visibility:

There's also a broader surface for documents, feedback, Google Drive connectors, and groups. The admin routes under /admin/api/* back the Inspector.

CRYS — the agent

CRYS is the agent harness that works on top of CRYSTAL's memory. It plans, researches, and executes — and can spin up its own sub-agents through the same multi-tier cognition workflow, inheriting the information barriers that keep self-generated work honest.

CRYS is mode-agnostic: coding is one mode among many, not the defining frame. The same reflection loops, knowledge tiers, verify loops, and gap handling apply whatever the task. Relevant configuration:

Inspector — the console

The Inspector is CRYSTAL's web console — the window into everything the memory holds. It lets you browse crystals, manage knowledge, resolve conflicts, review the query log, watch agents work, and track cost per customer and session.

Running it

The Inspector is a small, separate service: it serves the built single-page app behind nginx and reverse-proxies the app's API calls to CRYSTAL. Point it at your API with one environment variable:

docker build -f deploy/inspector/Dockerfile -t crystal-inspector .
docker run -p 8080:8080 \
  -e API_UPSTREAM=https://your-crystal-api.example \
  crystal-inspector

For development, npm run dev inside frontend/ does the same job, with VITE_API_TARGET pointing at your API. The Inspector is hosted for the managed platform and fully self-hostable alongside your own deployment.