Three Tiers of Memory, One Write Path

CommsCrew claims its agents write in your organization's voice. Everything behind that claim lives in one layer, and that layer has to answer three unrelated questions before a single token is generated.

backend/app_new/crew/memory_service.py keeps them as three tiers. Explicit memory is what a human typed: the brand voice profile, org facts, executive profiles. Implicit memory is what the system inferred from watching edits. Semantic memory is a vector corpus searched by cosine similarity, exposed to the agents as a tool called search_memory in backend/app_new/crew/base_agent.py, which means whatever ranks in it is what the crew gets to know.

The first two tiers are string assembly over ordinary tables. The third is where the interesting constraints are.

Every vector carries the name of the model that made it

backend/app_new/services/embeddings.py defines the seam between "what turns text into a vector" and everything that stores or searches one. It is a Protocol with three implementations, and the reason it is a Protocol rather than a get_embedding(text) helper is a single fact about vectors:

PROVIDER_DIMENSIONS: dict[str, int] = {
    "deterministic": 64,
    "local": 384,
    "voyage": 1024,
}

Vectors from different models are not comparable. At different widths, cosine similarity is undefined rather than merely wrong. At matching widths, two models still embed into unrelated spaces, so the number you get back is arithmetic performed on noise. A helper function would have made swapping providers look free, and the symptom would have been "the crew retrieves irrelevant memories sometimes," which is close to unfalsifiable in production.

So selection is explicit, and there is no fallback path at all:

    choice = (settings.EMBEDDING_PROVIDER or "").strip().lower()
    if choice not in _PROVIDERS:
        raise ValueError(
            f"EMBEDDING_PROVIDER={choice!r} is not one of {sorted(_PROVIDERS)}. "
            "It is required and has no default on purpose: a fallback would let "
            "vectors from two providers land in one table, which silently "
            "corrupts every similarity score computed against them."
        )

The failure mode a "try Voyage, fall back to local" path produces is a table where the vectors came from whichever provider happened to be reachable that day, discoverable only by noticing that retrieval quality drifted.

The deployed provider is local: BAAI/bge-small-en-v1.5 via fastembed, 384 dimensions, ONNX on CPU, no API key. That choice has a price the docstring states plainly. About 150 MB of model on disk and about 310 MB resident once loaded. On the 2 GB box this whole product runs on, that is a material fraction of memory, which is why the model loads lazily and a deployment that never embeds never pays for it.

One helper writes three columns, or none

backend/app_new/services/memory_embedding.py exists because three columns on OrgMemoryEmbedding are only meaningful together: embedding, embedding_provider, embedding_dim. Both write paths in the product go through it, the pasted-example endpoint in backend/app_new/api/v1/memory.py and the website scanner.

A row that holds a vector but no provider name is compared against nothing, ever, and it looks embedded everywhere a human might check.

That row is strictly worse than a row with no vector at all. It renders as embedded in the database, in GET /memory/examples, and in the dashboard's memory count, while retrieval skips it every single query. Writing all three fields in one function is what makes that state unreachable.

The body of the loop is where the care shows:

    if len(vectors) != len(pending):
        # Never zip a short result against the rows: the vectors would silently
        # shift by one and every row after the gap would be filed under someone
        # else's text, which no test and no user would ever notice.
        logger.warning(
            "memory.embedding_count_mismatch",
            provider=provider.name,
            expected=len(pending),
            received=len(vectors),
            rows_left_unembedded=len(pending),
        )
        return 0

    embedded = 0
    for row, vector in zip(pending, vectors, strict=True):
        if not vector or len(vector) != provider.dimension:
            # Storing a vector whose length disagrees with the provider's
            # declared dimension poisons retrieval for the whole org: cosine
            # raises on it and the row is skipped every query thereafter.
            # Leaving it unembedded is the recoverable failure.
            logger.warning(
                "memory.embedding_dimension_mismatch",
                provider=provider.name,
                expected_dim=provider.dimension,
                received_dim=len(vector) if vector else 0,
            )
            continue
        row.embedding = [float(x) for x in vector]
        row.embedding_provider = provider.name
        row.embedding_dim = provider.dimension
        embedded += 1

    return embedded

Two decisions in that block are worth naming.

The function never raises. If the provider is down, the key expired, or the model is still downloading, the user's text is still stored, unembedded and logged at WARNING, and backend/scripts/backfill_embeddings.py finds exactly those rows later. Failing the write instead would convert a degraded-retrieval incident into data loss, which is the worse of the two by a distance.

The rows are not added to a session and not committed. The caller owns the transaction, so the vector and the text land in the same commit rather than in two that can half-succeed. That is why the endpoint calls embed_memory_row(row) before db.add(row).

The docstring carries a warning that matters more than it looks. Never-raising is correct for a user-facing write and inverted for anything seeding a corpus it is about to measure: a fixture that partially embeds produces a plausible number for a smaller corpus than intended. Those callers are told to treat a return below len(rows) as fatal.

One attribute, two storage types, one score scale

backend/app_new/models/memory.py declares the vector column once:

    embedding: Mapped[list | None] = mapped_column(
        JSON(none_as_null=True).with_variant(Vector(active_dimension()), "postgresql")
    )

On Postgres that is vector(384) with an HNSW index. On SQLite it is a JSON float array. Every read and write site is identical; only the retrieval query branches on dialect.

Three details in that one line each came from something breaking.

none_as_null=True is there because SQLAlchemy's JSON type stores Python None as the JSON text null by default. On SQLite, WHERE embedding IS NULL then matches nothing, so any backfill keyed on that predicate reports zero work forever. Postgres's vector column has no such quirk, so the bug would have been permanent in dev and invisible in production.

active_dimension() is read at DDL time, which makes the honest consequence explicit: changing EMBEDDING_PROVIDER is a migration, not a config change. A plain JSON column would have accepted vectors of any width and let two providers coexist in one table undetected.

HNSW rather than IVFFlat, in backend/app_new/migrate.py, because IVFFlat must be trained against representative data and this table starts empty. An IVFFlat index built on an empty table is worse than no index.

The Postgres query path is short, and one line in it is the whole point:

        column = type_coerce(OrgMemoryEmbedding.embedding, Vector(provider.dimension))
        distance = column.cosine_distance(query_vec).label("distance")
        try:
            result = await self.db.execute(
                select(OrgMemoryEmbedding, distance)
                .where(OrgMemoryEmbedding.organization_id == self.org_id)
                .where(OrgMemoryEmbedding.embedding_provider == provider.name)
                .where(OrgMemoryEmbedding.embedding.isnot(None))
                .where(OrgMemoryEmbedding.embedding_dim == provider.dimension)
                .order_by(distance)
                .limit(limit)
            )
            rows = result.all()

with_variant produces the right DDL but SQLAlchemy still exposes the base type's comparator, so <=> is not on the column and the HNSW index is unreachable. type_coerce re-types the expression for this query only, which is what makes the operator, and therefore the index, actually apply. Without it the code looks correct, runs, returns results, and never touches the index.

<=> is cosine distance, so score = 1.0 - distance on the way out, matching the scale the Python path returns. Same inputs, same ordering, same score scale. Only the place the arithmetic happens differs.

The dual path has a real cost: dev and CI exercise Python while production exercises SQL. Two divergences have already opened between the branches. The first was this embedding_dim predicate, present on the Python side and missing from the pgvector one, so every test asserted behavior production did not have. The second was the provider-switch warning below, which lived only on the Python branch: production returned an empty result with no log while every local run warned. So CI runs the Postgres branch for real against pgvector/pgvector:pg16 rather than trusting it, and that job is what surfaced the second one.

Empty and unreadable are different answers

search_similar returns [] in two unrelated situations: the org has written nothing, or a provider switch left every memory tagged with a model this deployment no longer uses, so the whole corpus sits on disk unreachable.

The list cannot tell those apart, and that ambiguity independently produced a bug in two separate workstreams: a benchmark reporting sub-millisecond scans over zero rows, and an eval gate passing over an empty database. Both authors wrote their own private probe, which is the signal that the API was missing something.

corpus_stats is the supported way to ask:

        readable = (
            await self.db.execute(
                select(func.count())
                .select_from(OrgMemoryEmbedding)
                .where(OrgMemoryEmbedding.organization_id == self.org_id)
                .where(OrgMemoryEmbedding.embedding_provider == provider.name)
                .where(OrgMemoryEmbedding.embedding.isnot(None))
                .where(OrgMemoryEmbedding.embedding_dim == provider.dimension)
            )
        ).scalar_one()
        return {
            "total": total,
            "readable": readable,
            "unreadable": total - readable,
            "active_provider": provider.name,
        }

Note that readable repeats the exact predicate the scoring loop applies per row, not just the provider label. Counting on the label alone overstates the corpus, and overstating is the flattering direction: a caller checking readable before trusting an empty result would have been reassured by rows that are never scored. A genuinely new org reads 0 and 0, which is a different and legitimate answer.

When retrieval comes back empty and the corpus is not, both branches log with the row count and the remedy. It costs one COUNT on a path that returned nothing anyway.

Knowing when not to reach for the vector index

The Writer and Strategist get a "relevant past content" block appended after the last cache breakpoint, so per-query retrieval never invalidates the cached org-context prefix. Building that block used to call semantic search unconditionally. With no user query it passed the literal string "content examples brand voice writing style", a bag of keywords resembling no natural question, which embedding models rank badly.

The retrieval evals measured it at recall@5 of 0.43. Two of the five slots injected into every Writer call held documents that were not brand examples.

The fix in backend/app_new/crew/memory_service.py is four lines:

        if query:
            similar = await self.search_similar(query, limit=5)
        else:
            similar = await self._brand_examples(limit=5)

"Which of my documents are brand examples" is a question the content_type column answers exactly. _brand_examples makes no embedding call and returns no score field, because those rows were not scored against anything and reporting a number there would repeat the mistake this work removed.

An edit becomes a rule, and the rule has to be read

The implicit tier is the loop the product name rests on. A human edits a draft. backend/app_new/services/learning.py diffs it, asks Claude for a category and a one-line summary, and stores the enriched signal. Every third signal in a category, _apply_learning_to_profile folds a learned note into the brand voice profile:

    if category in ("tone", "style"):
        attrs = normalize_tone_attributes(profile.tone_attributes)
        learned_list = list(attrs["learned"])
        learned_list.append(learned_note)
        attrs["learned"] = learned_list[-5:]  # keep last 5
        profile.tone_attributes = attrs

    elif category == "vocabulary":
        vocab = dict(profile.vocabulary or {})
        notes = list(vocab.get("learned_notes", []))
        notes.append(learned_note)
        vocab["learned_notes"] = notes[-5:]
        profile.vocabulary = vocab

    elif category in ("length", "structure"):
        style = dict(profile.sentence_style or {})
        notes = list(style.get("learned_notes", []))
        notes.append(learned_note)
        style["learned_notes"] = notes[-5:]
        profile.sentence_style = style

Five categories, three JSON sub-keys. And get_brand_voice_summary is the only code in the system that turns a profile into a system prompt.

It read one of the three.

Tone and style reached the model. Vocabulary, length and structure corrections were captured, counted, and rendered in the UI at GET /memory/learning as "what your crew has learned," then dropped before any model saw them. The feature worked end to end everywhere a human would look and stopped one function short of the prompt.

The reader now walks all three:

        if profile.vocabulary:
            preferred = profile.vocabulary.get("preferred", [])
            avoided = profile.vocabulary.get("avoided", [])
            if preferred:
                parts.append(f"Preferred words: {', '.join(preferred[:10])}")
            if avoided:
                parts.append(f"Avoided words: {', '.join(avoided[:10])}")
            for note in profile.vocabulary.get("learned_notes", []):
                parts.append(f"Learned vocabulary: {note}")
        if profile.sentence_style:
            description = profile.sentence_style.get("description")
            if description:
                parts.append(f"Style: {description}")
            for note in profile.sentence_style.get("learned_notes", []):
                parts.append(f"Learned style: {note}")

The description guard is part of the same fix. A profile carrying only learned notes had been appending a bare "Style: " to every system prompt.

The test that pins this parametrizes all five categories through write, then read, then assembled prompt. It fails 4 of 6 against the pre-fix reader and passes 6 of 6 after, which is the only evidence that makes it a regression test rather than decoration.

The failure mode this guards against is the one a coding agent produces most readily. A write path, a read surface, and a UI label can all be individually correct and still describe a feature that does not exist.

What it measures, and what it does not

Replacing the old keyword search with real embeddings moved three numbers, measured over 34 documents and 17 queries:

recall@5          0.28 -> 0.87
MRR               0.20 -> 0.69
score separation  0.00 -> 0.15

That 0.00 is not rounding. The old implementation returned a hardcoded score: 0.5 for every hit, so any caller thresholding on it was thresholding on a constant, and its separation was zero by construction. Separation is in the suite specifically to fail a retriever with perfect recall and perfect MRR whose scores carry no information.

Now the counted limit. Recall@5 of 0.871 is the headline, and here is what it costs to say honestly. The gate is calibrated only at k=5: at k=1 the same healthy retriever scores 0.478 and the gate false-alarms, and at k=20 it scores 1.000 and the gate asserts nothing. One query in the suite, q-funding ("how much did we raise and from whom?"), fails outright on both retrievers and is kept failing on purpose, because a suite where every case passes has stopped measuring. And separation of 0.151 is real signal that is not a usable threshold: the query with no correct answer came back at a top score of 0.536, while the gate that would catch a genuine collapse sits at 0.70, set there only because bge-small's cosine scores for unrelated English do not approach zero. Ranking works. Deciding whether to answer at all is a capability this layer does not have and does not claim.

Two more honest edges. The SQLite path is O(n) in the org's rows, measured at 84.9 ms for 1,000 rows at 384 dimensions and 928.8 ms for 10,000. Interpolating between those two points puts the 100 ms budget at about 1,200 rows, which is where in-process cosine stops paying, and those timings are a floor because they were measured against in-memory SQLite with no network. And nothing on the product surface tells a user that an unembedded row is invisible to the crew. Every path that leaves one logs it, which helps an operator and not a customer.

The one-liner: a memory the crew can actually use is one where the text, the vector, and the model's name were written in the same breath and read by the same function.

Next in the series: post 5, the evaluation layer, and what it takes to turn a claim like "the crew learns your voice" into a number a gate can fail on.

The views expressed here are my own and are not related to or reflective of my work or any organization I am affiliated with.