← Library

One Mind, Many Compartments

An agent worth having remembers what you told it, in full, including the parts you would not say twice. The same agent talks to someone else tomorrow. Both have to be true at once. Getting there means separating two questions almost every memory system stores as one, and then being disciplined about which of the two is allowed to decide anything.

Prefer to listen? This design is also a 19-minute podcast, Securing the AI Cognitive Layer, recorded as a companion to this piece.

Start with what you want, because most writing on this subject starts with the fear and ends up describing an agent nobody would use.

You want an agent that knows you. Not shallowly, with your name and time zone on file. In the sense that you told it the real situation, the constraint you are slightly embarrassed by, the thing that went wrong last spring, and it remembers. Every conversation after that starts from there instead of from nothing. That accumulated understanding is not a side effect of the product. It is the product. An assistant that forgets you between conversations is a search box with better manners.

Now the uncomfortable half. That same agent talks to your colleague on Tuesday. The honest question, the one people ask silently and rarely out loud, is not "is my data encrypted." It is:

What, structurally, stops it from telling the next person what I told it?

Encryption does not answer that. Encryption protects you from someone who steals the disk. It does nothing about an agent holding your material in working memory while it composes an answer for somebody else.

Two non-answers

The first easy way out is an agent that refuses to retain anything personal. Nothing stored, nothing leaked. It is airtight and worthless, because it solves the problem by deleting the feature. The reason to give an agent memory is to hold what matters, and what matters is usually what you would not want repeated.

The second is access control: label each memory with who it belongs to, check the label on the way out. This is what most systems do, and it fails for a reason worth stating precisely.

Access control requires that every path that reads memory remembers to check. Not most paths. Every one, including the ones written next year by someone who never read the design. A memory system has many: the lookup tool, the browsing list, search, export, the admin view.

And one path has no interface at all. Modern agents pull relevant fragments into working context automatically, before answering, on relevance alone. Nobody clicked anything. Nothing appeared in a list. A passage simply became part of what the agent was thinking with while replying to someone else. If that step is scoped wrong, the system leaks with no user action, no tool call, and no log line resembling an access event.

A permission you have to remember to check is a permission you will eventually forget to check.

We should be careful about what replaces it, though, because there is a tempting overclaim here. You cannot make authorization disappear. Something must still decide what an agent may reach. What you can do is stop scattering that decision across every feature and force all of it through one mandatory boundary that features cannot opt out of. That is a real and large improvement, and it is a different claim from "no permissions."

The question almost everyone stores as one field

Here is the root error, and once you see it the rest of the architecture follows rather than being invented.

Every durable memory has to answer two independent questions:

The case that proves they are different is the one that breaks naive designs immediately. Alice tells the agent that Bob is thinking about leaving.

If you file that memory under whoever was talking, it becomes Alice's, and the agent is now quietly accumulating a dossier on Bob that Bob cannot see, cannot correct, and cannot delete, stored under someone else's name. If you file it under whoever it is about, and stop there, the agent may one day repeat Alice's private assessment back to Bob.

Both are wrong because they are answers to different questions. The subject is Bob, because the interests at stake are his. The audience is Alice, because she is the one who said it. Both facts get recorded, and each governs only its own domain.

The compartment is defined by audience, never by subject

This is the part it is easy to get wrong even after separating the two questions, and it is worth being blunt about why.

The tempting move is to make the compartment a person: everything about Bob lives in Bob's compartment. But look at what Bob's compartment would then contain. Something Bob told the agent, visible to Bob. Alice's remark about Bob, visible only to Alice. Something Carol mentioned, visible only to Carol. To use any Bob-related knowledge while talking to Alice, the agent would have to open Bob's compartment and filter its contents by audience on the way out.

That is a permission check on every read. It is precisely the design we just rejected, reintroduced one level down.

So the physical compartment is the audience scope. Not the person the memory concerns, but the set of people entitled to encounter it:

Alice's remark about Bob lives physically in the Alice-and-agent compartment. Bob appears in its subject index, which is a separate structure entirely.

And that separation buys something important. Subject is never consulted during retrieval. Working out who a memory concerns is an interpretive act, and interpretive acts are fallible. Because subject has no authority over what the agent can reach, a mistake there cannot expose one compartment to another conversation. Subject does govern rights and the review that clears an onward disclosure, so an unresolved subject fails closed: nothing is shared on a guess, and a late resolution cancels any approval computed before it. The retrieval boundary never rests on a judgment call.

"About nobody" and "shareable" are unrelated

A related trap is treating subjectless information as automatically general. It is not, and the examples are immediate.

An internal credential is about nobody and must be tightly held. A confidential acquisition plan concerns a company, not a person, and belongs to a handful of people. A published biography is entirely about a person and is available to the world. A quirk in a public API is about a product and belongs to everyone.

Once compartments are audience-shaped, this stops being a special case. Subject and audience are simply independent, as the design said they were from the start.

Scopes form a lattice, and the bottom of it is real

Order the compartments by who is in them. Alice-private is narrower than the Alice-and-Bob room, which is narrower than the team.

Now the rule that does most of the work in practice: anything derived from several inputs lands in the narrowest compartment compatible with all of them. Combine something Alice told you privately with an organization-wide document and the result is Alice-private, because that is the only audience entitled to both halves.

Follow that to its conclusion and you find the bottom of the lattice. Combine something Alice said privately with something Bob said privately and the resulting audience is nobody at all. That compartment exists, it is a real destination, and nothing in it can ever surface to anyone. An agent should not be able to manufacture a usable conclusion by mixing two people's confidences, and here it structurally cannot.

The consequence people skip is what "derived" covers. It is not just summaries. It is beliefs, plans, extracted entities, tool arguments, agent-to-agent messages, cached responses, evaluation data, and embeddings. An embedding of a private passage is that passage, in a form that is still searchable. So there is no global vector index. Indexes are compartmented like everything else. A system that partitions its database and shares one embedding space has not partitioned anything.

The audience is not the people in the conversation

Here is the failure that survives even a correct retrieval design, and it is the one most likely to bite in a product that actually does things.

Alice asks the agent to email Bob. Bob is not in the conversation. He is nonetheless about to receive whatever the agent produces. The same is true of posting to a channel, updating a shared record, writing into a document, creating an invitation, filing a ticket, or handing work to another agent that has its own principals.

So the audience that matters is not the participant list. It is everyone who ends up exposed to the result: participants, recipients, anyone who can read the destination, anyone who observes the side effect, and any counterpart agent along with the people it answers to.

A memory may inform an execution only when that entire set is entitled to it.

Stated flatly, that sounds like it would make the agent useless: your private context could never inform anything you send. The resolution is a distinction worth naming, because it is what keeps the system pleasant to use.

Generate under scope. Disclose under audience.

When the agent drafts something and hands it back to you for approval, the only person exposed is you. It may draw on everything you have ever told it. Nothing has been disclosed, because you are the one deciding what leaves. The moment you send it, sending becomes its own decision, and that decision is judged by where the draft's ingredients came from, never by re-reading the finished words. Text can be paraphrased; provenance cannot. If every interest in the draft's ingredients is yours and the destination's own use and retention rules are compatible, sending is one click. If something in it came from someone else, concerns someone else, or belongs to the organization rather than to you, the relevant person or policy is what clears it. Your click records responsibility. It does not grant permission you never had.

The rule tightens exactly where it should. If the agent is acting autonomously, with no human reading the output before it lands, then the recipients are in the audience from the first token, and your private material never enters the context at all. Review is what makes richer context safe, and its absence is what makes it unsafe.

The same discipline covers what the agent looks up, which is where most designs quietly leak. Suppose your private question sends the agent to a company system, and the answer it fetches is something the whole company could technically read. The fetched answer is not neutral: which record came back, and why, encodes your question. So a lookup made on your behalf inherits both sides: the result carries your compartment's walls and whatever restrictions the source itself had, and its audience is the overlap of the two. A lookup never loses either side merely because one of them is broader. What the agent retrieves because of you is part of what it knows about you.

And at the very end of every path there is one door. Every reply, email, post, document write, and message to another agent leaves through a single delivery gate that checks, at the moment of sending, that everyone about to be exposed is entitled to what is leaving, and that the destination still is what it was when the decision was made. A channel that gained a member between approval and delivery is a different channel. One door, checked last, is what makes all the earlier discipline mean something at the boundary where it actually counts.

Rooms change, and history does not follow

Compartments with more than two members raise a question with no default-correct answer, so it has to be decided rather than discovered.

Alice and Bob have been working with an agent in a shared room. Charlie joins. Does Charlie inherit the history?

For a conversational room, no. Membership grants access from the moment of joining forward. The prior history stays with the people who were present for it, because they spoke on that basis. For an institutional compartment, a project or a team or an organization, the answer flips: the content is organizational rather than confidential-between-participants, and joining the team is meant to give you the team's context.

Departure differs the same way. Leaving a conversational room ends access to everything after that moment, while access to the stretches you were actually present for may remain, since you already held them. Leaving an institutional scope can revoke access to the stored history as well. Neither policy can retract what someone already received, and neither erases the record that they were present, because that record is what makes later questions answerable.

How something becomes general

Some of what an agent should know really is about the work rather than about anyone, and some of that can only be learned from working with people. That crossing is the interesting part, and the naive version of it fails badly.

The naive version strips the names and promotes. Watch it fail:

"When someone has run a small nonprofit in a mid-size Midwestern city for about eight years and is burned out, pushing them toward growth targets makes them disengage."

No name. Possibly one person it could describe, identifiable to anyone who knows the community, and it now reports something they may have told nobody else. People are rarely re-identified by name. They are re-identified by the intersection of three or four unremarkable attributes.

The deeper problem is that it came from one conversation. A generalization drawn from one person is not a generalization. It is that person's data wearing a costume. Rewording it changes nothing, because the information content is unchanged.

So do not ask the unanswerable question. "Is this text free of identifying information" is a semantic judgment made case by case, millions of times, by a system that will be wrong sometimes without saying which times. Ask instead how many independent people a pattern comes from, and then apply the test that does the real work: remove any single contributor and the pattern must still hold. If it collapses when one person is taken away, it was never a pattern. It was that person, restated.

We should name this accurately rather than reach for a more impressive label. This is support with leave-one-out stability. It resembles k-anonymity and it is not the same thing and does not inherit its guarantees. It does not defeat membership inference, composition across several lessons, or an attacker who already knows a lot about the group. It is a strong practical filter, not a proof.

And one thing this crossing is not: an escape hatch from the bottom of the lattice. An artifact with no audience, made by mixing two people's confidences, stays unusable forever. What becomes general starts upstream, as a constrained candidate drawn independently inside each relationship, and only those candidates are ever compared.

Counting people is not the same as counting evidence

The hard part of corroboration is not arithmetic. It is deciding what counts as independent.

Five people are not five sources if they all read the same memo, or all report the same meeting, or all work for the manager who told them. And there is a failure unique to agents that deserves its own name: if the agent itself suggested something to all five, and all five later reflect it back, that is not five confirmations. That is an echo of its own voice, arriving with the appearance of consensus.

So independence has to be assessed across several dimensions at once, including whether the contributor had already been exposed to the very claim they are now supposedly confirming. A contribution that traces back to the agent counts for nothing.

Two further separations matter, and collapsing either is a mistake we would rather avoid than apologize for later.

Qualifying is not being true. A pattern can pass every privacy test and be wrong. Many people independently repeat things that are false, local, temporary, or an artifact of one environment. Whether something may be shared and whether it should be believed are different questions with different evidence, and the system keeps them in different fields.

Promotion changes what the agent knows, never what it is permitted to do. No amount of corroboration can turn a learned pattern into an instruction, a security rule, or an autonomous action policy. This closes the door on a real attack, where coordinated users try to install a preference by repetition, and it matches a boundary this platform already holds elsewhere: capability compounds automatically, and identity changes only with an owner's consent.

One more decision, which the shape of the model makes obvious once stated. A pattern that has proven itself inside one organization is promoted to that organization's compartment. It does not become universal simply because it qualified. Crossing between customers is a different act requiring different authority, and the default is that it does not happen.

Forgetting the way back without losing the ability to erase

When a pattern crosses into a broader compartment, it must not carry a path home. If the general claim cites the conversations behind it, then anyone who reaches the claim is one step from the personal material, and you are relying on permissions again. That is a door with a good lock where the requirement was a wall.

Which raises a tension most privacy writing skips. If nothing points back, and someone exercises their right to be forgotten, how do you remove their contribution?

The answer is that these are two systems, and only one of them belongs to the agent.

The shared claim holds text and operational metadata: how strongly it is supported, what contradicts it, where it applies, when it should be reviewed, what supersedes it. It holds no citations and no source list. Separately, outside the cognitive layer entirely, a ledger holds a pseudonymous mapping, enough to locate every claim an erasure request touches, plus proof that each claim qualified. It is not a browsable list of who said what. The agent cannot read that ledger. It is not part of the world the agent reasons in, it never enters a prompt, and it is not in an administrator's view either, because knowing who contributed to a pattern can itself say something about them.

So erasure works. The ledger locates every claim a person supported, and any claim that no longer survives the leave-one-out test without them is withdrawn. The wall holds, because the traversal happens somewhere the agent has no access to. Compliance can walk the path. The agent cannot.

And erasure goes all the way down. The design seals each stored item under its own key, held in a vault that backups never capture. Deleting the item destroys that key, which makes every copy, including the ones sitting in immutable backups, permanently unreadable. "Deleted" does not mean hidden from the interface while a backup remembers. It means the ciphertext outlives its key, which is to say it means nothing at all.

Seeing what is held about you, without exposing who said it

A privacy claim the protected person cannot verify is a marketing claim. So a person can ask what the system holds concerning them, and get a real answer.

But this is exactly where a careless design undoes its own work. Alice's private remark about Bob has Bob as its subject. If subject access simply returned every raw memory tagged with your name, Bob would learn that Alice reported on him, and Alice's confidence would be broken by the very feature meant to protect Bob.

Subject rights are therefore served by a separate path, not by a query over storage. It can tell you that information concerning you exists and what it concerns. It can let you contest or correct it. It can suppress or remove it. What it does not do is hand you another person's words as though they were your record. When rights genuinely conflict, because information belongs to more than one person at once, that is a case for a decision rather than an automatic answer, and the honest design says so instead of pretending the conflict does not exist.

The same logic governs sharing. Nothing widens in place. A disclosure creates a new artifact in a broader compartment while the original stays exactly as restricted as it was. When you are both the source and the subject, that is a single deliberate act and it should be one click. When more than one person's interests are in the same sentence, one of them cannot unilaterally speak for the other, and the system routes it accordingly.

Where "administrators cannot read it" actually stops

An absent screen is not a guarantee. If the claim is that the person running an agent cannot read your compartment, something has to enforce it.

That means compartments carry their own keys, and the process serving a conversation receives temporary decrypt capability for exactly what that conversation was authorized to reach, never custody of a compartment's keys. It means there is no general-purpose query endpoint: the assembly boundary is the only reader that can put private content in front of an agent, and the few isolated services that must touch content for rights, deletion, repair, or promotion operate under one-use, purpose-bound grants with no conversational or human-facing output at all. It means impersonation is not a feature. It means support tooling sees metadata and whatever a user chooses to share in a session, not a back door. It means logs and traces carry opaque identifiers and scope-keyed commitments rather than content, since a redacted architecture with verbose logging is not redacted.

And it means being precise about who "administrator" excludes. Against the owner of an agent and against application administrators, this is a real structural guarantee. Against the operators of the infrastructure itself, it is as strong as the separation between key custody and operations, and no stronger. Claiming more than that would require hardware isolation we would have to actually build. We would rather draw the line where it really falls.

Then where does the intelligence come from?

A fair objection to everything above: if the bar for shared knowledge is this high, the shared compartment will be nearly empty, and the agent will be private and stupid.

That objection is correct about the arithmetic and wrong about the architecture, because it assumes the only way an agent gets smart is by mining the people it talks to.

Most of what makes an agent genuinely capable never touches anyone's confidence. It is the documentation it was given. The tools and their behavior. The standards, playbooks, and hard-won doctrine its owner deliberately wrote down. Reference material brought in at its own audience and scope. Deliberately authored doctrine and public technical material is the main supply, and it never needed mining from anyone's confidences. Arbitrary documents and tool results are not automatically general just because they did not arrive as conversation: they keep whatever restrictions and taint they came with.

Promoted lessons are the garnish. Real, valuable, slow, and additive. Which produces a property worth stating plainly: if the promotion machinery never worked at all, the system would still be correct and still be useful. It would simply be less clever. A privacy guarantee that depends on the most speculative component working is not a guarantee, and this one does not.

What this does not claim

We would rather name the limits than have a reader find them.

What the structure does buy is worth being equally exact about. Another person's relationship-private material is not present in your conversation to be retrieved, no configuration puts it there, and everything derived from it stays as restricted as its narrowest ingredient. That removes the class of failure that ordinarily dominates this problem and leaves a much better question in its place.

The principle

An agent should hold everything you told it privately and be incapable of repeating it outside that relationship, unless a disclosure you authorize creates something new.

Not unwilling, and not configured not to. Incapable, because your compartment is not part of the world it inhabits when it is speaking to someone else, because anything it derives from you inherits your walls, and because the only routes out are a decision you make or a pattern that has stopped being about you.

One mind. Many compartments. The agent gets wiser everywhere, and what you told it stays exactly where you told it.

Part of the thinqOS science series.

It holds everything you told it, and cannot repeat it.

Read the point of view, or get into the private preview.

Request Private PreviewRead the Point of View →