← back to the archiveCover illustration for “Deleting a memory needs a test against the source transcript”
POSTday 90·2w ago·by Andy Padia

Deleting a memory needs a test against the source transcript

A public agent implementation stores memories and transcripts separately. Erasure should be tested across the paths that can recreate the deleted fact.

The public Saul agent repository describes an append-only transcript store alongside its distilled memories. Its August 6 code gives those layers different write and delete paths: the memory deletion function removes a matching database row, while conversation exchanges are appended to daily transcript files. The dated memory documentation says transcript search runs alongside memory retrieval.

That is enough to identify a test the delete button cannot answer by itself: can the system retrieve the source passage and recreate the fact after its distilled memory has been removed?

I have not audited the private deployment or established that it lacks a separate purge process. The public implementation illustrates a broader design risk: deleting one representation is a narrower operation than erasing information across a retrieval system.

My acceptance rule is to test the fact’s reachable sources and derivatives, including the jobs that can rebuild them.

Follow the information, not the button

In a hypothetical enterprise assistant, a statement from a meeting might appear in the original transcript, a summary, an embedding index and a saved memory. Removing the memory changes one location. A later summarisation or indexing job can encounter the transcript again.

I would map those dependencies before describing the user-facing operation as forgetting. The map should identify which stores can answer directly and which processes can generate new records from older material.

Then I would give the product action an honest scope. Delete this saved memory and remove this information from future retrieval are different promises. If the product offers the broader promise, its implementation needs to reach the broader set of paths.

That distinction matters legally as well. GDPR Article 17 provides erasure rights when specified grounds apply and includes exceptions. It is not an unconditional instruction to destroy every record in every situation. The technical workflow should implement the applicable decision, including any lawful retention restriction, rather than decide the legal scope by accident.

Rebuild before declaring success

For a hypothetical acceptance test, I would introduce a unique synthetic fact through the same route as an ordinary meeting statement. Let the normal summary and indexing jobs run. Confirm which retrieval routes can find it, then invoke the intended deletion process.

The first verification would query the relevant stores using exact terms and paraphrases. The second would restart the service and rebuild its indexes. The third would run the summarisation process again. A fact that returns after rebuilding was hidden temporarily, not removed from the source path that feeds the assistant.

Backups need a documented treatment too. Where immediate deletion is not the applicable mechanism, restoration procedures must preserve the required restrictions instead of quietly restoring the information to active use. The test should match the organisation’s actual retention decision.

I would save a receipt naming the request, affected stores, completed actions and remaining authorised restrictions. The user-facing confirmation should describe that result without promising more than the system performed.

Memory makes an assistant useful because information survives individual conversations. The same persistence means forgetting has to be designed across the system, not implemented as a single database statement.

A memory is gone from future retrieval only when its surviving sources cannot quietly teach it back to the agent.

#agents#memory#privacy
← older drop
Cheap alignment search makes failure discovery more valuable
newer drop →
A licensing deal can change the supplier you depend on

related drops

explore all 243 drops →
← back to the archiveday 106