aimu.memory¶
Semantic facts, path-based documents, and a shared abstract base.
aimu.memory.MemoryStore ¶
Bases: ABC
Abstract interface for memory store implementations.
All implementations must support storing, searching, deleting, and listing content. Concrete subclasses add implementation-specific methods on top of this baseline.
search
abstractmethod
¶
Return up to n_results strings relevant to query.
Concrete stores may accept additional store-specific keyword arguments (e.g.
SemanticMemoryStore.search(..., max_distance=...)); those are not part of this
interface, so code typed against MemoryStore should not rely on them.
delete
abstractmethod
¶
Remove content by identifier (path, exact string, or UUID).
aimu.memory.SemanticMemoryStore ¶
SemanticMemoryStore(collection_name: str = 'memories', persist_path: Optional[str] = None, embedding_client: Optional[Any] = None)
Bases: MemoryStore
Persistent store for factual memories backed by ChromaDB.
Memories are natural-language strings in subject-predicate-object form, e.g. "Paul works at Google" or "Sarah is the sister of Emma". ChromaDB embeds each fact so that topic-based retrieval ("work", "family life") finds semantically relevant facts without exact-string matching.
Thread-safe: every public method is serialized on a re-entrant per-instance
lock, so one store can be shared across concurrent threads or turns (e.g. an
async agent dispatching sync memory tools via asyncio.to_thread).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
collection_name
|
str
|
ChromaDB collection name (default: "memories"). |
'memories'
|
persist_path
|
Optional[str]
|
Directory path for persistent storage. If None, uses an in-memory ephemeral client. |
None
|
embedding_client
|
Optional[Any]
|
Optional AIMU embedding client (e.g. from
|
None
|
Examples:
>>> store = SemanticMemoryStore()
>>> store.store("Paul works at Google")
>>> store.store("Paul is married to Sarah")
>>> store.store("Sarah is the sister of Emma")
>>> store.search("work") # ["Paul works at Google"]
>>> store.search("family life") # marriage / sibling facts
>>> client = aimu.embedding_client("openai:text-embedding-3-small")
>>> store = SemanticMemoryStore(embedding_client=client)
store ¶
Store a fact string in the memory store.
The fact should be a natural-language sentence in subject-predicate-object form, e.g. "Paul works at Google" or "Sarah is the sister of Emma".
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
content
|
str
|
The fact string to store. |
required |
search ¶
Retrieve facts semantically related to a query.
Uses ChromaDB's vector similarity search so broad topics like "work" or "family life" match relevant facts without exact-string matching.
Note: max_distance is specific to this store and is not part of the
:class:~aimu.memory.base.MemoryStore interface; code typed against the base
MemoryStore (or a DocumentStore, which has no distance notion) cannot pass
it. Type the variable as SemanticMemoryStore to use it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
query
|
str
|
Natural-language query (e.g. "work", "family"). |
required |
n_results
|
int
|
Maximum number of facts to return. |
10
|
max_distance
|
float | None
|
Optional cosine-distance cutoff (0 = identical, 2 = maximally dissimilar). Facts with a distance above this threshold are excluded. Typical useful range: 0.3 (tight) – 0.6 (loose). Defaults to None (no cutoff). |
None
|
Returns:
| Type | Description |
|---|---|
list[str]
|
List of fact strings ordered by relevance. |
delete ¶
Remove all stored facts that exactly match the given string.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
identifier
|
str
|
The exact fact string to remove. |
required |
aimu.memory.DocumentStore ¶
DocumentStore(persist_path: Optional[str] = None, *, index_path: Optional[str] = None, embedding_client: Optional[Any] = None, max_file_bytes: int = 50000000, extensions: Optional[Collection[str]] = None)
Bases: MemoryStore
Path-addressed document store backed by the filesystem or an in-memory dict.
Implements the :class:MemoryStore abstract interface so it can be swapped
with :class:SemanticMemoryStore in any application, and adds the richer
path-based API in the spirit of Anthropic's memory tool (not its interface).
Thread-safe. Writes (write, edit, delete, store) are serialized on a re-entrant
per-instance lock, so edit -> read + write stays atomic. Reads are not serialized behind
each other: reading a file, and extracting a PDF's text, happen outside the lock, which is taken only to
consult and fill the caches, so one slow PDF never holds up a read or a write of another file. Searches
of the passage index are serialized with each other.
With persist_path set, the directory is the single source of truth: reads, listings, and searches hit the filesystem on each call, so a document copied into the directory by hand shows up without restarting the process. Files that are neither UTF-8 text nor a PDF with a text layer (an image, a binary artifact), and files over max_file_bytes, are not documents; they are skipped, with a warning naming the file, and dot-files are ignored outright. So is anything whose real path lies outside the directory (a symlink out of it), which also cannot be written or deleted through the store.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
persist_path
|
Optional[str]
|
Root directory for persistent storage, |
None
|
index_path
|
Optional[str]
|
Directory for the passage index behind :meth: |
None
|
embedding_client
|
Optional[Any]
|
An AIMU embedding client ( |
None
|
max_file_bytes
|
int
|
Files larger than this are unreadable rather than read, and :meth: |
50000000
|
extensions
|
Optional[Collection[str]]
|
The file suffixes that are documents, matched case-insensitively, e.g.
|
None
|
Examples:
>>> store = DocumentStore()
>>> store.write("/preferences.md", "Use concise responses.")
>>> store.read("/preferences.md")
'Use concise responses.'
>>> store.edit("/preferences.md", "concise", "detailed")
>>> store.search_full_text("detailed")
[{"path": "/preferences.md", "content": "Use detailed responses."}]
write ¶
Create or overwrite a document at path.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str
|
Memory path, e.g. |
required |
content
|
str
|
Text content to store (≤ 100 KB recommended). |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
If path is a PDF, if its suffix is not one of the store's extensions, if it resolves outside the folder, or if content is over max_file_bytes once encoded as UTF-8. |
read ¶
Return the content of the document at path.
Raises:
| Type | Description |
|---|---|
KeyError
|
If no document exists at path. :meth: |
edit ¶
Replace the one occurrence of old_str with new_str in the document at path.
old_str must appear exactly once. Zero matches and two-or-more matches both raise, and nothing is written either way: replacing the first of several matches answers a question the caller did not ask, and it does so silently, in the file, where the mistake is discovered later by whoever reads it. Include enough surrounding text to make the match unique.
To genuinely intend the first of several, say so where it can be seen::
store.write(path, store.read(path).replace(old_str, new_str, 1))
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str
|
Memory path of the document to edit. |
required |
old_str
|
str
|
Exact substring to find, unique within the document. |
required |
new_str
|
str
|
Replacement text. |
required |
Raises:
| Type | Description |
|---|---|
KeyError
|
If no document exists at path. |
ValueError
|
If old_str is absent, or appears more than once. |
list_paths ¶
Return all memory paths, optionally filtered by prefix.
Reads every file to leave out the ones that are not documents, which on a large folder is slow
the first time (each PDF is extracted) and afterwards still opens every text file. To list a large
folder, :meth:listing reads nothing.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
prefix
|
Optional[str]
|
Only return paths that start with this string. Pass None (default) to return all paths. |
None
|
Returns:
| Type | Description |
|---|---|
list[str]
|
Sorted list of path strings. |
listing ¶
List the folder from its metadata alone: no file is opened, so this costs a walk however large it is.
Where :meth:list_paths reads every file to leave out the ones that are not documents, this lists
every candidate file as a document unless it is already known not to be one: over
max_file_bytes (from its size), or found unreadable by an earlier read (a search, an index
build, :meth:read, :meth:unreadable) and unchanged since. So a file nothing has read yet is
listed, and turns up in unreadable once something has read it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
prefix
|
Optional[str]
|
Only list paths that start with this string. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
A |
DocumentListing
|
class: |
search_full_text ¶
Case-insensitive substring search across all document contents.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
query
|
str
|
Search string. |
required |
n_results
|
int
|
Maximum number of results to return. |
10
|
Returns:
| Type | Description |
|---|---|
list[dict]
|
List of |
list[dict]
|
documents, ordered by path. |
unreadable ¶
Return the files in the store's directory that cannot be read as documents, with why.
These are files someone put in the directory that never become documents: a scanned or
password-protected PDF, a Word export, a binary artifact. They are excluded from
:meth:list_paths and :meth:search_full_text, so this is what lets a caller tell the
difference between "you gave me nothing" and "what you gave me cannot be read", and say why.
Dot-files are not reported: they are not documents anyone placed here, and nor are files
whose suffix extensions does not allow.
Rescans the directory, reading every file; returns {} for an ephemeral store, which has no
directory. :meth:listing reports the reasons already on record without reading.
Returns:
| Type | Description |
|---|---|
dict[str, str]
|
Mapping of normalized memory path to a human-readable reason. |
unreadable_reason ¶
Why the file at path exists but is not a readable document, or None if it is one or is absent.
:meth:read raises KeyError for both a missing document and a file it cannot read; this tells
them apart, which is what a caller about to write to path needs, since writing would replace a
file it cannot see. Reads the one file. A file whose suffix extensions does not allow is no
candidate, so it gets None: :meth:write refuses that path anyway.
search_passages ¶
Rank passages of every document against query, best first.
Where :meth:search_full_text returns whole documents containing a substring, this returns
chunks of documents (about 1200 characters, each section or PDF page chunked on its own), each
with its location, ranked by BM25 over the query's words. Any query is accepted: punctuation and
search-syntax characters are dropped, never raised on, and common English words are ignored.
With an embedding_client, passages are also ranked by vector similarity, which finds one that
says the same thing in other words, and the two rankings are fused by reciprocal rank (see
:func:aimu.rag.reciprocal_rank_fusion). A failing embedding model makes the search keyword-only
and is reported by :meth:index_status; it does not raise, and neither does a file that fails to
index, which is left out and reported the same way.
The folder is the source of truth. A metadata walk before each search re-indexes only files
whose size or modification time changed, so a file edited, added, or deleted out-of-band is
reflected at once. A file that could not be opened (its permissions, or a cloud-synced file not
downloaded yet) is tried again a minute later, changed or not. The index is built on the first
call. A large build, and all embedding, carry on in a background thread, so a search during one
ranks what is indexed so far; check :meth:index_status to tell a partial answer from a complete one.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
query
|
str
|
Natural-language or keyword query. |
required |
n_results
|
int
|
Maximum number of passages to return. |
8
|
Returns:
| Type | Description |
|---|---|
list[Passage]
|
class: |
Raises:
| Type | Description |
|---|---|
ImportError
|
If an embedding_client was given and chromadb is not installed. |
index_status ¶
Report how much of the store the passage index covers, and what it cannot read.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
refresh
|
bool
|
Bring the index in step with the folder first (a metadata walk). Pass False straight after a search, which has just done that, to report on the same walk; the first call refreshes regardless, since there is no index before it. |
True
|
close ¶
Stop the passage index's background work and release its database, if one was opened.
The store stays usable; the next search reopens the index.
unreadable_paths ¶
Return the sorted memory paths of files that cannot be read as documents.
See :meth:unreadable for the reasons.
store ¶
Store content at an auto-assigned path (/note-{uuid}.md).
Use :meth:write directly when you need control over the path.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
content
|
str
|
Text content to store. |
required |
search ¶
Full-text search; returns a list of matching content strings.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
query
|
str
|
Search string. |
required |
n_results
|
int
|
Maximum number of results. |
10
|
Returns:
| Type | Description |
|---|---|
list[str]
|
List of content strings from matching documents. |
delete ¶
Delete the document at identifier (treated as a memory path).
No-op if the path does not exist, or names a file whose suffix extensions does not allow, which is no document of this store's.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
identifier
|
str
|
Memory path of the document to remove. |
required |
aimu.memory.Passage
dataclass
¶
One chunk of a document, as a search returns it.
Attributes:
| Name | Type | Description |
|---|---|---|
path |
str
|
The document's store path, e.g. |
location |
str
|
Where in the document the chunk sits: a heading path ( |
text |
str
|
The chunk itself. |
score |
float
|
Relevance, higher is better. Comparable only within one search. |
digest |
str
|
SHA-256 hex of the whole document's text when it was indexed. Two passages from copies of the same file share it, and a digest that no longer matches the file means the passage is stale. |
aimu.memory.IndexStatus
dataclass
¶
IndexStatus(files: int, indexed: int, embedded: int, unreadable: dict[str, str], problems: tuple[IndexProblem, ...] = (), vectors: bool = False, failed: int = 0)
How much of a document folder the index covers.
Attributes:
| Name | Type | Description |
|---|---|---|
files |
int
|
Documents in the folder: every file, less those found unreadable. |
indexed |
int
|
Documents whose current text is keyword-searchable. Below files while a build is running. |
embedded |
int
|
Indexed documents whose passages also have vectors (always 0 when vectors is False). |
unreadable |
dict[str, str]
|
Files in the folder that cannot be documents, mapped to why. |
problems |
tuple[IndexProblem, ...]
|
What is wrong, if anything, one :class: |
vectors |
bool
|
Whether the index has an embedding client, so embedded counts toward complete. |
failed |
int
|
Documents that failed to index and are left out until they change. Counted in files, never
in indexed, and not work still to come: |
complete
property
¶
Whether the index has nothing left to do, so a search that finds nothing found nothing.
Files that failed to index are the exception: they wait for a change, not for the index, so a store
whose only gap is failed files is complete, and its "file" problem is what says they are missing.
error
property
¶
Every problem's message, joined by "; ", or None when there are none.
has_problem ¶
Whether a problem of kind ("embedding", "file", "worker") is reported.
aimu.memory.IndexProblem
dataclass
¶
One thing wrong with an index, as :attr:IndexStatus.problems reports it.
Attributes:
| Name | Type | Description |
|---|---|---|
kind |
Literal['embedding', 'file', 'worker']
|
|
message |
str
|
What happened, in a sentence fit to show a person or a model. |
aimu.memory.DocumentListing ¶
Bases: NamedTuple
What :meth:DocumentStore.listing returns.
Attributes:
| Name | Type | Description |
|---|---|---|
paths |
list[str]
|
Sorted memory paths of the folder's documents, including any not read yet. |
unreadable |
dict[str, str]
|
Memory path to reason, for the files already found not to be documents. |
aimu.memory.CollectionPassage
dataclass
¶
A passage from :func:search_collections, labeled with the collection it came from.
Attributes:
| Name | Type | Description |
|---|---|---|
collection |
str
|
The key of the store the passage was found in. |
passage |
Passage
|
The passage itself. |
also_in |
tuple[tuple[str, str], ...]
|
|
aimu.memory.search_collections ¶
search_collections(stores: Mapping[str, DocumentStore], query: str, n_results: int = 8, *, rerank: bool = False, on_error: Optional[Callable[[str, Exception], None]] = None) -> list[CollectionPassage]
Search several stores at once and return one ranking of passages, each labeled with its collection.
Each store ranks its own passages (:meth:DocumentStore.search_passages); the rankings are merged
by reciprocal rank fusion, which uses only ranks, so stores of different sizes and scoring need no
common scale. A passage held by several stores (the same document text at the same location, e.g. one
paper filed in two collections) is returned once, at its best rank, with the others in also_in.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
stores
|
Mapping[str, DocumentStore]
|
Collection name to store. The mapping's order breaks ties and orders |
required |
query
|
str
|
Natural-language or keyword query. |
required |
n_results
|
int
|
Maximum number of passages to return. |
8
|
rerank
|
bool
|
Reorder the fused top |
False
|
on_error
|
Optional[Callable[[str, Exception], None]]
|
Called with a collection's name and the exception when its store's search raises; that store is then left out and the rest are still searched. Without it, the exception propagates, so one broken store fails the whole search. |
None
|
Returns:
| Type | Description |
|---|---|
list[CollectionPassage]
|
class: |