aimu.context¶
What a request to a model holds, as plain functions over a conversation's list[dict]. No
ContextPolicy class: an Agent(compaction=...) applies a compaction to what each request
sends, and the history stays whole. See how-to: manage context.
aimu.context.count_tokens ¶
Estimate the token count of messages.
The default counter is len(text) // 4, a rule-of-thumb estimate -- it is not a
real tokenizer and will be wrong (usually within 20-30%) for any specific model. Exact
counts exist only after the fact, via client.last_usage on a real request. Pass
counter= (any Callable[[str], int], e.g. a real tokenizer's encode+len)
for accuracy.
text is each message JSON-serialized (structural overhead included, since it rides
along in the real request) with AIMU's own inert bookkeeping keys (timestamp,
thinking, provenance) stripped first, since a provider never sees those. See the
module docstring for the full rationale.
aimu.context.Compaction
module-attribute
¶
What Agent(compaction=...) takes: the conversation's full history in, what one request sends out.
It runs on every request of a run, on a copy of the history, so it must not mutate its argument,
and it should depend only on its argument (and its own settings): the next request calls it again
on a longer history, and a strategy that remembered earlier calls could not be swapped for another
without losing track. :mod:aimu.context.compaction holds the ones AIMU ships.
aimu.context.compaction¶
The strategies, and the ready-made compactions built from them, bundled the way
aimu.tools.builtin bundles tools.
Ready-made compactions¶
aimu.context.compaction.mask ¶
A compaction that runs :func:mask_messages with these settings on every request.
options are :func:mask_messages' keyword arguments, checked now rather than on the first
request, so a misspelled one raises here.
aimu.context.compaction.trim ¶
A compaction that runs :func:trim_messages with these settings on every request.
options are :func:trim_messages' keyword arguments, checked now rather than on the first
request, so a misspelled one raises here.
aimu.context.compaction.chain ¶
A compaction that runs each of compactions in turn, each on the previous one's result.
The way to combine strategies, as builtin.web + [my_tool] combines tools: masking first,
trimming as the backstop, chain(mask(n), trim(n)). Put a mask ahead of anything that drops
messages, since its notes name positions in the list it is given (see the module docstring).
Strategies¶
aimu.context.compaction.mask_messages ¶
mask_messages(messages: list[dict], max_tokens: int, *, keep_first: int = 1, keep_last: int = 2, counter: Optional[Callable[[str], int]] = None, note: NoteFn = default_note) -> list[dict]
Return messages with old content replaced by notes until they fit within max_tokens.
Every message keeps its place and role, so the list is the same length, its shape is the
history's shape (roles alternate wherever they did), and a tool call never loses its result:
only content changes. Content is masked in :data:MASK_ORDER, oldest first within each kind,
and masking stops the moment the list fits:
tool: a tool result's content becomes the note. Aget_messageresult holds another message's content, so its note names that message (following reads of reads), never the read: a note naming the read sent a model to read its own read.image: a message's image blocks become one text block carrying the note, beside any text.assistant: a reply's text becomes the note, and any tool-call argument longer than 200 characters becomes a short placeholder (the call itself stays).user: a user message's content becomes the note.
System messages are never masked, nor the first keep_first and last keep_last messages
after them (extended outward to whole tool-call groups, like :func:trim_messages): in an agent
run the first is the task. A message is masked only if its note is shorter than what it
replaces. messages is never mutated. If every candidate is masked and the list still does not
fit, it is returned as masked; follow with :func:trim_messages (see :func:chain) to bound it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
messages
|
list[dict]
|
The conversation, OpenAI message-dict format. |
required |
max_tokens
|
int
|
The budget to fit, per |
required |
keep_first
|
int
|
Leading non-system messages never masked. |
1
|
keep_last
|
int
|
Trailing messages never masked. |
2
|
counter
|
Optional[Callable[[str], int]]
|
Optional real tokenizer; see :func: |
None
|
note
|
NoteFn
|
|
default_note
|
aimu.context.compaction.default_note ¶
The note :func:mask_messages puts in place of masked content, naming the way back.
kind is one of :data:MASK_ORDER. The text says what is missing and where, that it is a
context saving rather than a gap in the conversation, and which call returns it.
aimu.context.compaction.MASK_ORDER
module-attribute
¶
aimu.context.compaction.trim_messages ¶
trim_messages(messages: list[dict], max_tokens: int, *, keep_system: bool = True, keep_first: int = 1, keep_last: int = 2, counter: Optional[Callable[[str], int]] = None) -> list[dict]
Return a new, trimmed message list that fits within max_tokens (estimated).
Always returns a new list; messages (and its dicts) are never mutated. A conversation already at or under max_tokens is returned unchanged (a copy) -- this is a no-op, not a forced rewrite.
Dropping proceeds oldest-first over the non-system messages, one indivisible group at a
time: an assistant message carrying tool_calls is always dropped together with every
tool message answering it, never separately. This is the invariant that justifies this
function over a hand-written messages[-n:] slice, which produces exactly that illegal shape.
Dropping leaves no trace, so as a compaction it belongs behind :func:mask_messages (see
:func:chain), where it only removes messages masking has already reduced to notes.
The first non-system message is kept by default (keep_first=1). In an agent run it is
the task, the user message agent.run(task) appended, and oldest-first dropping would
otherwise remove it before anything else, leaving the agent working without knowing what
it was asked. That failure raises nothing: the run still returns a plausible answer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
messages
|
list[dict]
|
The conversation, OpenAI message-dict format. |
required |
max_tokens
|
int
|
The token budget to trim down to, per |
required |
keep_system
|
bool
|
When |
True
|
keep_first
|
int
|
The number of leading messages to always protect from dropping,
counted after the system messages when |
1
|
keep_last
|
int
|
The number of trailing messages (not exchanges) to always protect from dropping, extended outward to whole tool-call groups when the raw count would otherwise split one. |
2
|
counter
|
Optional[Callable[[str], int]]
|
Optional real tokenizer; see :func: |
None
|
aimu.context.compaction.summarize_messages ¶
summarize_messages(client: Any, messages: list[dict], *, keep_first: int = 1, keep_last: int = 2, prompt: Optional[str] = None) -> list[dict]
Replace the middle of a conversation with a one-message LLM summary.
Any {"role": "system"} messages are always preserved unchanged, and are returned at
the front (a mid-conversation system message is hoisted, the same reordering
:func:trim_messages does under keep_system=True). Of the rest, the first
keep_first and the last keep_last messages (each extended outward to whole
tool-call groups, the same invariant :func:trim_messages enforces) are kept verbatim.
Everything between them (the "prefix") is rendered to a plain transcript, summarized by one
client.generate() call, and replaced with a single {"role": "system", ...} message
carrying the summary text -- system, rather than user or assistant, because it is
out-of-band context neither party actually said, not a real conversational turn.
keep_first=1 is the default for the reason it is on :func:trim_messages: in an agent
run the first non-system message is the task, and a summarizer may paraphrase it or leave
it out. The kept first messages are still shown to the summarizer, labeled as background
it should not summarize, because the instruction asks for what a continuation will need
and the task is what that need is relative to. Models tend to restate the task in the
summary anyway, a small duplication. The returned order is system messages, then the
summary, then the kept first messages, then the tail. The summary goes ahead of the kept
first messages, though it describes what came after them, because several local chat
templates reject a system message anywhere but the start of the conversation.
client is any object exposing generate(prompt: str) -> str (a plain
:class:~aimu.models.BaseModelClient, or a :class:~aimu.agents.Agent via
agent.as_model_client()); this function takes it as an argument rather than
constructing one itself, so it stays free of any provider dependency. There is no
ready-made compaction for it; see the module docstring.
When nothing lies between the kept first and last messages, no summarization call is made and the conversation is returned unchanged (a new list; messages is never mutated).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
client
|
Any
|
Anything with a |
required |
messages
|
list[dict]
|
The conversation, OpenAI message-dict format. |
required |
keep_first
|
int
|
Leading non-system messages to keep verbatim; |
1
|
keep_last
|
int
|
Trailing messages (not exchanges) to keep verbatim; see above. |
2
|
prompt
|
Optional[str]
|
Optional instruction prepended to the rendered transcript, replacing
:data: |
None
|
aimu.context.compaction.DEFAULT_SUMMARIZE_PROMPT
module-attribute
¶
DEFAULT_SUMMARIZE_PROMPT = 'Summarize the conversation below in a few sentences. Preserve concrete facts, decisions, and any open questions a continuation of this conversation would still need to know.'