Skip to content

aimu.context

What a request to a model holds, as plain functions over a conversation's list[dict]. No ContextPolicy class: an Agent(compaction=...) applies a compaction to what each request sends, and the history stays whole. See how-to: manage context.

aimu.context.count_tokens

count_tokens(messages: list[dict], *, counter: Optional[Callable[[str], int]] = None) -> int

Estimate the token count of messages.

The default counter is len(text) // 4, a rule-of-thumb estimate -- it is not a real tokenizer and will be wrong (usually within 20-30%) for any specific model. Exact counts exist only after the fact, via client.last_usage on a real request. Pass counter= (any Callable[[str], int], e.g. a real tokenizer's encode+len) for accuracy.

text is each message JSON-serialized (structural overhead included, since it rides along in the real request) with AIMU's own inert bookkeeping keys (timestamp, thinking, provenance) stripped first, since a provider never sees those. See the module docstring for the full rationale.

aimu.context.Compaction module-attribute

Compaction = Callable[[list[dict]], list[dict]]

What Agent(compaction=...) takes: the conversation's full history in, what one request sends out.

It runs on every request of a run, on a copy of the history, so it must not mutate its argument, and it should depend only on its argument (and its own settings): the next request calls it again on a longer history, and a strategy that remembered earlier calls could not be swapped for another without losing track. :mod:aimu.context.compaction holds the ones AIMU ships.

aimu.context.compaction

The strategies, and the ready-made compactions built from them, bundled the way aimu.tools.builtin bundles tools.

Ready-made compactions

aimu.context.compaction.mask

mask(max_tokens: int, **options: Any) -> Compaction

A compaction that runs :func:mask_messages with these settings on every request.

options are :func:mask_messages' keyword arguments, checked now rather than on the first request, so a misspelled one raises here.

aimu.context.compaction.trim

trim(max_tokens: int, **options: Any) -> Compaction

A compaction that runs :func:trim_messages with these settings on every request.

options are :func:trim_messages' keyword arguments, checked now rather than on the first request, so a misspelled one raises here.

aimu.context.compaction.chain

chain(*compactions: Compaction) -> Compaction

A compaction that runs each of compactions in turn, each on the previous one's result.

The way to combine strategies, as builtin.web + [my_tool] combines tools: masking first, trimming as the backstop, chain(mask(n), trim(n)). Put a mask ahead of anything that drops messages, since its notes name positions in the list it is given (see the module docstring).

Strategies

aimu.context.compaction.mask_messages

mask_messages(messages: list[dict], max_tokens: int, *, keep_first: int = 1, keep_last: int = 2, counter: Optional[Callable[[str], int]] = None, note: NoteFn = default_note) -> list[dict]

Return messages with old content replaced by notes until they fit within max_tokens.

Every message keeps its place and role, so the list is the same length, its shape is the history's shape (roles alternate wherever they did), and a tool call never loses its result: only content changes. Content is masked in :data:MASK_ORDER, oldest first within each kind, and masking stops the moment the list fits:

  • tool: a tool result's content becomes the note. A get_message result holds another message's content, so its note names that message (following reads of reads), never the read: a note naming the read sent a model to read its own read.
  • image: a message's image blocks become one text block carrying the note, beside any text.
  • assistant: a reply's text becomes the note, and any tool-call argument longer than 200 characters becomes a short placeholder (the call itself stays).
  • user: a user message's content becomes the note.

System messages are never masked, nor the first keep_first and last keep_last messages after them (extended outward to whole tool-call groups, like :func:trim_messages): in an agent run the first is the task. A message is masked only if its note is shorter than what it replaces. messages is never mutated. If every candidate is masked and the list still does not fit, it is returned as masked; follow with :func:trim_messages (see :func:chain) to bound it.

Parameters:

Name Type Description Default
messages list[dict]

The conversation, OpenAI message-dict format.

required
max_tokens int

The budget to fit, per counter (or the default estimate).

required
keep_first int

Leading non-system messages never masked.

1
keep_last int

Trailing messages never masked.

2
counter Optional[Callable[[str], int]]

Optional real tokenizer; see :func:~aimu.context.count_tokens.

None
note NoteFn

note(index, message, kind) -> str, the text put in place of masked content. index is the message's position in messages, which is its position in the history when this runs as a compaction. The default, :func:default_note, names get_message(index).

default_note

aimu.context.compaction.default_note

default_note(index: int, message: dict, kind: str) -> str

The note :func:mask_messages puts in place of masked content, naming the way back.

kind is one of :data:MASK_ORDER. The text says what is missing and where, that it is a context saving rather than a gap in the conversation, and which call returns it.

aimu.context.compaction.MASK_ORDER module-attribute

MASK_ORDER = ('tool', 'image', 'assistant', 'user')

aimu.context.compaction.trim_messages

trim_messages(messages: list[dict], max_tokens: int, *, keep_system: bool = True, keep_first: int = 1, keep_last: int = 2, counter: Optional[Callable[[str], int]] = None) -> list[dict]

Return a new, trimmed message list that fits within max_tokens (estimated).

Always returns a new list; messages (and its dicts) are never mutated. A conversation already at or under max_tokens is returned unchanged (a copy) -- this is a no-op, not a forced rewrite.

Dropping proceeds oldest-first over the non-system messages, one indivisible group at a time: an assistant message carrying tool_calls is always dropped together with every tool message answering it, never separately. This is the invariant that justifies this function over a hand-written messages[-n:] slice, which produces exactly that illegal shape.

Dropping leaves no trace, so as a compaction it belongs behind :func:mask_messages (see :func:chain), where it only removes messages masking has already reduced to notes.

The first non-system message is kept by default (keep_first=1). In an agent run it is the task, the user message agent.run(task) appended, and oldest-first dropping would otherwise remove it before anything else, leaving the agent working without knowing what it was asked. That failure raises nothing: the run still returns a plausible answer.

Parameters:

Name Type Description Default
messages list[dict]

The conversation, OpenAI message-dict format.

required
max_tokens int

The token budget to trim down to, per counter (or the default estimate; see :func:~aimu.context.count_tokens).

required
keep_system bool

When True (default), every {"role": "system"} message is kept regardless of budget, and is moved to the front of the returned list. That is invisible for the usual single leading system prompt, but a system message that sat mid-conversation (an earlier summary inserted by :func:summarize_messages, say) comes back ahead of the turns it originally followed. When False, system messages are ordinary droppable messages like any other and order is preserved.

True
keep_first int

The number of leading messages to always protect from dropping, counted after the system messages when keep_system is True and from the very first message otherwise. Extended outward to whole tool-call groups, like keep_last. Pass 0 for plain oldest-first dropping.

1
keep_last int

The number of trailing messages (not exchanges) to always protect from dropping, extended outward to whole tool-call groups when the raw count would otherwise split one.

2
counter Optional[Callable[[str], int]]

Optional real tokenizer; see :func:~aimu.context.count_tokens.

None

aimu.context.compaction.summarize_messages

summarize_messages(client: Any, messages: list[dict], *, keep_first: int = 1, keep_last: int = 2, prompt: Optional[str] = None) -> list[dict]

Replace the middle of a conversation with a one-message LLM summary.

Any {"role": "system"} messages are always preserved unchanged, and are returned at the front (a mid-conversation system message is hoisted, the same reordering :func:trim_messages does under keep_system=True). Of the rest, the first keep_first and the last keep_last messages (each extended outward to whole tool-call groups, the same invariant :func:trim_messages enforces) are kept verbatim. Everything between them (the "prefix") is rendered to a plain transcript, summarized by one client.generate() call, and replaced with a single {"role": "system", ...} message carrying the summary text -- system, rather than user or assistant, because it is out-of-band context neither party actually said, not a real conversational turn.

keep_first=1 is the default for the reason it is on :func:trim_messages: in an agent run the first non-system message is the task, and a summarizer may paraphrase it or leave it out. The kept first messages are still shown to the summarizer, labeled as background it should not summarize, because the instruction asks for what a continuation will need and the task is what that need is relative to. Models tend to restate the task in the summary anyway, a small duplication. The returned order is system messages, then the summary, then the kept first messages, then the tail. The summary goes ahead of the kept first messages, though it describes what came after them, because several local chat templates reject a system message anywhere but the start of the conversation.

client is any object exposing generate(prompt: str) -> str (a plain :class:~aimu.models.BaseModelClient, or a :class:~aimu.agents.Agent via agent.as_model_client()); this function takes it as an argument rather than constructing one itself, so it stays free of any provider dependency. There is no ready-made compaction for it; see the module docstring.

When nothing lies between the kept first and last messages, no summarization call is made and the conversation is returned unchanged (a new list; messages is never mutated).

Parameters:

Name Type Description Default
client Any

Anything with a generate(prompt: str) -> str method.

required
messages list[dict]

The conversation, OpenAI message-dict format.

required
keep_first int

Leading non-system messages to keep verbatim; 0 summarizes from the first one.

1
keep_last int

Trailing messages (not exchanges) to keep verbatim; see above.

2
prompt Optional[str]

Optional instruction prepended to the rendered transcript, replacing :data:DEFAULT_SUMMARIZE_PROMPT.

None

aimu.context.compaction.DEFAULT_SUMMARIZE_PROMPT module-attribute

DEFAULT_SUMMARIZE_PROMPT = 'Summarize the conversation below in a few sentences. Preserve concrete facts, decisions, and any open questions a continuation of this conversation would still need to know.'