docs blog library
github

Context management

Context management controls what Jazz sends to the model on each request: conversation messages, instructions, tool definitions, and tool results.

Each model has a context window, the amount of text it can receive in one request. As a conversation grows, Jazz makes room by removing older tool output and summarizing older messages. The agent’s saved work state keeps the task’s objective and decisions available.

Managing conversation history

Jazz manages context automatically. By default:

Context used Action
50% Replace older large tool results with retrieval pointers, keeping recent results intact
70% Warn that the context is filling and ask the agent to save its findings
80% Summarize older history while keeping system instructions, pinned messages, and recent exchanges
95% Trim older messages if the history still exceeds the limit

These percentages use the model’s effective context window, lowered by the agent’s llm.maxContextTokens when configured. The warning and compaction thresholds can be changed in Jazz configuration.

Compaction replaces older exchanges with a summary so the agent can continue the same task. Jazz shows when it happens. It uses an additional model call and can lose detail; llm.summarizerModel selects the model for that call. The agent can also request compaction with summarize_context.

In the terminal, /compact shows its current phase and a spinner beside the context meter while work is in progress. The percentage is context usage, not a completion percentage; the meter’s drain during summarization is an estimate. The final result reports the measured context usage after compaction.

When the summarizer model is the conversation’s own model and its provider prices cached input or has reported cache hits, automatic and manual compaction can ask for the summary as one more request of the conversation: same system prompt, same tools, same messages, then one final instruction, with tool calls disabled. That request shares its prefix with the step before it, so the provider can serve most of it from cache instead of billing the largest call of the run as fully uncached input. Jazz falls back to a separate summarizer when llm.summarizerModel names another model, the provider has neither cache pricing nor observed cache hits, the request would not leave room in the window for the summary, the history changed since the last request, summarize_context triggers it, or the model returns a tool call anyway. The step’s cache rate is computed like any other request.

Manual /compact can reuse the same prefix after a turn in the current process. Jazz keeps the last offered tool schemas alongside its bounded cache ledger, without retaining another copy of the transcript. It uses the standalone summarizer if no prior request is available, the system prompt or earlier messages changed, pruning rewrote old results, attachments prevent a safe prefix match, or the aligned request fails. When an earlier completed request ended at the summary boundary, Jazz can omit the retained recent tail, extending through paired tool results when necessary. Automatic compaction uses the same completed-prefix checks and can also omit its retained recent tail. Otherwise it keeps the full matching request prefix: some hybrid backends cannot reuse arbitrary shorter prefixes. Local providers can qualify through actual reported cache hits even without published cache pricing. Automatic compaction keeps its full-history path when attachments prevent safe shortening. Its default 80% trigger is unchanged. Reasoning remains at the agent’s configured level in both paths; cache reuse reduces repeated input processing, while generating the summary still takes model time.

Trimming removes older messages without summarizing them. Jazz uses it when compaction cannot free enough space or the summarizer fails. Tool calls and their results stay paired, and Jazz tells you when history is discarded.

Keeping findings available

Work state records the task’s objective, decisions, open questions, and next step. Todos track progress and verification. Both survive compaction.

Jazz saves large tool outputs separately. When an older output is removed from context, or an output is too long to show whole, the agent can read the saved copy a page at a time with retrieve_tool_result. If saving failed, it must rerun the tool.

Use memory for facts needed across conversations and files or the scratchpad for longer drafts and research material.

Setting run limits

A run defaults to 100 iterations, where each iteration is a model call and any tools it requests. Cost, token, and duration caps are unset until you configure them. Set limits globally, in a workflow, or for one jazz run invocation.

Jazz warns the agent as its budgets fill, giving it time to consolidate findings and finish. Iteration, cost, and token limits are checked between iterations, so an in-flight call can cross them. A duration cap interrupts running work at its deadline.

See Budgets for configuration, enforcement, and spend reporting.

Detecting stalled work

Jazz monitors repeated tool calls. If fewer than 40% of the last ten calls have distinct names and arguments, it asks the agent to change approach. If repetition continues through another full window, Jazz stops the run and reports stalled: true in jazz run --json.

Calls with different arguments count as different work. This detects exact repetition; it may miss an agent repeating the same approach with slightly different arguments.

Continuing beyond a run

Goals continue an objective across successive turns. Remote handoff moves a conversation to your SSH server so it can continue after you close your laptop.

Read the prompt-cache hit rate

Providers bill input they already hold in a prompt cache at a lower rate and answer faster. Jazz shows how much of each request was served from cache, so you can tell a cheap turn from an expensive one.

  • The turn receipt after each answer ends with cache 82% when the provider reported cache figures. Press Ctrl+O on a multi-step turn for the per-step rates.
  • /context shows a cache row for the last request in the conversation.
  • jazz run --json adds cacheHitRate (0 to 1) and cacheMissReason to tokenUsage, plus cacheHitRatePartial: true when some requests reported no cache figures.
  • Plain-text output (showMetrics) appends the request’s figure to its [Tokens: ...] line.

The rate is cache reads divided by all input tokens of the request. A provider that reports no cache figures shows unknown, or no cache figure on the receipt, never 0%. When only some requests of a turn or run reported them, the figure covers those requests and reads cache 90% (partial).

When the rate is below 50%, the receipt adds what changed in the cacheable prefix compared with the previous request in the conversation:

Reason Meaning
first request Nothing was sent before in this process, so nothing could be cached.
prefix unchanged Only messages were appended. The provider still missed (see below).
tools changed The tool definitions differ, for example after a deferred tool was unlocked.
system prompt changed The system prompt differs: date, memory, instructions, or skills changed.
history rewritten Earlier messages changed: compaction, clearing old tool results, or trimming.
model changed A different model answered, so its cache is separate.
provider changed A different provider answered.

Jazz compares what it sends. A changed prefix is a likely cause of a miss, not a proof, and an unchanged prefix does not promise a hit: providers expire cached prefixes on their own. Anthropic drops them after 5 minutes without a request, so a pause between turns shows a low rate with prefix unchanged. Requests seconds apart inside one tool loop are where the rate is expected to be high. Run with --debug to log each request’s classification and the hashes that changed; prompt content is never logged.

Adjust the context limit

For an agent that reads many large files, set llm.maxContextTokens in its config to compact earlier. For example:

{ "llm": { "maxContextTokens": 64000 } }

Choose a limit no larger than the model’s supported window. Earlier compaction reduces the history sent per request, but may require the agent to retrieve or reread more details. See Agent configuration for the full file format.

machine-readable: /jazz/docs/concepts/context-management.md · /llms.txt · /llms-full.txt