fix(runtime): auto-compaction must measure the live context, not a running total

`maybe_auto_compact` gated on `usage_tracker.cumulative_usage().input_tokens`,
which is wrong in two independent ways:

1. It ignores cached input. With prompt caching a warm call reports
   `input_tokens: 2` and puts the real context in `cache_read_input_tokens`,
   so the counter barely moved and auto-compaction never fired at all,
   however large the conversation grew. Backends without prompt caching
   report the whole prompt as `input_tokens`, so the bug was invisible there.

2. A sum across calls is not a context measurement, and it never decreases.
   Once it did cross the threshold, every subsequent iteration compacted
   again, permanently, even after compaction had shrunk the context.

Gate on the latest call's total input instead — input + cache_creation +
cache_read — which is the live context window size. It rises with the real
context and falls back below the threshold once compaction has done its job,
so no counter reset is needed.

`CLAUDE_CODE_AUTO_COMPACT_INPUT_TOKENS` now means what it reads like:
compact when the live context reaches N tokens. The default of 100_000 is
unchanged and still lands at 50% of a 200k window.

Adds `TokenUsage::total_input_tokens()` (saturating) and four tests: the
cached-usage shape that never fired, non-latching across three turns, and
two unit tests for the new helper.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013bVvRYn7CQKR6hT2j1L6Cs
(cherry picked from commit aebba4e77eddf585fc5b21123967dbc4ebb6e6a1)
This commit is contained in:
reacechalmers authored and clawdbot committed 2026-10-12 22:58:19 +00:00
1 parent 7a1391fbc1
commit a25717aded
2 files changed
+187 -1

No files matched your search

+48
View File
@@ -89,6 +89,19 @@ impl TokenUsage {
+ self.cache_read_input_tokens
}
/// Total tokens the provider had to read to serve this call, i.e. the live size of the
/// context window.
///
/// `input_tokens` alone is NOT that number. With prompt caching a warm call reports
/// `input_tokens: 2` and puts the real bulk in `cache_read_input_tokens`, so anything
/// budgeting against the context window must sum all three input counters.
#[must_use]
pub fn total_input_tokens(self) -> u32 {
self.input_tokens
.saturating_add(self.cache_creation_input_tokens)
.saturating_add(self.cache_read_input_tokens)
}
#[must_use]
pub fn estimate_cost_usd(self) -> UsageCostEstimate {
self.estimate_cost_usd_with_pricing(ModelPricing::default_sonnet_tier())
@@ -243,6 +256,41 @@ mod tests {
assert_eq!(tracker.cumulative_usage().total_tokens(), 48);
}
#[test]
fn total_input_tokens_includes_cached_context() {
// The shape a warm prompt-cached call actually reports: input_tokens is ~nothing and
// the real context size is in the cache counters.
let cached = TokenUsage {
input_tokens: 2,
output_tokens: 1_133,
cache_creation_input_tokens: 1_453,
cache_read_input_tokens: 111_186,
};
assert_eq!(cached.total_input_tokens(), 112_641);
// Output tokens are not part of the input budget.
assert_ne!(cached.total_input_tokens(), cached.total_tokens());
// A backend without prompt caching reports the whole prompt as input_tokens.
let uncached = TokenUsage {
input_tokens: 29_056,
output_tokens: 73,
cache_creation_input_tokens: 0,
cache_read_input_tokens: 0,
};
assert_eq!(uncached.total_input_tokens(), 29_056);
}
#[test]
fn total_input_tokens_saturates_instead_of_overflowing() {
let usage = TokenUsage {
input_tokens: u32::MAX,
output_tokens: 0,
cache_creation_input_tokens: u32::MAX,
cache_read_input_tokens: u32::MAX,
};
assert_eq!(usage.total_input_tokens(), u32::MAX);
}
#[test]
fn computes_cost_summary_lines() {
let usage = TokenUsage {