mirror of
https://github.com/instructkr/claw-code.git
synced 2026-11-23 00:00:02 +08:00
Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b06b6b32d4 | ||
|
|
7491124d9a | ||
|
|
f61b45d10d |
No files matched your search
@@ -161,52 +161,52 @@ GJC sessions have **three entry points**:
|
|||||||
|
|
||||||
### claw-code Status
|
### claw-code Status
|
||||||
|
|
||||||
**Current** (as of PR #3330): Partial. SDK transport merged (see SDK Wire Protocol). Local session lifecycle exists in `runtime::Session`. Remaining: operation handlers, session router with durable lookup, invocation reconciliation store, terminal scope tracking.
|
**Current**: The SDK now exposes `sdk::session::SessionRouter` with durable `turn.create`, `turn.prompt`, `turn.steer`, `turn.abort_and_prompt`, and paginated `turn.list` handling. It bridges `runtime::Session`/`ConversationRuntime`, streams ordered lifecycle events, persists invocation outcomes for reconciliation, and stores terminal session scope by correlation key. The WebSocket transport remains a client transport; a remotely hosted broker and automatic recovery of an unknown post-crash outcome remain outside this slice.
|
||||||
|
|
||||||
**Files**:
|
**Files**:
|
||||||
- `rust/crates/runtime/src/session.rs` (1,961 LOC)
|
- `rust/crates/runtime/src/session.rs` (1,961 LOC)
|
||||||
- `rust/crates/runtime/src/conversation.rs` (1,878 LOC)
|
- `rust/crates/runtime/src/conversation.rs` (conversation loop and lifecycle event bridge)
|
||||||
- `rust/crates/runtime/src/session_control.rs` (file ops, migration)
|
- `rust/crates/runtime/src/session_control.rs` (file ops, migration)
|
||||||
- `rust/crates/runtime/src/lib.rs` (re-exports `Session`, `SessionStore`, `ConversationRuntime`)
|
- `rust/crates/runtime/src/lib.rs` (re-exports `Session`, `SessionStore`, `ConversationRuntime`)
|
||||||
|
- `rust/crates/sdk/src/session/` (router, event stream, reconciliation, terminal scope)
|
||||||
|
|
||||||
**Lines**: ~3,900 LOC (session + conversation)
|
**Lines**: ~3,900 LOC (session + conversation)
|
||||||
|
|
||||||
**Reality**:
|
**Reality**:
|
||||||
- `Session` struct holds in-memory state (messages, usage, context).
|
- `Session` holds persisted conversation state and `SessionStore` resolves session IDs in the workspace namespace.
|
||||||
- `SessionStore` reads/writes `.claw/sessions/{id}/` directory.
|
- `ConversationRuntime` drives the local agent loop and emits lifecycle events to an optional callback.
|
||||||
- `ConversationRuntime` drives the local agent loop (sync).
|
- `sdk::session::SessionRouter` dispatches the five turn operations and exposes a per-invocation event stream.
|
||||||
- No WebSocket or remote access.
|
- Invocation terminal outcomes and correlation-key terminal scopes are durable; startup recovery marks unfinished invocations uncertain for later reconciliation.
|
||||||
- No invocation reconciliation or uncertain outcome recovery.
|
- A remote WebSocket server/broker and inference of an outcome that was never durably recorded are still absent.
|
||||||
- No terminal scope tracking (per-session-per-correlation).
|
|
||||||
|
|
||||||
### Gap Assessment
|
### Gap Assessment
|
||||||
|
|
||||||
| Component | GJC | claw-code | Gap | Complexity |
|
| Component | GJC | claw-code | Gap | Complexity |
|
||||||
|-----------|-----|-----------|-----|------------|
|
|-----------|-----|-----------|-----|------------|
|
||||||
| Remote Session Router | 2,500 LOC | — | 100% | High |
|
| Remote Session Router | 2,500 LOC | Local router/dispatch | Partial | High |
|
||||||
| Invocation Reconciliation | 1,200 LOC | — | 100% | High |
|
| Invocation Reconciliation | 1,200 LOC | Durable invocation records | Partial | High |
|
||||||
| Terminal Scope Tracking | 600 LOC | — | 100% | Medium |
|
| Terminal Scope Tracking | 600 LOC | Implemented | — | Medium |
|
||||||
| Async Event Streaming | 800 LOC | Partial | 60% | Medium |
|
| Async Event Streaming | 800 LOC | Local per-invocation stream | Partial | Medium |
|
||||||
| Uncertain Outcome Recovery | 1,000 LOC | — | 100% | High |
|
| Uncertain Outcome Recovery | 1,000 LOC | Unfinished records recovered as uncertain | Partial | High |
|
||||||
| **Total** | **6,100 LOC** | **3,900** | **2,200 LOC** | — |
|
| **Total** | **6,100 LOC** | **Session lifecycle slice implemented** | **Remote broker remains** | — |
|
||||||
|
|
||||||
### Proposed Implementation
|
### Proposed Implementation
|
||||||
|
|
||||||
**Owner Crate**: `sdk` (new, or extend `runtime`)
|
**Owner Crate**: `sdk` (new, or extend `runtime`)
|
||||||
|
|
||||||
**Module Structure** (add to `sdk/session/`):
|
**Module Structure** (`sdk/session/`):
|
||||||
```
|
```
|
||||||
sdk/session/
|
sdk/session/
|
||||||
├── router.rs (session routing, durable lookup)
|
├── mod.rs (session routing and operation dispatch)
|
||||||
├── reconciliation.rs (uncertain → terminal recovery)
|
├── reconciliation.rs (uncertain → terminal recovery)
|
||||||
├── terminal_scope.rs (per-correlation state retention)
|
├── terminal_scope.rs (per-correlation state retention)
|
||||||
└── events.rs (event publishing, reconciliation)
|
└── events.rs (ordered lifecycle event types)
|
||||||
```
|
```
|
||||||
|
|
||||||
**Integration Points**:
|
**Integration Points**:
|
||||||
- Bridge `runtime::Session` into `sdk::Session` wrapper
|
- Bridge `runtime::Session` and `ConversationRuntime` into SDK lifecycle operations
|
||||||
- Add `runtime::SessionStore` extensions for reconciliation data
|
- Persist reconciliation and terminal-scope data alongside the session namespace
|
||||||
- Extend `ConversationRuntime` with event emission
|
- Emit lifecycle callbacks from `ConversationRuntime`
|
||||||
|
|
||||||
**Size Estimate**: 2,500 LOC (Rust, new + modifications)
|
**Size Estimate**: 2,500 LOC (Rust, new + modifications)
|
||||||
|
|
||||||
|
|||||||
Generated
+2
@@ -2337,8 +2337,10 @@ name = "sdk"
|
|||||||
version = "0.1.4"
|
version = "0.1.4"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"futures-util",
|
"futures-util",
|
||||||
|
"runtime",
|
||||||
"serde",
|
"serde",
|
||||||
"serde_json",
|
"serde_json",
|
||||||
|
"tempfile",
|
||||||
"tokio",
|
"tokio",
|
||||||
"tokio-tungstenite",
|
"tokio-tungstenite",
|
||||||
"tracing",
|
"tracing",
|
||||||
|
|||||||
@@ -1,8 +1,8 @@
|
|||||||
use std::collections::BTreeMap;
|
use std::collections::{BTreeMap, VecDeque};
|
||||||
use std::fmt::{Display, Formatter};
|
use std::fmt::{Display, Formatter};
|
||||||
use std::sync::{
|
use std::sync::{
|
||||||
atomic::{AtomicBool, Ordering},
|
atomic::{AtomicBool, Ordering},
|
||||||
Arc,
|
Arc, Mutex,
|
||||||
};
|
};
|
||||||
|
|
||||||
use serde_json::{Map, Value};
|
use serde_json::{Map, Value};
|
||||||
@@ -47,6 +47,30 @@ pub enum AssistantEvent {
|
|||||||
MessageStop,
|
MessageStop,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Runtime lifecycle event emitted while processing a conversation turn.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub enum ConversationEvent {
|
||||||
|
AgentBegin {
|
||||||
|
session_id: String,
|
||||||
|
},
|
||||||
|
ToolUse {
|
||||||
|
tool_use_id: String,
|
||||||
|
tool_name: String,
|
||||||
|
input: String,
|
||||||
|
},
|
||||||
|
ToolResult {
|
||||||
|
tool_use_id: String,
|
||||||
|
tool_name: String,
|
||||||
|
output: String,
|
||||||
|
is_error: bool,
|
||||||
|
},
|
||||||
|
AgentEnd {
|
||||||
|
session_id: String,
|
||||||
|
interrupted: bool,
|
||||||
|
output: Option<String>,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
/// Prompt-cache telemetry captured from the provider response stream.
|
/// Prompt-cache telemetry captured from the provider response stream.
|
||||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
pub struct PromptCacheEvent {
|
pub struct PromptCacheEvent {
|
||||||
@@ -92,6 +116,103 @@ impl TurnInterruptSignal {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Default)]
|
||||||
|
struct TurnControlState {
|
||||||
|
queued_steers: VecDeque<String>,
|
||||||
|
closed: bool,
|
||||||
|
aborted: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Shared steering and abort control for a running conversation turn.
|
||||||
|
///
|
||||||
|
/// Clones share one queue and lifecycle state. Steering is accepted until
|
||||||
|
/// `finish_or_take_steer` closes the signal or `abort` is called.
|
||||||
|
#[derive(Debug, Clone, Default)]
|
||||||
|
pub struct TurnControlSignal {
|
||||||
|
state: Arc<Mutex<TurnControlState>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl TurnControlSignal {
|
||||||
|
#[must_use]
|
||||||
|
pub fn new() -> Self {
|
||||||
|
Self::default()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Queue a steering prompt while the signal is accepting input.
|
||||||
|
pub fn steer(&self, prompt: impl Into<String>) -> bool {
|
||||||
|
let mut state = self
|
||||||
|
.state
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||||
|
if state.closed || state.aborted {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
state.queued_steers.push_back(prompt.into());
|
||||||
|
true
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Remove one queued steering prompt, if present.
|
||||||
|
#[must_use]
|
||||||
|
pub fn take_steer(&self) -> Option<String> {
|
||||||
|
let mut state = self
|
||||||
|
.state
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||||
|
if state.aborted {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
state.queued_steers.pop_front()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Drain one queued prompt, or atomically close the signal when empty.
|
||||||
|
#[must_use]
|
||||||
|
pub fn finish_or_take_steer(&self) -> Option<String> {
|
||||||
|
let mut state = self
|
||||||
|
.state
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||||
|
if state.aborted {
|
||||||
|
state.queued_steers.clear();
|
||||||
|
state.closed = true;
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
if let Some(prompt) = state.queued_steers.pop_front() {
|
||||||
|
return Some(prompt);
|
||||||
|
}
|
||||||
|
state.closed = true;
|
||||||
|
None
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Abort the active turn and reject further steering prompts.
|
||||||
|
pub fn abort(&self) {
|
||||||
|
let mut state = self
|
||||||
|
.state
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||||
|
state.aborted = true;
|
||||||
|
state.closed = true;
|
||||||
|
state.queued_steers.clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Close steering without classifying the turn as aborted.
|
||||||
|
pub fn close(&self) {
|
||||||
|
let mut state = self
|
||||||
|
.state
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||||
|
state.closed = true;
|
||||||
|
state.queued_steers.clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[must_use]
|
||||||
|
pub fn is_aborted(&self) -> bool {
|
||||||
|
self.state
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(std::sync::PoisonError::into_inner)
|
||||||
|
.aborted
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Minimal streaming API contract required by [`ConversationRuntime`].
|
/// Minimal streaming API contract required by [`ConversationRuntime`].
|
||||||
pub trait ApiClient {
|
pub trait ApiClient {
|
||||||
fn stream(&mut self, request: ApiRequest) -> Result<Vec<AssistantEvent>, RuntimeError>;
|
fn stream(&mut self, request: ApiRequest) -> Result<Vec<AssistantEvent>, RuntimeError>;
|
||||||
@@ -179,6 +300,8 @@ pub struct ConversationRuntime<C, T> {
|
|||||||
auto_compaction_input_tokens_threshold: u32,
|
auto_compaction_input_tokens_threshold: u32,
|
||||||
hook_abort_signal: HookAbortSignal,
|
hook_abort_signal: HookAbortSignal,
|
||||||
turn_interrupt_signal: TurnInterruptSignal,
|
turn_interrupt_signal: TurnInterruptSignal,
|
||||||
|
turn_control_signal: Option<TurnControlSignal>,
|
||||||
|
event_callback: Option<Arc<dyn Fn(ConversationEvent) + Send + Sync>>,
|
||||||
hook_progress_reporter: Option<Box<dyn HookProgressReporter>>,
|
hook_progress_reporter: Option<Box<dyn HookProgressReporter>>,
|
||||||
session_tracer: Option<SessionTracer>,
|
session_tracer: Option<SessionTracer>,
|
||||||
}
|
}
|
||||||
@@ -234,6 +357,8 @@ where
|
|||||||
auto_compaction_input_tokens_threshold: auto_compaction_threshold_from_env(),
|
auto_compaction_input_tokens_threshold: auto_compaction_threshold_from_env(),
|
||||||
hook_abort_signal: HookAbortSignal::default(),
|
hook_abort_signal: HookAbortSignal::default(),
|
||||||
turn_interrupt_signal: TurnInterruptSignal::default(),
|
turn_interrupt_signal: TurnInterruptSignal::default(),
|
||||||
|
turn_control_signal: None,
|
||||||
|
event_callback: None,
|
||||||
hook_progress_reporter: None,
|
hook_progress_reporter: None,
|
||||||
session_tracer: None,
|
session_tracer: None,
|
||||||
}
|
}
|
||||||
@@ -273,6 +398,21 @@ where
|
|||||||
self
|
self
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[must_use]
|
||||||
|
pub fn with_event_callback(
|
||||||
|
mut self,
|
||||||
|
callback: Arc<dyn Fn(ConversationEvent) + Send + Sync>,
|
||||||
|
) -> Self {
|
||||||
|
self.event_callback = Some(callback);
|
||||||
|
self
|
||||||
|
}
|
||||||
|
|
||||||
|
#[must_use]
|
||||||
|
pub fn with_turn_control_signal(mut self, turn_control_signal: TurnControlSignal) -> Self {
|
||||||
|
self.turn_control_signal = Some(turn_control_signal);
|
||||||
|
self
|
||||||
|
}
|
||||||
|
|
||||||
#[must_use]
|
#[must_use]
|
||||||
pub fn with_hook_progress_reporter(
|
pub fn with_hook_progress_reporter(
|
||||||
mut self,
|
mut self,
|
||||||
@@ -377,6 +517,58 @@ where
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn emit_conversation_event(&self, event: ConversationEvent) {
|
||||||
|
if let Some(callback) = &self.event_callback {
|
||||||
|
callback(event);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn turn_control_is_aborted(&self) -> bool {
|
||||||
|
let is_aborted = self
|
||||||
|
.turn_control_signal
|
||||||
|
.as_ref()
|
||||||
|
.is_some_and(TurnControlSignal::is_aborted);
|
||||||
|
if is_aborted {
|
||||||
|
self.turn_interrupt_signal.interrupt();
|
||||||
|
}
|
||||||
|
is_aborted
|
||||||
|
}
|
||||||
|
|
||||||
|
fn turn_is_interrupted(&self) -> bool {
|
||||||
|
self.turn_control_is_aborted() || self.turn_interrupt_signal.is_interrupted()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn append_queued_steer(&mut self) -> Result<bool, RuntimeError> {
|
||||||
|
let Some(prompt) = self
|
||||||
|
.turn_control_signal
|
||||||
|
.as_ref()
|
||||||
|
.and_then(TurnControlSignal::take_steer)
|
||||||
|
else {
|
||||||
|
return Ok(false);
|
||||||
|
};
|
||||||
|
self.session
|
||||||
|
.push_user_text(prompt)
|
||||||
|
.map_err(|error| RuntimeError::new(error.to_string()))?;
|
||||||
|
Ok(true)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn emit_tool_result_event(&self, message: &ConversationMessage) {
|
||||||
|
if let Some(ContentBlock::ToolResult {
|
||||||
|
tool_use_id,
|
||||||
|
tool_name,
|
||||||
|
output,
|
||||||
|
is_error,
|
||||||
|
}) = message.blocks.first()
|
||||||
|
{
|
||||||
|
self.emit_conversation_event(ConversationEvent::ToolResult {
|
||||||
|
tool_use_id: tool_use_id.clone(),
|
||||||
|
tool_name: tool_name.clone(),
|
||||||
|
output: output.clone(),
|
||||||
|
is_error: *is_error,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[allow(clippy::too_many_lines)]
|
#[allow(clippy::too_many_lines)]
|
||||||
pub fn run_turn(
|
pub fn run_turn(
|
||||||
&mut self,
|
&mut self,
|
||||||
@@ -384,7 +576,13 @@ where
|
|||||||
mut prompter: Option<&mut dyn PermissionPrompter>,
|
mut prompter: Option<&mut dyn PermissionPrompter>,
|
||||||
) -> Result<TurnSummary, RuntimeError> {
|
) -> Result<TurnSummary, RuntimeError> {
|
||||||
let user_input = user_input.into();
|
let user_input = user_input.into();
|
||||||
|
let session_id = self.session.session_id.clone();
|
||||||
|
self.emit_conversation_event(ConversationEvent::AgentBegin {
|
||||||
|
session_id: session_id.clone(),
|
||||||
|
});
|
||||||
|
let mut interrupted = false;
|
||||||
|
let mut final_output = None;
|
||||||
|
let outcome = (|| {
|
||||||
// ROADMAP #38: Session-health canary - probe if context was compacted
|
// ROADMAP #38: Session-health canary - probe if context was compacted
|
||||||
if self.session.compaction.is_some() {
|
if self.session.compaction.is_some() {
|
||||||
if let Err(error) = self.run_session_health_probe() {
|
if let Err(error) = self.run_session_health_probe() {
|
||||||
@@ -406,10 +604,9 @@ where
|
|||||||
let mut prompt_cache_events = Vec::new();
|
let mut prompt_cache_events = Vec::new();
|
||||||
let mut iterations = 0;
|
let mut iterations = 0;
|
||||||
let mut auto_compaction = None;
|
let mut auto_compaction = None;
|
||||||
let mut interrupted = false;
|
|
||||||
|
|
||||||
loop {
|
loop {
|
||||||
if self.turn_interrupt_signal.is_interrupted() {
|
if self.turn_is_interrupted() {
|
||||||
self.record_turn_interrupted(iterations, "before_request");
|
self.record_turn_interrupted(iterations, "before_request");
|
||||||
interrupted = true;
|
interrupted = true;
|
||||||
break;
|
break;
|
||||||
@@ -430,7 +627,7 @@ where
|
|||||||
let events = match self.api_client.stream(request) {
|
let events = match self.api_client.stream(request) {
|
||||||
Ok(events) => events,
|
Ok(events) => events,
|
||||||
Err(error) => {
|
Err(error) => {
|
||||||
if self.turn_interrupt_signal.is_interrupted() {
|
if self.turn_is_interrupted() {
|
||||||
// The client aborted because the user interrupted the
|
// The client aborted because the user interrupted the
|
||||||
// turn; any partial response is discarded and the stop
|
// turn; any partial response is discarded and the stop
|
||||||
// is reported as an interruption rather than a failure.
|
// is reported as an interruption rather than a failure.
|
||||||
@@ -450,6 +647,7 @@ where
|
|||||||
return Err(error);
|
return Err(error);
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
final_output = assistant_message_text(&assistant_message);
|
||||||
if let Some(usage) = usage {
|
if let Some(usage) = usage {
|
||||||
self.usage_tracker.record(usage);
|
self.usage_tracker.record(usage);
|
||||||
}
|
}
|
||||||
@@ -482,11 +680,28 @@ where
|
|||||||
}
|
}
|
||||||
|
|
||||||
if pending_tool_uses.is_empty() {
|
if pending_tool_uses.is_empty() {
|
||||||
|
if self.turn_control_is_aborted() {
|
||||||
|
self.record_turn_interrupted(iterations, "after_request");
|
||||||
|
interrupted = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
if self.append_queued_steer()? {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if self.turn_control_is_aborted() {
|
||||||
|
self.record_turn_interrupted(iterations, "after_request");
|
||||||
|
interrupted = true;
|
||||||
|
}
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
|
||||||
for (tool_use_id, tool_name, input) in pending_tool_uses {
|
for (tool_use_id, tool_name, input) in pending_tool_uses {
|
||||||
if interrupted || self.turn_interrupt_signal.is_interrupted() {
|
self.emit_conversation_event(ConversationEvent::ToolUse {
|
||||||
|
tool_use_id: tool_use_id.clone(),
|
||||||
|
tool_name: tool_name.clone(),
|
||||||
|
input: input.clone(),
|
||||||
|
});
|
||||||
|
if interrupted || self.turn_is_interrupted() {
|
||||||
// Every pending tool_use must still receive a tool_result
|
// Every pending tool_use must still receive a tool_result
|
||||||
// so the session stays valid for the next request.
|
// so the session stays valid for the next request.
|
||||||
if !interrupted {
|
if !interrupted {
|
||||||
@@ -502,6 +717,7 @@ where
|
|||||||
self.session
|
self.session
|
||||||
.push_message(result_message.clone())
|
.push_message(result_message.clone())
|
||||||
.map_err(|error| RuntimeError::new(error.to_string()))?;
|
.map_err(|error| RuntimeError::new(error.to_string()))?;
|
||||||
|
self.emit_tool_result_event(&result_message);
|
||||||
tool_results.push(result_message);
|
tool_results.push(result_message);
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
@@ -551,15 +767,28 @@ where
|
|||||||
)
|
)
|
||||||
};
|
};
|
||||||
|
|
||||||
let result_message = match permission_outcome {
|
let result_message = if self.turn_is_interrupted() {
|
||||||
|
self.record_turn_interrupted(iterations, "before_tool_dispatch");
|
||||||
|
interrupted = true;
|
||||||
|
ConversationMessage::tool_result(
|
||||||
|
tool_use_id,
|
||||||
|
tool_name,
|
||||||
|
"Interrupted by user before this tool could run.",
|
||||||
|
true,
|
||||||
|
)
|
||||||
|
} else {
|
||||||
|
match permission_outcome {
|
||||||
PermissionOutcome::Allow => {
|
PermissionOutcome::Allow => {
|
||||||
self.record_tool_started(iterations, &tool_name);
|
self.record_tool_started(iterations, &tool_name);
|
||||||
let (mut output, mut is_error) =
|
let (mut output, mut is_error) = match self
|
||||||
match self.tool_executor.execute(&tool_name, &effective_input) {
|
.tool_executor
|
||||||
|
.execute(&tool_name, &effective_input)
|
||||||
|
{
|
||||||
Ok(output) => (output, false),
|
Ok(output) => (output, false),
|
||||||
Err(error) => (error.to_string(), true),
|
Err(error) => (error.to_string(), true),
|
||||||
};
|
};
|
||||||
output = merge_hook_feedback(pre_hook_result.messages(), output, false);
|
output =
|
||||||
|
merge_hook_feedback(pre_hook_result.messages(), output, false);
|
||||||
|
|
||||||
let post_hook_result = if is_error {
|
let post_hook_result = if is_error {
|
||||||
self.run_post_tool_use_failure_hook(
|
self.run_post_tool_use_failure_hook(
|
||||||
@@ -589,7 +818,12 @@ where
|
|||||||
|| post_hook_result.is_cancelled(),
|
|| post_hook_result.is_cancelled(),
|
||||||
);
|
);
|
||||||
|
|
||||||
ConversationMessage::tool_result(tool_use_id, tool_name, output, is_error)
|
ConversationMessage::tool_result(
|
||||||
|
tool_use_id,
|
||||||
|
tool_name,
|
||||||
|
output,
|
||||||
|
is_error,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
PermissionOutcome::Deny { reason } => ConversationMessage::tool_result(
|
PermissionOutcome::Deny { reason } => ConversationMessage::tool_result(
|
||||||
tool_use_id,
|
tool_use_id,
|
||||||
@@ -597,17 +831,31 @@ where
|
|||||||
merge_hook_feedback(pre_hook_result.messages(), reason, true),
|
merge_hook_feedback(pre_hook_result.messages(), reason, true),
|
||||||
true,
|
true,
|
||||||
),
|
),
|
||||||
|
}
|
||||||
};
|
};
|
||||||
self.session
|
self.session
|
||||||
.push_message(result_message.clone())
|
.push_message(result_message.clone())
|
||||||
.map_err(|error| RuntimeError::new(error.to_string()))?;
|
.map_err(|error| RuntimeError::new(error.to_string()))?;
|
||||||
|
self.emit_tool_result_event(&result_message);
|
||||||
self.record_tool_finished(iterations, &result_message);
|
self.record_tool_finished(iterations, &result_message);
|
||||||
tool_results.push(result_message);
|
tool_results.push(result_message);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if !interrupted && self.turn_control_is_aborted() {
|
||||||
|
self.record_turn_interrupted(iterations, "after_tool");
|
||||||
|
interrupted = true;
|
||||||
|
}
|
||||||
if interrupted {
|
if interrupted {
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
if self.append_queued_steer()? {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if self.turn_control_is_aborted() {
|
||||||
|
self.record_turn_interrupted(iterations, "after_tool");
|
||||||
|
interrupted = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
let summary = TurnSummary {
|
let summary = TurnSummary {
|
||||||
@@ -624,6 +872,13 @@ where
|
|||||||
}
|
}
|
||||||
|
|
||||||
Ok(summary)
|
Ok(summary)
|
||||||
|
})();
|
||||||
|
self.emit_conversation_event(ConversationEvent::AgentEnd {
|
||||||
|
session_id,
|
||||||
|
interrupted,
|
||||||
|
output: final_output,
|
||||||
|
});
|
||||||
|
outcome
|
||||||
}
|
}
|
||||||
|
|
||||||
#[must_use]
|
#[must_use]
|
||||||
@@ -885,6 +1140,18 @@ fn build_assistant_message(
|
|||||||
))
|
))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn assistant_message_text(message: &ConversationMessage) -> Option<String> {
|
||||||
|
let output = message
|
||||||
|
.blocks
|
||||||
|
.iter()
|
||||||
|
.filter_map(|block| match block {
|
||||||
|
ContentBlock::Text { text } => Some(text.as_str()),
|
||||||
|
_ => None,
|
||||||
|
})
|
||||||
|
.collect::<String>();
|
||||||
|
(!output.is_empty()).then_some(output)
|
||||||
|
}
|
||||||
|
|
||||||
fn flush_text_block(text: &mut String, blocks: &mut Vec<ContentBlock>) {
|
fn flush_text_block(text: &mut String, blocks: &mut Vec<ContentBlock>) {
|
||||||
if !text.is_empty() {
|
if !text.is_empty() {
|
||||||
blocks.push(ContentBlock::Text {
|
blocks.push(ContentBlock::Text {
|
||||||
@@ -956,9 +1223,9 @@ impl ToolExecutor for StaticToolExecutor {
|
|||||||
mod tests {
|
mod tests {
|
||||||
use super::{
|
use super::{
|
||||||
build_assistant_message, parse_auto_compaction_threshold, ApiClient, ApiRequest,
|
build_assistant_message, parse_auto_compaction_threshold, ApiClient, ApiRequest,
|
||||||
AssistantEvent, AutoCompactionEvent, ConversationRuntime, PromptCacheEvent, RuntimeError,
|
AssistantEvent, AutoCompactionEvent, ConversationEvent, ConversationRuntime,
|
||||||
StaticToolExecutor, ToolExecutor, TurnInterruptSignal,
|
PromptCacheEvent, RuntimeError, StaticToolExecutor, ToolExecutor, TurnControlSignal,
|
||||||
DEFAULT_AUTO_COMPACTION_INPUT_TOKENS_THRESHOLD,
|
TurnInterruptSignal, DEFAULT_AUTO_COMPACTION_INPUT_TOKENS_THRESHOLD,
|
||||||
};
|
};
|
||||||
use crate::compact::CompactionConfig;
|
use crate::compact::CompactionConfig;
|
||||||
use crate::config::{RuntimeFeatureConfig, RuntimeHookConfig};
|
use crate::config::{RuntimeFeatureConfig, RuntimeHookConfig};
|
||||||
@@ -972,7 +1239,7 @@ mod tests {
|
|||||||
use crate::ToolError;
|
use crate::ToolError;
|
||||||
use std::fs;
|
use std::fs;
|
||||||
use std::path::PathBuf;
|
use std::path::PathBuf;
|
||||||
use std::sync::Arc;
|
use std::sync::{Arc, Mutex};
|
||||||
use std::time::{SystemTime, UNIX_EPOCH};
|
use std::time::{SystemTime, UNIX_EPOCH};
|
||||||
use telemetry::{MemoryTelemetrySink, SessionTracer, TelemetryEvent};
|
use telemetry::{MemoryTelemetrySink, SessionTracer, TelemetryEvent};
|
||||||
|
|
||||||
@@ -1045,6 +1312,300 @@ mod tests {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn emits_ordered_lifecycle_events_for_tool_calls() {
|
||||||
|
struct ToolThenDoneApi {
|
||||||
|
calls: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl ApiClient for ToolThenDoneApi {
|
||||||
|
fn stream(
|
||||||
|
&mut self,
|
||||||
|
_request: ApiRequest,
|
||||||
|
) -> Result<Vec<AssistantEvent>, RuntimeError> {
|
||||||
|
self.calls += 1;
|
||||||
|
if self.calls == 1 {
|
||||||
|
Ok(vec![
|
||||||
|
AssistantEvent::ToolUse {
|
||||||
|
id: "tool-event-1".to_string(),
|
||||||
|
name: "echo".to_string(),
|
||||||
|
input: "payload".to_string(),
|
||||||
|
},
|
||||||
|
AssistantEvent::MessageStop,
|
||||||
|
])
|
||||||
|
} else {
|
||||||
|
Ok(vec![
|
||||||
|
AssistantEvent::TextDelta("done".to_string()),
|
||||||
|
AssistantEvent::MessageStop,
|
||||||
|
])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let session = Session::new();
|
||||||
|
let session_id = session.session_id.clone();
|
||||||
|
let captured_events = Arc::new(Mutex::new(Vec::new()));
|
||||||
|
let callback_events = Arc::clone(&captured_events);
|
||||||
|
let mut runtime = ConversationRuntime::new(
|
||||||
|
session,
|
||||||
|
ToolThenDoneApi { calls: 0 },
|
||||||
|
StaticToolExecutor::new().register("echo", |input| Ok(input.to_string())),
|
||||||
|
PermissionPolicy::new(PermissionMode::DangerFullAccess),
|
||||||
|
vec!["system".to_string()],
|
||||||
|
)
|
||||||
|
.with_event_callback(Arc::new(move |event| {
|
||||||
|
callback_events
|
||||||
|
.lock()
|
||||||
|
.expect("event mutex should not be poisoned")
|
||||||
|
.push(event);
|
||||||
|
}));
|
||||||
|
|
||||||
|
runtime
|
||||||
|
.run_turn("start", None)
|
||||||
|
.expect("tool turn should complete");
|
||||||
|
|
||||||
|
assert_eq!(
|
||||||
|
*captured_events
|
||||||
|
.lock()
|
||||||
|
.expect("event mutex should not be poisoned"),
|
||||||
|
vec![
|
||||||
|
ConversationEvent::AgentBegin {
|
||||||
|
session_id: session_id.clone(),
|
||||||
|
},
|
||||||
|
ConversationEvent::ToolUse {
|
||||||
|
tool_use_id: "tool-event-1".to_string(),
|
||||||
|
tool_name: "echo".to_string(),
|
||||||
|
input: "payload".to_string(),
|
||||||
|
},
|
||||||
|
ConversationEvent::ToolResult {
|
||||||
|
tool_use_id: "tool-event-1".to_string(),
|
||||||
|
tool_name: "echo".to_string(),
|
||||||
|
output: "payload".to_string(),
|
||||||
|
is_error: false,
|
||||||
|
},
|
||||||
|
ConversationEvent::AgentEnd {
|
||||||
|
session_id,
|
||||||
|
interrupted: false,
|
||||||
|
output: Some("done".to_string()),
|
||||||
|
},
|
||||||
|
]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn consumes_steering_queued_during_a_model_request_and_preserves_late_steers() {
|
||||||
|
struct SteeringApi {
|
||||||
|
control: TurnControlSignal,
|
||||||
|
calls: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl ApiClient for SteeringApi {
|
||||||
|
fn stream(&mut self, request: ApiRequest) -> Result<Vec<AssistantEvent>, RuntimeError> {
|
||||||
|
self.calls += 1;
|
||||||
|
if self.calls == 1 {
|
||||||
|
// Queue from another thread while this model request is
|
||||||
|
// still inside `stream`.
|
||||||
|
let control = self.control.clone();
|
||||||
|
assert!(std::thread::spawn(move || control.steer("follow-up"))
|
||||||
|
.join()
|
||||||
|
.expect("steering thread should finish"));
|
||||||
|
return Ok(vec![
|
||||||
|
AssistantEvent::ToolUse {
|
||||||
|
id: "steer-tool".to_string(),
|
||||||
|
name: "echo".to_string(),
|
||||||
|
input: "tool input".to_string(),
|
||||||
|
},
|
||||||
|
AssistantEvent::MessageStop,
|
||||||
|
]);
|
||||||
|
}
|
||||||
|
if self.calls == 2 {
|
||||||
|
assert!(request
|
||||||
|
.messages
|
||||||
|
.iter()
|
||||||
|
.any(|message| message.role == MessageRole::Tool));
|
||||||
|
assert!(request.messages.iter().any(|message| {
|
||||||
|
message.role == MessageRole::User
|
||||||
|
&& message.blocks.iter().any(|block| {
|
||||||
|
matches!(block, ContentBlock::Text { text } if text == "follow-up")
|
||||||
|
})
|
||||||
|
}));
|
||||||
|
return Ok(vec![
|
||||||
|
AssistantEvent::TextDelta("second response".to_string()),
|
||||||
|
AssistantEvent::MessageStop,
|
||||||
|
]);
|
||||||
|
}
|
||||||
|
unreachable!("the queued steer should only cause one extra request")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let control = TurnControlSignal::new();
|
||||||
|
let mut runtime = ConversationRuntime::new(
|
||||||
|
Session::new(),
|
||||||
|
SteeringApi {
|
||||||
|
control: control.clone(),
|
||||||
|
calls: 0,
|
||||||
|
},
|
||||||
|
StaticToolExecutor::new().register("echo", |input| Ok(input.to_string())),
|
||||||
|
PermissionPolicy::new(PermissionMode::DangerFullAccess),
|
||||||
|
vec!["system".to_string()],
|
||||||
|
)
|
||||||
|
.with_turn_control_signal(control.clone());
|
||||||
|
|
||||||
|
let summary = runtime
|
||||||
|
.run_turn("start", None)
|
||||||
|
.expect("steering turn should complete");
|
||||||
|
|
||||||
|
assert_eq!(summary.iterations, 2);
|
||||||
|
assert!(runtime.session().messages.iter().any(|message| {
|
||||||
|
message.role == MessageRole::User
|
||||||
|
&& message.blocks.iter().any(
|
||||||
|
|block| matches!(block, ContentBlock::Text { text } if text == "follow-up"),
|
||||||
|
)
|
||||||
|
}));
|
||||||
|
assert!(control.steer("late steer"));
|
||||||
|
assert_eq!(
|
||||||
|
control.finish_or_take_steer().as_deref(),
|
||||||
|
Some("late steer")
|
||||||
|
);
|
||||||
|
assert_eq!(control.finish_or_take_steer(), None);
|
||||||
|
assert!(!control.steer("rejected after finish"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn aborting_turn_control_uses_existing_interrupted_outcome() {
|
||||||
|
struct AbortApi {
|
||||||
|
control: TurnControlSignal,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl ApiClient for AbortApi {
|
||||||
|
fn stream(
|
||||||
|
&mut self,
|
||||||
|
_request: ApiRequest,
|
||||||
|
) -> Result<Vec<AssistantEvent>, RuntimeError> {
|
||||||
|
self.control.abort();
|
||||||
|
Err(RuntimeError::new("request aborted"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let control = TurnControlSignal::new();
|
||||||
|
let events = Arc::new(Mutex::new(Vec::new()));
|
||||||
|
let callback_events = Arc::clone(&events);
|
||||||
|
let mut runtime = ConversationRuntime::new(
|
||||||
|
Session::new(),
|
||||||
|
AbortApi {
|
||||||
|
control: control.clone(),
|
||||||
|
},
|
||||||
|
StaticToolExecutor::new(),
|
||||||
|
PermissionPolicy::new(PermissionMode::DangerFullAccess),
|
||||||
|
vec!["system".to_string()],
|
||||||
|
)
|
||||||
|
.with_turn_control_signal(control.clone())
|
||||||
|
.with_event_callback(Arc::new(move |event| {
|
||||||
|
callback_events
|
||||||
|
.lock()
|
||||||
|
.expect("event mutex should not be poisoned")
|
||||||
|
.push(event);
|
||||||
|
}));
|
||||||
|
|
||||||
|
let summary = runtime
|
||||||
|
.run_turn("start", None)
|
||||||
|
.expect("control abort should be reported as an interruption");
|
||||||
|
|
||||||
|
assert!(summary.interrupted);
|
||||||
|
assert!(control.is_aborted());
|
||||||
|
assert!(!control.steer("too late"));
|
||||||
|
assert!(matches!(
|
||||||
|
events
|
||||||
|
.lock()
|
||||||
|
.expect("event mutex should not be poisoned")
|
||||||
|
.last(),
|
||||||
|
Some(ConversationEvent::AgentEnd {
|
||||||
|
interrupted: true,
|
||||||
|
..
|
||||||
|
})
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn abort_during_permission_prompt_prevents_tool_dispatch() {
|
||||||
|
struct ToolApi;
|
||||||
|
|
||||||
|
impl ApiClient for ToolApi {
|
||||||
|
fn stream(
|
||||||
|
&mut self,
|
||||||
|
_request: ApiRequest,
|
||||||
|
) -> Result<Vec<AssistantEvent>, RuntimeError> {
|
||||||
|
Ok(vec![
|
||||||
|
AssistantEvent::ToolUse {
|
||||||
|
id: "pending-tool".to_string(),
|
||||||
|
name: "dangerous".to_string(),
|
||||||
|
input: "{}".to_string(),
|
||||||
|
},
|
||||||
|
AssistantEvent::MessageStop,
|
||||||
|
])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
struct BlockingAllowPrompter {
|
||||||
|
started: std::sync::mpsc::SyncSender<()>,
|
||||||
|
release: std::sync::mpsc::Receiver<()>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl PermissionPrompter for BlockingAllowPrompter {
|
||||||
|
fn decide(&mut self, _request: &PermissionRequest) -> PermissionPromptDecision {
|
||||||
|
self.started.send(()).expect("notify permission prompt");
|
||||||
|
self.release.recv().expect("wait for approval");
|
||||||
|
PermissionPromptDecision::Allow
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let control = TurnControlSignal::new();
|
||||||
|
let calls = Arc::new(std::sync::atomic::AtomicUsize::new(0));
|
||||||
|
let tool_calls = Arc::clone(&calls);
|
||||||
|
let executor = StaticToolExecutor::new().register("dangerous", move |_| {
|
||||||
|
tool_calls.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
|
||||||
|
Ok("executed".to_string())
|
||||||
|
});
|
||||||
|
let (started_tx, started_rx) = std::sync::mpsc::sync_channel(1);
|
||||||
|
let (release_tx, release_rx) = std::sync::mpsc::sync_channel(1);
|
||||||
|
let abort_control = control.clone();
|
||||||
|
let abort_thread = std::thread::spawn(move || {
|
||||||
|
started_rx
|
||||||
|
.recv_timeout(std::time::Duration::from_secs(2))
|
||||||
|
.expect("permission prompt should start");
|
||||||
|
abort_control.abort();
|
||||||
|
release_tx.send(()).expect("release permission prompt");
|
||||||
|
});
|
||||||
|
let mut prompter = BlockingAllowPrompter {
|
||||||
|
started: started_tx,
|
||||||
|
release: release_rx,
|
||||||
|
};
|
||||||
|
let mut runtime = ConversationRuntime::new(
|
||||||
|
Session::new(),
|
||||||
|
ToolApi,
|
||||||
|
executor,
|
||||||
|
PermissionPolicy::new(PermissionMode::Prompt),
|
||||||
|
vec!["system".to_string()],
|
||||||
|
)
|
||||||
|
.with_turn_control_signal(control);
|
||||||
|
let summary = runtime
|
||||||
|
.run_turn("start", Some(&mut prompter))
|
||||||
|
.expect("aborted approval should be a normal interruption");
|
||||||
|
abort_thread.join().expect("abort control thread");
|
||||||
|
let messages = runtime.session().messages.clone();
|
||||||
|
|
||||||
|
assert!(summary.interrupted);
|
||||||
|
assert_eq!(calls.load(std::sync::atomic::Ordering::SeqCst), 0);
|
||||||
|
assert!(matches!(
|
||||||
|
&messages[2].blocks[0],
|
||||||
|
ContentBlock::ToolResult {
|
||||||
|
is_error: true,
|
||||||
|
output,
|
||||||
|
..
|
||||||
|
} if output.contains("Interrupted by user")
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn runs_user_to_tool_to_result_loop_end_to_end_and_tracks_usage() {
|
fn runs_user_to_tool_to_result_loop_end_to_end_and_tracks_usage() {
|
||||||
let api_client = ScriptedApiClient { call_count: 0 };
|
let api_client = ScriptedApiClient { call_count: 0 };
|
||||||
@@ -2127,8 +2688,12 @@ mod tests {
|
|||||||
|
|
||||||
// given
|
// given
|
||||||
let interrupt = TurnInterruptSignal::new();
|
let interrupt = TurnInterruptSignal::new();
|
||||||
|
let session = Session::new();
|
||||||
|
let session_id = session.session_id.clone();
|
||||||
|
let captured_events = Arc::new(Mutex::new(Vec::new()));
|
||||||
|
let callback_events = Arc::clone(&captured_events);
|
||||||
let mut runtime = ConversationRuntime::new(
|
let mut runtime = ConversationRuntime::new(
|
||||||
Session::new(),
|
session,
|
||||||
ToolUseApi {
|
ToolUseApi {
|
||||||
interrupt: interrupt.clone(),
|
interrupt: interrupt.clone(),
|
||||||
},
|
},
|
||||||
@@ -2137,7 +2702,13 @@ mod tests {
|
|||||||
PermissionPolicy::new(PermissionMode::DangerFullAccess),
|
PermissionPolicy::new(PermissionMode::DangerFullAccess),
|
||||||
vec!["system".to_string()],
|
vec!["system".to_string()],
|
||||||
)
|
)
|
||||||
.with_turn_interrupt_signal(interrupt);
|
.with_turn_interrupt_signal(interrupt)
|
||||||
|
.with_event_callback(Arc::new(move |event| {
|
||||||
|
callback_events
|
||||||
|
.lock()
|
||||||
|
.expect("event mutex should not be poisoned")
|
||||||
|
.push(event);
|
||||||
|
}));
|
||||||
|
|
||||||
// when
|
// when
|
||||||
let summary = runtime
|
let summary = runtime
|
||||||
@@ -2164,6 +2735,32 @@ mod tests {
|
|||||||
runtime.session().messages[2].blocks[0],
|
runtime.session().messages[2].blocks[0],
|
||||||
ContentBlock::ToolResult { is_error: true, .. }
|
ContentBlock::ToolResult { is_error: true, .. }
|
||||||
));
|
));
|
||||||
|
assert_eq!(
|
||||||
|
*captured_events
|
||||||
|
.lock()
|
||||||
|
.expect("event mutex should not be poisoned"),
|
||||||
|
vec![
|
||||||
|
ConversationEvent::AgentBegin {
|
||||||
|
session_id: session_id.clone(),
|
||||||
|
},
|
||||||
|
ConversationEvent::ToolUse {
|
||||||
|
tool_use_id: "tool-1".to_string(),
|
||||||
|
tool_name: "add".to_string(),
|
||||||
|
input: "2,2".to_string(),
|
||||||
|
},
|
||||||
|
ConversationEvent::ToolResult {
|
||||||
|
tool_use_id: "tool-1".to_string(),
|
||||||
|
tool_name: "add".to_string(),
|
||||||
|
output: "Interrupted by user before this tool could run.".to_string(),
|
||||||
|
is_error: true,
|
||||||
|
},
|
||||||
|
ConversationEvent::AgentEnd {
|
||||||
|
session_id,
|
||||||
|
interrupted: true,
|
||||||
|
output: Some("Running the tool.".to_string()),
|
||||||
|
},
|
||||||
|
]
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|||||||
@@ -84,8 +84,8 @@ pub use config_validate::{
|
|||||||
};
|
};
|
||||||
pub use conversation::{
|
pub use conversation::{
|
||||||
auto_compaction_threshold_from_env, ApiClient, ApiRequest, AssistantEvent, AutoCompactionEvent,
|
auto_compaction_threshold_from_env, ApiClient, ApiRequest, AssistantEvent, AutoCompactionEvent,
|
||||||
ConversationRuntime, PromptCacheEvent, RuntimeError, StaticToolExecutor, ToolError,
|
ConversationEvent, ConversationRuntime, PromptCacheEvent, RuntimeError, StaticToolExecutor,
|
||||||
ToolExecutor, TurnInterruptSignal, TurnSummary,
|
ToolError, ToolExecutor, TurnControlSignal, TurnInterruptSignal, TurnSummary,
|
||||||
};
|
};
|
||||||
pub use file_ops::{
|
pub use file_ops::{
|
||||||
edit_file, edit_file_in_workspace, glob_search, glob_search_in_workspace, grep_search,
|
edit_file, edit_file_in_workspace, glob_search, glob_search_in_workspace, grep_search,
|
||||||
|
|||||||
@@ -13,3 +13,10 @@ serde = { version = "1", features = ["derive"] }
|
|||||||
uuid = { version = "1", features = ["v4", "serde"] }
|
uuid = { version = "1", features = ["v4", "serde"] }
|
||||||
tracing = "0.1"
|
tracing = "0.1"
|
||||||
url = "2"
|
url = "2"
|
||||||
|
runtime = { path = "../runtime" }
|
||||||
|
|
||||||
|
[dev-dependencies]
|
||||||
|
tempfile = "3"
|
||||||
|
|
||||||
|
[lints]
|
||||||
|
workspace = true
|
||||||
@@ -15,6 +15,8 @@ pub enum SdkErrorCode {
|
|||||||
Unauthorized,
|
Unauthorized,
|
||||||
EndpointCredentialForbidden,
|
EndpointCredentialForbidden,
|
||||||
Unavailable,
|
Unavailable,
|
||||||
|
/// Requested session state conflicts with an active invocation
|
||||||
|
Conflict,
|
||||||
ConnectionError,
|
ConnectionError,
|
||||||
ConnectionClosed,
|
ConnectionClosed,
|
||||||
Timeout,
|
Timeout,
|
||||||
@@ -37,6 +39,7 @@ impl SdkErrorCode {
|
|||||||
Self::Unauthorized => "unauthorized",
|
Self::Unauthorized => "unauthorized",
|
||||||
Self::EndpointCredentialForbidden => "endpoint_credential_forbidden",
|
Self::EndpointCredentialForbidden => "endpoint_credential_forbidden",
|
||||||
Self::Unavailable => "unavailable",
|
Self::Unavailable => "unavailable",
|
||||||
|
Self::Conflict => "conflict",
|
||||||
Self::ConnectionError => "connection_error",
|
Self::ConnectionError => "connection_error",
|
||||||
Self::ConnectionClosed => "connection_closed",
|
Self::ConnectionClosed => "connection_closed",
|
||||||
Self::Timeout => "timeout",
|
Self::Timeout => "timeout",
|
||||||
@@ -59,6 +62,7 @@ impl SdkErrorCode {
|
|||||||
"unauthorized" => Self::Unauthorized,
|
"unauthorized" => Self::Unauthorized,
|
||||||
"endpoint_credential_forbidden" => Self::EndpointCredentialForbidden,
|
"endpoint_credential_forbidden" => Self::EndpointCredentialForbidden,
|
||||||
"unavailable" => Self::Unavailable,
|
"unavailable" => Self::Unavailable,
|
||||||
|
"conflict" => Self::Conflict,
|
||||||
"connection_error" => Self::ConnectionError,
|
"connection_error" => Self::ConnectionError,
|
||||||
"connection_closed" => Self::ConnectionClosed,
|
"connection_closed" => Self::ConnectionClosed,
|
||||||
"timeout" => Self::Timeout,
|
"timeout" => Self::Timeout,
|
||||||
@@ -132,6 +136,14 @@ impl SdkError {
|
|||||||
Self::new(SdkErrorCode::Unavailable, message)
|
Self::new(SdkErrorCode::Unavailable, message)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub fn not_found(message: impl Into<String>) -> Self {
|
||||||
|
Self::new(SdkErrorCode::NotFound, message)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn conflict(message: impl Into<String>) -> Self {
|
||||||
|
Self::new(SdkErrorCode::Conflict, message)
|
||||||
|
}
|
||||||
|
|
||||||
pub fn connection_error(message: impl Into<String>) -> Self {
|
pub fn connection_error(message: impl Into<String>) -> Self {
|
||||||
Self::new(SdkErrorCode::ConnectionError, message)
|
Self::new(SdkErrorCode::ConnectionError, message)
|
||||||
}
|
}
|
||||||
@@ -209,6 +221,13 @@ mod tests {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn conflict_error_code_round_trips_on_the_wire() {
|
||||||
|
let error = SdkError::conflict("session is active");
|
||||||
|
assert_eq!(error.code.to_string(), "conflict");
|
||||||
|
assert_eq!(SdkErrorCode::from_wire("conflict"), SdkErrorCode::Conflict);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn unknown_remote_code_is_preserved_with_typed_internal_code() {
|
fn unknown_remote_code_is_preserved_with_typed_internal_code() {
|
||||||
let error = SdkError::from_remote("future_error", "failed");
|
let error = SdkError::from_remote("future_error", "failed");
|
||||||
|
|||||||
@@ -14,9 +14,16 @@
|
|||||||
pub mod error;
|
pub mod error;
|
||||||
pub mod frame;
|
pub mod frame;
|
||||||
pub mod registry;
|
pub mod registry;
|
||||||
|
pub mod session;
|
||||||
pub mod transport;
|
pub mod transport;
|
||||||
|
|
||||||
pub use error::{SdkError, SdkErrorCode, SdkResult, SdkSentRecord};
|
pub use error::{SdkError, SdkErrorCode, SdkResult, SdkSentRecord};
|
||||||
pub use frame::{FrameType, SdkFrame};
|
pub use frame::{FrameType, SdkFrame};
|
||||||
pub use registry::{OperationMetadata, OperationRegistry, Schema};
|
pub use registry::{OperationMetadata, OperationRegistry, Schema};
|
||||||
|
pub use session::{
|
||||||
|
InvocationOutcome, InvocationReconciliationStore, InvocationRecord, InvocationStatus,
|
||||||
|
RuntimeSessionRunner, SessionDispatch, SessionEvent, SessionEventStream, SessionEventType,
|
||||||
|
SessionInvocation, SessionPage, SessionRouter, SessionRunResult, SessionSummary,
|
||||||
|
SessionTurnRunner, TerminalScopeRecord, TerminalScopeStore,
|
||||||
|
};
|
||||||
pub use transport::{SdkClient, SdkClientConfig};
|
pub use transport::{SdkClient, SdkClientConfig};
|
||||||
@@ -263,6 +263,89 @@ impl OperationRegistry {
|
|||||||
)),
|
)),
|
||||||
);
|
);
|
||||||
|
|
||||||
|
// SDK session lifecycle operations.
|
||||||
|
for (name, description, required) in [
|
||||||
|
(
|
||||||
|
"turn.create",
|
||||||
|
"Create a durable SDK session",
|
||||||
|
Vec::<&str>::new(),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"turn.prompt",
|
||||||
|
"Prompt a session and stream lifecycle events",
|
||||||
|
Vec::<&str>::new(),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"turn.steer",
|
||||||
|
"Steer an active session invocation",
|
||||||
|
vec!["invocationId"],
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"turn.abort_and_prompt",
|
||||||
|
"Abort the active invocation and start a replacement prompt",
|
||||||
|
Vec::<&str>::new(),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"turn.list",
|
||||||
|
"List persisted SDK sessions",
|
||||||
|
Vec::<&str>::new(),
|
||||||
|
),
|
||||||
|
] {
|
||||||
|
let mut properties = serde_json::Map::new();
|
||||||
|
properties.insert("sessionId".to_string(), json!({ "type": "string" }));
|
||||||
|
properties.insert("correlationKey".to_string(), json!({ "type": "string" }));
|
||||||
|
properties.insert("invocationId".to_string(), json!({ "type": "string" }));
|
||||||
|
properties.insert("prompt".to_string(), json!({ "type": "string" }));
|
||||||
|
properties.insert(
|
||||||
|
"override".to_string(),
|
||||||
|
json!({ "type": ["string", "object", "array"] }),
|
||||||
|
);
|
||||||
|
properties.insert(
|
||||||
|
"request".to_string(),
|
||||||
|
json!({ "type": ["string", "object", "array"] }),
|
||||||
|
);
|
||||||
|
properties.insert(
|
||||||
|
"input".to_string(),
|
||||||
|
json!({ "type": ["string", "object", "array"] }),
|
||||||
|
);
|
||||||
|
properties.insert("tools".to_string(), json!({ "type": "array" }));
|
||||||
|
properties.insert("settings".to_string(), json!({ "type": "object" }));
|
||||||
|
properties.insert("model".to_string(), json!({ "type": "string" }));
|
||||||
|
properties.insert("cursor".to_string(), json!({ "type": "string" }));
|
||||||
|
properties.insert(
|
||||||
|
"limit".to_string(),
|
||||||
|
json!({ "type": "integer", "minimum": 1 }),
|
||||||
|
);
|
||||||
|
let mut input_schema = json!({
|
||||||
|
"type": "object",
|
||||||
|
"properties": properties,
|
||||||
|
"required": required,
|
||||||
|
});
|
||||||
|
let prompt_alternatives = json!([
|
||||||
|
{ "required": ["prompt"] },
|
||||||
|
{ "required": ["request"] },
|
||||||
|
{ "required": ["input"] },
|
||||||
|
]);
|
||||||
|
match name {
|
||||||
|
"turn.prompt" | "turn.abort_and_prompt" => {
|
||||||
|
input_schema["anyOf"] = prompt_alternatives;
|
||||||
|
}
|
||||||
|
"turn.steer" => {
|
||||||
|
input_schema["anyOf"] = json!([
|
||||||
|
{ "required": ["override"] },
|
||||||
|
{ "required": ["prompt"] },
|
||||||
|
{ "required": ["request"] },
|
||||||
|
{ "required": ["input"] },
|
||||||
|
]);
|
||||||
|
}
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
|
registry.register(
|
||||||
|
OperationMetadata::new(name, description)
|
||||||
|
.with_input_schema(Schema::new(format!("Input for {name}"), input_schema)),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
registry
|
registry
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -397,6 +480,20 @@ mod tests {
|
|||||||
assert!(registry.has_operation("events.replay"));
|
assert!(registry.has_operation("events.replay"));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn standard_registry_has_session_lifecycle_operations() {
|
||||||
|
let registry = OperationRegistry::with_standard_operations();
|
||||||
|
for operation in [
|
||||||
|
"turn.create",
|
||||||
|
"turn.prompt",
|
||||||
|
"turn.steer",
|
||||||
|
"turn.abort_and_prompt",
|
||||||
|
"turn.list",
|
||||||
|
] {
|
||||||
|
assert!(registry.has_operation(operation), "missing {operation}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn registry_validation_catches_missing_required_fields() {
|
fn registry_validation_catches_missing_required_fields() {
|
||||||
let registry = OperationRegistry::with_standard_operations();
|
let registry = OperationRegistry::with_standard_operations();
|
||||||
|
|||||||
@@ -0,0 +1,31 @@
|
|||||||
|
//! Event stream types for session lifecycle operations.
|
||||||
|
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use serde_json::Value;
|
||||||
|
|
||||||
|
/// Lifecycle events published for one SDK turn invocation.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "snake_case")]
|
||||||
|
pub enum SessionEventType {
|
||||||
|
TurnAccepted,
|
||||||
|
AgentBegin,
|
||||||
|
ToolUse,
|
||||||
|
ToolResult,
|
||||||
|
AgentEnd,
|
||||||
|
TurnSteered,
|
||||||
|
TurnEnded,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One ordered session lifecycle event.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "camelCase")]
|
||||||
|
pub struct SessionEvent {
|
||||||
|
pub event_id: String,
|
||||||
|
pub event_type: SessionEventType,
|
||||||
|
pub session_id: String,
|
||||||
|
pub invocation_id: String,
|
||||||
|
pub sequence: u64,
|
||||||
|
pub timestamp_ms: u64,
|
||||||
|
pub durable: bool,
|
||||||
|
pub payload: Value,
|
||||||
|
}
|
||||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,673 @@
|
|||||||
|
//! Durable invocation state for recovering SDK sessions across process restarts.
|
||||||
|
|
||||||
|
use std::{
|
||||||
|
fs::{self, OpenOptions},
|
||||||
|
io::Write,
|
||||||
|
path::{Path, PathBuf},
|
||||||
|
sync::Mutex,
|
||||||
|
time::{SystemTime, UNIX_EPOCH},
|
||||||
|
};
|
||||||
|
|
||||||
|
#[cfg(unix)]
|
||||||
|
use std::fs::File;
|
||||||
|
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use uuid::Uuid;
|
||||||
|
|
||||||
|
use crate::error::{SdkError, SdkResult};
|
||||||
|
|
||||||
|
static RECONCILIATION_WRITE_LOCK: Mutex<()> = Mutex::new(());
|
||||||
|
|
||||||
|
/// Lifecycle state persisted for an invocation.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "snake_case")]
|
||||||
|
pub enum InvocationStatus {
|
||||||
|
Accepted,
|
||||||
|
Running,
|
||||||
|
Uncertain,
|
||||||
|
Terminal,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Result captured when an invocation reaches a terminal state.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "snake_case")]
|
||||||
|
pub enum InvocationOutcome {
|
||||||
|
Completed {
|
||||||
|
message_count: usize,
|
||||||
|
},
|
||||||
|
Aborted {
|
||||||
|
message_count: usize,
|
||||||
|
},
|
||||||
|
Failed {
|
||||||
|
message: String,
|
||||||
|
message_count: usize,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Persisted lifecycle and reconciliation data for one invocation.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct InvocationRecord {
|
||||||
|
pub invocation_id: String,
|
||||||
|
pub session_id: String,
|
||||||
|
pub correlation_key: Option<String>,
|
||||||
|
pub status: InvocationStatus,
|
||||||
|
pub accepted_at_ms: u64,
|
||||||
|
pub updated_at_ms: u64,
|
||||||
|
pub outcome: Option<InvocationOutcome>,
|
||||||
|
pub uncertainty_reason: Option<String>,
|
||||||
|
pub terminal_event: Option<serde_json::Value>,
|
||||||
|
pub terminal_scope_revision: Option<u64>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// File-backed store for invocation lifecycle reconciliation.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct InvocationReconciliationStore {
|
||||||
|
root: PathBuf,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl InvocationReconciliationStore {
|
||||||
|
/// Creates a store handle without creating its root directory.
|
||||||
|
#[must_use]
|
||||||
|
pub fn new(root: impl Into<PathBuf>) -> Self {
|
||||||
|
Self { root: root.into() }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Inserts a new invocation record. Existing IDs are not overwritten.
|
||||||
|
pub fn insert(&self, record: InvocationRecord) -> SdkResult<()> {
|
||||||
|
let _guard = RECONCILIATION_WRITE_LOCK
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||||
|
let path = self.record_path(&record.invocation_id)?;
|
||||||
|
if self.get(&record.invocation_id)?.is_some() {
|
||||||
|
return Err(SdkError::invalid_input(format!(
|
||||||
|
"invocation '{}' already exists",
|
||||||
|
record.invocation_id
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
atomic_write_json(&path, &record)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Marks a non-terminal invocation as running.
|
||||||
|
pub fn mark_running(&self, invocation_id: &str) -> SdkResult<()> {
|
||||||
|
let _guard = RECONCILIATION_WRITE_LOCK
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||||
|
let mut record = self.required_record(invocation_id)?;
|
||||||
|
if record.status != InvocationStatus::Accepted {
|
||||||
|
return Err(SdkError::conflict(format!(
|
||||||
|
"invocation '{}' cannot start from state {:?}",
|
||||||
|
invocation_id, record.status
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
record.status = InvocationStatus::Running;
|
||||||
|
record.uncertainty_reason = None;
|
||||||
|
record.updated_at_ms = updated_at_ms(record.updated_at_ms);
|
||||||
|
self.write_record(&record)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Marks a non-terminal invocation as uncertain with a recovery reason.
|
||||||
|
pub fn mark_uncertain(&self, invocation_id: &str, reason: impl Into<String>) -> SdkResult<()> {
|
||||||
|
let reason = reason.into();
|
||||||
|
self.update_nonterminal(invocation_id, move |record| {
|
||||||
|
record.status = InvocationStatus::Uncertain;
|
||||||
|
record.uncertainty_reason = Some(reason);
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Records a terminal result. Once terminal, a record cannot be changed again.
|
||||||
|
pub fn record_terminal(
|
||||||
|
&self,
|
||||||
|
invocation_id: &str,
|
||||||
|
outcome: InvocationOutcome,
|
||||||
|
terminal_event: Option<serde_json::Value>,
|
||||||
|
) -> SdkResult<()> {
|
||||||
|
self.record_terminal_with_scope_revision(invocation_id, outcome, terminal_event, None)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Records a terminal result together with its durable correlation-scope revision.
|
||||||
|
pub fn record_terminal_with_scope_revision(
|
||||||
|
&self,
|
||||||
|
invocation_id: &str,
|
||||||
|
outcome: InvocationOutcome,
|
||||||
|
terminal_event: Option<serde_json::Value>,
|
||||||
|
terminal_scope_revision: Option<u64>,
|
||||||
|
) -> SdkResult<()> {
|
||||||
|
let _guard = RECONCILIATION_WRITE_LOCK
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||||
|
let mut record = self.required_record(invocation_id)?;
|
||||||
|
if record.status == InvocationStatus::Terminal {
|
||||||
|
return Err(SdkError::invalid_input(format!(
|
||||||
|
"invocation '{}' is already terminal",
|
||||||
|
invocation_id
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
if record
|
||||||
|
.outcome
|
||||||
|
.as_ref()
|
||||||
|
.is_some_and(|pending| pending != &outcome)
|
||||||
|
|| record
|
||||||
|
.terminal_event
|
||||||
|
.as_ref()
|
||||||
|
.is_some_and(|pending| Some(pending) != terminal_event.as_ref())
|
||||||
|
{
|
||||||
|
return Err(SdkError::conflict(format!(
|
||||||
|
"invocation '{}' has a different pending terminal result",
|
||||||
|
invocation_id
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
record.status = InvocationStatus::Terminal;
|
||||||
|
record.outcome = Some(outcome);
|
||||||
|
record.uncertainty_reason = None;
|
||||||
|
record.terminal_event = terminal_event;
|
||||||
|
record.terminal_scope_revision = terminal_scope_revision;
|
||||||
|
record.updated_at_ms = updated_at_ms(record.updated_at_ms);
|
||||||
|
self.write_record(&record)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Persists a known turn result before its correlation scope is updated.
|
||||||
|
///
|
||||||
|
/// The record remains non-terminal until the scope update succeeds, allowing
|
||||||
|
/// startup or a later reconciliation read to safely finish the transition.
|
||||||
|
pub fn record_pending_terminal(
|
||||||
|
&self,
|
||||||
|
invocation_id: &str,
|
||||||
|
outcome: InvocationOutcome,
|
||||||
|
terminal_event: serde_json::Value,
|
||||||
|
) -> SdkResult<()> {
|
||||||
|
let _guard = RECONCILIATION_WRITE_LOCK
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||||
|
let mut record = self.required_record(invocation_id)?;
|
||||||
|
if record.status == InvocationStatus::Terminal {
|
||||||
|
return Err(SdkError::invalid_input(format!(
|
||||||
|
"invocation '{}' is already terminal",
|
||||||
|
invocation_id
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
if record.outcome.is_some() || record.terminal_event.is_some() {
|
||||||
|
return Err(SdkError::conflict(format!(
|
||||||
|
"invocation '{}' already has a pending terminal result",
|
||||||
|
invocation_id
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
record.outcome = Some(outcome);
|
||||||
|
record.terminal_event = Some(terminal_event);
|
||||||
|
record.terminal_scope_revision = None;
|
||||||
|
record.updated_at_ms = updated_at_ms(record.updated_at_ms);
|
||||||
|
self.write_record(&record)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Loads one invocation record, if present.
|
||||||
|
pub fn get(&self, invocation_id: &str) -> SdkResult<Option<InvocationRecord>> {
|
||||||
|
let path = self.record_path(invocation_id)?;
|
||||||
|
read_record(&path, invocation_id)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Loads all invocation records in stable invocation-ID order.
|
||||||
|
pub fn list(&self) -> SdkResult<Vec<InvocationRecord>> {
|
||||||
|
let entries = match fs::read_dir(&self.root) {
|
||||||
|
Ok(entries) => entries,
|
||||||
|
Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(Vec::new()),
|
||||||
|
Err(error) => {
|
||||||
|
return Err(io_error(
|
||||||
|
"read invocation store directory",
|
||||||
|
&self.root,
|
||||||
|
error,
|
||||||
|
))
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
let mut records = Vec::new();
|
||||||
|
for entry in entries {
|
||||||
|
let entry = entry
|
||||||
|
.map_err(|error| io_error("read invocation store entry", &self.root, error))?;
|
||||||
|
if !entry
|
||||||
|
.file_type()
|
||||||
|
.map_err(|error| io_error("inspect invocation store entry", &entry.path(), error))?
|
||||||
|
.is_file()
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
let path = entry.path();
|
||||||
|
if path.extension() != Some(std::ffi::OsStr::new("json")) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let invocation_id = path
|
||||||
|
.file_stem()
|
||||||
|
.and_then(|stem| stem.to_str())
|
||||||
|
.ok_or_else(|| SdkError::invalid_input("invocation filename is not valid UTF-8"))?;
|
||||||
|
validate_invocation_id(invocation_id)?;
|
||||||
|
if let Some(record) = read_record(&path, invocation_id)? {
|
||||||
|
records.push(record);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
records.sort_by(|left, right| left.invocation_id.cmp(&right.invocation_id));
|
||||||
|
Ok(records)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Loads records associated with the exact correlation key.
|
||||||
|
pub fn list_by_correlation_key(&self, key: &str) -> SdkResult<Vec<InvocationRecord>> {
|
||||||
|
Ok(self
|
||||||
|
.list()?
|
||||||
|
.into_iter()
|
||||||
|
.filter(|record| record.correlation_key.as_deref() == Some(key))
|
||||||
|
.collect())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Durably marks accepted and running records uncertain after a restart.
|
||||||
|
///
|
||||||
|
/// Returns the records changed by this recovery pass; already uncertain and
|
||||||
|
/// terminal records are left untouched and are not included in the result.
|
||||||
|
pub fn reconcile_unfinished(&self) -> SdkResult<Vec<InvocationRecord>> {
|
||||||
|
let _guard = RECONCILIATION_WRITE_LOCK
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||||
|
let mut recovered = Vec::new();
|
||||||
|
for mut record in self.list()? {
|
||||||
|
if matches!(
|
||||||
|
record.status,
|
||||||
|
InvocationStatus::Accepted | InvocationStatus::Running
|
||||||
|
) {
|
||||||
|
record.status = InvocationStatus::Uncertain;
|
||||||
|
record.uncertainty_reason =
|
||||||
|
Some("recovered unfinished invocation after restart".into());
|
||||||
|
record.updated_at_ms = updated_at_ms(record.updated_at_ms);
|
||||||
|
self.write_record(&record)?;
|
||||||
|
recovered.push(record);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(recovered)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn record_path(&self, invocation_id: &str) -> SdkResult<PathBuf> {
|
||||||
|
validate_invocation_id(invocation_id)?;
|
||||||
|
Ok(self.root.join(format!("{invocation_id}.json")))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn required_record(&self, invocation_id: &str) -> SdkResult<InvocationRecord> {
|
||||||
|
self.get(invocation_id)?.ok_or_else(|| {
|
||||||
|
SdkError::invalid_input(format!("invocation '{invocation_id}' does not exist"))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn update_nonterminal(
|
||||||
|
&self,
|
||||||
|
invocation_id: &str,
|
||||||
|
update: impl FnOnce(&mut InvocationRecord),
|
||||||
|
) -> SdkResult<()> {
|
||||||
|
let _guard = RECONCILIATION_WRITE_LOCK
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||||
|
let mut record = self.required_record(invocation_id)?;
|
||||||
|
if record.status == InvocationStatus::Terminal {
|
||||||
|
return Err(SdkError::invalid_input(format!(
|
||||||
|
"invocation '{}' is already terminal",
|
||||||
|
invocation_id
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
update(&mut record);
|
||||||
|
record.updated_at_ms = updated_at_ms(record.updated_at_ms);
|
||||||
|
self.write_record(&record)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn write_record(&self, record: &InvocationRecord) -> SdkResult<()> {
|
||||||
|
let path = self.record_path(&record.invocation_id)?;
|
||||||
|
atomic_write_json(&path, record)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Atomically serializes a value beside its target and renames it into place.
|
||||||
|
pub(super) fn atomic_write_json<T: Serialize + ?Sized>(path: &Path, value: &T) -> SdkResult<()> {
|
||||||
|
let bytes = serde_json::to_vec(value)
|
||||||
|
.map_err(|error| SdkError::internal(format!("serialize {}: {error}", path.display())))?;
|
||||||
|
let parent = path
|
||||||
|
.parent()
|
||||||
|
.filter(|parent| !parent.as_os_str().is_empty())
|
||||||
|
.unwrap_or_else(|| Path::new("."));
|
||||||
|
fs::create_dir_all(parent)
|
||||||
|
.map_err(|error| io_error("create store directory", parent, error))?;
|
||||||
|
let filename = path
|
||||||
|
.file_name()
|
||||||
|
.ok_or_else(|| SdkError::invalid_input("store path has no filename"))?;
|
||||||
|
let temporary_path = parent.join(format!(
|
||||||
|
".{}.{}.tmp",
|
||||||
|
filename.to_string_lossy(),
|
||||||
|
Uuid::new_v4()
|
||||||
|
));
|
||||||
|
|
||||||
|
let write_result = (|| -> std::io::Result<()> {
|
||||||
|
let mut temporary = OpenOptions::new()
|
||||||
|
.write(true)
|
||||||
|
.create_new(true)
|
||||||
|
.open(&temporary_path)?;
|
||||||
|
temporary.write_all(&bytes)?;
|
||||||
|
temporary.sync_all()?;
|
||||||
|
drop(temporary);
|
||||||
|
fs::rename(&temporary_path, path)?;
|
||||||
|
#[cfg(unix)]
|
||||||
|
File::open(parent)?.sync_all()?;
|
||||||
|
Ok(())
|
||||||
|
})();
|
||||||
|
|
||||||
|
match write_result {
|
||||||
|
Ok(()) => Ok(()),
|
||||||
|
Err(error) => {
|
||||||
|
let _ = fs::remove_file(&temporary_path);
|
||||||
|
Err(io_error("atomically write JSON", path, error))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn read_record(path: &Path, expected_id: &str) -> SdkResult<Option<InvocationRecord>> {
|
||||||
|
let bytes = match fs::read(path) {
|
||||||
|
Ok(bytes) => bytes,
|
||||||
|
Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None),
|
||||||
|
Err(error) => return Err(io_error("read invocation record", path, error)),
|
||||||
|
};
|
||||||
|
let record: InvocationRecord = serde_json::from_slice(&bytes).map_err(|error| {
|
||||||
|
SdkError::internal(format!(
|
||||||
|
"parse invocation record {}: {error}",
|
||||||
|
path.display()
|
||||||
|
))
|
||||||
|
})?;
|
||||||
|
validate_invocation_id(&record.invocation_id)?;
|
||||||
|
if record.invocation_id != expected_id {
|
||||||
|
return Err(SdkError::internal(format!(
|
||||||
|
"invocation record ID '{}' does not match filename ID '{expected_id}'",
|
||||||
|
record.invocation_id
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
Ok(Some(record))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn validate_invocation_id(invocation_id: &str) -> SdkResult<()> {
|
||||||
|
if invocation_id.is_empty()
|
||||||
|
|| invocation_id.len() > 200
|
||||||
|
|| !invocation_id
|
||||||
|
.bytes()
|
||||||
|
.all(|byte| byte.is_ascii_alphanumeric() || byte == b'-' || byte == b'_')
|
||||||
|
{
|
||||||
|
return Err(SdkError::invalid_input(format!(
|
||||||
|
"unsafe invocation ID '{invocation_id}'"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn updated_at_ms(previous: u64) -> u64 {
|
||||||
|
let now = SystemTime::now()
|
||||||
|
.duration_since(UNIX_EPOCH)
|
||||||
|
.unwrap_or_default()
|
||||||
|
.as_millis()
|
||||||
|
.min(u128::from(u64::MAX)) as u64;
|
||||||
|
now.max(previous.saturating_add(1))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn io_error(action: &str, path: &Path, error: std::io::Error) -> SdkError {
|
||||||
|
SdkError::internal(format!("{action} '{}': {error}", path.display()))
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
struct TestDirectory(PathBuf);
|
||||||
|
|
||||||
|
impl TestDirectory {
|
||||||
|
fn new() -> Self {
|
||||||
|
let path = std::env::temp_dir().join(format!("sdk-session-{}", Uuid::new_v4()));
|
||||||
|
fs::create_dir(&path).expect("create test directory");
|
||||||
|
Self(path)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Drop for TestDirectory {
|
||||||
|
fn drop(&mut self) {
|
||||||
|
let _ = fs::remove_dir_all(&self.0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn accepted_record(invocation_id: &str) -> InvocationRecord {
|
||||||
|
InvocationRecord {
|
||||||
|
invocation_id: invocation_id.into(),
|
||||||
|
session_id: "session-a".into(),
|
||||||
|
correlation_key: Some("request-a".into()),
|
||||||
|
status: InvocationStatus::Accepted,
|
||||||
|
accepted_at_ms: 10,
|
||||||
|
updated_at_ms: 10,
|
||||||
|
outcome: None,
|
||||||
|
uncertainty_reason: None,
|
||||||
|
terminal_event: None,
|
||||||
|
terminal_scope_revision: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn constructor_has_no_filesystem_side_effects() {
|
||||||
|
let root = std::env::temp_dir().join(format!("sdk-session-{}", Uuid::new_v4()));
|
||||||
|
let _store = InvocationReconciliationStore::new(&root);
|
||||||
|
assert!(!root.exists());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn invocation_records_survive_store_restart() {
|
||||||
|
let root = TestDirectory::new();
|
||||||
|
let record = accepted_record("invocation-a");
|
||||||
|
InvocationReconciliationStore::new(&root.0)
|
||||||
|
.insert(record.clone())
|
||||||
|
.expect("insert record");
|
||||||
|
|
||||||
|
let loaded = InvocationReconciliationStore::new(&root.0)
|
||||||
|
.get("invocation-a")
|
||||||
|
.expect("load record")
|
||||||
|
.expect("record exists");
|
||||||
|
assert_eq!(loaded, record);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn running_transition_only_accepts_an_accepted_invocation() {
|
||||||
|
let root = TestDirectory::new();
|
||||||
|
let store = InvocationReconciliationStore::new(&root.0);
|
||||||
|
store
|
||||||
|
.insert(accepted_record("invocation-a"))
|
||||||
|
.expect("insert");
|
||||||
|
store.mark_running("invocation-a").expect("mark running");
|
||||||
|
assert_eq!(
|
||||||
|
store.get("invocation-a").unwrap().unwrap().status,
|
||||||
|
InvocationStatus::Running
|
||||||
|
);
|
||||||
|
assert!(store.mark_running("invocation-a").is_err());
|
||||||
|
store
|
||||||
|
.mark_uncertain("invocation-a", "recovered")
|
||||||
|
.expect("mark uncertain");
|
||||||
|
assert!(store.mark_running("invocation-a").is_err());
|
||||||
|
assert_eq!(
|
||||||
|
store.get("invocation-a").unwrap().unwrap().status,
|
||||||
|
InvocationStatus::Uncertain
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn uncertain_invocation_can_become_terminal_only_once() {
|
||||||
|
let root = TestDirectory::new();
|
||||||
|
let store = InvocationReconciliationStore::new(&root.0);
|
||||||
|
store
|
||||||
|
.insert(accepted_record("invocation-a"))
|
||||||
|
.expect("insert");
|
||||||
|
store
|
||||||
|
.mark_uncertain("invocation-a", "runtime disconnected")
|
||||||
|
.expect("mark uncertain");
|
||||||
|
store
|
||||||
|
.record_terminal(
|
||||||
|
"invocation-a",
|
||||||
|
InvocationOutcome::Completed { message_count: 2 },
|
||||||
|
Some(serde_json::json!({"type": "completed"})),
|
||||||
|
)
|
||||||
|
.expect("record terminal");
|
||||||
|
|
||||||
|
let terminal = store.get("invocation-a").expect("load").expect("exists");
|
||||||
|
assert_eq!(terminal.status, InvocationStatus::Terminal);
|
||||||
|
assert_eq!(terminal.uncertainty_reason, None);
|
||||||
|
assert_eq!(
|
||||||
|
terminal.outcome,
|
||||||
|
Some(InvocationOutcome::Completed { message_count: 2 })
|
||||||
|
);
|
||||||
|
assert!(store
|
||||||
|
.record_terminal(
|
||||||
|
"invocation-a",
|
||||||
|
InvocationOutcome::Aborted { message_count: 0 },
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
.is_err());
|
||||||
|
assert_eq!(store.get("invocation-a").expect("reload"), Some(terminal));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn recovery_marks_only_accepted_and_running_records_uncertain() {
|
||||||
|
let root = TestDirectory::new();
|
||||||
|
let store = InvocationReconciliationStore::new(&root.0);
|
||||||
|
store
|
||||||
|
.insert(accepted_record("accepted"))
|
||||||
|
.expect("insert accepted");
|
||||||
|
let mut running = accepted_record("running");
|
||||||
|
running.status = InvocationStatus::Running;
|
||||||
|
store.insert(running).expect("insert running");
|
||||||
|
let mut terminal = accepted_record("terminal");
|
||||||
|
terminal.status = InvocationStatus::Terminal;
|
||||||
|
terminal.outcome = Some(InvocationOutcome::Aborted { message_count: 0 });
|
||||||
|
store.insert(terminal.clone()).expect("insert terminal");
|
||||||
|
|
||||||
|
let recovered = store.reconcile_unfinished().expect("reconcile");
|
||||||
|
assert_eq!(recovered.len(), 2);
|
||||||
|
assert!(recovered
|
||||||
|
.iter()
|
||||||
|
.all(|record| record.status == InvocationStatus::Uncertain));
|
||||||
|
assert_eq!(
|
||||||
|
store.get("terminal").expect("load terminal"),
|
||||||
|
Some(terminal)
|
||||||
|
);
|
||||||
|
assert!(store
|
||||||
|
.reconcile_unfinished()
|
||||||
|
.expect("repeat recovery")
|
||||||
|
.is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn pending_terminal_result_survives_uncertainty_and_can_be_finalized() {
|
||||||
|
let root = TestDirectory::new();
|
||||||
|
let store = InvocationReconciliationStore::new(&root.0);
|
||||||
|
store
|
||||||
|
.insert(accepted_record("pending"))
|
||||||
|
.expect("insert accepted record");
|
||||||
|
store.mark_running("pending").expect("mark running");
|
||||||
|
store
|
||||||
|
.record_pending_terminal(
|
||||||
|
"pending",
|
||||||
|
InvocationOutcome::Completed { message_count: 3 },
|
||||||
|
serde_json::json!({"eventType": "turn_ended"}),
|
||||||
|
)
|
||||||
|
.expect("record pending result");
|
||||||
|
store
|
||||||
|
.mark_uncertain("pending", "terminal scope store was unavailable")
|
||||||
|
.expect("mark uncertain");
|
||||||
|
|
||||||
|
let pending = store.get("pending").unwrap().unwrap();
|
||||||
|
assert_eq!(pending.status, InvocationStatus::Uncertain);
|
||||||
|
assert_eq!(
|
||||||
|
pending.outcome,
|
||||||
|
Some(InvocationOutcome::Completed { message_count: 3 })
|
||||||
|
);
|
||||||
|
store
|
||||||
|
.record_terminal_with_scope_revision(
|
||||||
|
"pending",
|
||||||
|
pending.outcome.unwrap(),
|
||||||
|
pending.terminal_event,
|
||||||
|
Some(1),
|
||||||
|
)
|
||||||
|
.expect("finalize after recovery");
|
||||||
|
assert_eq!(
|
||||||
|
store.get("pending").unwrap().unwrap().status,
|
||||||
|
InvocationStatus::Terminal
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn concurrent_terminal_and_uncertain_transitions_preserve_terminal_result() {
|
||||||
|
let root = TestDirectory::new();
|
||||||
|
let store = InvocationReconciliationStore::new(&root.0);
|
||||||
|
store
|
||||||
|
.insert(accepted_record("concurrent"))
|
||||||
|
.expect("insert accepted record");
|
||||||
|
store.mark_running("concurrent").expect("mark running");
|
||||||
|
|
||||||
|
let barrier = std::sync::Arc::new(std::sync::Barrier::new(3));
|
||||||
|
let terminal_store = store.clone();
|
||||||
|
let terminal_barrier = std::sync::Arc::clone(&barrier);
|
||||||
|
let terminal = std::thread::spawn(move || {
|
||||||
|
terminal_barrier.wait();
|
||||||
|
terminal_store.record_terminal(
|
||||||
|
"concurrent",
|
||||||
|
InvocationOutcome::Completed { message_count: 1 },
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
});
|
||||||
|
let uncertain_store = store.clone();
|
||||||
|
let uncertain_barrier = std::sync::Arc::clone(&barrier);
|
||||||
|
let uncertain = std::thread::spawn(move || {
|
||||||
|
uncertain_barrier.wait();
|
||||||
|
uncertain_store.mark_uncertain("concurrent", "raced with terminal write")
|
||||||
|
});
|
||||||
|
barrier.wait();
|
||||||
|
let _ = terminal.join().expect("terminal writer thread");
|
||||||
|
let _ = uncertain.join().expect("uncertain writer thread");
|
||||||
|
|
||||||
|
let record = store.get("concurrent").unwrap().unwrap();
|
||||||
|
assert_eq!(record.status, InvocationStatus::Terminal);
|
||||||
|
assert_eq!(
|
||||||
|
record.outcome,
|
||||||
|
Some(InvocationOutcome::Completed { message_count: 1 })
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn concurrent_duplicate_inserts_have_only_one_winner() {
|
||||||
|
let root = TestDirectory::new();
|
||||||
|
let store = InvocationReconciliationStore::new(&root.0);
|
||||||
|
let barrier = std::sync::Arc::new(std::sync::Barrier::new(3));
|
||||||
|
let first_store = store.clone();
|
||||||
|
let first_barrier = std::sync::Arc::clone(&barrier);
|
||||||
|
let first = std::thread::spawn(move || {
|
||||||
|
first_barrier.wait();
|
||||||
|
first_store.insert(accepted_record("duplicate"))
|
||||||
|
});
|
||||||
|
let second_store = store.clone();
|
||||||
|
let second_barrier = std::sync::Arc::clone(&barrier);
|
||||||
|
let second = std::thread::spawn(move || {
|
||||||
|
second_barrier.wait();
|
||||||
|
second_store.insert(accepted_record("duplicate"))
|
||||||
|
});
|
||||||
|
barrier.wait();
|
||||||
|
let first = first.join().expect("first insert thread");
|
||||||
|
let second = second.join().expect("second insert thread");
|
||||||
|
|
||||||
|
assert_ne!(first.is_ok(), second.is_ok());
|
||||||
|
assert_eq!(
|
||||||
|
store.get("duplicate").unwrap().unwrap().invocation_id,
|
||||||
|
"duplicate"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn malformed_ids_and_json_are_rejected() {
|
||||||
|
let root = TestDirectory::new();
|
||||||
|
let store = InvocationReconciliationStore::new(&root.0);
|
||||||
|
assert!(store.get("../escape").is_err());
|
||||||
|
assert!(store.insert(accepted_record("bad/id")).is_err());
|
||||||
|
|
||||||
|
fs::write(root.0.join("malformed.json"), "{").expect("write malformed record");
|
||||||
|
assert!(store.get("malformed").is_err());
|
||||||
|
assert!(store.list().is_err());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,310 @@
|
|||||||
|
//! Durable mapping from SDK correlation keys to their active terminal session.
|
||||||
|
|
||||||
|
use std::{
|
||||||
|
fs,
|
||||||
|
path::PathBuf,
|
||||||
|
sync::Mutex,
|
||||||
|
time::{SystemTime, UNIX_EPOCH},
|
||||||
|
};
|
||||||
|
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
|
||||||
|
use crate::error::{SdkError, SdkResult};
|
||||||
|
|
||||||
|
use super::reconciliation::atomic_write_json;
|
||||||
|
|
||||||
|
const MAX_CORRELATION_KEY_BYTES: usize = 512;
|
||||||
|
static TERMINAL_SCOPE_WRITE_LOCK: Mutex<()> = Mutex::new(());
|
||||||
|
|
||||||
|
/// Persisted terminal-session scope for one correlation key.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct TerminalScopeRecord {
|
||||||
|
pub correlation_key: String,
|
||||||
|
pub session_id: String,
|
||||||
|
pub revision: u64,
|
||||||
|
pub updated_at_ms: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// File-backed mapping of correlation keys to terminal sessions.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct TerminalScopeStore {
|
||||||
|
root: PathBuf,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl TerminalScopeStore {
|
||||||
|
/// Creates a store handle without creating its root directory.
|
||||||
|
#[must_use]
|
||||||
|
pub fn new(root: impl Into<PathBuf>) -> Self {
|
||||||
|
Self { root: root.into() }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Sets the session associated with a correlation key.
|
||||||
|
pub fn set(
|
||||||
|
&self,
|
||||||
|
correlation_key: impl Into<String>,
|
||||||
|
session_id: impl Into<String>,
|
||||||
|
) -> SdkResult<TerminalScopeRecord> {
|
||||||
|
let correlation_key = correlation_key.into();
|
||||||
|
validate_correlation_key(&correlation_key)?;
|
||||||
|
let _guard = TERMINAL_SCOPE_WRITE_LOCK
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||||
|
let previous = self.read_record(&correlation_key)?;
|
||||||
|
let revision = previous
|
||||||
|
.as_ref()
|
||||||
|
.map_or(Some(1), |record| record.revision.checked_add(1))
|
||||||
|
.ok_or_else(|| SdkError::internal("terminal scope revision is exhausted"))?;
|
||||||
|
let previous_updated_at = previous.map_or(0, |record| record.updated_at_ms);
|
||||||
|
let record = TerminalScopeRecord {
|
||||||
|
correlation_key: correlation_key.clone(),
|
||||||
|
session_id: session_id.into(),
|
||||||
|
revision,
|
||||||
|
updated_at_ms: current_timestamp_ms().max(previous_updated_at.saturating_add(1)),
|
||||||
|
};
|
||||||
|
atomic_write_json(&self.record_path(&correlation_key), &record)?;
|
||||||
|
Ok(record)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Rebuilds a mapping from durable invocation history without moving it backward.
|
||||||
|
pub fn restore_if_newer(
|
||||||
|
&self,
|
||||||
|
correlation_key: &str,
|
||||||
|
session_id: &str,
|
||||||
|
revision: u64,
|
||||||
|
) -> SdkResult<()> {
|
||||||
|
validate_correlation_key(correlation_key)?;
|
||||||
|
let _guard = TERMINAL_SCOPE_WRITE_LOCK
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||||
|
let previous = self.read_record(correlation_key)?;
|
||||||
|
match previous.as_ref() {
|
||||||
|
Some(record) if record.revision > revision => return Ok(()),
|
||||||
|
Some(record) if record.revision == revision && record.session_id == session_id => {
|
||||||
|
return Ok(())
|
||||||
|
}
|
||||||
|
Some(record) if record.revision == revision => {
|
||||||
|
return Err(SdkError::internal(format!(
|
||||||
|
"terminal scope revision {revision} conflicts for key '{correlation_key}'"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
|
let record = TerminalScopeRecord {
|
||||||
|
correlation_key: correlation_key.to_string(),
|
||||||
|
session_id: session_id.to_string(),
|
||||||
|
revision,
|
||||||
|
updated_at_ms: current_timestamp_ms().max(
|
||||||
|
previous
|
||||||
|
.map_or(0, |record| record.updated_at_ms)
|
||||||
|
.saturating_add(1),
|
||||||
|
),
|
||||||
|
};
|
||||||
|
atomic_write_json(&self.record_path(correlation_key), &record)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Loads the session associated with a correlation key, if present.
|
||||||
|
pub fn get(&self, correlation_key: &str) -> SdkResult<Option<TerminalScopeRecord>> {
|
||||||
|
validate_correlation_key(correlation_key)?;
|
||||||
|
self.read_record(correlation_key)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn read_record(&self, correlation_key: &str) -> SdkResult<Option<TerminalScopeRecord>> {
|
||||||
|
let path = self.record_path(correlation_key);
|
||||||
|
let bytes = match fs::read(&path) {
|
||||||
|
Ok(bytes) => bytes,
|
||||||
|
Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None),
|
||||||
|
Err(error) => {
|
||||||
|
return Err(SdkError::internal(format!(
|
||||||
|
"read terminal scope '{}': {error}",
|
||||||
|
path.display()
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let record: TerminalScopeRecord = serde_json::from_slice(&bytes).map_err(|error| {
|
||||||
|
SdkError::internal(format!(
|
||||||
|
"parse terminal scope '{}': {error}",
|
||||||
|
path.display()
|
||||||
|
))
|
||||||
|
})?;
|
||||||
|
if record.correlation_key != correlation_key {
|
||||||
|
return Err(SdkError::internal(format!(
|
||||||
|
"terminal scope key does not match requested key for '{}'",
|
||||||
|
path.display()
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
Ok(Some(record))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn record_path(&self, correlation_key: &str) -> PathBuf {
|
||||||
|
const MAX_COMPONENT_BYTES: usize = 180;
|
||||||
|
let encoded = encode_key_as_hex(correlation_key);
|
||||||
|
let mut path = self.root.clone();
|
||||||
|
let mut remaining = encoded.as_str();
|
||||||
|
while remaining.len() > MAX_COMPONENT_BYTES {
|
||||||
|
let (component, tail) = remaining.split_at(MAX_COMPONENT_BYTES);
|
||||||
|
path.push(component);
|
||||||
|
remaining = tail;
|
||||||
|
}
|
||||||
|
path.push(format!("{remaining}.json"));
|
||||||
|
path
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn validate_correlation_key(correlation_key: &str) -> SdkResult<()> {
|
||||||
|
if correlation_key.len() <= MAX_CORRELATION_KEY_BYTES {
|
||||||
|
Ok(())
|
||||||
|
} else {
|
||||||
|
Err(SdkError::invalid_input("correlation key exceeds 512 bytes"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn encode_key_as_hex(key: &str) -> String {
|
||||||
|
const HEX_DIGITS: &[u8; 16] = b"0123456789abcdef";
|
||||||
|
let mut encoded = String::with_capacity(key.len() * 2);
|
||||||
|
for byte in key.as_bytes() {
|
||||||
|
encoded.push(char::from(HEX_DIGITS[usize::from(*byte >> 4)]));
|
||||||
|
encoded.push(char::from(HEX_DIGITS[usize::from(*byte & 0x0f)]));
|
||||||
|
}
|
||||||
|
encoded
|
||||||
|
}
|
||||||
|
|
||||||
|
fn current_timestamp_ms() -> u64 {
|
||||||
|
SystemTime::now()
|
||||||
|
.duration_since(UNIX_EPOCH)
|
||||||
|
.unwrap_or_default()
|
||||||
|
.as_millis()
|
||||||
|
.min(u128::from(u64::MAX)) as u64
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use std::fs;
|
||||||
|
use uuid::Uuid;
|
||||||
|
|
||||||
|
struct TestDirectory(PathBuf);
|
||||||
|
|
||||||
|
impl TestDirectory {
|
||||||
|
fn new() -> Self {
|
||||||
|
let path = std::env::temp_dir().join(format!("sdk-terminal-scope-{}", Uuid::new_v4()));
|
||||||
|
fs::create_dir(&path).expect("create test directory");
|
||||||
|
Self(path)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Drop for TestDirectory {
|
||||||
|
fn drop(&mut self) {
|
||||||
|
let _ = fs::remove_dir_all(&self.0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn terminal_scope_survives_store_restart() {
|
||||||
|
let root = TestDirectory::new();
|
||||||
|
TerminalScopeStore::new(&root.0)
|
||||||
|
.set("request-1", "session-1")
|
||||||
|
.expect("set terminal scope");
|
||||||
|
|
||||||
|
let loaded = TerminalScopeStore::new(&root.0)
|
||||||
|
.get("request-1")
|
||||||
|
.expect("load terminal scope")
|
||||||
|
.expect("scope exists");
|
||||||
|
assert_eq!(loaded.correlation_key, "request-1");
|
||||||
|
assert_eq!(loaded.session_id, "session-1");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn scope_revisions_increase_and_recovery_never_moves_backward() {
|
||||||
|
let root = TestDirectory::new();
|
||||||
|
let store = TerminalScopeStore::new(&root.0);
|
||||||
|
let first = store.set("request-1", "session-1").unwrap();
|
||||||
|
let second = store.set("request-1", "session-2").unwrap();
|
||||||
|
assert_eq!(second.revision, first.revision + 1);
|
||||||
|
assert!(second.updated_at_ms > first.updated_at_ms);
|
||||||
|
|
||||||
|
store
|
||||||
|
.restore_if_newer("request-1", "session-1", first.revision)
|
||||||
|
.expect("stale scope restoration should be ignored");
|
||||||
|
assert_eq!(
|
||||||
|
store.get("request-1").unwrap().unwrap().session_id,
|
||||||
|
"session-2"
|
||||||
|
);
|
||||||
|
assert!(store
|
||||||
|
.restore_if_newer("request-1", "other-session", second.revision)
|
||||||
|
.is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn encoded_keys_are_isolated_and_cannot_traverse_paths() {
|
||||||
|
let root = TestDirectory::new();
|
||||||
|
let store = TerminalScopeStore::new(&root.0);
|
||||||
|
store.set("same", "session-a").expect("set first scope");
|
||||||
|
store
|
||||||
|
.set("../same", "session-b")
|
||||||
|
.expect("set path-like key");
|
||||||
|
store
|
||||||
|
.set("../sämé", "session-c")
|
||||||
|
.expect("set Unicode path-like key");
|
||||||
|
|
||||||
|
assert_eq!(
|
||||||
|
store
|
||||||
|
.get("same")
|
||||||
|
.expect("get first")
|
||||||
|
.expect("exists")
|
||||||
|
.session_id,
|
||||||
|
"session-a"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
store
|
||||||
|
.get("../same")
|
||||||
|
.expect("get path-like key")
|
||||||
|
.expect("exists")
|
||||||
|
.session_id,
|
||||||
|
"session-b"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
store
|
||||||
|
.get("../sämé")
|
||||||
|
.expect("get Unicode path-like key")
|
||||||
|
.expect("exists")
|
||||||
|
.session_id,
|
||||||
|
"session-c"
|
||||||
|
);
|
||||||
|
assert_eq!(fs::read_dir(&root.0).expect("read root").count(), 3);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn malformed_terminal_scope_json_is_rejected() {
|
||||||
|
let root = TestDirectory::new();
|
||||||
|
let store = TerminalScopeStore::new(&root.0);
|
||||||
|
let path = root
|
||||||
|
.0
|
||||||
|
.join(format!("{}.json", encode_key_as_hex("request")));
|
||||||
|
fs::write(path, "{").expect("write malformed scope");
|
||||||
|
assert!(store.get("request").is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn long_hex_keys_use_safe_path_components() {
|
||||||
|
let root = TestDirectory::new();
|
||||||
|
let store = TerminalScopeStore::new(&root.0);
|
||||||
|
let key = "k".repeat(512);
|
||||||
|
store
|
||||||
|
.set(&key, "session-long")
|
||||||
|
.expect("long key should persist");
|
||||||
|
assert_eq!(
|
||||||
|
store.get(&key).expect("get long key").unwrap().session_id,
|
||||||
|
"session-long"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn oversized_correlation_keys_are_rejected() {
|
||||||
|
let root = TestDirectory::new();
|
||||||
|
let store = TerminalScopeStore::new(&root.0);
|
||||||
|
let key = "k".repeat(513);
|
||||||
|
assert!(store.set(&key, "session-long").is_err());
|
||||||
|
assert!(store.get(&key).is_err());
|
||||||
|
assert!(!root.0.exists() || fs::read_dir(&root.0).unwrap().next().is_none());
|
||||||
|
}
|
||||||
|
}
|
||||||
File diff suppressed because it is too large.
Load diff
Reference in new issue
Block a user