Observability: FailoverClient.get_stats() exposes aggregate counters
(requests / failovers / cooldown_skips / exhausted, snapshot copy);
new on_failover(from_name, to_name, reason) callback fires on every
failover decision — to_name=None marks chain exhaustion — for future
admin-panel/frontend wiring, wrapped so callback exceptions never
affect the request path.
Timeout budget: profile field "failover_timeout_budget" caps the total
seconds spent walking the chain (default unlimited; values <= 0 are
rejected to None with a warning). Once elapsed, remaining profiles are
skipped — prevents worst-case N x timeout stacking on longer chains.
Exhaustion is counted once per call via a single end-of-loop increment
(an initial double count at the budget branch was caught by the new
budget test and folded into the unified counter).
Retry-After: TransientLLMError now carries retry_after parsed from the
numeric Retry-After response header (non-numeric values like HTTP-dates
are ignored); FailoverClient._mark_fail uses it to extend that
profile's cooldown beyond the default 60s, respecting server-side
throttling on 429s. _fail_at entries became (fail_time, cooldown)
tuples; three existing tests that poke the internal map were updated.
Config: create_llm_client passes profile.failover_timeout_budget into
the FailoverClient constructor.
Tests: new TestL4cObservabilityAndBudget (13 cases: stats on
success/exhaustion/cooldown-skip, snapshot copy, hook called with
chain-tail None, hook exception safety, budget stops chain via mocked
monotonic, invalid budget ignored, config field plumbing, default
no-budget regression, Retry-After parse/ignore/extend-cooldown). Full
suite 355 passed.
FailoverClient.stream_chat now walks the chain but only fails over
while no event has been produced yet — replay before the first
reasoning/content/tool_call event is invisible to the user. Once any
event is out, failures propagate directly (replay would duplicate
frontend output and corrupt tool-call accumulation).
DeepSeekClient.stream_chat classifies non-200 responses: 429/5xx raise
TransientLLMError (failover-eligible), auth/param 4xx keep raising
RuntimeError. Chain exhaustion re-raises the last exception so the
existing rollback/error handling in think_and_respond catches it; an
all-cooling chain raises RuntimeError.
Cancel-flag guard: LLMClient.cancel() sets _cancelled because the
RequestException raised by force-closing the connection is
indistinguishable from a real network fault. FailoverClient.cancel()
is now defined explicitly (overriding __getattr__ delegation),
broadcasts to every client in the chain and sets the wrapper flag;
stream_chat/chat_completion reset it per call and suppress failover
when it is set — otherwise interrupting would make the system hop to
another vendor and keep generating against the user's intent.
Tests: new TestL4bStreamFailover (12 cases: failover before first
event on TransientLLMError/ConnectionError, no failover after first
event, cancel never fails over on both paths, cancel broadcast, chain
exhaustion, all-cooling, cooldown skip, success isolation, transient/
auth status classification). Full suite 342 passed.
When a profile declares "fallback": ["kimi", "local"], create_llm_client
now returns a FailoverClient wrapper holding the ordered chain. For the
idempotent non-streaming path (chat_completion — conversation
compression, light subagents):
- Fail over on transient errors only: HTTP 429/5xx (now raised as
TransientLLMError by DeepSeekClient) and connection errors/timeouts
(previously swallowed into None by chat_completion's catch-all, now
re-raised). Auth/param 4xx failures keep returning None with no
failover — config problems stay loud, consistent with the H1/H2
error-early philosophy.
- Failed profiles enter a 60s in-process cooldown and are skipped
until it expires; chain exhaustion returns None per the existing
failure contract.
- Config structure errors fail at construction (non-list fallback,
unknown profile name, self-reference — H2-style). Environmental
problems (e.g. fallback profile missing its API key) skip that
fallback with a warning instead of blocking the primary.
- Unspecified attributes delegate to the primary client via
__getattr__ (name/model/api_key/cancel), so DeepSeekChat and
set_client keep working unchanged. stream_chat still delegates to
the primary directly; first-token failover is L4b.
create_llm_client refactored: single-profile construction extracted
into _construct_profile_client, shared by the main path and the
fallback chain (flat-api path untouched).
Tests: new TestL4aChatCompletionFailover (15 cases: chain build,
validation, 429/5xx/connection failover, 401 no-failover, exhaustion,
cooldown skip/expiry, missing-key skip with warning, transient raise,
4xx None regression, stream delegation); full suite 330 passed.
L1 display name: client.name now resolves to the profile's
display_name field, falling back to the profile key — the dialogue
context no longer labels Kimi/Qwen sessions "DeepSeek". The legacy
flat api section keeps "DeepSeek" (zero breakage).
_resolve_model_profile now returns (profile, model, profile_name).
L2 prefix escape: "model:NAME" forces bare-model semantics even when
a profile happens to be named NAME; "profile:NAME" forces profile
semantics and raises a clear error (listing available profiles) when
missing, instead of silently falling back to bare-name behavior.
Empty prefixes are rejected. Wired through --model / create_llm_client
/ subagent model params.
L3 profile temperature: new profile field "temperature" becomes the
client's default_temperature. chat_completion/stream_chat treat
temperature=None as "use profile default, else the old hardcoded
0.3/0.7", so explicit caller values still win. DeepSeekChat and
run_light_subagent resolve CLI > profile > constants.DEFAULT_TEMPERATURE.
L5 reasoning field aliases: _parse_delta recognizes
reasoning_content / reasoning / thinking / thinking_content (first
non-empty wins), so non-DeepSeek vendors' thinking streams are no
longer silently dropped.
L4 (cross-profile failover) intentionally deferred: streaming
failover needs request-replay semantics and is feature-level work.
Tests: new TestLBatch (16 cases); 7 existing unpack sites updated to
the 3-tuple. Full suite 315 passed.
Profile-level reasoning_enabled was attached as an instance attribute
after construction only when a profile matched; the base class never
declared it and consumers probed with double getattr. Any code that
accidentally set client.reasoning_enabled would silently override the
global switch.
The flag is now an explicit LLMClient constructor parameter with
three-state semantics: True/False (profile-level explicit, or the
global api.reasoning_enabled value concretized when the profile omits
the field) and None (no profile -> consumer falls back to global).
create_llm_client passes it at construction and no longer mutates the
client afterwards; DeepSeekChat._reasoning_enabled simplifies to a
single getattr probe that still tolerates clients without the attr.
Tests: TestM3ReasoningConstructor (4 cases) covering profile True,
profile-omitted -> global concretized, base default None, and the
None -> global fallback at the consumer; one legacy assertion updated
to the new contract (attribute exists, value None). Full suite 299
passed.
Profiles previously had no validation, so config errors surfaced far
from their origin and cryptically:
- explicit profile missing `model` fell back to the profile NAME as the
model (a profile named "local" would send "model": "local")
- default profile missing `model` with no model param produced
"model": null -> API 400
- missing `api_url` surfaced as requests InvalidURL
_resolve_model_profile now runs _validate_profile (dict type + api_url)
before reading fields and _require_model on the effective model,
raising ValueErrors that name the profile, the missing field, and the
config.json path to fix. Bare model names still legally substitute for
a default profile's missing model field.
Tests: new TestH2SchemaValidation (6 cases: missing model/api_url,
null-model via default profile, bare-name substitution, non-dict
profile, e2e create_llm_client); full suite 282 passed.
~/.cert/deepseek.key is a DeepSeek-only credential; when a models profile
specified its own api_key_env (e.g. MOONSHOT_API_KEY) but the env var was
unset, resolve_api_key silently read that file and sent the DeepSeek key
as Bearer to a third-party endpoint (cross-vendor credential leak).
resolve_api_key gains allow_key_file (default True for the legacy flat
api section); create_llm_client passes allow_key_file=profile is None so
any profile path raises a clear RuntimeError instead, pointing users to
set the env var or use "" for local keyless endpoints.
Tests: new TestH1KeyIsolation (4 cases) with a mocked readable
deepseek.key proving the profile path errors out while the flat api
section still reads the file; full suite 276 passed.
qing