[verified] fix: preserve tracker precision eligibility
This commit is contained in:
27
.scratch/proxy-stream-cancellation/PRD.md
Normal file
27
.scratch/proxy-stream-cancellation/PRD.md
Normal file
@@ -0,0 +1,27 @@
|
||||
# PRD: Streaming proxy cancellation delivery
|
||||
|
||||
## Overview
|
||||
|
||||
A tracker proxy route can be selected and registered as active, but a direct SSE upstream may not deliver its first frame to the downstream client until the upstream response completes. A dashboard cancellation request made after the upstream emitted and flushed an SSE frame can therefore arrive after the tracker has already finalized and unregistered the proxy, returning 404.
|
||||
|
||||
## Goal
|
||||
|
||||
Preserve immediate SSE frame delivery and keep the active-proxy record cancelable until the streaming response actually completes or is canceled.
|
||||
|
||||
## Quality gates
|
||||
|
||||
- Run the focused tracker streaming/cancellation tests.
|
||||
- Run full `python -m pytest -q` and record unrelated failures exactly.
|
||||
- Do not weaken billing, disconnect handling, proxy-inflight accounting, or cancel authorization.
|
||||
|
||||
## User story
|
||||
|
||||
### PSC-001: Cancel an active direct SSE proxy
|
||||
|
||||
As a dashboard operator, I need a cancellation request to reach an active direct SSE proxy after its first streamed frame, so I can stop long-running inference without a race-induced 404.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Do not change route selection or quantization policy.
|
||||
- Do not alter relay transport behavior unless needed to share the same cancellation invariant.
|
||||
- Do not introduce client-visible buffering.
|
||||
@@ -0,0 +1,34 @@
|
||||
# PSC-001 — Direct SSE cancellation race
|
||||
|
||||
Status: ready-for-agent
|
||||
Priority: High
|
||||
|
||||
## Problem
|
||||
|
||||
`tests/test_tracker_routing.py::test_tracker_dashboard_can_cancel_inflight_proxy` registers a streaming upstream node. The upstream writes and flushes the first SSE frame, then waits three seconds. The tracker’s client does not receive that frame until after the upstream has completed; by then `_unregister_active_proxy()` has run and the dashboard cancellation endpoint returns 404.
|
||||
|
||||
Observed trace:
|
||||
|
||||
```text
|
||||
proxy route selected
|
||||
proxy connected
|
||||
proxy progress ... elapsed_seconds≈3
|
||||
proxy complete ... elapsed_seconds≈3
|
||||
POST /v1/proxy/requests/<id>/cancel → 404
|
||||
```
|
||||
|
||||
This is a production cancellation/delivery race, not a stale test: the endpoint promises to cancel active proxy work, and the upstream had already emitted a first stream frame before cancellation was attempted.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] A direct SSE upstream frame is relayed and flushed to the client before the upstream completes.
|
||||
- [ ] After that first frame, `/v1/proxy/requests/{request_id}/cancel` returns 200 while the stream is active.
|
||||
- [ ] Cancellation closes/stops the upstream safely, finalizes inflight accounting exactly once, and records the cancellation event.
|
||||
- [ ] Cancel authorization remains unchanged.
|
||||
- [ ] Client disconnect and normal SSE completion retain current billing/throughput behavior.
|
||||
- [ ] Regression test is deterministic and does not rely on timing races longer than necessary.
|
||||
- [ ] Focused tracker routing tests and full pytest are run with unrelated failures documented.
|
||||
|
||||
## Likely seam
|
||||
|
||||
Inspect direct streaming behavior around `_handle_proxy_chat` upstream reads (`packages/tracker/meshnet_tracker/server.py`, roughly lines 3953–4019). The direct response path writes/flushed each line, but current HTTP response buffering/reading prevents the line from being observed until stream end. Fix delivery at the proxy transport boundary; do not paper over it by retaining completed proxy records indefinitely.
|
||||
27
.scratch/routing-compatibility-regression/PRD.md
Normal file
27
.scratch/routing-compatibility-regression/PRD.md
Normal file
@@ -0,0 +1,27 @@
|
||||
# PRD: Legacy routing compatibility regression
|
||||
|
||||
## Overview
|
||||
|
||||
The tracker’s new request-precision gate rejects legacy node registrations that omit `quantization`. This conflicts with the explicit `capability_policy=compat` rollout: an older node without a capability report remains routable, but an older node without a quantization field is silently excluded from proxy, pinned-route, benchmark, billing, and latency paths.
|
||||
|
||||
## Goal
|
||||
|
||||
Restore backward-compatible routing for legacy registrations while preserving fail-closed behavior for explicitly declared invalid/unsupported quantization values.
|
||||
|
||||
## Quality gates
|
||||
|
||||
- Run targeted routing, billing, benchmark, pricing, validation, and latency tests.
|
||||
- Run `python -m pytest -q` before completion and record any unrelated failures exactly.
|
||||
- Preserve the admission-policy invariant: invalid capability reports remain non-routable; absent reports route only under `compat`.
|
||||
|
||||
## User story
|
||||
|
||||
### RCR-001: Legacy registration precision fallback
|
||||
|
||||
As an operator upgrading a mixed fleet, I need nodes that predate the `quantization` registration field to serve default-precision requests under `compat`, so the tracker rollout does not make otherwise healthy legacy nodes dark.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Do not weaken `enforce` capability policy.
|
||||
- Do not treat explicitly malformed or unsupported quantization values as valid.
|
||||
- Do not change requested-precision semantics for nodes that declare a supported precision.
|
||||
@@ -0,0 +1,38 @@
|
||||
# RCR-001 — Legacy registration precision fallback
|
||||
|
||||
Status: ready-for-agent
|
||||
Priority: Critical
|
||||
|
||||
## Problem
|
||||
|
||||
The tracker now filters proxy and pinned-route candidates through `_quantization_satisfies(node.quantization, requested_quantization)`. A registration produced before the protocol added singular `quantization` is normalized to `quantizations=["bfloat16"]`, but leaves `node.quantization is None`; routing then fails every request, including the default `bfloat16` request.
|
||||
|
||||
The defect is therefore at protocol normalization: registration correctly records an available plural capability but never chooses a compatible active precision for the legacy entry.
|
||||
|
||||
This is inconsistent with the explicit capability admission rollout policy: an absent capability report is routable under `compat`. Logs show `capability=absent routable=True`, while `/v1/chat/completions` returns 503 or a pinned route returns 409 because the separate precision gate excludes the same legacy node.
|
||||
|
||||
## Evidence
|
||||
|
||||
Representative tight repros fail on current `master`:
|
||||
|
||||
- `tests/test_billing_ledger.py::test_proxy_chat_bills_credited_client_and_credits_node` → HTTP 503
|
||||
- `tests/test_tracker_routing.py::test_tracker_proxy_accepts_hf_model_alias_from_quickstart` → HTTP 503
|
||||
- `tests/test_manual_route_benchmark.py::test_pinned_route_uses_named_node` → HTTP 409
|
||||
- `tests/test_model_speed_latency.py::test_tracker_records_increasing_hop_latency_for_model_and_hardware` → HTTP 409 for every parameter
|
||||
|
||||
Each fixture registers a complete, healthy legacy node without `quantization`; tracker registration logs show it as routable under `compat`.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Legacy nodes omitting the `quantization` field are treated as the historical default precision for compatibility purposes.
|
||||
- [ ] Explicit `null`, invalid, or unsupported declared precision never receives that fallback and remains excluded from routing, assignment, and coverage calculations, including after managed shard assignment/rebalancing.
|
||||
- [ ] Raft replication preserves singular `quantization` and plural `quantizations`; follower routing has identical precision eligibility to the leader.
|
||||
- [ ] `capability_policy=enforce` still excludes absent capability reports regardless of precision fallback.
|
||||
- [ ] Automatic proxy routing, pinned routes, benchmark routes, and route-stat sampling use the same compatibility rule.
|
||||
- [ ] Add behavior-level tracker HTTP tests for legacy omission and explicit invalid declaration.
|
||||
- [ ] Targeted billing, routing, benchmark, model-speed, pricing, forfeiture, and TOPLOC dispatch tests pass.
|
||||
- [ ] Run the full suite and document any failures not caused by this issue.
|
||||
|
||||
## Implementation notes
|
||||
|
||||
Keep the compatibility decision at the protocol-normalization boundary, not scattered into individual route paths. A normalized legacy default can preserve existing `_quantization_satisfies` semantics while differentiating an omitted field from an explicitly invalid value.
|
||||
29
.scratch/routing-compatibility-regression/prd.json
Normal file
29
.scratch/routing-compatibility-regression/prd.json
Normal file
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"name": "Legacy routing compatibility regression",
|
||||
"description": "Restore compat-policy routing for legacy node registrations that omit quantization without weakening validation for invalid declared values.",
|
||||
"branchName": "ralph/routing-compatibility-regression",
|
||||
"userStories": [
|
||||
{
|
||||
"id": "RCR-001",
|
||||
"title": "Legacy registration precision fallback",
|
||||
"description": "As an operator upgrading a mixed fleet, I need nodes that predate the quantization registration field to serve default-precision requests under compat, so otherwise healthy legacy nodes do not become dark.",
|
||||
"acceptanceCriteria": [
|
||||
"A legacy registration without quantization is eligible for a bfloat16 request when capability policy is compat.",
|
||||
"The same registration is excluded under capability policy enforce because its capability report is absent.",
|
||||
"A node that explicitly declares null, an invalid, or an unsupported quantization remains excluded from routing, assignment, and coverage calculations, including after managed shard assignment/rebalancing.",
|
||||
"Raft-applied registrations preserve both singular quantization and plural quantizations so follower routing matches leader routing.",
|
||||
"Automatic proxy routing, pinned routes, manual benchmarks, billing proxy paths, and route-latency sampling retain their existing behavior for legacy registrations.",
|
||||
"Tests cover the legacy/no-field case and explicit-invalid-field case at the tracker HTTP seam.",
|
||||
"Targeted routing, billing, benchmark, pricing, validation, and latency tests pass.",
|
||||
"python -m pytest -q is run and unrelated failures are recorded exactly."
|
||||
],
|
||||
"priority": 1,
|
||||
"passes": true,
|
||||
"dependsOn": [],
|
||||
"completionNotes": "Completed by agent"
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"updatedAt": "2026-07-13T07:25:08.460Z"
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user