What Basalt changes in a Blackwell local inference stack
Basalt narrows local inference to Qwen3.8-Flash-Next on Blackwell. Its memory and concurrency controls matter more than copying the largest throughput number.
AI in production. Reliability in practice.
Latest edition ·The latest analysis
Run a candidate log classifier beside the current route, preserve complete outcomes, and promote only the slices whose mistakes and review cost are understood.
Read the storyFrom the publication
Basalt narrows local inference to Qwen3.8-Flash-Next on Blackwell. Its memory and concurrency controls matter more than copying the largest throughput number.
A host that launches codex exec must own the process, thread lease, JSONL trace, cancellation and artifact checks. Resume is a lifecycle decision, not a shell shortcut.
A client-side SSE probe separates first answer text from normal completion. Fixed arrivals and outcome counts show whether a serving change delivers more useful answers.
Infermeld makes a mixed-GPU llama.cpp launch inspectable. Its recorded failures show why separate device allocations and completed workload checks matter more than adding VRAM capacities.
Field guides
Follow a request. Review the code. Find out what actually happened.
Explore the guidesKeep the traces that explain a failure
Tail sampling selects recorded traces by error, latency and other evidence. A checkout timeout shows how routing, waiting time and memory affect the trace a responder can retrieve.
Give AI code review a useful job
CodeRabbit brings AI feedback into pull requests, editors and the CLI. A retry loop that exceeds its caller’s deadline shows the kind of finding that can repay review time.
Recover when a tool call times out
A missing response can leave a worker already created. Durable operation IDs and the destination’s idempotency contract let a replacement executor recover the original request.
Put it into practice
Free reliability tools. No account needed.
Open the workbenchThe AI tools guide
Coding assistants, code review and agent supervision.
Browse the guideCompanies & products