Opens in a new tab
News

LFM2.5’s small active count is only part of the local-agent job

In brief

An engineer’s local assistant may calculate less for each reply while still needing room for the whole model. Follow the tradeoff from loading to a useful answer.

5 min read

Is this the right starting point?
Before you read
Basic model-inference and agent-workflow concepts.
What you will take away
Distinguish active-parameter compute from full-model memory and evaluate a local model against the complete tool task.
Sources
A paper-cut architectural archive keeps many expert chambers available while four colored routes activate for one token.
On this page4 sections

An engineer wants a workstation assistant to read an alert, open the relevant runbook and suggest the next check. The model promises less calculation for each bit of its reply, but the engineer still has to load the whole file. A faster calculation is useful only after the model fits somewhere the machine can reach it.

That is the distinction behind Liquid AI's October 13 LFM2.5-24B-A2B release. Its complete set of learned numerical settings contains 24 billion parameters, while roughly 2.45 billion are active for each token, a piece of input or output. The smaller active part can reduce calculation. The rest still needs storage. We will follow this hypothetical assistant from loading the model to finishing the engineer's investigation.

Follow the token through the selected experts

The model card describes 64 experts in each mixture-of-experts block and top-four routing. A router chooses four sets of learned calculations for a token instead of running all 64. The next token may need another selection.

That saves per-token computation compared with a dense 24B model; it does not create a 2.45B checkpoint. Weight placement in fast memory, system RAM or a slower offload tier affects how quickly each selection becomes available. Quantization changes the space those stored numbers occupy.

The 131,072-token context adds another cost. Prefill processes the input before the response starts, and attention retains state in a key-value cache. Longer input adds work and memory. This reasoning-only model can also produce reasoning tokens before its final answer. Sparse expert routing does not remove those stages.

A diagram separates LFM2.5's 24 billion total parameters from the 2.45 billion parameters active for each token, with a router selecting four experts from a larger resident pool.
Each token uses only selected parts of the model. The unused parts still need storage, ready for a later token that may select them.

Read the performance table as a shortlist

Liquid reports 184 decoded tokens per second on Apple M5 Max and 114 on AMD Ryzen AI Max+ 395 under its tested quantization, runtime and prompt conditions. Those are vendor measurements, not a universal speed guarantee.

For the engineer's assistant, choosing the runbook lookup and supplying the right service name is one job. Repairing the diagnostic's Python code is another. A result about calling tools cannot settle both. The Berkeley Function Calling Leaderboard, or BFCL, evaluates tool calls; Liquid's table reports LFM2.5 and Qwen3.6-35B-A3B essentially level on its BFCLv4 comparison.

The rest of Liquid's table has the same mix of strengths and limits. LFM2.5 leads the compared models on AutomationBench and leads that Qwen model on the ToolSandbox and LiveClaw agent tests. Qwen leads the listed browser-work comparison, WebArena Verified, the knowledge-heavy GPQA Diamond test, and the coding tests SWE-Bench Verified and SciCode. These names identify different published tests, not successive steps our workstation has passed.

Liquid recommends retrieval or web search for knowledge-heavy work. For the engineer, that means checking what happens after the model chooses a lookup: did the right runbook arrive, and did the proposed diagnostic follow it? The tool call is useful because of the work it enables.

A result belongs to the model, harness, settings and conditions that produced it. Use the table to choose candidates, then compare representative work with the relevant tools, hardware and acceptance checks held steady. The benchmark-and-harness guide explains when a complete-setup comparison is the useful one.

Test the workstation from cold load to final answer

In a hypothetical incident-triage task, the model reads a sanitized alert, retrieves a runbook, calls a read-only diagnostic and proposes the next check. A fast warm reply covers only part of that wait.

TestKeep fixedMeasureWhat it reveals
Cold loadCheckpoint format and host stateLoad time, peak memory, page-ins, failuresWhether the full model fits the planned memory path
Warm tool loopHarness, tools, prompt and acceptance rubricTime to first token, dispatch latency, retries, accepted tasksWhether sparse execution improves the actual workflow
Long-context replayDocument set and arrival patternPrefill time, cache growth, throughput, latency percentilesWhether the 131K context is usable at the target objective
Failure recoveryInjected timeout and malformed tool resultRepeated actions, bounded failures, destination stateWhether the agent remains safe when a tool outcome is uncertain

An accepted result chooses the appropriate diagnostic and preserves evidence for its next step. Fluent prose does not repair a wrong tool, repeated action or missing observation. The inference compatibility suite covers protocol behavior separately.

Trace where local work still leaves the machine

Local weights remove one cloud-model call. Retrieval, search, telemetry, package downloads and tools may still send data elsewhere. Follow the task before calling the whole agent private.

Review licensing too. Liquid distributes this model under LFM Open License v1.0. Its license page describes a $10 million annual-revenue commercial-use boundary and the need for larger entities to obtain a separate license. Open weights are not unrestricted enterprise permission.

The engineer also has to choose a model file that the serving program can read. A downloaded file is not yet a running assistant. For example, after choosing a supported GGUF build and compatible engine, first check that it loads, then send the runbook lookup and inspect the returned tool name and arguments. A successful load answers the file-format question; the lookup answers an application question.

The release lists native, FP8, GGUF, ONNX and MLX forms, with documented routes through Transformers, vLLM, SGLang and llama.cpp. Use the documented pairing for the selected file rather than treating those names as interchangeable parts. Record that pair with its versions, then repeat the small lookup after upgrades. Model integrity, resource limits and an unavailable serving endpoint are now maintenance work for the team operating the workstation.

Return to the triage workstation. Can it load the full model, read the supplied evidence and complete the diagnostic loop without unacceptable delay? The smaller active path makes that trial promising. The larger stored model explains why its memory path still belongs in the result.

Source context

Source links appear within this article. Read them alongside the author’s analysis and evaluate the guidance against your environment.

Report an error or outdated detail

Keep reading

Where to go next