PulsarLM keeps context small by dropping old token pages

PulsarLM takes a different approach to a long conversation: instead of keeping every old token in the model’s active context, it removes pages that receive little attention. The session can continue with a smaller working record, but something the agent learned earlier may no longer be present when a later decision needs it.

The experimental inference engine appeared in a LocalLLaMA discussion reviewed on October 15. Its primary repository describes page scoring, selective eviction and a snapshot cache. This is an explanation of that documented design, not a benchmark or an assertion that the project was first released today.

What leaves the working context

In a transformer, the attention mechanism lets the current computation use information from earlier token positions. PulsarLM’s documented custom kernels accumulate attention scores for groups of tokens, called pages. Periodically, low-scoring pages leave the active context; the system prompt and recent pages are retained. Rather than writing a prose summary of older exchanges, this changes which token pages remain available during inference.

That difference matters when comparing it with conversation summarization. A summary can lose a condition by paraphrasing it badly. An eviction system can lose access to the condition because the page containing it is no longer retained. Neither mechanism should be evaluated only by the size of the resulting context.

Low attention is evidence about how the current computation is using a page. It is not an independent determination that the page will never matter again. A resource name mentioned once can remain consequential even while a long series of unrelated log queries attracts most of the subsequent attention. The useful question is what happens when that earlier fact becomes relevant again.

Follow one old constraint into a later task

Consider a hypothetical read-only assistant investigating storage failures. Early in the conversation, the responder explains that archive-b contains the only remaining incident capture and must be retained until its export is verified. The investigation then produces many turns of fresh measurements. Much later, someone asks which old captures could be removed to free capacity.

The old archive condition now changes the right answer. If it is still in context, the assistant can explain why archive-b is excluded. If its page has been evicted, the harness needs a reliable way to retrieve the maintained retention record before making that recommendation. Continuing the conversation without an error does not tell the evaluator which of those paths occurred.

Evaluate a shortened incident record against the constraints that must survive, including rejected explanations and protected resources, before counting the tokens removed. In this example, that means inspecting the proposed cleanup set and its supporting evidence. A fluent account of recent storage errors would not compensate for including the protected capture.

The permission boundary remains outside the conversation. This assistant has no deletion tool, and any later executor should independently enforce the archive’s retention state. That prevents a mistaken recommendation from becoming permission to remove the only copy. It also lets the evaluation distinguish a memory failure from a control that correctly refused the resulting action.

Snapshots solve a different problem from durable recall

The repository explains that removing earlier tokens disrupts ordinary prefix reuse, so it keeps context snapshots from which a conversation can resume. Its documented default state cache is pinned host memory, is bounded, and is not persisted. A snapshot of the working context should therefore not be treated as a durable archive of everything the agent once saw.

For the storage investigation, keep the authoritative capture-retention record and any permitted evidence outside that volatile state. Then a replay can ask whether retrieval brings the relevant condition back when cleanup becomes the task. Also test what the harness does when the record cannot be retrieved: it should leave that capture’s disposition unresolved rather than infer that an absent restriction means deletion is allowed.

These are proposed evaluation cases, not results from running PulsarLM. Compare the same investigation with ordinary full-context handling and with eviction plus the intended retrieval mechanism. Record which conditions were available, which were retrieved, and which recommendations changed. Preserve all attempted cases, including a retrieval failure. The Active Memory evaluation article explores related retrieval questions, while this trial isolates the consequence of removing token pages at inference time.

Where the experiment fits

PulsarLM is interesting when a long-running local session is expensive to keep active and the application can maintain its important records separately. The repository currently describes limited model coverage and a narrow tested hardware scope. Check its current requirements and implemented models before planning a trial; a package existing for a platform is not evidence that the project tested that platform.

We have not installed the engine or independently reproduced its throughput figures. Those project measurements also do not answer whether the archive condition survives this workload. A short, disposable replay with synthetic records is more informative for that decision than carrying an unrelated tokens-per-second result into a production recommendation.

If the workflow cannot recover a decision-critical condition after eviction, keep that class of work on an approach that preserves the needed evidence while the retrieval path is repaired. If the task is a disposable chat with no consequential action or retained operational state, the cost of forgetting can be much lower. The application’s obligation determines how demanding the test should be.

For the hypothetical investigator, success is a smaller active context that still leads to an evidence-backed exclusion of archive-b, or an explicit unresolved answer when that evidence is unavailable. That is a useful result a page-eviction trial can establish. An indefinitely continuing session, by itself, cannot.