Strategies compared¶

convopack ships five built-in strategies. This notebook runs each one on the same history and shows how they differ.

Strategy	What it does	When to use
Recency	Keep newest chunks that fit.	Default chat assistant.
FirstFit	Keep oldest chunks that fit.	System prompt + few-shots dominate.
SummaryEvict	Drop oldest, insert a summary.	Long sessions, semantic recall needed.
Importance	Score per message; drop lowest.	App-specific signals (starred, admin, etc.).
SemanticDedup	Drop near-duplicates first.	Repetitive agents.

In [ ]:

from convopack import (
    FirstFit,
    Importance,
    Message,
    Packer,
    Recency,
    SemanticDedup,
    SummaryEvict,
    TextBlock,
)

from convopack import ( FirstFit, Importance, Message, Packer, Recency, SemanticDedup, SummaryEvict, TextBlock, )

1. A shared history¶

Twelve turns. Some are early-stable (system + first user question), some are repetitive, some are recent.

In [ ]:

history = [
    Message(role="system", content=[TextBlock(text="Be concise and helpful.")]),
    Message(role="user", content=[TextBlock(text="What is convopack?")]),
    Message(role="assistant", content=[TextBlock(text="A context-window packer for LLMs.")]),
    Message(role="user", content=[TextBlock(text="random middle question 1 - filler text " * 4)]),
    Message(role="assistant", content=[TextBlock(text="middle answer 1 - filler text " * 4)]),
    Message(role="user", content=[TextBlock(text="random middle question 1 - filler text " * 4)]),  # near-duplicate of #4
    Message(role="assistant", content=[TextBlock(text="middle answer 1 - filler text " * 4)]),  # near-duplicate of #5
    Message(role="user", content=[TextBlock(text="random middle question 2 - filler text " * 4)]),
    Message(role="assistant", content=[TextBlock(text="middle answer 2 - filler text " * 4)]),
    Message(role="user", content=[TextBlock(text="latest user question - what just happened?")]),
]
len(history)

history = [ Message(role="system", content=[TextBlock(text="Be concise and helpful.")]), Message(role="user", content=[TextBlock(text="What is convopack?")]), Message(role="assistant", content=[TextBlock(text="A context-window packer for LLMs.")]), Message(role="user", content=[TextBlock(text="random middle question 1 - filler text " * 4)]), Message(role="assistant", content=[TextBlock(text="middle answer 1 - filler text " * 4)]), Message(role="user", content=[TextBlock(text="random middle question 1 - filler text " * 4)]), # near-duplicate of #4 Message(role="assistant", content=[TextBlock(text="middle answer 1 - filler text " * 4)]), # near-duplicate of #5 Message(role="user", content=[TextBlock(text="random middle question 2 - filler text " * 4)]), Message(role="assistant", content=[TextBlock(text="middle answer 2 - filler text " * 4)]), Message(role="user", content=[TextBlock(text="latest user question - what just happened?")]), ] len(history)

2. Run each strategy under the same budget¶

We tighten the budget so eviction is forced and the strategies make visibly different choices.

In [ ]:

BUDGET = 80

def run(name, strategy, **packer_kwargs):
    p = Packer(budget=BUDGET, tokenizer="approx", strategy=strategy, **packer_kwargs)
    r = p.pack(history)
    print(f"=== {name} ===")
    print(f"kept {len(r.kept)} of {len(history)}, tokens {r.token_count}/{r.budget}")
    for m in r.kept:
        print(f"  [{m.role:>10}] {m.text()[:50]}")
    print()

run("Recency", Recency(), pin=("system", "last_user"))
run("FirstFit", FirstFit(), pin=("system",))

BUDGET = 80 def run(name, strategy, **packer_kwargs): p = Packer(budget=BUDGET, tokenizer="approx", strategy=strategy, **packer_kwargs) r = p.pack(history) print(f"=== {name} ===") print(f"kept {len(r.kept)} of {len(history)}, tokens {r.token_count}/{r.budget}") for m in r.kept: print(f" [{m.role:>10}] {m.text()[:50]}") print() run("Recency", Recency(), pin=("system", "last_user")) run("FirstFit", FirstFit(), pin=("system",))

Recency keeps system + the latest user question + as much tail as fits. FirstFit keeps system + the early turns, dropping the tail.

3. SummaryEvict¶

A real summariser would call an LLM. Here we use a deterministic fake so the notebook stays runnable offline.

In [ ]:

def fake_summary(messages):
    return f"({len(messages)} earlier turns about convopack, filler text, and middle questions)"

run("SummaryEvict", SummaryEvict(fake_summary), pin=("system", "last_user"))

def fake_summary(messages): return f"({len(messages)} earlier turns about convopack, filler text, and middle questions)" run("SummaryEvict", SummaryEvict(fake_summary), pin=("system", "last_user"))

Notice the synthetic [summary] (...) message inserted at the head of the kept list. Older turns are dropped but their content is summarised in-place.

4. Importance with a custom scorer¶

Application-specific signal: a starred message must always survive.

In [ ]:

# Star one of the middle turns to demonstrate the override.
history[3].metadata["starred"] = True

def scorer(msg):
    if msg.metadata.get("starred"):
        return 1000.0
    if msg.role == "system":
        return 100.0
    if msg.role == "user":
        return 10.0
    return 1.0

run("Importance", Importance(scorer=scorer), pin=("system",))

# Star one of the middle turns to demonstrate the override. history[3].metadata["starred"] = True def scorer(msg): if msg.metadata.get("starred"): return 1000.0 if msg.role == "system": return 100.0 if msg.role == "user": return 10.0 return 1.0 run("Importance", Importance(scorer=scorer), pin=("system",))

5. SemanticDedup¶

Drop near-duplicate turns first, then defer to a fallback (default Recency). We use a deterministic char-bigram embedder here so the notebook runs without API keys.

In [ ]:

from collections import Counter


class BigramEmbedder:
    """Tiny deterministic embedder over [a-z0-9 ]."""
    VOCAB = "abcdefghijklmnopqrstuvwxyz0123456789 "
    def embed(self, text):
        text = text.lower()
        counts = Counter(text[i:i+2] for i in range(len(text)-1))
        vec = [float(counts.get(a+b, 0)) for a in self.VOCAB for b in self.VOCAB]
        norm = sum(x*x for x in vec) ** 0.5 or 1.0
        return [x/norm for x in vec]
    def embed_batch(self, texts):
        return [self.embed(t) for t in texts]

run("SemanticDedup", SemanticDedup(BigramEmbedder(), threshold=0.95), pin=("system", "last_user"))

from collections import Counter class BigramEmbedder: """Tiny deterministic embedder over [a-z0-9 ].""" VOCAB = "abcdefghijklmnopqrstuvwxyz0123456789 " def embed(self, text): text = text.lower() counts = Counter(text[i:i+2] for i in range(len(text)-1)) vec = [float(counts.get(a+b, 0)) for a in self.VOCAB for b in self.VOCAB] norm = sum(x*x for x in vec) ** 0.5 or 1.0 return [x/norm for x in vec] def embed_batch(self, texts): return [self.embed(t) for t in texts] run("SemanticDedup", SemanticDedup(BigramEmbedder(), threshold=0.95), pin=("system", "last_user"))

The two near-duplicate middle exchanges collapse into one (the first occurrence survives, the second is dropped). After that, the fallback Recency handles the remaining budget.

Picking a strategy¶

• Default to Recency. It's cheap, predictable, and the right answer most of the time.
• Switch to SummaryEvict when silent eviction would harm UX (e.g. long agents that need to remember earlier instructions).
• Use Importance when you have application-level signals (starred, admin, web-search results).
• Stack SemanticDedup in front of any of these when an agent or user is repeating itself.
• FirstFit is a niche tool: useful when the early system prompt + few-shots are doing the heavy lifting.