narratoflow¶
Compress huge LLM input context into dense intermediate representations. Pay fewer tokens, keep the meaning.
narratoflow (PyPI name; import as narrato) is an open-source Python library (Apache-2.0) for shrinking long source text before sending it to an LLM. It targets any workload where the input dwarfs the output and tokens dominate cost — RAG retrieval contexts, narrative generation, transcript summarisation, long-document QA.
The library has a generic, language- and domain-neutral core. Common starting points ship as named profiles (rag-en, narrative-no, news-en, …) so you do not have to choose every argument up front. See Profiles.
Why¶
LLM input is priced per token, and a long source document — say a 20-page Norwegian transcript that feeds a 200-word narrative — burns most of the budget before the model has written anything.
narratoflow trades a tiny bit of fidelity for a large reduction in input tokens by passing your downstream LLM a dense, machine-friendly representation instead of the raw text.
The intermediate representation does not need to be human-readable. It just needs to be:
- Cheap to produce.
- Decodable by the downstream LLM into a faithful output.
- Smaller in tokens than the original.
Measured¶
On the bundled Norwegian short-story benchmark (~500 words, gpt-4o-mini extractor → gpt-4o target):
| metric | value |
|---|---|
| input tokens | 693 |
| compressed tokens | 392 |
| token reduction | 43% |
| cost savings | 13% |
| LLM-judge quality | 8/10 |
Cost savings grow sharply with input size — on a 10k-token source, expect ~74% savings.
Install¶
Then: