I burned all my tokens researching how to save tokens
The author embarked on a quest to tame the runaway costs and unreliable outputs of AI agents, ironically burning through a significant token budget in the process. This deep technical dive details a sophisticated multi-model orchestration strategy that drastically cuts token usage and enhances research quality. It resonates with Hacker News by offering practical, real-world solutions to common pain points in agentic AI workflows, prompting a vigorous debate on the true value and cost-effectiveness of AI tools.
The Lowdown
Bartosz Kotrys recounts his initial struggle with AI agent costs, where a simple deep research task consumed his Claude Max 5x plan in just 30 minutes without yielding results. This expensive lesson spurred him to develop a more efficient, multi-model pipeline for AI research.
Key strategies and findings include:
- Leveraging Existing Subscriptions: Instead of solely relying on one expensive model, he integrated Claude, Codex, and Antigravity, using a shared memory plugin (
claude-mem) to allow all tools to contribute to a common knowledge base without additional subscription costs. - Strategic Model Allocation: A tiered approach assigns specific roles to different LLM models based on their cost and capability. Cheaper models like Claude Sonnet 5 handle initial finding, while more accurate ones like Claude Opus 4.8 verify claims. The most expensive, Claude Fable 5, is reserved for high-level tasks like judging and planning, and Codex (GPT-5.5) and Antigravity (Gemini 3.1 Pro) serve as headless subagents for tool execution and second opinions.
- Cost-Saving Fallback Mechanism: A simple Bash script orchestrates calls to external vendor models. If a subagent hits its usage limit, the system automatically falls back to Claude models, ensuring continuous operation and extending research time tenfold without extra expense.
- Hallucination Reduction: To improve trustworthiness, strict rules were implemented: claims are verified by a different model, all findings require a primary source URL and quote, and numbers must be directly verifiable. These rules were continuously refined based on observed errors.
- Optimized Deep Research: The initial
/deep-researchtool, which was highly token-intensive, is now used as a final, focused step to refine and deepen already verified findings, drastically reducing its token consumption and increasing its utility.
Ultimately, this hybrid approach, combining human oversight with an intelligently orchestrated agent pipeline, resulted in a trustworthy knowledge base. The author concludes that while agents handle heavy lifting, human curation and continuous pipeline improvement are crucial for quality, finding a balance between AI-driven efficiency and human-validated reliability.
The Gossip
Orchestrating Optimal Outputs
This theme highlights the article's core strategy: using multiple AI models strategically to optimize both cost and performance. Commenters laud the effectiveness of routing tasks to cheaper, specialized models, leveraging diverse provider subscriptions, and using intelligent fallback mechanisms. The author's approach of dedicating specific models to roles like 'find,' 'verify,' and 'judge' resonated as a practical solution to manage token burn and enhance output quality.
Halting Hallucinations & Fostering Trust
The persistent problem of AI 'hallucinations' is a central point of discussion. Commenters debate the true efficacy of rule-based systems and multi-model verification in reducing factual errors. While some express skepticism, arguing that hallucinations can't be fully 'fixed' by mere rules, the author clarifies that his method 'significantly reduced' them. The conversation also touches on the meta-challenge of verifying 'primary sources' that might themselves be LLM-generated, creating a feedback loop of potential misinformation.
The Cloud vs. Local AI Conundrum
A lively debate ensues regarding the cost-effectiveness and practicality of running LLMs locally versus relying on cloud providers. Some advocate for a hybrid approach, using local models for routine tasks and cloud models for complex ones, or for privacy reasons, especially for corporate intellectual property. Others contend that for individuals, the high hardware investment for local models often outweighs token savings, while cloud providers offer superior economies of scale and parallel processing, making them more financially viable in most scenarios.
Shipping vs. Slop: The AI Productivity Debate
A significant portion of the comments questions whether AI is genuinely enabling people to 'ship' tangible products or merely generating 'slop' (low-quality, unpolished output). Skeptics demand concrete evidence ('receipts') beyond mere blog posts, while numerous others share specific examples of how AI has dramatically accelerated their development of features, internal tools, migrations, and personal projects. This theme highlights the ongoing tension between AI's perceived potential and the often-debated reality of its contribution to production-ready value.
Contextual Complications & Cache Killers
This discussion delves into the subtle yet significant costs associated with context management and caching within LLM interactions. Commenters highlight that seemingly beneficial strategies, such as context compaction or dynamically altering prompt prefixes, can paradoxically increase token usage by invalidating crucial caches. The author's finding that a mid-session tool change re-bills the entire cached prefix at full price resonates, leading to advice to either minimize dynamic elements or trust smarter models to manage context and summarization efficiently.