Talk by Rajkumar Sakthivel (Tesco) on reducing AI coding token usage by roughly 94% using a local code index. The team found their AI coding tools were sending ~45,000 tokens of context per query when only ~5,000 were useful, so they built a local retrieval layer that structures code into AST-aware chunks (tree-sitter), combines vector search with keyword matching, and uses a lightweight relationship layer to follow execution across files. Everything runs locally with no data leaving the machine, and one index supports multiple AI tools. The talk covers initial mistakes, why context matters more than model tuning, the system architecture, and real benchmarks and trade-offs — including the hardest problem of detecting when retrieval is wrong, where a simple heuristic outperformed LLM-based scoring and threshold tuning. # Channel Identity Observations: "We Cut 94% of AI Coding Tokens With a Local Code Index" ## Explicit Self-Description Evidence - **Speaker identifies himself by name only**: > "Hey, I'm Raj." — later referenced as "Rajkumar Sakthivel, Tesco" in the title, though the transcript itself contains no Tesco affiliation statement. - **Collaborator named**: > "Me and my friend Foss, we are building project together." Foss is mentioned multiple times as co-builder (> "Here is the hard part Foss and I spend most of the time on"; > "something Foss pointed out"). - **Talk format**: The video is a conference/meetup talk — "In this talk, is about what we built and what we learned" and references to slides ("This is the most important slide," "The command is on screen"). ## Intended Audience - **Developers using AI coding tools daily**, specifically those paying for token costs: - > "We are using AI coding tools everyday. Cloud code, cursor, co-pilot, code X, normal stuff." - > "One month our AI bill was fine. Next month, huge." - **Cost-conscious engineering teams**: the framing repeatedly centers on token spend and savings ("12.4 million tokens saved. Nearly 186 not spent"). ## Claimed Scope of the Content - **Practical engineering solution, not theory**: the talk claims to present a working tool: - > "we started to building something to fix it. In this talk, is about what we built and what we learned." - **Quantified, reproducible claims**: - > "Without using our tool, 83 uh K tokens per questions. With our tools, uh 4.9 K tokens per questions. That is 94% less." - > "These numbers are real. Test is public. You can run it yourself." - **Open-source offering**: > "It's free, open source. You will see the code on screen. Try it, see the number, tell us what you saved." (Tool referred to as "CCE.") ## Demonstrated Presentation Patterns (Observable) - **Transparency about limitations**: the speaker voluntarily discloses weaknesses: - > "I want to be honest about the limits... Real savings are lower than 94%." - > "We tested on large projects with 396 files. The recall dropped almost zero." - **Simple analogies and mental models**: pizza analogy for wasted context (> "It's like ordering a pizza and paying for extra nine pizzas you don't eat"); cost breakdown framing (> "90% of your AI cost is input... Only 10% is output"). - **Story-driven structure**: opens with a personal cost-shock narrative ("We panicked") before the technical content. - **Iterative problem-solving narrative**: three failed/successful attempts (short prompts, model settings, output compression) presented before the solution. - **Value statement**: > "The answer was not a better model. The answer was sending less." and > "simple formula beats the complex model most of the time." ## Claims vs. Demonstrated Distinction - **Creator claims**: 94% token reduction, 90% accuracy, 0.4ms scoring, public test — these are asserted with cited methodology but not verifiable within the video itself. - **Demonstrated**: methodical structure, honest limitation disclosure, and audience-oriented cost framing are observable presentation behaviors, not claims. # Delivery Observation — "We Cut 94% of AI Coding Tokens With a Local Code Index" ## Tone & Overall Style - **Conversational, storytelling opening**: The talk begins as a personal narrative, not a technical briefing. > "Hey, I'm Raj. I want to tell a story. Me and my friend Foss, we are building project together." - **Plain, simple vocabulary throughout**: Short sentences and everyday words dominate. > "One month our AI bill was fine. Next month, huge." - **Direct, imperative advice lines** used as punchlines: > "Fix the input. That's where your money goes." ## Recurring Phrases & Verbal Habits - **"Me and my friend Foss" / "Me and Foss"** — repeated framing of the work as a two-person collaboration (lines 1, 45, 328, 603). - **"This is the most important slide"** — explicit signposting of key moments: > "This is the most important slide. 90% of your AI cost is input." - **Repetition for emphasis** — restating the core lesson in multiple forms: > "Simple choices work better than complex one." and "The lesson we learned, simple formula beats the complex model most of the time." - **Frequent fillers and self-corrections**: "uh" appears many times ("Uh money is in the input", "we run two searches at at the same time", "83 uh K tokens"), and stumbles like "It's It's like ordering a pizza" — suggesting somewhat unpolished live delivery. - **Non-native grammar patterns** consistently present ("we started to building something", "if they are actually relevant?", "authentic" phrasing like "This is the beauty"). ## Teaching Habits & Structure - **Numbered step-by-step explanations**: > "Here how it works. Five steps. Step one, we read the code and break into small pieces..." - **Analogy use** to explain cost: > "It's like ordering a pizza and paying for extra nine pizzas you don't eat every time." - **Failure-then-solution structure**: lists three failed attempts before the fix ("First, we changed our prompt... Second, we change the model setting... Third, uh output compression"). - **Concrete numbers emphasized**: > "45,000 tokens of context, but the part of actually mattered is about 5,000 only." > "247 queries. 12.4 million tokens saved." - **Explicit honesty/caveat section** — a deliberate teaching choice to state limits: > "I want to be honest about the limits." - **Call to action with verification invitation**: > "These numbers are real. Test is public. You can run it yourself." ## Pattern vs. Isolated Moments (Confidence Notes) - **Stable patterns** (repeated evidence): simple vocabulary, step numbering, number-heavy claims, honesty framing, collaborative "me and Foss" framing. - **Less certain / isolated**: The pizza analogy appears only once — it's a delivery habit hint but not a confirmed pattern. The "uh" fillers are frequent enough to be a stable feature of live delivery, but transcription noise cannot be ruled out for exact counts. # Position Record: "We Cut 94% of AI Coding Tokens With a Local Code Index" — Rajkumar Sakthivel, Tesco ## Speaker - **Raj (Rajkumar Sakthivel)** is the sole identified speaker throughout. He repeatedly credits his collaborator **Foss** for specific observations and decisions. --- ## 1. Problem Statement - Raj and Foss use AI coding tools (**Claude Code, Cursor, Copilot, CodeX**) daily on a shared project. - Their AI bill spiked one month with **no change in tools or project** — only more usage. - **Core finding:** Most cost came not from AI "thinking" but from **sending too much input context** — irrelevant files sent every query. > "It's like ordering a pizza and paying for extra nine pizzas you don't eat every time." - **Measured example:** A typical query sent ~45,000 tokens of context; only ~5,000 mattered. --- ## 2. Failed Approaches (as stated) 1. **Shorter prompts / relevant-code-only prompts** — Failed. The 45K tokens were already sent before the prompt was read; cost already incurred. 2. **Model settings (max tokens, temperature)** — Failed. These change **output, not input**, and "money is in the input." 3. **Output compression** — Worked (75% output reduction), but insufficient: output is only ~10% of cost. --- ## 3. Central Claim: Cost Distribution - **~90% of AI cost is input**; ~10% is output. - Therefore cutting output 75% saves only ~8% total; **cutting input 94% saves ~61% total**. > "Fix the input. That's where your money goes." --- ## 4. The Solution: Local Search Layer A layer between the codebase and AI tools, in five steps: 1. **Chunking:** Break code into meaningful units (functions, classes, methods), not random chunks. 2. **Dual search (biggest saving):** Run **meaning-based (semantic)** and **word-based (keyword)** search simultaneously, combined. 3. **Result shrinking:** Keep function name + description; 50-line function → 5 lines. 4. **Call-graph tracking:** Trace which functions call which. 5. **Scored filtering:** Low-scoring results are not sent at all. - **Everything runs locally; nothing goes to the cloud.** ### Why two searches - Semantic search misses exact names (may return a different `auth` function for "authenticate user"). - Keyword search misses related ideas (misses "sign in" when searching "login flow"). - **Individually, each misses ~1 in 4 results; together, ~1 in 10.** --- ## 5. Relevance Scoring (attributed to extensive work by Raj and Foss) - Problem: search results may be irrelevant; a confidently wrong answer is "worse than no answer." - **Rejected approaches:** - AI self-judging — too slow (+2–3 seconds). - Fixed score limit — too simple; short questions score low even on perfect matches. - **What worked:** formula of **50% meaning score, 30% keyword score, 20% code recency**, with an adaptive limit; runs in **0.4 ms**, no extra AI calls. > "Simple formula beats the complex model most of the time." --- ## 6. Reported Results - **Test:** FastAPI, 53 files, 20 real developer questions. - Without tool: **83K tokens/question**. - With tool: **4.9K tokens/question** (94% less). - With extra compression: **523 tokens/question**; accuracy ~90% right-code retrieval. - Claim: numbers are real, test is public and reproducible. - **Real-project savings report:** 247 queries, 12.4M tokens saved, ~£186 not spent; **84% of savings from search layer**, rest from compression. Not an estimate — tool measures per-query counterfactual and multiplies by model price. --- ## 7. Stated Limitations (Raj is explicit) - **94% is a worst-case baseline** (full-file reading every time); real tools like Claude Code are smarter, so **real savings are lower**. Full-file baseline was chosen only for consistent measurement. - **Large mixed codebases:** on a 396-file project, **recall dropped almost to zero**. Works well when files do one thing; struggles with multi-purpose files. - **Small fast search model:** fast (<1s re-indexing), but a bigger model would find more. **"We chose speed over perfection."** --- ## 8. Cross-Tool Index and Memory - Observation Raj credits to **Foss**: tools (Claude Code, Cursor, Copilot) each start fresh and share nothing — you re-explain the codebase repeatedly. - **Solution:** one shared local index + persistent memory, so knowledge learned in one tool carries to the next session. --- ## 9. Concluding Position - The answer to high bills was **not a better model** but **sending less**. - Models may account for ~30% of cost; **~70% is the input fed to them**. - Model choice (Opus vs. Sonnet) "matters less than you think." - Tool is **free, open source**, installable via one command (`CCE`); invites others to run it and report savings. # Video Speaker Attribution Record ## Video - **Title:** We Cut 94% of AI Coding Tokens With a Local Code Index - Rajkumar Sakthivel, Tesco - **Video ID:** `01a15ca8-dd57-7242-af41-80e94d755c5c` ## Speakers & Roles | Speaker | Role | Evidence | |---|---|---| | **Raj** (Rajkumar Sakthivel) | **Creator / Presenter** | Self-introduced: *"Hey, I'm Raj. I want to tell a story."* Delivers the entire talk. | | **Foss** (friend/collaborator) | **Referenced co-builder — not a speaker** | Mentioned repeatedly as project partner; *"Here is something Foss pointed out earlier on."* No direct speech in transcript. | ## Referenced Sources & Claims - **Tested project:** FastAPI (open-source), cited with figures: 53 files, 20 questions, 83K → 4.9K tokens per question (94% reduction), 523 tokens with compression, ~90% accuracy. Claim: *"Test is public. You can run it yourself."* - **Tools referenced:** Claude Code ("cloud code"), Cursor, Copilot, CodeX. - **Models referenced:** Opus, Sonnet (named in cost comparison only). - **Reported metrics:** 247 queries, 12.4 million tokens saved, ~186 currency units not spent, 84% of savings from search layer. ## Promotional Segments — Flagged - **Closing promotional segment:** Speaker promotes his own tool: - *"One command to try it, CCE. It's free, open source."* - *"Run it on your project for a week. See your own number."* - *"Try it, see the number, tell us what you saved."* ## Uncertainty & Limitations (as stated by speaker) - 94% figure is **worst-case** (full-file baseline); *"Real savings are lower than 94%."* - Recall dropped "almost zero" on a 396-file mixed codebase. - Small fast model used for search; bigger model would find more. ## Unavailable Evidence Channels - **No curated metadata supplied** (no comment availability, description, or timestamps). - **No timestamps present** in transcript. - Foss's identity and exact role details are not established beyond "friend/collaborator." Hey, I'm Raj. I want to tell a story. Me and my friend Foss, we are building project together. We are using AI coding tools everyday. Cloud code, cursor, co-pilot, code X, normal stuff. One month our AI bill was fine. Next month, huge. We did nothing different. Same project, same tools, just more of it. We panicked. We looked what was happening and we found something surprising. Most of the money was not the AI thinking. Most of it was sending too much context. Files the AI don't need. Context is important. Code that was not relevant sent anyway every time. So, me and my friend Foss, we started to building something to fix it. In this talk, is about what we built and what we learned. Every AI coding tools uh do the same thing. It send your code to the model as a context. And the tools thinks more context is better. We measured typical query on our project. It was sending 45,000 tokens of context, but the part of actually mattered is about 5,000 only. Other 40,000 tokens are not useful, but we paid for them every single query. It's It's like ordering a pizza and paying for extra nine pizzas you don't eat every time. We tried three things before we found what works. First, we changed our prompt be short, only show relevant code. Sounds good. But, it does not work. The model already got 45,000 tokens before it read the prompt. Cost already happened. Second, we change the model setting like a max token temperature. Same problem. These changes the output, not the input. Uh money is in the input. Third, uh output compression. This one actually works. We told the model to write short answers. It cut the output 75%. But, output only uh about 10% of the cost. So, 75% of a small number still small number, not enough. We need to fix the input. This is the most important slide. 90% of your AI cost is input. Files, search results, context you send in. Only 10% is output. The code the AI writes back. So, if you cut the output by 75%, you can save about eight 8% total. But, if you cut input by 94%, you can save about 61% total. Same math, but different result. Fix the input. That's where your money goes. We built a local search layer. It sits between your code base and the AI. Instead of sending whole files, the AI search and index. It gets back only small piece of code actually it needs. Here how it works. Five steps. Step one, we read the code and break into small pieces, functions, classes, methods. Not a random chunks, proper piece that makes sense. Step two, we run two searches at at the same time. One such find the code by meaning. One such find the code by exact words. Then combine the results. This is where big saving comes from. Steps three, we can shrink their results even more. Keep only the function name and the description. Cut 50-line function down to five lines. Step four, uh we track the connection with the which function called which. So, if you find one piece of code, you can find everything connected to it. Step five, every results get score. If the score is too low, we don't send it. No bad context. Everything runs on your machine. Nothing goes to the cloud. This is the beauty. Why do we run two searches instead of one? Because each one has a weakness. Meaning-based search is good at finding related ideas, but it misses exact names. You will search for um authenticate user function, and it might show you different auth function instead, because they are similar meaning. The word-based search is good at exact names, but it misses related ideas. You search for login flow, and it misses everything that says sign in. By themselves, both searches miss about one in four results. Together, they miss about one in 10. They fix each other weakness spots. Here is the hard part Foss and I spend most of the time on. The search finds results, but they actually relevant? Sometimes search results returns 10 results, and none of them are right. If they use the bad results, it gives confident wrong answer. The worse than no answer. We tried asking AI to judge its own results, too slow. Add 2-3 seconds every time. We tried to fix the score limit, too simple. Short questions score low even when the match is perfect. It worked. Some simple formula, 50% meaning score, 30% keyword score, 20% how the recent code is. And the limit adjust based on the current result. It runs 0.4 milliseconds, no extra AI calls needed. The lesson we learned, simple formula beats the complex model most of the time. We need numbers, not just stories. So, here are ours. We tested an open-source real project, FastAPI, 53 files, 20 real questions a developer would ask. Without using our tool, 83 uh K tokens per questions. With our tools, uh 4.9 K tokens per questions. That is 94% less. With the extra compression on top, 523 tokens per question. And the accuracy still um find the right code 90% of the code. These numbers are real. Test is public. You can run it yourself. The command is on screen. I want to be honest about the limits. The 94% again the worst case, reading full files every time. In a real life, the tools like a cloud code already smarter than that. Real savings are lower than 94%. We use full file base because it is the only one we can measure the same way every time. Big mixed code base are hard. We tested on large projects with 396 files. The recall dropped almost zero. If your files each do one thing, it works well. If your files do many things, it struggles. We use a small fast model for search. It's quick. Re-indexing takes under a second. But the bigger model would find more. We chose speed over perfection. Simple choices work better than complex one. Small database instead of big infrastructure. Two searches instead of one fancy one. Local instead of cloud. Simple one. Here is something Foss pointed out earlier on. We use many tools. Cloud code for hard problems. Cursor for quick edits. Copilot for small completions. Each tool starts fresh every time. They do not share anything. You explain the same code base to three different days. We built one shared index so all your tools connect to it. Same search, same results for all of them. And we added memory. When one tool learns something about your project, that knowledge stays. Next session, different tool, same project, the context is already there. We explain the code base once. Every tool remembered. This is the saving report on real project. 247 queries. 12.4 million tokens saved. Nearly 186 not spent. Most of the savings, 84% came from search layer. The rest of them, compression. This is not an estimate. The tool tracks every query. It compares what would have been sent against what was sent. Then it multiplies by the model price. Run it on your project for a week. See your own number. Me and Foss built this because we had a huge bill and no good answer. The answer was not a better model. The answer was sending less. We argue about which model is best, Opus or Sonnet. But the models may be 30% of the cost, but other 70% is what you feed it. Fix the input, the model choice matters less than you think. One command to try it, CCE. It's free, open source. You will see the code on screen. Try it, see the number, tell us what you saved. Thanks. Happy coding.