The wrong passage reaches the model
Chunking ignores document structure, keyword search is missing, and nothing reranks. No prompt fixes missing evidence.
Production RAG & LLM systems
Your RAG demo works.
Your production system doesn't.
I build retrieval systems that cite sources, refuse when the evidence is weak, and pass a measurable evaluation — in 14 days.
Fine-tuned Mistral-7B
Hybrid retrieval
Evaluation-first
Citations & refusal
Why it breaks
It's almost never the model.
Three failure modes account for most RAG projects that never make it to production.
Nobody defined "correct"
Without a golden question set and retrieval metrics, quality is judged by vibes. Degradation goes unnoticed until users complain.
Confident answers, no citations
In legal, compliance, and finance, an unsupported answer is a liability. The system must cite — and refuse when it can't.
The 14-day build
From messy documents to a system you can trust.
Fixed scope. Measurable acceptance test. You own the code and the evaluation.
-
Days 1–2
Discovery & success criteria
Document audit, retrieval requirements, and the questions your system must answer correctly. We define "done" before writing code.
-
Days 3–5
Ingestion & indexing
Parsing, metadata-aware chunking, embeddings, and a vector index. Structure preserved (act, section, page) so retrieval can filter.
-
Days 6–8
Retrieval that actually retrieves
Hybrid search (dense + keyword), reranking, and metadata filters. Measured with recall@k and MRR, not intuition.
-
Days 9–11
Grounded generation
Prompt design, citation mapping, and refusal behavior when evidence is weak. No unsupported claims.
-
Days 12–13
Evaluation harness
A golden set your team can re-run. Retrieval hit rate, faithfulness, latency, and cost per query — in one report.
-
Day 14
Deployment & handoff
Dockerized FastAPI service, docs, and a recorded walkthrough. You can run it without me.
Proof of work
E-Wakil — a legal AI assistant for Nepali law
A domain-specific assistant that answers questions over statutes and case law with citations. Built on a fine-tuned Mistral-7B and a hybrid retrieval pipeline, with refusal behavior for unsupported questions.
- Fine-tuning: LoRA on domain legal text, constrained GPU budget
- Retrieval: metadata-aware chunks, dense + keyword, reranking
- Trust: citation verification and refusal when evidence is missing
- Evaluation: golden-set recall@k, MRR, faithfulness, latency, cost
Documents
↓
Parse & chunk
↓
Hybrid retrieval
↓
Rerank + filter
↓
Grounded generation
↓
Citations + refusal
↓
Evaluation report
Open source
RAG Eval Kit
A dependency-free evaluation harness for retrieval pipelines. Measure recall@k, precision@k, MRR, and nDCG on your own corpus — before and after every change.
python -m rag_eval.cli --corpus data/corpus.jsonl --queries data/queries.jsonl
Recall@kmeasured
MRRmeasured
nDCGmeasured
BM25 + dense hybridincluded
Zero dependenciesyes
This is for you if…
- Your AI answers on private documents are unreliable
- You need citations for legal, compliance, or finance
- You have documents and a real accuracy requirement
- You want to measure quality, not guess
Not for you if…
- You need a ChatGPT wrapper in 24 hours
- You want a guaranteed 100% accuracy
- There is no document corpus
- Budget is under $500
Pricing
Start with a diagnosis, not a proposal.
Every engagement begins with a paid, fixed-scope audit so we both know what "working" means.
RAG Readiness Audit
$500
3 days · credited toward the build
- Document and retrieval audit
- 25-question golden set
- Failure breakdown by layer
- Written recommendation
Production RAG MVP
$2,500+
14 days · fixed scope
- Ingestion + metadata chunking
- Hybrid retrieval + reranking
- Citations and refusal behavior
- Evaluation harness
- Dockerized deployment + handoff
Fine-tuning sprints ($1,500+) and monthly retainers ($1,500+/mo) available after the audit.
Questions
Can you guarantee 100% accuracy?
No, and you should be suspicious of anyone who does. I guarantee a measured system: we define the evaluation set together and you see exactly where it fails before scaling it.
We already tried RAG and it didn't work.
That's common, and it's usually an evaluation problem. Nobody defined what "correct" meant, so retrieval degradation went unnoticed. The audit rebuilds the eval set from your real questions and shows whether it's fixable in a week or needs a redesign.
Why not just use ChatGPT?
For public information, you can. The moment the answer must come from your private documents, with citations, and without hallucinating, you need retrieval you control.
What do you need from me?
A sample of real documents, the questions your team actually asks, and one technical contact. Everything else I handle.
Do you sign NDAs?
Yes. Send yours over; work starts after it's signed.
20 minutes · free
Tell me the question your system gets wrong.
I'll tell you why it's happening and what I'd test first — whether we work together or not. No pitch deck.
- Specific diagnosis, not a sales call
- Bring one real failure case
- You keep the notes