Tagged: Evals

Articles

• Part 1Retrieval & RAG

  Designing a RAG System for 10 Million Documents

  How to size, build and measure retrieval-augmented generation over 10M documents: ingestion, hybrid retrieval, reranking, ACLs, cost and evals.

• Part 2Retrieval & RAG

  Debugging a RAG System That Gives Wrong Answers

  A stage-by-stage method for localizing RAG failures: build a replayable failure set, attribute each case to a stage, fix the biggest bucket, and gate on it.

• Part 3Retrieval & RAG

  Chunking Legal Contracts for Retrieval

  Why fixed-size chunks fail on contracts, and how to chunk by clause with context headers, defined terms, cross-references and amendment lineage.

• Part 4Retrieval & RAG

  BM25 vs Dense vs Hybrid Search: When to Use Each

  How lexical, dense, learned-sparse and late-interaction retrieval fail differently, how to fuse them with RRF, and how to choose on your own traffic.

• Part 5Retrieval & RAG

  Adding a Reranker and Proving It Helped

  How to add a cross-encoder reranker to a RAG pipeline, size it, and prove the gain with paired offline evals, end-to-end checks and a guarded A/B test.

• Part 6Cost, Latency & Serving

  Cutting LLM Cost by 70% Without Losing Quality

  How to stack prompt caching, model routing, batch APIs and token hygiene into a 70-80% cost cut, with an eval gate that proves quality held.

• Part 9Cost, Latency & Serving

  Routing Requests Between a Cheap and a Strong Model

  Design an LLM router that sends each request to the cheapest good-enough model: rules, learned routers, cascades, threshold tuning and drift monitoring.

• Part 10Agents & Tool Use

  Building Tool-Using Agents That Work Reliably

  Tool-use reliability is a systems property: tool design, a guarded control loop, permission gates, and pass^k evals on verifiable end states.

• Part 11Agents & Tool Use

  Detecting and Recovering from Agent Loops

  Classify why an agent loops, add harness guardrails and better tool feedback, and recover with an escalation ladder you can measure.

• Part 12Agents & Tool Use

  Designing Long-Term Memory for an LLM Assistant

  A tiered memory design with explicit write, read and forgetting policies, plus the benchmarks and production signals that show it works.

• Part 13Agents & Tool Use

  Guaranteeing Valid JSON Output from LLMs

  Enforce JSON Schema at decode time, design schemas that keep quality high, and add validation, repair and truncation handling on top.

• Part 14Evaluation & Reliability

  Detecting and Reducing LLM Hallucinations

  Separate hallucination types, detect them with claim-level verification and uncertainty signals, and reduce them without making the model useless.

• Part 15Evaluation & Reliability

  Building an Eval Suite for a Customer Support Bot

  Build support-bot evals from real failures: layered offline tests, calibrated judges, statistically sound release gates and a production flywheel.

• Part 16Evaluation & Reliability

  Making LLM-as-a-Judge Trustworthy: Calibration, Bias and Drift

  Treat an LLM judge like a measurement instrument: calibrate it against human labels, remove its known biases, and size your eval sets for real error bars.

• Part 19Fine-tuning & Adaptation

  Fine-Tuning vs RAG vs Prompting: An Eval-First Decision Procedure

  Diagnose whether your gap is knowledge or behavior, then climb from prompting to RAG to fine-tuning only when the cheaper rung plateaus.

• Part 20Fine-tuning & Adaptation

  Fine-Tuning a Small Model with LoRA on 5,000 Examples

  A practical LoRA recipe for an 8B model on 5K examples: data hygiene, hyperparameters, a real GPU memory budget, and eval-driven checkpoint selection.

• Part 23Production Engineering

  Tracing Every Step of a Multi-Agent Workflow

  Model multi-agent runs as one OpenTelemetry trace: GenAI span conventions, context propagation across queues, private content storage and tail sampling.

• Part 24Production Engineering

  Rolling Back a Bad Prompt Change in Production

  Make prompt rollback a sub-minute alias flip, catch regressions with canaries and guardrail metrics, and clean up what the bad prompt left behind.