Skip to content

RAG lab: eight build-it-yourself notebooks with self-checks, solutions and instructor notes - #17

Merged
kchandan merged 3 commits into
mainfrom
rag-lab-build
Oct 29, 2026
Merged

kchandan merged 3 commits into
mainfrom
rag-lab-build

Conversation

@kchandan

Copy link
Copy Markdown
Member

What

The local RAG lab was a tour of a finished library (RAGLabClient). Learners now build each part themselves, one notebook at a time, with a self-check after every exercise.

# Notebook Learner writes
1 chunking fixed and paragraph-aware chunkers
2 embeddings cosine similarity, ranking
3 vector store numpy store (add, search)
4 generation grounded prompt, full pipeline
5 Chroma and filters ChromaStore, metadata filters
6 hybrid search keyword score, hybrid ranking
7 evaluation hit rate and MRR scorer, settings grid, own cases
8 break it newest-version filter, refusal rule, injection detection, safer prompt

09_sec_filings_rag is the old SEC notebook, unchanged and optional.

  • One source file per notebook holds both the stub and the solution (labagent/ai/rag/lab/), so learner and instructor versions cannot drift.
  • labagent rag init creates the lab folder (README, nine notebooks, eight sample docs). --solutions adds solutions/ for instructors.
  • Five new sample documents (eight in total).
  • Docs: docs/rag-local.md rewritten, new docs/rag-lab-instructor.md, README line updated.
  • Setup note: the pip bundled with macOS Python 3.9 cannot do pip install -e here, so the docs say to upgrade pip first.

Tested

  • All 8 solution notebooks run through a real Jupyter kernel with every check passing.
  • Tests execute every solution notebook and assert that every student notebook stops at a failed check, not a crash. Execution tests skip when sentence-transformers or chromadb is missing (so CI, which installs only the dev extra, runs the structural tests only).
  • Full suite: 473 passed, 1 skipped (before the last two small commits; the RAG and hygiene tests were rerun after them).

Not tested

  • Real LLM answers (Groq, Ollama): only the mock provider was exercised.
  • Timings in the instructor notes are estimates, not measured with learners.
  • Windows and Python 3.11 or newer.
  • The abstain threshold in notebook 8 is tuned to this model and corpus, with a thin margin.

🤖 Generated with Claude Code

kchandan and others added 3 commits October 29, 2026 11:16
…es, correct settings

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…s and instructor notes

The local lab was a tour of a finished library. Learners now write the chunker, cosine
similarity, a numpy vector store, the prompt, a Chroma store, hybrid search, an evaluation
and defences against stale, uncovered and hostile documents. Each notebook source holds the
stub and the solution, so the learner and instructor versions cannot drift apart.

- labagent rag init creates README, eight notebooks, the SEC notebook as 09, eight sample docs
- labagent rag init --solutions adds solutions/ for instructors
- tests execute every solution notebook and check that every student notebook stops at a failed check

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@kchandan
kchandan merged commit ea13e78 into main Oct 29, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant