Systematically Improving RAG Applications

Categories: Marketing & Sales
Wishlist Share

About Course

Systematically Improving RAG Applications

Stop building RAG systems that impress in demos but disappoint in production

Transform your retrieval from “good enough” to “mission-critical” in weeks, not months. Most RAG systems stall in prototype purgatory: they demo well, but fail on complex queries—eroding trust and wasting engineering time. The difference isn’t just better tech, but a systematic mindset.

With the RAG Flywheel, you’ll:
✅ Pinpoint failures with synthetic evals
✅ Fine-tune embeddings for 20–40% gains
✅ Collect 5x more user feedback
✅ Segment queries to target high-impact fixes
✅ Build multimodal indices for docs, tables, images
✅ Route queries to the best retriever automatically

Week by week, you move from vague “make it better” to clear metrics, focused improvements, and compounding value. Real-world results include +20% accuracy from re-ranking, +14% with cross-encoders, and $50M revenue boosts from better search.

Join 400+ engineers applying this framework in production. Instructor Jason Liu has built multimodal retrieval and recommendation systems at Facebook, Stitch Fix, and through consulting—experience that shaped this practical, battle-tested approach.

What you’ll learn

Follow a repeatable process to continually evaluate and improve your RAG application

Analyze and Diagnose RAG System Performance

  •  

    Evaluate retrieval quality using precision, recall, and MRR metrics to identify system weaknesses

  •  

    Differentiate between leading metrics (experiments run) and lagging metrics (customer satisfaction) to drive actionable improvements

  •  

    Design synthetic data generation pipelines that enable rapid experimentation without waiting for user data

Construct Data-Driven Improvement Frameworks

  •  

    Create comprehensive evaluation datasets using LLMs to generate realistic query-answer pairs

  •  

    Establish baselines using tools like LanceDB to benchmark different retrieval implementations

Design Specialized Search Systems

  •  

    Develop multimodal retrieval systems that handle documents, images, tables, and structured data

  •  

    Synthesize lexical (BM25), semantic (embeddings), and metadata-based search for optimal results

Optimize Query Understanding & Routing

  •  

    Extract structured information from diverse data sources to enable precise filtering

  •  

    Classify queries using domain expertise and few-shot classifiers to improve routing accuracy

Show More

Course Content

Topic 1: Cohort 1
Module containing 16 video lessons

  • Lesson 1: Cohort 1 Guest Lectures Building Dynamic Ai Mem
    55:44
  • Lesson 2: Cohort 1 Guest Lectures Text Chunking In Rag Es
    01:01:29
  • Lesson 3: Cohort 1 Guest Lectures Custom Rag Evaluations
    53:37
  • Lesson 4: Cohort 1 Guest Lectures Multimodal Rag Hybrid
    59:04
  • Lesson 5: Cohort 1 Guest Lectures Leveraging User Feedbac
    55:16
  • Lesson 6: Cohort 1 Guest Lectures Cohere S Guide To Rag C
    01:08:23
  • Lesson 7: Cohort 1 Guest Lectures Boosting Bm 25 With Gene
    58:12
  • Lesson 8: Cohort 1 Office Hours Beyond Dense Embeddings Expl
    25:31
  • Lesson 9: Cohort 1 Office Hours Building Resilient Rag Syste
    01:00:08
  • Lesson 10: Cohort 1 Office Hours Building Scalable Systems Wi
    59:00
  • Lesson 11: Cohort 1 Office Hours Data Flywheels And Fine Tu
    29:16
  • Lesson 12: Cohort 1 Office Hours Evaluating Agent Performance
    47:42
  • Lesson 13: Cohort 1 Office Hours Handling Real Time Insights
    51:36
  • Lesson 14: Cohort 1 Office Hours Optimizing Planner And Feedb
    44:24
  • Lesson 15: Cohort 1 Office Hours Smart Routing And Query Opti
    42:22
  • Lesson 16: Cohort 1 Office Hours Using Ai To Streamline Query
    44:32

Topic 2: Cohort 2
Module containing 6 video lessons

Topic 3: Main Content
Module containing 19 video lessons

Topic 4: Workshops
Module containing 6 video lessons

Download Full Course Resources Size 31MB

Student Ratings & Reviews

No Review Yet
No Review Yet