Systematically Improving RAG Applications

Categories: Marketing & Sales
Wishlist Share
Share Course
Page Link
Share On Social Media

About Course

Systematically Improving RAG Applications

Stop building RAG systems that impress in demos but disappoint in production

Transform your retrieval from “good enough” to “mission-critical” in weeks, not months. Most RAG systems stall in prototype purgatory: they demo well, but fail on complex queries—eroding trust and wasting engineering time. The difference isn’t just better tech, but a systematic mindset.

With the RAG Flywheel, you’ll:
✅ Pinpoint failures with synthetic evals
✅ Fine-tune embeddings for 20–40% gains
✅ Collect 5x more user feedback
✅ Segment queries to target high-impact fixes
✅ Build multimodal indices for docs, tables, images
✅ Route queries to the best retriever automatically

Week by week, you move from vague “make it better” to clear metrics, focused improvements, and compounding value. Real-world results include +20% accuracy from re-ranking, +14% with cross-encoders, and $50M revenue boosts from better search.

Join 400+ engineers applying this framework in production. Instructor Jason Liu has built multimodal retrieval and recommendation systems at Facebook, Stitch Fix, and through consulting—experience that shaped this practical, battle-tested approach.

What you’ll learn

Follow a repeatable process to continually evaluate and improve your RAG application

Analyze and Diagnose RAG System Performance

  •  

    Evaluate retrieval quality using precision, recall, and MRR metrics to identify system weaknesses

  •  

    Differentiate between leading metrics (experiments run) and lagging metrics (customer satisfaction) to drive actionable improvements

  •  

    Design synthetic data generation pipelines that enable rapid experimentation without waiting for user data

Construct Data-Driven Improvement Frameworks

  •  

    Create comprehensive evaluation datasets using LLMs to generate realistic query-answer pairs

  •  

    Establish baselines using tools like LanceDB to benchmark different retrieval implementations

Design Specialized Search Systems

  •  

    Develop multimodal retrieval systems that handle documents, images, tables, and structured data

  •  

    Synthesize lexical (BM25), semantic (embeddings), and metadata-based search for optimal results

Optimize Query Understanding & Routing

  •  

    Extract structured information from diverse data sources to enable precise filtering

  •  

    Classify queries using domain expertise and few-shot classifiers to improve routing accuracy

Show More

Course Content

Topic 1: Cohort 1
Module containing 16 video lessons

  • Lesson 1: Cohort 1 Guest Lectures Building Dynamic Ai Mem
    55:44
  • Lesson 2: Cohort 1 Guest Lectures Text Chunking In Rag Es
    01:01:29
  • Lesson 3: Cohort 1 Guest Lectures Custom Rag Evaluations
    53:37
  • Lesson 4: Cohort 1 Guest Lectures Multimodal Rag Hybrid
    59:04
  • Lesson 5: Cohort 1 Guest Lectures Leveraging User Feedbac
    55:16
  • Lesson 6: Cohort 1 Guest Lectures Cohere S Guide To Rag C
    01:08:23
  • Lesson 7: Cohort 1 Guest Lectures Boosting Bm 25 With Gene
    58:12
  • Lesson 8: Cohort 1 Office Hours Beyond Dense Embeddings Expl
    25:31
  • Lesson 9: Cohort 1 Office Hours Building Resilient Rag Syste
    01:00:08
  • Lesson 10: Cohort 1 Office Hours Building Scalable Systems Wi
    59:00
  • Lesson 11: Cohort 1 Office Hours Data Flywheels And Fine Tu
    29:16
  • Lesson 12: Cohort 1 Office Hours Evaluating Agent Performance
    47:42
  • Lesson 13: Cohort 1 Office Hours Handling Real Time Insights
    51:36
  • Lesson 14: Cohort 1 Office Hours Optimizing Planner And Feedb
    44:24
  • Lesson 15: Cohort 1 Office Hours Smart Routing And Query Opti
    42:22
  • Lesson 16: Cohort 1 Office Hours Using Ai To Streamline Query
    44:32

Topic 2: Cohort 2
Module containing 6 video lessons

Topic 3: Main Content
Module containing 19 video lessons

Topic 4: Workshops
Module containing 6 video lessons

Download Full Course Resources Size 31MB

Student Ratings & Reviews

No Review Yet
No Review Yet