What is the RAG Pattern? Implementing Retrieval-Augmented Generation on Databricks

RAG (Retrieval-Augmented Generation) is an architecture that combines an LLM's generation capability with retrieval over external knowledge, suppressing hallucinations while producing up-to-date, accurate answers. On the Databricks GenAI Engineer exam, roughly 30-40% of the questions cover RAG, testing your understanding of architecture design, component selection, and evaluation methods.

Overall RAG Architecture

A RAG pipeline consists of two stages: a retrieval phase and a generation phase. The user's query is converted into an Embedding, Vector Search retrieves related documents, and the prompt is then sent to the LLM together with that context.

┌─────────────────────────────────────────────────────────────┐
│                    RAG Pipeline                             │
│                                                             │
│  User Query                                                 │
│      │                                                      │
│      ▼                                                      │
│  ┌──────────────┐    ┌──────────────────┐                   │
│  │ Embedding    │───▶│ Vector Search    │                   │
│  │ Model        │    │ (similarity)     │                   │
│  └──────────────┘    └───────┬──────────┘                   │
│                              │ Top-K chunks                 │
│                              ▼                              │
│                     ┌──────────────────┐                    │
│                     │ Prompt Template  │                    │
│                     │ (Query+Context)  │                    │
│                     └───────┬──────────┘                    │
│                             │                               │
│                             ▼                               │
│                     ┌──────────────────┐                    │
│                     │ LLM (generation) │                    │
│                     └───────┬──────────┘                    │
│                             │                               │
│                             ▼                               │
│                        Response                             │
└─────────────────────────────────────────────────────────────┘

This pipeline design lets the LLM access information it did not see at training time (internal documents, fresh data, etc.) and produce grounded answers.

Comparing Chunking Strategies

Before storing documents in Vector Search, you need to split them into appropriately sized chunks. Chunking strategy directly affects retrieval accuracy, so the right choice depends on the characteristics of the document.

The GenAI Engineer exam asks which chunking strategy to pick based on the type of document. For example, Recursive fits a document with heading structure (like an internal wiki), while Fixed-size suits a collection of short FAQs.

Choosing an Embedding Model

Embedding models convert text into a vector space and form the foundation of similarity search. On Databricks, you can use models served by the Foundation Model API or call external APIs.

If you want to stay entirely within Databricks, the GTE model on the Foundation Model API is the easiest to integrate. You can also use the External Model feature to call external APIs such as OpenAI.

Configuring Vector Search

Databricks Vector Search is a managed vector database that offers similarity search under Unity Catalog-integrated access control. The choice of index type has a major impact on RAG architecture operability.

Prompt Engineering Templates

In RAG, the prompt you hand to the LLM determines answer quality. Passing the retrieved context and the user's query in a structured way helps suppress hallucinations.

# Example RAG prompt template

prompt_template = """
You are an assistant that answers based on internal documents.
Use ONLY the context below to answer the question.
If the context does not contain the answer, reply "Information not found."

## Context
{context}

## Question
{query}

## Answer
"""

# Running it on Databricks Foundation Model API
import mlflow.deployments

client = mlflow.deployments.get_deploy_client("databricks")

response = client.predict(
    endpoint="databricks-meta-llama-3-1-70b-instruct",
    inputs={
        "messages": [
            {"role": "system", "content": "Answer based on the internal documents."},
            {"role": "user", "content": prompt_template.format(
                context=retrieved_context,
                query=user_query
            )}
        ],
        "max_tokens": 1024,
        "temperature": 0.1
    }
)

Setting temperature low (0.0-0.2) makes it easier to produce answers that stay faithful to the context. The exam tests how important it is to include a constraint like "do not use information outside the context" in the prompt template.

RAG Evaluation Metrics and MLflow evaluate()

To quantitatively evaluate RAG pipeline quality, use the following three metrics. On Databricks, they are built into MLflow evaluate().

import mlflow

# Run the RAG evaluation pipeline
eval_data = pd.DataFrame({
    "questions": ["What are Databricks cluster policies?", ...],
    "ground_truth": ["A feature that defines constraints on cluster creation settings.", ...],
    "retrieved_context": [retrieved_chunks_list, ...],
    "generated_answers": [rag_responses_list, ...]
})

results = mlflow.evaluate(
    data=eval_data,
    model_type="question-answering",
    evaluators="default",
    extra_metrics=[
        mlflow.metrics.genai.faithfulness(),
        mlflow.metrics.genai.relevance(),
    ]
)

print(results.metrics)
# {'faithfulness/v1/mean': 0.92, 'relevance/v1/mean': 0.88, ...}

MLflow evaluate() uses the LLM-as-a-Judge pattern, having a separate LLM score the answer quality. Each metric is rated on a 1-5 scale and can be used as a threshold-based quality gate for the pipeline.

RAG vs Fine-tuning Comparison

Key Topics on the GenAI Engineer Exam

• Choosing a chunking strategy: picking Fixed / Semantic / Recursive based on document characteristics
• Choosing an index type: criteria for Delta Sync Index vs Direct Vector Access Index
• Understanding evaluation metrics: definitions and measurement methods for Faithfulness, Relevance, and Context Precision
• RAG vs Fine-tuning: picking the right approach for the use case
• Prompt design: building a prompt template that includes context constraints
• MLflow evaluate(): evaluation methods for RAG pipeline quality

Roughly 30-40% of the GenAI Engineer exam covers RAG. It tests more than just architectural understanding — you need practical judgment about which component to choose in a given scenario.

Sample Question

Frequently Asked Questions