What is context precision in RAG evaluation
How it works
Context precision evaluates the quality of the retrieval step in a RAG pipeline by measuring the fraction of retrieved documents that are truly relevant to the query and correctly support the generated response. Imagine a researcher gathering references for a paper: context precision is like checking how many of those references actually contain useful information that backs up the claims made. High context precision means the retrieval system is precise, providing mostly relevant documents, which helps the language model generate accurate, grounded answers.
Concrete example
Suppose a RAG system retrieves 5 documents for a question, but only 3 of them are relevant and support the answer. The context precision is calculated as:
retrieved_docs = 5
relevant_docs = 3
context_precision = relevant_docs / retrieved_docs
print(f"Context Precision: {context_precision:.2f}")Context Precision: 0.60
| Retrieved Documents | Relevant Documents | Context Precision |
|---|---|---|
| 5 | 3 | 0.60 |
When to use it
Use context precision when you want to evaluate how well the retrieval component of a RAG system filters out irrelevant documents and provides precise context for generation. It is critical in applications where factual accuracy and grounding are essential, such as question answering, knowledge-based chatbots, and document summarization. Avoid relying solely on context precision if recall (retrieving all relevant documents) is also important, as it does not measure completeness.
Key Takeaways
- Context precision measures the fraction of retrieved documents that are relevant and support the generated answer in RAG systems.
- High context precision ensures the language model generates answers grounded in accurate and relevant information.
- Use context precision to evaluate retrieval quality when factual grounding is critical, but combine with recall metrics for completeness.
- It helps identify if the retrieval step is filtering out noise and providing precise context for generation.