- Keep the runs that come back right as golden questions, so later changes can’t quietly break them.
- Turn the ones that come back wrong into bad cases to fix.
Test Retrieval
On the knowledge base’s Retrieval Test page, type a question and run it. Ask full questions the way your users would, not just keywords. Run the same question at each retrieval depth, and pick the one that returns the right chunks most consistently as your knowledge base’s default. You can also attach images to a question. See Multimodal Retrieval for which models make that work. Every run stays under Records, so you can compare results before and after a settings change.Golden Questions
Some questions must return the right chunks no matter what you change later, such as the ones your users ask most. Keep those as golden questions, along with the chunks each one should return. Then each time you change a setting or model, run one evaluation to check that they all still pass. When a run returns the right chunks, click Keep golden question under its results, tick the chunks that must come back, and choose whether all of them or any one of them has to return for the question to pass. To add one without a test run: open the Quality page, click Add golden, enter the question, describe the chunk you expect, and click Find evidence to pick from the closest matches. If you already have a list of questions, import it there from a CSV file (up to 1 MB). Evaluations skip golden questions with these statuses:- Draft: it has no expected chunks yet. Edit it and pick them.
- Stale: an expected chunk no longer exists, for example because you re-uploaded its document or a sync removed that passage. Edit it and pick new chunks.
- Paused: an expected chunk’s document is disabled. It counts again once you enable the document.
Bad Cases
When the right chunk is missing or ranked too low, click Make bad case under the results and pick the reason. Each points to a different fix:- Retrieval miss: the chunk didn’t come back at all. First check how the document was indexed. If it looks fine, widen the search with a deeper retrieval depth, a higher top K, or a lower score threshold under Retrieval Settings. A wider search can also bring back more unrelated chunks.
- Low score: the chunk came back but ranked too low. The rerank model assigns the scores. Try a different one, which may rank the same chunk higher.