> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dify.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Multimodal Retrieval

> How model choices affect image indexing and image-based search in a knowledge base

Whether retrieval can work with images depends on the models you picked for the knowledge base. Start from what you want retrieval to do with images:

* **Text-to-image search**: a text question finds images in the knowledge base by what they show.

  Pick a vision-capable *embedding* model. A question like "the org chart" can then match a diagram by what it shows, even if it has no caption or alt text.

* **Image-to-image search**: an attached image finds visually similar images in the knowledge base.

  Pick a vision-capable *embedding* model, so similar images match even when they share no text.

* **Image-to-text search**: an attached image finds relevant text chunks.

  Pick a vision-capable *reasoning* model. It describes the image in words and searches that description against text.

For all three, pick vision-capable models for both embedding and reasoning. Either model can be changed later under **Settings** > **Retrieval**, but changing it re-indexes every document and costs tokens.

> [!NOTE]
>
> After switching from two text-only models, click **Re-index** on each document that has images once the change finishes.

## How Models Affect Image Behavior

| | Images are extracted | Text finds image | Image finds text | Image finds image |
| :- | :- | :- | :- | :- |
| **Both support vision** | Yes | By appearance and by caption or alt text | Deep and Research: yes.  Fast: no, image queries find images only. | Yes |
| **Embedding only** | Yes | By appearance and by caption or alt text | No | Yes |
| **Reasoning only** | Yes | By caption or alt text only | Yes | No |
| **Neither** | Markdown and HTML: only the caption or alt text.  Office files: no. | By caption or alt text only | No.  Image-only queries fail, and images on a text query are ignored. | No |

Captions and alt text on images in Markdown and HTML files are searchable regardless of model choice, so well-captioned sources improve image retrieval in every combination.

> [!NOTE]
>
> * Text inside images in PDF files is read when either model accepts images.
> * Text inside images in Word, PowerPoint, and Excel files isn't read, so put anything users will search for in the surrounding text.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.