A knowledge pipeline is a document processing workflow that transforms raw data into searchable knowledge bases. Think of orchestrating a workflow, now you can visually combine and configure different processing nodes and tools to optimize data processing for better accuracy and relevance.
Every knowledge pipeline normally follows a structured flow through four key steps:
Data Sources → Data Extraction → Data Processing → Knowledge Storage
Each step serves a specific purpose: gathering content from various sources, converting it to processable text, refining it for search, and storing it in a format that enables fast, accurate retrieval.
Dify provides built-in pipeline templates that is optimized for certain use cases, or you can also create knowledge pipelines from scratch. In this session, we will go through creating options, general process of building knowledge pipelines, and how to manage it.
Step 1: Create Knowledge Pipeline
Start from built-in templates, blank knowledge pipeline or import existing pipeline.
Step 2: Orchestrate Knowledge Pipeline
Get to know how the knowledge pipeline works, orchestrate different nodes and make sure it’s ready to use.
Step 3: Publish Knowledge Pipeline
Let’s make it ready for document processing.
Step 4: Upload Files
Add documents and process them into the searchable knowledge base.
Step 5: Manage and Use Knowledge Base
Maintain documents, test retrieval, modify settings, and more.
Last modified on June 25, 2026
