Reverse Model Request refers to the plugin's ability to make reverse requests to LLM capabilities within Dify, including all model types and features on the platform, such as TTS, Rerank, etc.
Note that requesting models requires passing a ModelConfig type parameter. Its structure can be referenced in Common Specification Definitions, and this structure will have slight differences for different types of models.
For example, for LLM type models, it needs to include completion_params and mode parameters. You can manually build this structure or use model-selector type parameters or configuration.
Note: If the model you're requesting doesn't have tool_call capability, the tools passed here won't take effect.
Example
If you want to request OpenAI's gpt-4o-mini model in a Tool, refer to the following example code:
from collections.abc import Generator
from typing import Any
from dify_plugin import Tool
from dify_plugin.entities.model.llm import LLMModelConfig
from dify_plugin.entities.tool import ToolInvokeMessage
from dify_plugin.entities.model.message import SystemPromptMessage, UserPromptMessage
class LLMTool(Tool):
def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage]:
response = self.session.model.llm.invoke(
model_config=LLMModelConfig(
provider='openai',
model='gpt-4o-mini',
mode='chat',
completion_params={}
),
prompt_messages=[
SystemPromptMessage(
content='you are a helpful assistant'
),
UserPromptMessage(
content=tool_parameters.get('query')
)
],
stream=True
)
for chunk in response:
if chunk.delta.message:
assert isinstance(chunk.delta.message.content, str)
yield self.create_text_message(text=chunk.delta.message.content)
Notice that the query parameter from tool_parameters is passed in the code.
Best Practices
It's not recommended to manually build LLMModelConfig. Instead, allow users to select their desired model in the UI. In this case, you can modify the tool's parameter list by adding a model parameter according to the following configuration:
identity:
name: llm
author: Dify
label:
en_US: LLM
zh_Hans: LLM
pt_BR: LLM
description:
human:
en_US: A tool for invoking a large language model
zh_Hans: 用于调用大型语言模型的工具
pt_BR: A tool for invoking a large language model
llm: A tool for invoking a large language model
parameters:
- name: prompt
type: string
required: true
label:
en_US: Prompt string
zh_Hans: 提示字符串
pt_BR: Prompt string
human_description:
en_US: used for searching
zh_Hans: 用于搜索网页内容
pt_BR: used for searching
llm_description: key words for searching
form: llm
- name: model
type: model-selector
scope: llm
required: true
label:
en_US: Model
zh_Hans: 使用的模型
pt_BR: Model
human_description:
en_US: Model
zh_Hans: 使用的模型
pt_BR: Model
llm_description: which Model to invoke
form: form
extra:
python:
source: tools/llm.py
Note that in this example, the model's scope is specified as llm, so users can only select llm type parameters. This allows you to modify the above example code as follows:
from collections.abc import Generator
from typing import Any
from dify_plugin import Tool
from dify_plugin.entities.model.llm import LLMModelConfig
from dify_plugin.entities.tool import ToolInvokeMessage
from dify_plugin.entities.model.message import SystemPromptMessage, UserPromptMessage
class LLMTool(Tool):
def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage]:
response = self.session.model.llm.invoke(
model_config=tool_parameters.get('model'),
prompt_messages=[
SystemPromptMessage(
content='you are a helpful assistant'
),
UserPromptMessage(
content=tool_parameters.get('query')
)
],
stream=True
)
for chunk in response:
if chunk.delta.message:
assert isinstance(chunk.delta.message.content, str)
yield self.create_text_message(text=chunk.delta.message.content)
Request Summary
You can request this endpoint to summarize a text. It will use the system model in your current workspace to summarize the text.
Entry:
self.session.model.summary
Endpoint:
text: The text to be summarized
instruction: Additional instructions you want to add, allowing you to stylize the summary
Note: The bytes stream returned by the TTS endpoint is an mp3 audio byte stream, with each iteration returning a complete audio. If you want to perform more in-depth processing tasks, please select an appropriate library.
