In addition to the system’s built-in content moderation types, Dify also supports user-defined content moderation rules. This method is suitable for developers customizing their own private deployments. For instance, in an enterprise internal customer service setup, it may be required that users, while querying or customer service agents while responding, not only avoid entering words related to violence, sex, and illegal activities but also avoid specific terms forbidden by the enterprise or violating internally established moderation logic. Developers can extend custom content moderation rules at the code level in a private deployment of Dify.
Here is an example of extending a Cloud Service
content moderation type, with the steps as follows:
To add a custom type Cloud Service
, create the relevant directories and files under the api/core/moderation
directory.
schema.json
: This file defines the frontend component specifications.cloud_service.py
code template where you can implement specific business logic.
Note: The class variable name must be the same as the custom type name, matching the directory and file names, and must be unique.
At this point, you can select the custom Cloud Service
content moderation extension type for debugging in the Dify application orchestration interface.
The schema.json
form validation method is called when the user clicks “Publish” to save the configuration.
config
form parameters
{variable}
custom variable of the forminputs_config
input moderation preset response
enabled
whether it is enabledpreset_response
input preset responseoutputs_config
output moderation preset response
enabled
whether it is enabledpreset_response
output preset responseInput validation function
inputs
: values passed by the end userquery
: the current input content of the end user in a conversation, a fixed parameter for conversational applications.ModerationInputsResult
flagged
: whether it violates the moderation rulesaction
: action to be taken
direct_output
: directly output the preset responseoverridden
: override the passed variable valuespreset_response
: preset response (returned only when action=direct_output)inputs
: values passed by the end user, with key as the variable name and value as the variable value (returned only when action=overridden)query
: overridden current input content of the end user in a conversation, a fixed parameter for conversational applications (returned only when action=overridden)Output validation function
text
: content output by the modelmoderation_for_outputs
: output validation function
text
: content of the LLM response. When the LLM output is streamed, this is the content in segments of 100 characters.ModerationOutputsResult
flagged
: whether it violates the moderation rulesaction
: action to be taken
direct_output
: directly output the preset responseoverridden
: override the passed variable valuespreset_response
: preset response (returned only when action=direct_output)text
: overridden content of the LLM response (returned only when action=overridden).Edit this page | Report an issue
In addition to the system’s built-in content moderation types, Dify also supports user-defined content moderation rules. This method is suitable for developers customizing their own private deployments. For instance, in an enterprise internal customer service setup, it may be required that users, while querying or customer service agents while responding, not only avoid entering words related to violence, sex, and illegal activities but also avoid specific terms forbidden by the enterprise or violating internally established moderation logic. Developers can extend custom content moderation rules at the code level in a private deployment of Dify.
Here is an example of extending a Cloud Service
content moderation type, with the steps as follows:
To add a custom type Cloud Service
, create the relevant directories and files under the api/core/moderation
directory.
schema.json
: This file defines the frontend component specifications.cloud_service.py
code template where you can implement specific business logic.
Note: The class variable name must be the same as the custom type name, matching the directory and file names, and must be unique.
At this point, you can select the custom Cloud Service
content moderation extension type for debugging in the Dify application orchestration interface.
The schema.json
form validation method is called when the user clicks “Publish” to save the configuration.
config
form parameters
{variable}
custom variable of the forminputs_config
input moderation preset response
enabled
whether it is enabledpreset_response
input preset responseoutputs_config
output moderation preset response
enabled
whether it is enabledpreset_response
output preset responseInput validation function
inputs
: values passed by the end userquery
: the current input content of the end user in a conversation, a fixed parameter for conversational applications.ModerationInputsResult
flagged
: whether it violates the moderation rulesaction
: action to be taken
direct_output
: directly output the preset responseoverridden
: override the passed variable valuespreset_response
: preset response (returned only when action=direct_output)inputs
: values passed by the end user, with key as the variable name and value as the variable value (returned only when action=overridden)query
: overridden current input content of the end user in a conversation, a fixed parameter for conversational applications (returned only when action=overridden)Output validation function
text
: content output by the modelmoderation_for_outputs
: output validation function
text
: content of the LLM response. When the LLM output is streamed, this is the content in segments of 100 characters.ModerationOutputsResult
flagged
: whether it violates the moderation rulesaction
: action to be taken
direct_output
: directly output the preset responseoverridden
: override the passed variable valuespreset_response
: preset response (returned only when action=direct_output)text
: overridden content of the LLM response (returned only when action=overridden).Edit this page | Report an issue