You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
gcgj-dify-1.7.0/api/core
QuantumGhost 349c3cf7b8
feat(api): Add image multimodal support for LLMNode (#17372)
Enhance `LLMNode` with multimodal capability, introducing support for
image outputs.

This implementation extracts base64-encoded images from LLM responses,
saves them to the storage service, and records the file metadata in the
`ToolFile` table. In conversations, these images are rendered as
markdown-based inline images.
Additionally, the images are included in the LLMNode's output as
file variables, enabling subsequent nodes in the workflow to utilize them.

To integrate file outputs into workflows, adjustments to the frontend code
are necessary.

For multimodal output functionality, updates to related model configurations
are required. Currently, this capability has been applied exclusively to
Google's Gemini models.

Close #15814.

Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
1 year ago
..
agent fix: metadata filter not work (#19020) 1 year ago
app feat(api): Add image multimodal support for LLMNode (#17372) 1 year ago
callback_handler fix retrival resource miss in chatflow (#18307) 1 year ago
entities fix: #18744 The model order defined in position.yaml in the Model Plugin is not taking effect. (#18756) 1 year ago
errors fix(core/errors): change base class of custom exceptions to ValueError (#11955) 1 year ago
extension feat: mypy for all type check (#10921) 1 year ago
external_data_tool Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 1 year ago
file feat(api): Add image multimodal support for LLMNode (#17372) 1 year ago
helper Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 1 year ago
llm_generator Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 1 year ago
memory fix: Update prompt message content types to use Literal and add union type for content (#17136) 1 year ago
model_runtime feat(api): Add image multimodal support for LLMNode (#17372) 1 year ago
moderation Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 1 year ago
ops refactor: Refactors repository imports structure (#18901) 1 year ago
plugin Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 1 year ago
prompt Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 1 year ago
rag immediately return initialed tiktokenizer instance and remove dead code in usage of tiktokenizer (#17957) 1 year ago
repositories refactor: Refactors repository imports structure (#18901) 1 year ago
tools feat(api): Add image multimodal support for LLMNode (#17372) 1 year ago
variables fix Unicode Escape Characters (#15318) 1 year ago
workflow feat(api): Add image multimodal support for LLMNode (#17372) 1 year ago
__init__.py
hosting_configuration.py chore(provider_manager): Update hosted model's name (#14334) 1 year ago
indexing_runner.py fix full-doc mode document doesn't reindex after enable or un_archive (#16737) 1 year ago
model_manager.py feat(api): Add image multimodal support for LLMNode (#17372) 1 year ago
provider_manager.py fix: perferred model provider not match with provider. (#18282) 1 year ago