You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
gcgj-dify-1.7.0/api/core
QuantumGhost 349c3cf7b8
feat(api): Add image multimodal support for LLMNode (#17372)
Enhance `LLMNode` with multimodal capability, introducing support for
image outputs.

This implementation extracts base64-encoded images from LLM responses,
saves them to the storage service, and records the file metadata in the
`ToolFile` table. In conversations, these images are rendered as
markdown-based inline images.
Additionally, the images are included in the LLMNode's output as
file variables, enabling subsequent nodes in the workflow to utilize them.

To integrate file outputs into workflows, adjustments to the frontend code
are necessary.

For multimodal output functionality, updates to related model configurations
are required. Currently, this capability has been applied exclusively to
Google's Gemini models.

Close #15814.

Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
9 months ago
..
agent fix: metadata filter not work (#19020) 9 months ago
app feat(api): Add image multimodal support for LLMNode (#17372) 9 months ago
callback_handler fix retrival resource miss in chatflow (#18307) 10 months ago
entities fix: #18744 The model order defined in position.yaml in the Model Plugin is not taking effect. (#18756) 10 months ago
errors fix(core/errors): change base class of custom exceptions to ValueError (#11955) 1 year ago
extension feat: mypy for all type check (#10921) 1 year ago
external_data_tool Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 9 months ago
file feat(api): Add image multimodal support for LLMNode (#17372) 9 months ago
helper Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 9 months ago
llm_generator Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 9 months ago
memory fix: Update prompt message content types to use Literal and add union type for content (#17136) 10 months ago
model_runtime feat(api): Add image multimodal support for LLMNode (#17372) 9 months ago
moderation Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 9 months ago
ops refactor: Refactors repository imports structure (#18901) 9 months ago
plugin Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 9 months ago
prompt Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 9 months ago
rag immediately return initialed tiktokenizer instance and remove dead code in usage of tiktokenizer (#17957) 9 months ago
repositories refactor: Refactors repository imports structure (#18901) 9 months ago
tools feat(api): Add image multimodal support for LLMNode (#17372) 9 months ago
variables fix Unicode Escape Characters (#15318) 11 months ago
workflow feat(api): Add image multimodal support for LLMNode (#17372) 9 months ago
__init__.py chore(api/core): apply ruff reformatting (#7624) 1 year ago
hosting_configuration.py chore(provider_manager): Update hosted model's name (#14334) 11 months ago
indexing_runner.py fix full-doc mode document doesn't reindex after enable or un_archive (#16737) 11 months ago
model_manager.py feat(api): Add image multimodal support for LLMNode (#17372) 9 months ago
provider_manager.py fix: perferred model provider not match with provider. (#18282) 10 months ago