Commit Graph

2544 Commits (db91643915a226da3d4a31aebb1433906f050eef)

Author SHA1 Message Date
xuzijie1995 db91643915 refactor(workflow): Rearchitect stream dependency logic for complex graphs
This commit addresses a critical issue where streaming output would fail in workflows with complex topologies, particularly those involving multiple conditional branches (if/else) that converge on a common node before the LLM and Answer nodes.

The root cause was twofold:
1. A bug in the branch pruning logic () that would incorrectly remove shared downstream nodes, leading to a premature emptying of the  list.
2. A flawed static dependency analysis () that could not correctly resolve dependencies for nodes that were part of multiple, mutually exclusive execution paths.

This refactor introduces a new, robust architecture for streaming dependency management based on the principle of "Static Pre-pruning + Dynamic Adjudication":

- **Fix**: The branch pruning logic in  is now non-recursive and conservative. It only prunes the immediate first node of an unreachable branch, preserving the integrity of shared downstream paths and join points.

- **Refactor**: The old static dependency analysis has been completely removed. This includes deleting the  attribute from the  entity and deleting the associated recursive dependency fetching methods (, ).

- **Feat**: A new method, , has been implemented in . This method performs a real-time, backward traversal of the graph from the streaming node, querying the runtime execution state () to dynamically validate if the *actual* dependency path has been successfully completed. This ensures that streaming decisions are based on the ground truth of the current execution, not a flawed static prediction.

- **Doc**: Added comprehensive docstrings and comments to the modified components to explain the new architecture and the rationale behind the changes.
10 months ago
xuzijie1995 b13ae784a4 fix(workflow): Resolve streaming failure on conditional join points
Improves the robustness of the workflow engine's streaming output by fixing two core issues that caused streaming to fail in complex topologies where multiple conditional branches merge.

**1. Corrected Runtime State Management ("Pruning"):**

The primary bug was located in the `_remove_unreachable_nodes` method. Its aggressive recursive "pruning" algorithm incorrectly removed shared downstream nodes (including LLM and Answer) when handling conditional branches that led to a join point. This prematurely emptied the `rest_node_ids` list, causing the stream processor to fail its initial state check.

The fix replaces the recursive logic with a more conservative, non-recursive approach that only prunes the immediate first node of an unreachable branch. This ensures the integrity of the `rest_node_ids` list throughout the workflow execution.

**2. Improved Static Dependency Analysis:**

A secondary, underlying issue was found in the static dependency analysis (`_recursive_fetch_answer_dependencies`). It incorrectly identified all upstream, mutually exclusive `If/Else` nodes as parallel dependencies of the Answer node.

The fix enhances this analysis by adding "join point awareness". The upward trace now stops when it encounters a node with more than one incoming edge, correctly identifying the join point itself as the dependency rather than its upstream branches.

Together, these changes ensure that streaming output remains reliable and predictable, even in complex workflows with reusable, multi-input nodes.
10 months ago
Bowen Liang 74940ad3f2
chore: code improvement for mcp_client and mcp_tools_manage_service (#22645) 10 months ago
Novice 17a8f1a0f1
fix: avoid using node_data.version for judgement tool node version (#22462)
Co-authored-by: JzoNg <jzongcode@gmail.com>
10 months ago
Aryan Raj ce794335e9
Fix/replace datetime patterns with naive utc now (#22654) 10 months ago
kurokobo a93db6d797
fix: correct tracing for workflows and chatflows for phoenix (#22547) 10 months ago
znn ed263aed9f
fix text splitter (#22596) 10 months ago
-LAN- 1f9cd99bc2
refactor: elegant event dispatch patterns (92% complexity reduction) (#22600)
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Claude <noreply@anthropic.com>
10 months ago
-LAN- 460a825ef1
refactor: decouple Node and NodeData (#22581)
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
10 months ago
-LAN- 1715dd4320
refactor: Fix some type error (#22594)
Signed-off-by: -LAN- <laipz8200@outlook.com>
10 months ago
-LAN- b88dd17fc1
feat(workflow_cycle_manager): Removes redundant repository methods and adds caching (#22597)
Signed-off-by: -LAN- <laipz8200@outlook.com>
10 months ago
helojo e7d80bf7bf
Fix: the pict type picture was not processed in the docx (#19305)
Co-authored-by: zqgame <zqgame@zqgame.local>
10 months ago
Kalo Chin 7a69b57823
Fix jinja2 variable naming inconsistencies (#22578) 10 months ago
Maries a4ef900916
Support OAuth Integration for Plugin Tools (#22550)
Co-authored-by: zxhlyh <jasonapring2015@outlook.com>
Co-authored-by: Yeuoly <admin@srmxy.cn>
10 months ago
Stream 4b2baeea65
fix: use model provided by user in prompt generator (#22541) (#22542)
Co-authored-by: stream <stream@dify.ai>
10 months ago
quicksand 10e6b11ff6
fix: code node check decimal precision (#22522) 10 months ago
NeatGuyCoding c3037c5491
minor code fix: remove duplicate type check branch (#22536) 10 months ago
jiangbo721 fb5c6dd644
chore: remove unused code (#22501)
Co-authored-by: 刘江波 <jiangbo721@163.com>
10 months ago
yihong d2933c2bfe
fix: drop dead code phase2 unused class (#22042)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
10 months ago
kenwoodjw 3587bd4040
fix mcp error not input (#22505)
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
10 months ago
Chieh Wang 30aa052a57
feat: Add Citations and Attributions to Agent Node (#18558)
Co-authored-by: oneness0 <2902216407@qq.com>
Co-authored-by: Novice <novice12185727@gmail.com>
10 months ago
heyszt 2c9ee54a16
fix aliyun trace session_id (#22468) 10 months ago
QuantumGhost 2c1ab4879f
refactor(api): Separate SegmentType for Integer/Float to Enable Pydantic Serialization (#22025)
refactor(api): Separate SegmentType for Integer/Float to Enable Pydantic Serialization (#22025)

This PR addresses serialization issues in the VariablePool model by separating the `value_type` tags for `IntegerSegment`/`FloatSegment` and `IntegerVariable`/`FloatVariable`. Previously, both Integer and Float types shared the same `SegmentType.NUMBER` tag, causing conflicts during serialization.

Key changes:
- Introduce distinct `value_type` tags for Integer and Float segments/variables
- Add `VariableUnion` and `SegmentUnion` types for proper type discrimination
- Leverage Pydantic's discriminated union feature for seamless serialization/deserialization
- Enable accurate serialization of data structures containing these types

Closes #22024.
10 months ago
NeatGuyCoding bf542233a9
minor fix: using Pydantic model_validate instead of deprecated parse_obj (#22239)
Signed-off-by: neatguycoding <15627489+NeatGuyCoding@users.noreply.github.com>
10 months ago
NeatGuyCoding 7bf3d2c8bf
fix(api): Fix potential thread leak in MCP `BaseSession` (#22169)
The `BaseSession` class in the `core/mcp/session` package uses `ThreadPoolExecutor` 
to run the receive loop but fails to properly clean up the executor and receiver 
future, leading to potential thread leaks.

This PR addresses this issue by:
- Initializing `_executor` and `_receiver_future` attributes to `None` for proper cleanup checks
- Adding graceful shutdown with a 5-second timeout in the `__exit__` method
- Ensuring the ThreadPoolExecutor is properly shut down to prevent resource leaks

This fix prevents memory leaks and hanging threads in long-running scenarios where 
multiple MCP sessions are created and destroyed.

Signed-off-by: neatguycoding <15627489+NeatGuyCoding@users.noreply.github.com>
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
10 months ago
Hao Cheng 32c541a9ed
fix: generate deterministic operationId for root endpoints without one (#19888) 10 months ago
quicksand 5247c19498
fix: code result included "error" field (#22392) 10 months ago
湛露先生 9823edd3a2
fix workflow node iterator . (#21008)
Signed-off-by: zhanluxianshen <zhanluxianshen@163.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
10 months ago
homejim 8e910d8c59
fix(plugin): introduce response_type parameter in plugin list API to enable paginated response support (#22251) 10 months ago
wanttobeamaster bf7b2c339b
tablestore vector support more method (#22225)
Co-authored-by: xiaozhiqing.xzq <xiaozhiqing.xzq@alibaba-inc.com>
10 months ago
quicksand ebb88bbe0b
improve opik workflow_trace span name to node name (#22356) 10 months ago
Jacky Wu 3e96c0c468
fix: close session before doing long latency operation (#22306) 10 months ago
-LAN- 6eb155ae69
feat(api/repo): Allow to config repository implementation (#21458)
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
10 months ago
Novice 1d54ffcf89
fix: error parsing object type parameters for code node (#22230) 10 months ago
K d5624ba671
fix: resolve Docker file URL networking issue for plugins (#21334) (#21382)
Co-authored-by: crazywoola <427733928@qq.com>
10 months ago
诗浓 e576b989b8
feat(tool): add support for API key authentication via query parameter (#21656) 10 months ago
NeatGuyCoding f929bfb94c
minor fix: remove duplicates, fix typo, and add restriction for get mcp server (#22170)
Signed-off-by: neatguycoding <15627489+NeatGuyCoding@users.noreply.github.com>
10 months ago
诗浓 f4df80e093
fix(custom_tool): omit optional parameters instead of setting them to None (#22171) 10 months ago
Novice 0e793a660d
fix: add the default value to the dark icon (#22149) 10 months ago
非法操作 7b2cab5767
feat: support ping method for MCP server (#22144) 10 months ago
Yeuoly 94a13d7d62
feat: add support for dark icons in provider and tool entities (#22081) 10 months ago
湛露先生 edf5fd28c9
update worklow events logs. (#19871)
Signed-off-by: zhanluxianshen <zhanluxianshen@163.com>
10 months ago
Novice 535fff62f3
feat: add MCP support (#20716)
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
10 months ago
baonudesifeizhai 18b58424ec
Fix: Resolve issue with json_output (#22053) 10 months ago
luckylhb90 a371390d6c
optimize: batch embedding and qdrant write_consistency_factor parameter (#21776)
Co-authored-by: hobo.l <hobo.l@binance.com>
10 months ago
-LAN- 4cb50f1809
feat(libs): Introduce `extract_tenant_id` (#22086)
Signed-off-by: -LAN- <laipz8200@outlook.com>
10 months ago
wlleiiwang 89b52471fb
Optimize the memory usage of Tencent Vector Database (#22079)
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
10 months ago
Yongtao Huang 521488f926
Remove tow unused files (#22022) 10 months ago
Davide Delbianco 816210d744
Expose LLM usage in workflows (#21766)
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
10 months ago
NeatGuyCoding 2ffbf5435d
minro fix: fix duplicate local import of ToolProviderType (#22013)
Signed-off-by: neatguycoding <15627489+NeatGuyCoding@users.noreply.github.com>
10 months ago