Commit Graph

3870 Commits (db91643915a226da3d4a31aebb1433906f050eef)

Author SHA1 Message Date
xuzijie1995 db91643915 refactor(workflow): Rearchitect stream dependency logic for complex graphs
This commit addresses a critical issue where streaming output would fail in workflows with complex topologies, particularly those involving multiple conditional branches (if/else) that converge on a common node before the LLM and Answer nodes.

The root cause was twofold:
1. A bug in the branch pruning logic () that would incorrectly remove shared downstream nodes, leading to a premature emptying of the  list.
2. A flawed static dependency analysis () that could not correctly resolve dependencies for nodes that were part of multiple, mutually exclusive execution paths.

This refactor introduces a new, robust architecture for streaming dependency management based on the principle of "Static Pre-pruning + Dynamic Adjudication":

- **Fix**: The branch pruning logic in  is now non-recursive and conservative. It only prunes the immediate first node of an unreachable branch, preserving the integrity of shared downstream paths and join points.

- **Refactor**: The old static dependency analysis has been completely removed. This includes deleting the  attribute from the  entity and deleting the associated recursive dependency fetching methods (, ).

- **Feat**: A new method, , has been implemented in . This method performs a real-time, backward traversal of the graph from the streaming node, querying the runtime execution state () to dynamically validate if the *actual* dependency path has been successfully completed. This ensures that streaming decisions are based on the ground truth of the current execution, not a flawed static prediction.

- **Doc**: Added comprehensive docstrings and comments to the modified components to explain the new architecture and the rationale behind the changes.
9 months ago
xuzijie1995 b13ae784a4 fix(workflow): Resolve streaming failure on conditional join points
Improves the robustness of the workflow engine's streaming output by fixing two core issues that caused streaming to fail in complex topologies where multiple conditional branches merge.

**1. Corrected Runtime State Management ("Pruning"):**

The primary bug was located in the `_remove_unreachable_nodes` method. Its aggressive recursive "pruning" algorithm incorrectly removed shared downstream nodes (including LLM and Answer) when handling conditional branches that led to a join point. This prematurely emptied the `rest_node_ids` list, causing the stream processor to fail its initial state check.

The fix replaces the recursive logic with a more conservative, non-recursive approach that only prunes the immediate first node of an unreachable branch. This ensures the integrity of the `rest_node_ids` list throughout the workflow execution.

**2. Improved Static Dependency Analysis:**

A secondary, underlying issue was found in the static dependency analysis (`_recursive_fetch_answer_dependencies`). It incorrectly identified all upstream, mutually exclusive `If/Else` nodes as parallel dependencies of the Answer node.

The fix enhances this analysis by adding "join point awareness". The upward trace now stops when it encounters a node with more than one incoming edge, correctly identifying the join point itself as the dependency rather than its upstream branches.

Together, these changes ensure that streaming output remains reliable and predictable, even in complex workflows with reusable, multi-input nodes.
9 months ago
Kushagra Singhal 383a79772c
Increased the character limitation (#22679)
Co-authored-by: crazywoola <427733928@qq.com>
9 months ago
Bowen Liang 74940ad3f2
chore: code improvement for mcp_client and mcp_tools_manage_service (#22645) 9 months ago
Novice 17a8f1a0f1
fix: avoid using node_data.version for judgement tool node version (#22462)
Co-authored-by: JzoNg <jzongcode@gmail.com>
9 months ago
znn 19c09d6111
enabling vector index prefix name via configuration files (#22661) 9 months ago
Jason Young 6248658c04
fix: resolve Redis mock import error in test configuration (#22663) 9 months ago
Jason Young 274142c4c2
test: add comprehensive unit tests for auth service module (#22662) 9 months ago
Aryan Raj ce794335e9
Fix/replace datetime patterns with naive utc now (#22654) 9 months ago
birkhoff 71d96b671b
feat: update VECTOR_STORE supported types in api/.env.example (#22617)
Co-authored-by: nicksarno <nicksarno@ztm-tech.com>
9 months ago
kurokobo a93db6d797
fix: correct tracing for workflows and chatflows for phoenix (#22547) 9 months ago
Jason Young f2389771cf
test: add comprehensive API key authentication service tests (#22572) 9 months ago
znn ed263aed9f
fix text splitter (#22596) 9 months ago
-LAN- 1f9cd99bc2
refactor: elegant event dispatch patterns (92% complexity reduction) (#22600)
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Claude <noreply@anthropic.com>
9 months ago
jiangbo721 ffee6f3288
fix: admin feedback uses the same method create_feedback (#22580)
Co-authored-by: 刘江波 <jiangbo721@163.com>
9 months ago
-LAN- 460a825ef1
refactor: decouple Node and NodeData (#22581)
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
9 months ago
kenwoodjw 61a5741c05
fix celery config (#22566)
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
9 months ago
-LAN- 1715dd4320
refactor: Fix some type error (#22594)
Signed-off-by: -LAN- <laipz8200@outlook.com>
9 months ago
-LAN- b88dd17fc1
feat(workflow_cycle_manager): Removes redundant repository methods and adds caching (#22597)
Signed-off-by: -LAN- <laipz8200@outlook.com>
9 months ago
hato_code 62586719b3
fix: remove redundant partial member list retrieval in dataset API (#15492) 9 months ago
helojo e7d80bf7bf
Fix: the pict type picture was not processed in the docx (#19305)
Co-authored-by: zqgame <zqgame@zqgame.local>
9 months ago
Kalo Chin 7a69b57823
Fix jinja2 variable naming inconsistencies (#22578) 9 months ago
Maries a4ef900916
Support OAuth Integration for Plugin Tools (#22550)
Co-authored-by: zxhlyh <jasonapring2015@outlook.com>
Co-authored-by: Yeuoly <admin@srmxy.cn>
9 months ago
Jason Young 74caebac32
test: add comprehensive OAuth authentication unit tests (#22528) 9 months ago
Stream 4b2baeea65
fix: use model provided by user in prompt generator (#22541) (#22542)
Co-authored-by: stream <stream@dify.ai>
9 months ago
NeatGuyCoding 93c27b134d
minor typo fix: remove debug code and fix typo (#22539) 9 months ago
NeatGuyCoding 853c97a910
minor bug fix: wrong default metrics endpoint (#22535) 9 months ago
quicksand 10e6b11ff6
fix: code node check decimal precision (#22522) 9 months ago
NeatGuyCoding c3037c5491
minor code fix: remove duplicate type check branch (#22536) 9 months ago
quicksand e4ae1e2b94
fix (ci) : remove test_url_signer (#22525) 9 months ago
zyssyz123 a4f421028c
Feat/change user email (#22213)
Co-authored-by: NFish <douxc512@gmail.com>
Co-authored-by: JzoNg <jzongcode@gmail.com>
Co-authored-by: Garfield Dai <dai.hai@foxmail.com>
9 months ago
jiangbo721 fb5c6dd644
chore: remove unused code (#22501)
Co-authored-by: 刘江波 <jiangbo721@163.com>
9 months ago
yihong d2933c2bfe
fix: drop dead code phase2 unused class (#22042)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
9 months ago
kenwoodjw 3587bd4040
fix mcp error not input (#22505)
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
9 months ago
heyszt f082452c9b
feat: add otel endpoint config (#22492) 9 months ago
Chieh Wang 30aa052a57
feat: Add Citations and Attributions to Agent Node (#18558)
Co-authored-by: oneness0 <2902216407@qq.com>
Co-authored-by: Novice <novice12185727@gmail.com>
9 months ago
qfl bdb9f29948
feat(app): support custom max_active_requests per app (#22073) 9 months ago
baonudesifeizhai d52fb18457
feat: auto-fill MCP server description with app description #22443 (#22477) 9 months ago
heyszt 2c9ee54a16
fix aliyun trace session_id (#22468) 9 months ago
QuantumGhost ddfd8c8525
feat(api): add UUIDv7 implementation in SQL and Python (#22058)
This PR introduces UUIDv7 implementations in both Python and SQL to establish the foundation for migrating from UUIDv4 to UUIDv7 as proposed in #19754.

ID generation algorithm of existing models are not changed, and new models should use UUIDv7 for ID generation.

Close #19754.
9 months ago
QuantumGhost 2c1ab4879f
refactor(api): Separate SegmentType for Integer/Float to Enable Pydantic Serialization (#22025)
refactor(api): Separate SegmentType for Integer/Float to Enable Pydantic Serialization (#22025)

This PR addresses serialization issues in the VariablePool model by separating the `value_type` tags for `IntegerSegment`/`FloatSegment` and `IntegerVariable`/`FloatVariable`. Previously, both Integer and Float types shared the same `SegmentType.NUMBER` tag, causing conflicts during serialization.

Key changes:
- Introduce distinct `value_type` tags for Integer and Float segments/variables
- Add `VariableUnion` and `SegmentUnion` types for proper type discrimination
- Leverage Pydantic's discriminated union feature for seamless serialization/deserialization
- Enable accurate serialization of data structures containing these types

Closes #22024.
9 months ago
NeatGuyCoding bf542233a9
minor fix: using Pydantic model_validate instead of deprecated parse_obj (#22239)
Signed-off-by: neatguycoding <15627489+NeatGuyCoding@users.noreply.github.com>
9 months ago
Jason Young 38106074b4
test: add comprehensive unit tests for console authentication and authorization decorators (#22439) 9 months ago
NeatGuyCoding 7bf3d2c8bf
fix(api): Fix potential thread leak in MCP `BaseSession` (#22169)
The `BaseSession` class in the `core/mcp/session` package uses `ThreadPoolExecutor` 
to run the receive loop but fails to properly clean up the executor and receiver 
future, leading to potential thread leaks.

This PR addresses this issue by:
- Initializing `_executor` and `_receiver_future` attributes to `None` for proper cleanup checks
- Adding graceful shutdown with a 5-second timeout in the `__exit__` method
- Ensuring the ThreadPoolExecutor is properly shut down to prevent resource leaks

This fix prevents memory leaks and hanging threads in long-running scenarios where 
multiple MCP sessions are created and destroyed.

Signed-off-by: neatguycoding <15627489+NeatGuyCoding@users.noreply.github.com>
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
9 months ago
Jacky Wu da53bf511f
chore: add SQLALCHEMY_POOL_USE_LIFO option and missing SQLALCHEMY_POOL_PRE_PING env default value. (#22371) 9 months ago
quicksand 14f79ee652
fix: create api workflow run repository error (#22422) 9 months ago
-LAN- df89629e04
fix: conversatino statistic including data from debugger (#22412)
Signed-off-by: -LAN- <laipz8200@outlook.com>
9 months ago
Hao Cheng 32c541a9ed
fix: generate deterministic operationId for root endpoints without one (#19888) 9 months ago
quicksand 5247c19498
fix: code result included "error" field (#22392) 9 months ago
湛露先生 9823edd3a2
fix workflow node iterator . (#21008)
Signed-off-by: zhanluxianshen <zhanluxianshen@163.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
9 months ago