Commit Graph

6512 Commits (37c534df212eefc0e6717c81635ef485478ca4b6)
 

Author SHA1 Message Date
jiangbo721 c70b0cb730
fix(docs): unify workflow_run_id style with other languages (#22642)
Co-authored-by: 刘江波 <jiangbo721@163.com>
11 months ago
yunqiqiliang b5a3f1d5e0 Fix remaining Python style and linting issues
- Fix line length violation in middleware config description
- Fix RUF013 type annotation to use union syntax
- Complete all Python style and linting fixes for CI checks
- Resolve formatter and linter warnings

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang c3851595d0 Fix MyPy type checking errors in ClickZetta vector implementation
- Add proper type annotations for Connection from clickzetta module
- Implement _ensure_connection() method to handle None connection checks
- Fix all database cursor access patterns to use proper null checking
- Add type annotation for result queue in _execute_write method
- Resolve factory method configuration issues with None value handling

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang f57fa13f1b fix: resolve CI linting issues and add missing newlines
- Fix all line length issues (120 character limit)
- Remove all trailing whitespace
- Add missing newlines at end of files
- Add CLICKZETTA_VOLUME_DIFY_PREFIX environment variable to docker-compose.yaml
- Ensure proper code formatting for all ClickZetta files

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang f3b1bdc04f fix: update .env.example configuration order and defaults
- Change default STORAGE_TYPE from 'local' to 'opendal' for consistency
- Move ClickZetta Volume configuration before S3 configuration for better organization
- Maintain clear grouping of storage configurations

🚀 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang 72dc2cce35 feat: integrate ClickZetta Volume storage with Vector DB
Merge ClickZetta Volume storage functionality into the ClickZetta Vector DB feature branch.

🎯 Combined Features:
- ClickZetta Vector Database integration
- ClickZetta Volume Storage (User/Table/External Volume types)
- Unified ClickZetta configuration and connection management
- Complete file operations with Volume storage backend
- Docker compose integration for both vector and storage features

🔧 Configuration Updates:
- Removed CLICKZETTA_VOLUME_PERMISSION_CHECK (disabled by default)
- Set default CLICKZETTA_VOLUME_TYPE to 'user' for better UX
- Use official Docker images for better compatibility
- Clean integration with existing ClickZetta vector configuration

📦 Volume Storage Features:
- Three volume types: User, Table, External Volume
- Complete file lifecycle management
- Configuration fallback to vector DB settings
- Comprehensive error handling and logging
- Integration tests for storage functionality

This creates a comprehensive ClickZetta integration supporting both vector database and file storage capabilities.

🚀 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang 2de316c557 feat: add ClickZetta Volume storage support
- Add three volume types: User, Table, and External Volume
- Complete file operations: upload, download, delete, list, stream
- Intelligent configuration management with fallback to vector DB settings
- Simplified user experience with 'user' as default volume type
- Comprehensive error handling and logging
- Docker integration with updated compose files
- Integration tests for all volume types
- Disabled complex permission checking for stability

🎯 Features:
- User Volume: Personal/small team use, simple configuration
- Table Volume: Enterprise multi-tenant with smart routing
- External Volume: Data lake integration with external storage
- Flexible configuration with environment variable support
- Complete file lifecycle management

🔧 Technical:
- Reuses existing ClickZetta connection configuration
- Pydantic-based configuration validation
- Comprehensive error handling and logging
- Performance-optimized with connection reuse
- Clean integration with Dify's storage architecture

🚀 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
Wu Tianwei dba42567b1
fix: restore globals dependency in package.json and pnpm-lock.yaml (#22625) 11 months ago
birkhoff 71d96b671b
feat: update VECTOR_STORE supported types in api/.env.example (#22617)
Co-authored-by: nicksarno <nicksarno@ztm-tech.com>
11 months ago
kurokobo a93db6d797
fix: correct tracing for workflows and chatflows for phoenix (#22547) 11 months ago
Jason Young f2389771cf
test: add comprehensive API key authentication service tests (#22572) 11 months ago
znn ed263aed9f
fix text splitter (#22596) 11 months ago
mudiko d37b6716cd
Fix #22508 (#22590) 11 months ago
Wu Tianwei b035f3f884
feat: convert components to dynamic imports for improved performance (#22614) 11 months ago
yunqiqiliang cb023189a9 fix: update docker-compose.yaml with correct Clickzetta defaults
- Update CLICKZETTA_SERVICE default to api.clickzetta.com
- Update CLICKZETTA_WORKSPACE default to quick_start
- Ensure consistency between docker/.env.example and docker-compose.yaml

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang 54cdfb6593 chore: update CLICKZETTA_WORKSPACE default value to quick_start
- Set default workspace to quick_start for better user experience
- Update both docker/.env.example and docker/docker-compose.yaml

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang ecbe555cb0 refactor: remove clickzetta/ folder and update service endpoint
- Remove clickzetta/ development folder from PR (add to .gitignore)
- Update CLICKZETTA_SERVICE from uat-api.clickzetta.com to api.clickzetta.com
- Update both docker/.env.example and docker/docker-compose.yaml for consistency

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
-LAN- 1f9cd99bc2
refactor: elegant event dispatch patterns (92% complexity reduction) (#22600)
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Claude <noreply@anthropic.com>
11 months ago
jiangbo721 ffee6f3288
fix: admin feedback uses the same method create_feedback (#22580)
Co-authored-by: 刘江波 <jiangbo721@163.com>
11 months ago
-LAN- 460a825ef1
refactor: decouple Node and NodeData (#22581)
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
11 months ago
github-actions[bot] 54c56f2d05
chore: translate i18n files (#22563)
Co-authored-by: Yeuoly <45712896+Yeuoly@users.noreply.github.com>
11 months ago
kenwoodjw 61a5741c05
fix celery config (#22566)
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
11 months ago
KVOJJJin 0e235e5872
Fix valid check of mcp server address (#22510) 11 months ago
kawabata-mcl d0bece1679
fix(docs): correct workflow API parameter name from workflow_id to workflow_run_id (#22587) 11 months ago
-LAN- 1715dd4320
refactor: Fix some type error (#22594)
Signed-off-by: -LAN- <laipz8200@outlook.com>
11 months ago
Asuka Minato 14513b7677
use nolyfill to reduce download size (#22589) 11 months ago
-LAN- b88dd17fc1
feat(workflow_cycle_manager): Removes redundant repository methods and adds caching (#22597)
Signed-off-by: -LAN- <laipz8200@outlook.com>
11 months ago
znn 3826b57424
remove node-info for non mcp (#22595) 11 months ago
hato_code 62586719b3
fix: remove redundant partial member list retrieval in dataset API (#15492) 11 months ago
helojo e7d80bf7bf
Fix: the pict type picture was not processed in the docx (#19305)
Co-authored-by: zqgame <zqgame@zqgame.local>
11 months ago
yunqiqiliang 0246f39564 CRITICAL FIX: Correct Docker image versions and optimize build process
- Add v1.6.0 version tag for stable releases
- Use correct official Dify web image: langgenius/dify-web:1.6.0
- Remove unnecessary Web image build (only API changes needed for ClickZetta)
- Update all documentation with accurate version numbers
- Ensure build script only creates required ClickZetta-specific images

This ensures users get the correct, tested versions and prevents version confusion.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang ee6ce138eb Add Docker Hub multi-architecture build infrastructure for community testing
- Add build-and-push-multiarch.sh script for creating Docker images
- Include ready-to-use docker-compose.clickzetta.yml for users
- Provide .env.clickzetta.example environment template
- Add comprehensive README.clickzetta.md user guide
- Update maintainer response with Docker Hub image information

This allows community testing before PR merge while maintaining security.
Images available at czqiliang/dify-clickzetta-{api,web}:latest

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang 8e707cace9 Fix recall testing and search functionality for ClickZetta integration
- Fix double JSON encoding issue in metadata parsing for all search methods
- Remove unnecessary dataset_id filters since each dataset has its own table
- Add robust metadata parsing with fallback for JSON decode errors
- Ensure document_id is always present for Dify's format_retrieval_documents
- Clean up debug logging while preserving essential error logs
- Support vector search, full-text search, and hybrid search in recall testing

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
Kalo Chin 7a69b57823
Fix jinja2 variable naming inconsistencies (#22578) 11 months ago
Asuka Minato 2423f97c72
remove overrides (#22575) 11 months ago
yunqiqiliang fcf8387f52 Fix SQL statement length issues and improve batch processing
- Add SQL length monitoring and automatic batch splitting
- Reduce default batch size from 100 to 20 to prevent large SQL statements
- Add detailed error logging for SQL execution failures
- Implement recursive batch splitting for oversized SQL statements
- Set 1MB limit for SQL statement length

This resolves issues where large batches create SQL statements that
exceed database limits, causing vector insertion failures.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang 8dea8766e9 Fix document content special characters causing SQL syntax errors
- Add specialized document content cleaning function
- Handle backticks, quotes, newlines, and control characters properly
- Replace problematic characters instead of just escaping them
- Normalize whitespace and remove control characters
- Fix "Syntax error at or near" issues from document content like shell commands

This resolves SQL syntax errors when documents contain shell scripts,
code snippets, or other text with special formatting characters.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang f116407045 Fix SQL injection vulnerabilities and character encoding issues
- Enhanced string escaping for SQL safety (backslashes, newlines, tabs)
- Added safe JSON formatting with ensure_ascii=True
- Implemented safe doc_id validation (alphanumeric + hyphens/underscores only)
- Protected all user input: document content, metadata, IDs, search queries
- Fixed potential SQL syntax errors from special characters in document content

This addresses "Syntax error at or near 'files'" errors that occur when
document content or metadata contains special characters that break SQL syntax.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang 9c2bf2b30f Fix SQL syntax errors with vector formatting
- Add safe vector formatting function to handle special float values
- Handle NaN, infinity values in vector embeddings
- Prevent SQL syntax errors from malformed VECTOR() statements
- Use consistent vector formatting across all SQL operations

This fixes "Syntax error at or near '{'" errors that occur when
vector embeddings contain special float values during knowledge
base construction.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang 1b7603deb1 Fix inverted index duplicate creation issue
- Add table existence check before creating indexes
- Improve error handling for ClickZetta specific error messages
- Remove duplicate _table_exists method definition
- Prevent high-frequency index creation attempts during bulk operations

This fixes the "already has index with the same type" errors during
large knowledge base construction with 600+ documents.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
Maries a4ef900916
Support OAuth Integration for Plugin Tools (#22550)
Co-authored-by: zxhlyh <jasonapring2015@outlook.com>
Co-authored-by: Yeuoly <admin@srmxy.cn>
11 months ago
yunqiqiliang 87df741069 Fix final Python style issues in Docker integration test
- Remove unused json import
- Fix import sorting with ruff auto-fix
- All Clickzetta files now pass style checks

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang c4d9e5c69f Add documentation section and clean up formatting
- Add clear section header for ClickZetta configuration
- Improve code organization and readability
- All lint checks should pass with latest fixes

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang 1b9f4dff71 Trigger CI after enabling GitHub Actions
Fork repository Actions now enabled - all lint fixes should be validated:
- Fixed unused imports in main vector file and test file
- Fixed import sorting issues
- Confirmed architecture compliance (all code in api/ directory)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang ca95158a6d Trigger CI: fix Python style issues
All lint issues should now be resolved:
- Removed unused imports from main vector file
- Removed unused typing.cast from test file
- Architecture compliance confirmed

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang a539c31b15 Fix Python style issue: remove unused typing.cast import
- Remove unused `typing.cast` import from test_clickzetta.py
- This resolves the Python Style CI check failure

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang b5157758cf Add maintainer response highlighting CI fixes and progress
- Document all resolved lint and style issues
- Report passing CI checks (Python Style, SuperLinter, Docker Compose)
- Emphasize architecture compliance (all code in api/ directory)
- Acknowledge remaining API test investigation
- Professional response showing responsiveness and progress

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang 91f4d58843 Add maintainer update: CI checks successfully fixed
- Document that lint errors and code style issues are resolved
- All required CI checks now passing (Docker Compose, SuperLinter, Python Style)
- API and VDB tests are running
- Confirm all code is within api/ directory as requested

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang 1e4c749df7 Add CI fixes summary and next steps guide
- Document fixed Python style issues
- Outline remaining CI checks to monitor
- Provide troubleshooting guide for potential failures
- Include testing strategies and next steps

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang ed139a49a3 Fix code style issues for CI checks
- Remove unused imports (time, VectorType)
- Replace logger.error with logger.exception for exception handling
- Remove redundant exception objects from logging.exception calls
- Ensure all Python style checks pass

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago