gcgj-dify-1.7.0

Commit Graph

Author	SHA1	Message	Date
yunqiqiliang	18230d12f9	Auto-format: Fix code style for CI compliance 🤖 Automated formatting applied by CI test script - Ensures 100% compliance with Python style guidelines - No functional changes, only formatting improvements Generated by: run_complete_ci_test.sh	11 months ago
yunqiqiliang	c3851595d0	Fix MyPy type checking errors in ClickZetta vector implementation - Add proper type annotations for Connection from clickzetta module - Implement _ensure_connection() method to handle None connection checks - Fix all database cursor access patterns to use proper null checking - Add type annotation for result queue in _execute_write method - Resolve factory method configuration issues with None value handling 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	11 months ago
yunqiqiliang	f57fa13f1b	fix: resolve CI linting issues and add missing newlines - Fix all line length issues (120 character limit) - Remove all trailing whitespace - Add missing newlines at end of files - Add CLICKZETTA_VOLUME_DIFY_PREFIX environment variable to docker-compose.yaml - Ensure proper code formatting for all ClickZetta files 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	11 months ago
yunqiqiliang	2de316c557	feat: add ClickZetta Volume storage support - Add three volume types: User, Table, and External Volume - Complete file operations: upload, download, delete, list, stream - Intelligent configuration management with fallback to vector DB settings - Simplified user experience with 'user' as default volume type - Comprehensive error handling and logging - Docker integration with updated compose files - Integration tests for all volume types - Disabled complex permission checking for stability 🎯 Features: - User Volume: Personal/small team use, simple configuration - Table Volume: Enterprise multi-tenant with smart routing - External Volume: Data lake integration with external storage - Flexible configuration with environment variable support - Complete file lifecycle management 🔧 Technical: - Reuses existing ClickZetta connection configuration - Pydantic-based configuration validation - Comprehensive error handling and logging - Performance-optimized with connection reuse - Clean integration with Dify's storage architecture 🚀 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	11 months ago
yunqiqiliang	8e707cace9	Fix recall testing and search functionality for ClickZetta integration - Fix double JSON encoding issue in metadata parsing for all search methods - Remove unnecessary dataset_id filters since each dataset has its own table - Add robust metadata parsing with fallback for JSON decode errors - Ensure document_id is always present for Dify's format_retrieval_documents - Clean up debug logging while preserving essential error logs - Support vector search, full-text search, and hybrid search in recall testing 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	11 months ago
yunqiqiliang	fcf8387f52	Fix SQL statement length issues and improve batch processing - Add SQL length monitoring and automatic batch splitting - Reduce default batch size from 100 to 20 to prevent large SQL statements - Add detailed error logging for SQL execution failures - Implement recursive batch splitting for oversized SQL statements - Set 1MB limit for SQL statement length This resolves issues where large batches create SQL statements that exceed database limits, causing vector insertion failures. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	11 months ago
yunqiqiliang	8dea8766e9	Fix document content special characters causing SQL syntax errors - Add specialized document content cleaning function - Handle backticks, quotes, newlines, and control characters properly - Replace problematic characters instead of just escaping them - Normalize whitespace and remove control characters - Fix "Syntax error at or near" issues from document content like shell commands This resolves SQL syntax errors when documents contain shell scripts, code snippets, or other text with special formatting characters. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	11 months ago
yunqiqiliang	f116407045	Fix SQL injection vulnerabilities and character encoding issues - Enhanced string escaping for SQL safety (backslashes, newlines, tabs) - Added safe JSON formatting with ensure_ascii=True - Implemented safe doc_id validation (alphanumeric + hyphens/underscores only) - Protected all user input: document content, metadata, IDs, search queries - Fixed potential SQL syntax errors from special characters in document content This addresses "Syntax error at or near 'files'" errors that occur when document content or metadata contains special characters that break SQL syntax. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	11 months ago
yunqiqiliang	9c2bf2b30f	Fix SQL syntax errors with vector formatting - Add safe vector formatting function to handle special float values - Handle NaN, infinity values in vector embeddings - Prevent SQL syntax errors from malformed VECTOR() statements - Use consistent vector formatting across all SQL operations This fixes "Syntax error at or near '{'" errors that occur when vector embeddings contain special float values during knowledge base construction. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	11 months ago
yunqiqiliang	1b7603deb1	Fix inverted index duplicate creation issue - Add table existence check before creating indexes - Improve error handling for ClickZetta specific error messages - Remove duplicate _table_exists method definition - Prevent high-frequency index creation attempts during bulk operations This fixes the "already has index with the same type" errors during large knowledge base construction with 600+ documents. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	11 months ago
yunqiqiliang	c4d9e5c69f	Add documentation section and clean up formatting - Add clear section header for ClickZetta configuration - Improve code organization and readability - All lint checks should pass with latest fixes 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	11 months ago
yunqiqiliang	ed139a49a3	Fix code style issues for CI checks - Remove unused imports (time, VectorType) - Replace logger.error with logger.exception for exception handling - Remove redundant exception objects from logging.exception calls - Ensure all Python style checks pass 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	11 months ago
yunqiqiliang	b201e5d502	feat: add Clickzetta vector database support - Add ClickzettaVector implementation with write queue for concurrent safety - Support vector similarity search using HNSW algorithm - Support full-text search with inverted indexes - Add comprehensive configuration and environment variables - Add unit and integration tests - Resolve dependency conflicts with clickzetta-connector-python 0.8.102 Co-authored-by: Claude <noreply@anthropic.com>	11 months ago

13 Commits (37c534df212eefc0e6717c81635ef485478ca4b6)