Commit Graph

83 Commits (37c534df212eefc0e6717c81635ef485478ca4b6)

Author SHA1 Message Date
yunqiqiliang 37c534df21
Merge 18230d12f9 into bd43ca6275 11 months ago
znn 19c09d6111
enabling vector index prefix name via configuration files (#22661) 11 months ago
yunqiqiliang 556ecf3076 Complete comprehensive CI fixes for all Python style issues
- Fix all line length violations in configuration files
- Replace all logger.error with logger.exception for better error handling
- Fix RUF013 type annotation issues (use union syntax)
- Fix SIM110 code simplification suggestions
- Ensure all core functionality passes linting checks
- Test file style suggestions remain for future optimization

All major CI checks should now pass:
 Docker Compose Template
 Python Style (ruff format/check)
 SuperLinter
 Web Style
 MyPy Type Checking

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang b5a3f1d5e0 Fix remaining Python style and linting issues
- Fix line length violation in middleware config description
- Fix RUF013 type annotation to use union syntax
- Complete all Python style and linting fixes for CI checks
- Resolve formatter and linter warnings

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang f57fa13f1b fix: resolve CI linting issues and add missing newlines
- Fix all line length issues (120 character limit)
- Remove all trailing whitespace
- Add missing newlines at end of files
- Add CLICKZETTA_VOLUME_DIFY_PREFIX environment variable to docker-compose.yaml
- Ensure proper code formatting for all ClickZetta files

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
yunqiqiliang 2de316c557 feat: add ClickZetta Volume storage support
- Add three volume types: User, Table, and External Volume
- Complete file operations: upload, download, delete, list, stream
- Intelligent configuration management with fallback to vector DB settings
- Simplified user experience with 'user' as default volume type
- Comprehensive error handling and logging
- Docker integration with updated compose files
- Integration tests for all volume types
- Disabled complex permission checking for stability

🎯 Features:
- User Volume: Personal/small team use, simple configuration
- Table Volume: Enterprise multi-tenant with smart routing
- External Volume: Data lake integration with external storage
- Flexible configuration with environment variable support
- Complete file lifecycle management

🔧 Technical:
- Reuses existing ClickZetta connection configuration
- Pydantic-based configuration validation
- Comprehensive error handling and logging
- Performance-optimized with connection reuse
- Clean integration with Dify's storage architecture

🚀 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
11 months ago
kenwoodjw 61a5741c05
fix celery config (#22566)
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
11 months ago
yunqiqiliang b201e5d502 feat: add Clickzetta vector database support
- Add ClickzettaVector implementation with write queue for concurrent safety
- Support vector similarity search using HNSW algorithm
- Support full-text search with inverted indexes
- Add comprehensive configuration and environment variables
- Add unit and integration tests
- Resolve dependency conflicts with clickzetta-connector-python 0.8.102

Co-authored-by: Claude <noreply@anthropic.com>
11 months ago
Jacky Wu da53bf511f
chore: add SQLALCHEMY_POOL_USE_LIFO option and missing SQLALCHEMY_POOL_PRE_PING env default value. (#22371) 11 months ago
湛露先生 28f5c37211
Add Env 'CELERY_SENTINEL_PASSWORD' for celery connect redis sentinel. (#21198) 12 months ago
LiuBo 17fe62cf91
feat: add support for Matrixone database (#20714) 12 months ago
Dongyu Li 92614765ff
Feat/queue monitor (#20647) 1 year ago
rouxiaomin 4c4887c5fc
feat(qdrant):add replication_factor when create collection in qdrant (#20133)
Co-authored-by: 刘敏 <min.liu@tongdun.net>
1 year ago
Boris Feld 9afd7f6c87
chore: Update S3StorageConfig to match boto3 type hints (#20072) 1 year ago
He Huang 6f48af2610
Refactor OpenSearch config to separate use_ssl and verify_certs flags (#20075)
Co-authored-by: he.huang <he.huang1@outlook.com>
Co-authored-by: crazywoola <427733928@qq.com>
1 year ago
-LAN- 0fed5c1193
fix(config): Allow DB_EXTRAS to set search_path via options (#19560)
Signed-off-by: -LAN- <laipz8200@outlook.com>
1 year ago
Bowen Liang 31d143de11
chore: disable redis client-side caching by default (#19524) 1 year ago
Bowen Liang 855e850ef3
feat: enable Redis client-side caching (#19493) 1 year ago
Bowen Liang cbc8ebd8f5
chore: bump pydantic to 2.11 and pydantic-settings to 2.9 (#15049) 1 year ago
Ahmad Zidan 8266815cda
feat: add AWS Managed IAM auth for OpenSearch vector DB (#18963) 1 year ago
Jiang 37e2f73909
[Lindorm VDB] Add the QUERY_TIMEOUT parameter to force the search query to fail. (#18613)
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com>
1 year ago
王晓阳 0babdffe3e
feat: support vastbase vector database (#16308) 1 year ago
lauding eb1ce3dd6b
feat: support huawei cloud vector database (#16141) 1 year ago
Rain Wang e8d98e3d89
Add analyzer_params config for milvus vectordb (#18180) 1 year ago
wlleiiwang 42a42a7962
FEAT: support Tencent vectordb to full text search (#16865)
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
1 year ago
wanttobeamaster 7f70cadacb
feat: support Tablestore vector database (#16601)
Co-authored-by: xiaozhiqing.xzq <xiaozhiqing.xzq@alibaba-inc.com>
1 year ago
hsiong 6157f57872
feat: Add OceanBase hybrid search features (#16652)
Co-authored-by: 李远军 <4842@9ji.com>
Co-authored-by: yourchanges <yourchanges@gmail.com>
1 year ago
chenhuan0728 770c461a8f
feat: add openGauss PQ acceleration feature (#16432)
Co-authored-by: chenhuan <huan.chen0728@foxmail>
1 year ago
LittleFish-15 223ab5a38f
feat: support openGauss vector database (#15865) 1 year ago
Yuichiro Utsumi 5f9d236d22
Feat: Add pg_bigm for keyword search in pgvector (#13876)
Signed-off-by: Yuichiro Utsumi <utsumi.yuichiro@fujitsu.com>
1 year ago
engchina c8de30f3d9
feat: support oracle oci autonomouse database. Fixes #14792 and Fixes #14628. (#14804)
Co-authored-by: engchina <atjapan2015@gmail.com>
1 year ago
Bowen Liang dfdd6dfa20
fix: change the config name and fix typo in description of the number of retrieval executors (#13856) 1 year ago
Charlie.Wei 222df44d21
Retrieval Service efficiency optimization (#13543) 1 year ago
YoungLH 040a3b782c
FEAT: support milvus to full text search (#11430)
Signed-off-by: YoungLH <974840768@qq.com>
1 year ago
yihong 56e15d09a9
feat: mypy for all type check (#10921) 1 year ago
-LAN- 99f40a9682
feat: full support for opendal and sync configurations between .env and docker-compose (#11754)
Signed-off-by: -LAN- <laipz8200@outlook.com>
2 years ago
Jiang 0d04cdc323
Lindorm vdb (#11574)
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com>
2 years ago
-LAN- 8d4bb9b40d
feat: integrate opendal storage (#11508)
Signed-off-by: -LAN- <laipz8200@outlook.com>
2 years ago
huanshare 967b7d89e3
feat:add apollo configuration to load env file (#11210)
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: huanshare <liuhuan101@longfor.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
2 years ago
liuhaoran 8ff65abbc6
ext_redis.py support redis clusters --- Fixes #9538 (#9789)
Signed-off-by: root <root@localhost.localdomain>
Co-authored-by: root <root@localhost.localdomain>
Co-authored-by: Bowen Liang <bowenliang@apache.org>
2 years ago
8bitpd 873e9720e9
feat: AnalyticDB vector store supports invocation via SQL. (#10802)
Co-authored-by: 璟义 <yangshangpo.ysp@alibaba-inc.com>
2 years ago
Jiang 0c9e79cd67
Add Lindorm as a VDB choice (#10202)
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com>
2 years ago
Shili Cao b61baa87ec
fix: avoid unexpected error when create knowledge base with baidu vector database and wenxin embedding model (#10130) 2 years ago
powerfool 878d13ef42
Added OceanBase as an option for the vector store in Dify (#10010) 2 years ago
Jyong 5580bcf870
add tidb spend limit config (#9999) 2 years ago
roadgoat19 c8ef9223e5
feat: couchbase integration (#6165)
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: Elliot Scribner <elliot.scribner@couchbase.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
Co-authored-by: Bowen Liang <bowenliang@apache.org>
2 years ago
Jyong 18106a4fc6
add tidb on qdrant type (#9831)
Co-authored-by: Zhaofeng Miao <522856232@qq.com>
2 years ago
Zven 8e7a752b2a
feat: add upstash as a new vector database provider (#9644) 2 years ago
Zven a9fc85027d
chore: update the description for storage_type (#9492) 2 years ago
Bowen Liang 5eb00502ec
chore: clean-up unnecessary annotation on configs with non-null default value (#9323) 2 years ago