Commit Graph

416 Commits (deploy/rag-dev)

Author SHA1 Message Date
yihong 65c7c01d90
fix: clean up two unreachable code (#20773)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
10 months ago
jefferyvvv 37c3283450
fix: opensearch vector search falls back to keyword search (#20723)
Co-authored-by: wenjun.gu <wenjun.gu@envision-energy.com>
10 months ago
jefferyvvv 4271602cfc
fix: opensearch metadata filtering returns empty (#20701)
Co-authored-by: wenjun.gu <wenjun.gu@envision-energy.com>
Co-authored-by: crazywoola <427733928@qq.com>
10 months ago
jefferyvvv 138ad6e8b3
fix: opensearch fulltext search with metadata filtering dsl error (#20702)
Co-authored-by: wenjun.gu <wenjun.gu@envision-energy.com>
10 months ago
kenwoodjw 01d500db14
fix: autocorrect everything in web (#20605)
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
10 months ago
jyong 9cdd2cbb27 r2 10 months ago
jyong 309fffd1e4 Merge branch 'main' into feat/r2
# Conflicts:
#	api/core/repositories/sqlalchemy_workflow_node_execution_repository.py
#	api/core/workflow/entities/node_entities.py
#	api/core/workflow/enums.py
10 months ago
jyong 4130c50643 r2 10 months ago
jyong 7b7f8ef51d r2 10 months ago
zhaobingshuang 3f7aa38d77
fix: #20560 When elasticsearch is used as the vector database, the Retrieval Test fails to filter the data after setting the Score Threshold, and the score of the recalled results is empty (#20561) 10 months ago
Cheney Zhang b4b59148dc
check zilliz cloud of full-text search (#20519) 10 months ago
Dongyu Li 1ea4459d9f
update knowledge base api (#20426) 10 months ago
-LAN- a6ea15e63c
Refactor/message cycle manage and knowledge retrieval (#20460)
Signed-off-by: -LAN- <laipz8200@outlook.com>
10 months ago
yihong 5a991295e0
fix: drop some type fixme (#20344) 10 months ago
-LAN- 482e50aae9
Refactor/remove db from cycle manager (#20455)
Signed-off-by: -LAN- <laipz8200@outlook.com>
10 months ago
jyong a025db137d Merge branch 'main' into feat/r2 10 months ago
jyong 7f59ffe7af r2 10 months ago
rouxiaomin 4c4887c5fc
feat(qdrant):add replication_factor when create collection in qdrant (#20133)
Co-authored-by: 刘敏 <min.liu@tongdun.net>
10 months ago
jyong 5fc2bc58a9 r2 10 months ago
jyong 35be8721b9 Merge branch 'main' into feat/r2 10 months ago
He Huang 6f48af2610
Refactor OpenSearch config to separate use_ssl and verify_certs flags (#20075)
Co-authored-by: he.huang <he.huang1@outlook.com>
Co-authored-by: crazywoola <427733928@qq.com>
11 months ago
wlleiiwang 7d230acf40
tencent vectordb compatible with version 1.1.3 and below (#20056)
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
11 months ago
-LAN- 3196dc2d61
refactor: Use typed SQLAlchemy base model and fix type errors (#19980)
Signed-off-by: -LAN- <laipz8200@outlook.com>
11 months ago
Darlewo 8cb3b4aef2
fix: multiple retrieve reranking_enabled switch (#19958) 11 months ago
jyong 82be119fec Merge branch 'main' into feat/r2 11 months ago
Amir Mohsen Asaran c9ee60e197
Feat(WaterCrawl error handling): add custom exceptions and error handling (#19948) 11 months ago
-LAN- 4977bb21ec
feat(workflow): domain model for workflow node execution (#19430)
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
11 months ago
jyong 4ff971c8a3 r2 11 months ago
jyong 360f8a3375 Merge branch 'main' into feat/r2 11 months ago
k-kayashima b292990075
Fix: Ensure unique index names for pgvector knowledge tables (#19672)
Co-authored-by: crazywoola <427733928@qq.com>
11 months ago
非法操作 085bd1aa93
chore: model.query change to db.session.query (#19551)
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
11 months ago
非法操作 14cd71ed0a
chore: all model.query replace to db.session.query (#19521) 11 months ago
非法操作 b00f94df64
fix: replace all dataset.Model.query to db.session.query(Model) (#19509) 11 months ago
湛露先生 1119790b02
clean rag word_extractor. (#19397)
Signed-off-by: zhanluxianshen <zhanluxianshen@163.com>
11 months ago
jyong 3f1363503b r2 11 months ago
Will bfa652f2d0
fix: metadata filtering condition variable unassigned; fix External K… (#19208) 11 months ago
jyong a998022c12 r2 11 months ago
jyong a25cc4e8af r2 11 months ago
Hao Cheng 9bcf837f17
fix: use only supported operators in metadata filter system prompts (#19195) 11 months ago
Will a212a63e6a
fix: time type metadata filtering error (#19192) 11 months ago
Bowen Liang 12c96b93d9
immediately return initialed tiktokenizer instance and remove dead code in usage of tiktokenizer (#17957) 11 months ago
QuantumGhost bd1bbfee4b
Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 11 months ago
Ahmad Zidan 8266815cda
feat: add AWS Managed IAM auth for OpenSearch vector DB (#18963) 11 months ago
Ethan 8b4ea01810
feat: support access milvus with token (#19034) 11 months ago
Panpan 83187b30c0
fix: fix rerank model runner usage (#19008) 11 months ago
Wesley b62eb61400
fix depth param issue for WaterCrawl (#18839) 11 months ago
Jiang 37e2f73909
[Lindorm VDB] Add the QUERY_TIMEOUT parameter to force the search query to fail. (#18613)
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com>
11 months ago
王晓阳 0babdffe3e
feat: support vastbase vector database (#16308) 11 months ago
Jyong e2cb7006c4
check metadata_filtering_conditions could be None in auto mode (#18548) 12 months ago
lauding eb1ce3dd6b
feat: support huawei cloud vector database (#16141) 12 months ago
tmuife 7b6523e54d
Update Oracle db connection library and change connection pool to single connection (#18466) 12 months ago
Rain Wang d2e3744ca3
Switching from CONSOLE_API_URL to FILES_URL in word_extractor.py (#18249) 12 months ago
Rain Wang 83f1aeec1d
Fix ORDER BY (score, id) error in api/core/rag/datasource/vdb/analyticdb/analyticdb_vector_sql.py line 249 (#18252) 12 months ago
Rain Wang e8d98e3d89
Add analyzer_params config for milvus vectordb (#18180) 12 months ago
Jyong 95283b4dd3
Feat/change split length method (#18097)
Co-authored-by: JzoNg <jzongcode@gmail.com>
12 months ago
YANG d119c7d629
ignore errors when creating duplicate indexes (#18069)
Co-authored-by: 璟义 <yangshangpo.ysp@alibaba-inc.com>
12 months ago
Jasonfish 1f722cde22
fix(api): Some params were ignored when creating empty Datasets through API (#17932) 12 months ago
Yongtao Huang 5d72003ebb
Remove dead code (#17899) 12 months ago
briqt 91cfa90503
Fix external knowledge Issues: (#17685) (#17843) 12 months ago
yihong f04d52c044
fix: autocorrect everything in api (#17859)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
12 months ago
wlleiiwang 9d20561af4
create db if not exists (#17796)
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
12 months ago
Jyong 8b3be4224d
revert batch query (#17707) 12 months ago
wlleiiwang f148f1efa2
fix: Check collection exists before drop it. (#17692)
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
12 months ago
Jyong abfcd9d3b6
fix segment query index not effect (#17704) 12 months ago
Han f1e4d5ed6c
Fix Performance Issues: (#17083)
Co-authored-by: Wang Han <wanghan@zhejianglab.org>
12 months ago
Steven Li abead647e2
fix: Extract docx file fails when the file contains an invalid link (#17576) 12 months ago
Amir Mohsen Asaran f54905e685
feat: Integrate WaterCrawl.dev as a new knowledge base provider (#16396)
Co-authored-by: crazywoola <427733928@qq.com>
1 year ago
wlleiiwang 42a42a7962
FEAT: support Tencent vectordb to full text search (#16865)
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
1 year ago
crazywoola 3e698074e7
Fix/17466 cannot create a knowledge base by adding files (#17470) 1 year ago
Panpan fc3f14c0ee
fix: keep image url (#17430) 1 year ago
Perfecto 16c722d1d8
fix: move hardcoded text to language settings (#16990) (#17133) 1 year ago
Jyong 6104b91d3f
add doc support in knowledge base for unstructured (#17352) 1 year ago
Jiang fd1e40d22e
Lindorm VDB bugfix (#17357)
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com>
1 year ago
Jiang ff388fe3e6
optimize lindorm vdb add_texts (#17212)
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com>
1 year ago
非法操作 44f911a0a8
chore: docstring not match the function parameter (#17162) 1 year ago
jiangbo721 a1aa325ce3
Chore/code format and Repair commit_id 3254018d more deleted codes and Fix naming error ambiguity between workflow_run_id and workflow_id (#17075)
Co-authored-by: 刘江波 <jiangbo721@163.com>
1 year ago
wanttobeamaster 7f70cadacb
feat: support Tablestore vector database (#16601)
Co-authored-by: xiaozhiqing.xzq <xiaozhiqing.xzq@alibaba-inc.com>
1 year ago
wlleiiwang a743d5dc71
feat: tencent vectordb: use grpc client and set upsert batch size (#16016)
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
1 year ago
Jyong 30792a1e1a
install pandoc (#16825) 1 year ago
yourchanges 59a86dabee
fix: fix missing oceanbase config enable_hybrid_search init (#16852)
Co-authored-by: 李远军 <4842@9ji.com>
1 year ago
Jyong 6a857e01f6
fix multiple metadata filter's confusing setting (#16771) 1 year ago
taokuizu 0c2a459c30
fix typo in _process_metadata_filter_func (#16780) 1 year ago
Jyong 2174225259
fix milvus filter search (#16725) 1 year ago
hsiong 6157f57872
feat: Add OceanBase hybrid search features (#16652)
Co-authored-by: 李远军 <4842@9ji.com>
Co-authored-by: yourchanges <yourchanges@gmail.com>
1 year ago
kenwoodjw a113356695
fix: pgvector metadata filter (#16688)
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
1 year ago
Jiang fc8c765215
Fix/vdb lindorm (#16660)
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com>
1 year ago
Jyong 86a1859d02
Metadata variable value fix (#16665) 1 year ago
Jyong 1be0d26c1f
fix metadata filter not affect in keyword-search and fulltext-search (#16644) 1 year ago
chenhuan0728 770c461a8f
feat: add openGauss PQ acceleration feature (#16432)
Co-authored-by: chenhuan <huan.chen0728@foxmail>
1 year ago
Jyong d135677c25
add vdb document id index (#16244)
Co-authored-by: crazywoola <427733928@qq.com>
1 year ago
Jyong a8879057c0
fix tidb metadata filter (#16237) 1 year ago
Jyong 81325df368
fix weaviate metadata filter (#16230) 1 year ago
Jyong b8ef3149ef
metadata expect value check error (#16210) 1 year ago
Jyong c3c957bb80
change recreate_collection function to create_collection (#16212) 1 year ago
Jyong abeaea4f79
Support knowledge metadata filter (#15982) 1 year ago
Jyong 33ba7e659b
fix vector db sql injection (#16096) 1 year ago
LittleFish-15 223ab5a38f
feat: support openGauss vector database (#15865) 1 year ago
huangzhuo1949 695a7400a9
fix:delete empty table bug (#15517)
Co-authored-by: huangzhuo <huangzhuo1@xiaomi.com>
1 year ago
Jyong 84a866028a
fix document could be None (#15818) 1 year ago
Yuichiro Utsumi 5f9d236d22
Feat: Add pg_bigm for keyword search in pgvector (#13876)
Signed-off-by: Yuichiro Utsumi <utsumi.yuichiro@fujitsu.com>
1 year ago