Commit Graph

67 Commits (cdc08a434f8e82809fa33ae4ebe8541a7ffc16a8)

Author SHA1 Message Date
takatost d1dbbc1e33
feat: backend model load balancing support (#4927) 2 years ago
Jyong b6631cd878
modify rerank and splitter code directory (#4924) 2 years ago
Jyong 233c4150d1
support images and tables extract from docx (#4619) 2 years ago
Rain Chen c255a20d7c
allow to config max segmentation tokens length for RAG document using environment variable (#4375) 2 years ago
Bowen Liang 04ad46dd31
chore: skip unnecessary key checks prior to accessing a dictionary (#4497) 2 years ago
LIU HONGWEI c227f3d985
feat: Deprecate datetime.utcnow() in favor of datetime.now(timezone.utc).replace(tzinfo=None) for better timezone handling (#3408) (#3416) 2 years ago
Jyong 33ea689861
fix detached instance error in keyword index create thread and fix question classifier node out of index error (#3219) 2 years ago
Jyong 283979fc46
fix keyword index error when storage source is S3 (#3182) 2 years ago
takatost 7753ba2d37
FEAT: NEW WORKFLOW ENGINE (#3160)
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: Yeuoly <admin@srmxy.cn>
Co-authored-by: JzoNg <jzongcode@gmail.com>
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
Co-authored-by: jyong <jyong@dify.ai>
Co-authored-by: nite-knite <nkCoding@gmail.com>
Co-authored-by: jyong <718720800@qq.com>
2 years ago
Jyong b0b0cc045f
add mutil-thread document embedding (#3016)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong 6454e1d644
chunk-overlap None check (#2781)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong 31070ffbca
fix qa index processor tenant id is None error (#2713)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Charlie.Wei fa7ba30ba3
Fix rebuild index&csv parsing (#2705)
Co-authored-by: luowei <glpat-EjySCyNjWiLqAED-YmwM>
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2 years ago
Jyong 5b953c1ef2
Fix some RAG bugs (#2570)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong 0620fa3094
Feat/vdb migrate command (#2562)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong 4be3087642
Fix/new RAG bugs (#2547)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong 91ea6fe4ee
Fix/langchain document schema (#2539)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong 6c4e6bf1d6
Feat/dify rag (#2528)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong 97fe817186
Fix/upload limit (#2521)
Co-authored-by: jyong <jyong@dify.ai>
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
2 years ago
Bowen Liang 063191889d
chore: apply ruff's pyupgrade linter rules to modernize Python code with targeted version (#2419) 2 years ago
crazywoola 243ca5b1e2
fix: typo in package path of core.splitter (#2411) 2 years ago
Bowen Liang 843280f82b
enhancement: introduce Ruff for Python linter for reordering and removing unused imports with automated pre-commit and sytle check (#2366) 2 years ago
takatost 9f637ead38
bump version to 0.5.3 (#2306) 2 years ago
KVOJJJin 89fcf4ea7c
Feat: chunk overlap supported (#2209)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
takatost 6cf93379b3
fix: split chunks return empty strings (#2197) 2 years ago
Jyong 869690c485
fix notion estimate (#2090)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong cb7a608d75
ascii filter Unicode U+FFFE (#2038)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong a63a9c7d45
text spliter length method use default embedding model tokenizer (#2011)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Bowen Liang cc9e74123c
improve: introduce isort for linting Python imports (#1983) 2 years ago
Jyong 24bdedf802
fix get embedding model provider in empty dataset (#1986)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong 4a3d15b6de
fix customer spliter character (#1915)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
takatost a938e1f184
fix: notion_indexing_estimate embedding_model_instance NPE (#1907) 2 years ago
Yeuoly 9134849744
fix: remove tiktoken from text splitter (#1876) 2 years ago
takatost d069c668f8
Model Runtime (#1858)
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
Co-authored-by: Garfield Dai <dai.hai@foxmail.com>
Co-authored-by: chenhe <guchenhe@gmail.com>
Co-authored-by: jyong <jyong@dify.ai>
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: Yeuoly <admin@srmxy.cn>
2 years ago
Jyong df1509983c
ppt & pptx improve (#1790)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong 5e34f938c1
Feat/add unstructured support (#1780)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
crazywoola 994fceece3
fix: qa regex (#1738) 2 years ago
Pascal M bc54cdc537
refactor: typo in dataset docstore (#1711) 2 years ago
Pascal M 5d10cf0fe6
fix: error Class 'builtins.list' is not mapped (#1710) 2 years ago
Jyong 4588831bff
Feat/add retriever rerank (#1560)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
crazywoola d0e1ea8f06
1506 remove duplicated code (#1511) 2 years ago
Garfield Dai 42a5b3ec17
feat: advanced prompt backend (#1301)
Co-authored-by: takatost <takatost@gmail.com>
2 years ago
Jyong 289c93d081
Feat/improve document delete logic (#1325)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
yezhwi 8b8e510bfe
fix: handle AttributeError for datasets and index (#1052) 2 years ago
Jyong a55ba6e614
Fix/ignore economy dataset (#1043)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong 2d604d9330
Fix/filter empty segment (#1004)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong 5623839c71
update document segment (#950)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
takatost 3a0a9e2d8f
fix: embedding get price definition missing (#922) 3 years ago
Krasus.Chen fd0fc8f4fe
Fix/price calc (#862) 3 years ago
Jyong db7156dafd
Feature/mutil embedding model (#908)
Co-authored-by: JzoNg <jzongcode@gmail.com>
Co-authored-by: jyong <jyong@dify.ai>
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
3 years ago