Commit Graph

41 Commits (cb7a608d75ac630ed2756ba95ef2b5f5ef95c03a)

Author SHA1 Message Date
Jyong cb7a608d75
ascii filter Unicode U+FFFE (#2038)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong a63a9c7d45
text spliter length method use default embedding model tokenizer (#2011)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Bowen Liang cc9e74123c
improve: introduce isort for linting Python imports (#1983) 2 years ago
Jyong 24bdedf802
fix get embedding model provider in empty dataset (#1986)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong 4a3d15b6de
fix customer spliter character (#1915)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
takatost a938e1f184
fix: notion_indexing_estimate embedding_model_instance NPE (#1907) 2 years ago
Yeuoly 9134849744
fix: remove tiktoken from text splitter (#1876) 2 years ago
takatost d069c668f8
Model Runtime (#1858)
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
Co-authored-by: Garfield Dai <dai.hai@foxmail.com>
Co-authored-by: chenhe <guchenhe@gmail.com>
Co-authored-by: jyong <jyong@dify.ai>
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: Yeuoly <admin@srmxy.cn>
2 years ago
Jyong df1509983c
ppt & pptx improve (#1790)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong 5e34f938c1
Feat/add unstructured support (#1780)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
crazywoola 994fceece3
fix: qa regex (#1738) 2 years ago
Pascal M bc54cdc537
refactor: typo in dataset docstore (#1711) 2 years ago
Pascal M 5d10cf0fe6
fix: error Class 'builtins.list' is not mapped (#1710) 2 years ago
Jyong 4588831bff
Feat/add retriever rerank (#1560)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
crazywoola d0e1ea8f06
1506 remove duplicated code (#1511) 2 years ago
Garfield Dai 42a5b3ec17
feat: advanced prompt backend (#1301)
Co-authored-by: takatost <takatost@gmail.com>
2 years ago
Jyong 289c93d081
Feat/improve document delete logic (#1325)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
yezhwi 8b8e510bfe
fix: handle AttributeError for datasets and index (#1052) 2 years ago
Jyong a55ba6e614
Fix/ignore economy dataset (#1043)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong 2d604d9330
Fix/filter empty segment (#1004)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
Jyong 5623839c71
update document segment (#950)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago
takatost 3a0a9e2d8f
fix: embedding get price definition missing (#922) 2 years ago
Krasus.Chen fd0fc8f4fe
Fix/price calc (#862) 3 years ago
Jyong db7156dafd
Feature/mutil embedding model (#908)
Co-authored-by: JzoNg <jzongcode@gmail.com>
Co-authored-by: jyong <jyong@dify.ai>
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
3 years ago
Jyong f207e180df
fix multi thread app context (#868)
Co-authored-by: jyong <jyong@dify.ai>
3 years ago
takatost f18ce203b5
feat: optimize error logging (#808) 3 years ago
takatost 5fa2161b05
feat: server multi models support (#799) 3 years ago
Jyong 174ebb51db
add qa thread control (#677) 3 years ago
Jyong 9eaae770a6
Feat/add thread control (#675) 3 years ago
Jyong ca60610306
logging qa error (#672) 3 years ago
Jyong 082f8b17ab
Feat/milvus support (#671)
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
Co-authored-by: JzoNg <jzongcode@gmail.com>
3 years ago
KVOJJJin cf93d8d6e2
Feat: Q&A format segmentation support (#668)
Co-authored-by: jyong <718720800@qq.com>
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
3 years ago
crazywoola 998f819b04
use sub to operate all (#475) 3 years ago
Jyong 2eea114ac0
fix special code (#473) 3 years ago
John Wang 3241e4015b
feat: upgrade langchain (#430)
Co-authored-by: jyong <718720800@qq.com>
3 years ago
Jyong 9253f72dea
Feat/dataset notion import (#392)
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
Co-authored-by: JzoNg <jzongcode@gmail.com>
3 years ago
lisaifei@cvte.com 0abd67288b
feat: support xlsx file parsing (#304)
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
3 years ago
Jyong 2bf48514bc
fix markdown parser (#230) 3 years ago
John Wang 0587ff0fba
fix: remove empty segment in splitter (#68) 3 years ago
John Wang 815f794eef
feat: optimize split rule when use custom split segment identifier (#35) 3 years ago
John Wang db896255d6 Initial commit 3 years ago