Commit Graph

59 Commits (bc7cc06572ad56ac8ebe430b31742ecbbac17841)

Author SHA1 Message Date
virgosoy 17cacf258e
fix: wrong element object (#9868) 1 year ago
Zixuan Cheng 88dec6ef2b
Added description for .ppt, specify the reason for unstructured.io (#9452)
Co-authored-by: crazywoola <427733928@qq.com>
1 year ago
Jyong 5f11fe521d
remove unstructured pdf extract (#9794) 1 year ago
Jyong 3e9d271b52
nltk security issue and upgrade unstructured (#9558) 1 year ago
-LAN- 5f12c17355
fix(core): use CreatedByRole enum for role consistency (#9607) 1 year ago
-LAN- e61752bd3a
feat/enhance the multi-modal support (#8818) 1 year ago
Bowen Liang 240b66d737
chore: avoid implicit optional in type annotations of method (#8727) 1 year ago
Zhaofeng Miao 369e1e6f58
feat(website-crawl): add jina reader as additional alternative for website crawling (#8761) 2 years ago
crazywoola bf64ff215b
fix: . is missing in file_extension (#8736) 2 years ago
Bowen Liang a1104ab97e
chore: refurish python code by applying Pylint linter rules (#8322) 2 years ago
Bowen Liang 6613b8f2e0
chore: fix unnecessary string concatation in single line (#8311) 2 years ago
Bowen Liang 40fb4d16ef
chore: refurbish Python code by applying refurb linter rules (#8296) 2 years ago
Bowen Liang c69f5b07ba
chore: apply ruff E501 line-too-long linter rule (#8275)
Co-authored-by: -LAN- <laipz8200@outlook.com>
2 years ago
takatost 56c90e212a
fix(workflow): missing content in the answer node stream output during iterations (#8292)
Co-authored-by: -LAN- <laipz8200@outlook.com>
2 years ago
Bowen Liang 0f14873255
chore: cleanup ruff flake8-simplify linter rules (#8286)
Co-authored-by: -LAN- <laipz8200@outlook.com>
2 years ago
Bowen Liang 2cf1187b32
chore(api/core): apply ruff reformatting (#7624) 2 years ago
Jyong af92f19291
filter excel empty sheet (#8194) 2 years ago
Nam Vu 2d7954c7da
Fix variable typo (#8084) 2 years ago
Jyong 01581dd35f
improve the notion table extract (#7925) 2 years ago
Jyong 6f33351eb3
ignore linked images when image id is none (#7890) 2 years ago
Bowen Liang ccb6ddd840
chore: bump Ruff to 0.5.7 (#7174) 2 years ago
Jyong 12095f8cd6
extract docx filter comment element (#7092) 2 years ago
chenxu9741 72c75b75cf
feat: Add hyperlink parsing to the DOCX document. (#7017) 2 years ago
yanghx c53875ce8c
fix #6902 .docx handles images within tables and handles cross-column tables (#6951) 2 years ago
Jyong cf258b7a67
add xlsx support hyperlink extract (#6722) 2 years ago
Yeuoly 79cb23e8ac
security/SSRF vulns (#6682) 2 years ago
灰灰 5e4ac11df3
fix: code block segmentation problem of markdown document (#6465) 2 years ago
Poorandy c8f5dfcf17
refactor(rag): switch to dify_config. (#6410)
Co-authored-by: -LAN- <laipz8200@outlook.com>
2 years ago
tangyoha 0cbbaf3f68
fix: markdown proc will remove image (#5855) 2 years ago
Matri a9ee52f2d7
Fix/firecrawl parameters issue (#6213) 2 years ago
Aurelius Huang f546db5437
fix: document truncation and loss in notion document sync (#5631)
Co-authored-by: Aurelius Huang <cm.huang@aftership.com>
2 years ago
Jyong 43335b5c87
delete the deprecated method (#5612) 2 years ago
Bowen Liang 39c14ec7c1
improve: unify Excel files parsing in either xls or xlsx file format by Pandas (#4965) 2 years ago
takatost 12c815c597
fix: ExtractSetting optional value missing None as default val (#5238) 2 years ago
Jyong ba5f8afaa8
Feat/firecrawl data source (#5232)
Co-authored-by: Nicolas <nicolascamara29@gmail.com>
Co-authored-by: chenhe <guchenhe@gmail.com>
Co-authored-by: takatost <takatost@gmail.com>
2 years ago
Bowen Liang f976740b57
improve: mordernizing validation by migrating pydantic from 1.x to 2.x (#4592) 2 years ago
Jyong 3b60c28b3a
deal the external image when extract docx image (#5024) 2 years ago
YC 9f8ca75a81
fixing a bug of handling header row when parsing xls file, and tune xls/xlsx parsing result to be more structured (#3600) 2 years ago
Bowen Liang 58db719a2c
dep: bump pandas from 1.x to 2.x (#4820) 2 years ago
Oliver Lee 176d91937d
fix 'NoneType' and new ContentType supported. (#4818) 2 years ago
yalei 026175c8f7
feat: update notion extractor (#3898)
Co-authored-by: duyalei <>
2 years ago
Jyong 233c4150d1
support images and tables extract from docx (#4619) 2 years ago
majian b5204111da
Add UNSTRUCTURED_API_KEY env support (#4369) 2 years ago
Charlie.Wei 97b65f9b4b
Optimize webscraper (#4392)
Co-authored-by: luowei <glpat-EjySCyNjWiLqAED-YmwM>
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2 years ago
Bowen Liang 7919596a21
fix: UP031 style rule violation (#3866) 2 years ago
Jyong 0737e930cb
chore: remove Langchain tools import (#3407) 2 years ago
chenxu9741 ad65c891e7
add xls file suport (#3321) 2 years ago
LiuVaayne b00466f025
feat:api Add support for extracting EPUB files in ExtractProcessor (#3254)
Co-authored-by: crazywoola <427733928@qq.com>
2 years ago
Jyong 6164604462
fix dataset retrival in dataset mode (#3334) 2 years ago
Jyong 9eba6ffdd4
Optimize csv and excel extract (#3155)
Co-authored-by: jyong <jyong@dify.ai>
2 years ago