Commit Graph

82 Commits (349c3cf7b838cd5c7f1adc535d3045cacda0eb79)

Author SHA1 Message Date
QuantumGhost 349c3cf7b8
feat(api): Add image multimodal support for LLMNode (#17372)
Enhance `LLMNode` with multimodal capability, introducing support for
image outputs.

This implementation extracts base64-encoded images from LLM responses,
saves them to the storage service, and records the file metadata in the
`ToolFile` table. In conversations, these images are rendered as
markdown-based inline images.
Additionally, the images are included in the LLMNode's output as
file variables, enabling subsequent nodes in the workflow to utilize them.

To integrate file outputs into workflows, adjustments to the frontend code
are necessary.

For multimodal output functionality, updates to related model configurations
are required. Currently, this capability has been applied exclusively to
Google's Gemini models.

Close #15814.

Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
9 months ago
kingpc 65b7a783fe
fix: metadata filter not work (#19020)
Co-authored-by: 金鹏程 <jinpengcheng01@corp.netease.com>
Co-authored-by: crazywoola <427733928@qq.com>
9 months ago
Will cb69cb2d64
fix weird syntax error (#18454) 10 months ago
briqt 91cfa90503
Fix external knowledge Issues: (#17685) (#17843) 10 months ago
crazywoola 3e698074e7
Fix/17466 cannot create a knowledge base by adding files (#17470) 10 months ago
Perfecto 16c722d1d8
fix: move hardcoded text to language settings (#16990) (#17133) 10 months ago
非法操作 44f911a0a8
chore: docstring not match the function parameter (#17162) 10 months ago
JimintheBox 8b89447549
Fix Custom Tool File Upload: Resolve Multiple Files Recognition and Multipart Boundary Issues (#14014)
Co-authored-by: crazywoola <427733928@qq.com>
10 months ago
StoneFancyX 2524f16525
support config filename in meta for create_blob_message (#15605)
Co-authored-by: StoneFancyX <kindbin@qq.com>
Co-authored-by: crazywoola <427733928@qq.com>
11 months ago
Will aef43910b1
fix: octet/stream => application/octet-stream (#15329)
Co-authored-by: crazywoola <427733928@qq.com>
11 months ago
crazywoola ce2dd22bd7
fix: typo `doc_metadat` (#14569) 11 months ago
yuhaowin 1e3197a1ea
Fixes 14217: database retrieve api and chat-messages api response doc_metadata (#14219) 11 months ago
Yeuoly 403e2d58b9
Introduce Plugins (#13836)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Signed-off-by: -LAN- <laipz8200@outlook.com>
Signed-off-by: xhe <xw897002528@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: takatost <takatost@gmail.com>
Co-authored-by: kurokobo <kuro664@gmail.com>
Co-authored-by: Novice Lee <novicelee@NoviPro.local>
Co-authored-by: zxhlyh <jasonapring2015@outlook.com>
Co-authored-by: AkaraChen <akarachen@outlook.com>
Co-authored-by: Yi <yxiaoisme@gmail.com>
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: JzoNg <jzongcode@gmail.com>
Co-authored-by: twwu <twwu@dify.ai>
Co-authored-by: Hiroshi Fujita <fujita-h@users.noreply.github.com>
Co-authored-by: AkaraChen <85140972+AkaraChen@users.noreply.github.com>
Co-authored-by: NFish <douxc512@gmail.com>
Co-authored-by: Wu Tianwei <30284043+WTW0313@users.noreply.github.com>
Co-authored-by: 非法操作 <hjlarry@163.com>
Co-authored-by: Novice <857526207@qq.com>
Co-authored-by: Hiroki Nagai <82458324+nagaihiroki-git@users.noreply.github.com>
Co-authored-by: Gen Sato <52241300+halogen22@users.noreply.github.com>
Co-authored-by: eux <euxuuu@gmail.com>
Co-authored-by: huangzhuo1949 <167434202+huangzhuo1949@users.noreply.github.com>
Co-authored-by: huangzhuo <huangzhuo1@xiaomi.com>
Co-authored-by: lotsik <lotsik@mail.ru>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
Co-authored-by: nite-knite <nkCoding@gmail.com>
Co-authored-by: Jyong <76649700+JohnJyong@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: gakkiyomi <gakkiyomi@aliyun.com>
Co-authored-by: CN-P5 <heibai2006@gmail.com>
Co-authored-by: CN-P5 <heibai2006@qq.com>
Co-authored-by: Chuehnone <1897025+chuehnone@users.noreply.github.com>
Co-authored-by: yihong <zouzou0208@gmail.com>
Co-authored-by: Kevin9703 <51311316+Kevin9703@users.noreply.github.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Boris Feld <lothiraldan@gmail.com>
Co-authored-by: mbo <himabo@gmail.com>
Co-authored-by: mabo <mabo@aeyes.ai>
Co-authored-by: Warren Chen <warren.chen830@gmail.com>
Co-authored-by: JzoNgKVO <27049666+JzoNgKVO@users.noreply.github.com>
Co-authored-by: jiandanfeng <chenjh3@wangsu.com>
Co-authored-by: zhu-an <70234959+xhdd123321@users.noreply.github.com>
Co-authored-by: zhaoqingyu.1075 <zhaoqingyu.1075@bytedance.com>
Co-authored-by: 海狸大師 <86974027+yenslife@users.noreply.github.com>
Co-authored-by: Xu Song <xusong.vip@gmail.com>
Co-authored-by: rayshaw001 <396301947@163.com>
Co-authored-by: Ding Jiatong <dingjiatong@gmail.com>
Co-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: JasonVV <jasonwangiii@outlook.com>
Co-authored-by: le0zh <newlight@qq.com>
Co-authored-by: zhuxinliang <zhuxinliang@didiglobal.com>
Co-authored-by: k-zaku <zaku99@outlook.jp>
Co-authored-by: luckylhb90 <luckylhb90@gmail.com>
Co-authored-by: hobo.l <hobo.l@binance.com>
Co-authored-by: jiangbo721 <365065261@qq.com>
Co-authored-by: 刘江波 <jiangbo721@163.com>
Co-authored-by: Shun Miyazawa <34241526+miya@users.noreply.github.com>
Co-authored-by: EricPan <30651140+Egfly@users.noreply.github.com>
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: sino <sino2322@gmail.com>
Co-authored-by: Jhvcc <37662342+Jhvcc@users.noreply.github.com>
Co-authored-by: lowell <lowell.hu@zkteco.in>
Co-authored-by: Boris Polonsky <BorisPolonsky@users.noreply.github.com>
Co-authored-by: Ademílson Tonato <ademilsonft@outlook.com>
Co-authored-by: Ademílson Tonato <ademilson.tonato@refurbed.com>
Co-authored-by: IWAI, Masaharu <iwaim.sub@gmail.com>
Co-authored-by: Yueh-Po Peng (Yabi) <94939112+y10ab1@users.noreply.github.com>
Co-authored-by: Jason <ggbbddjm@gmail.com>
Co-authored-by: Xin Zhang <sjhpzx@gmail.com>
Co-authored-by: yjc980121 <3898524+yjc980121@users.noreply.github.com>
Co-authored-by: heyszt <36215648+hieheihei@users.noreply.github.com>
Co-authored-by: Abdullah AlOsaimi <osaimiacc@gmail.com>
Co-authored-by: Abdullah AlOsaimi <189027247+osaimi@users.noreply.github.com>
Co-authored-by: Yingchun Lai <laiyingchun@apache.org>
Co-authored-by: Hash Brown <hi@xzd.me>
Co-authored-by: zuodongxu <192560071+zuodongxu@users.noreply.github.com>
Co-authored-by: Masashi Tomooka <tmokmss@users.noreply.github.com>
Co-authored-by: aplio <ryo.091219@gmail.com>
Co-authored-by: Obada Khalili <54270856+obadakhalili@users.noreply.github.com>
Co-authored-by: Nam Vu <zuzoovn@gmail.com>
Co-authored-by: Kei YAMAZAKI <1715090+kei-yamazaki@users.noreply.github.com>
Co-authored-by: TechnoHouse <13776377+deephbz@users.noreply.github.com>
Co-authored-by: Riddhimaan-Senapati <114703025+Riddhimaan-Senapati@users.noreply.github.com>
Co-authored-by: MaFee921 <31881301+2284730142@users.noreply.github.com>
Co-authored-by: te-chan <t-nakanome@sakura-is.co.jp>
Co-authored-by: HQidea <HQidea@users.noreply.github.com>
Co-authored-by: Joshbly <36315710+Joshbly@users.noreply.github.com>
Co-authored-by: xhe <xw897002528@gmail.com>
Co-authored-by: weiwenyan-dev <154779315+weiwenyan-dev@users.noreply.github.com>
Co-authored-by: ex_wenyan.wei <ex_wenyan.wei@tcl.com>
Co-authored-by: engchina <12236799+engchina@users.noreply.github.com>
Co-authored-by: engchina <atjapan2015@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: 呆萌闷油瓶 <253605712@qq.com>
Co-authored-by: Kemal <kemalmeler@outlook.com>
Co-authored-by: Lazy_Frog <4590648+lazyFrogLOL@users.noreply.github.com>
Co-authored-by: Yi Xiao <54782454+YIXIAO0@users.noreply.github.com>
Co-authored-by: Steven sun <98230804+Tuyohai@users.noreply.github.com>
Co-authored-by: steven <sunzwj@digitalchina.com>
Co-authored-by: Kalo Chin <91766386+fdb02983rhy@users.noreply.github.com>
Co-authored-by: Katy Tao <34019945+KatyTao@users.noreply.github.com>
Co-authored-by: depy <42985524+h4ckdepy@users.noreply.github.com>
Co-authored-by: 胡春东 <gycm520@gmail.com>
Co-authored-by: Junjie.M <118170653@qq.com>
Co-authored-by: MuYu <mr.muzea@gmail.com>
Co-authored-by: Naoki Takashima <39912547+takatea@users.noreply.github.com>
Co-authored-by: Summer-Gu <37869445+gubinjie@users.noreply.github.com>
Co-authored-by: Fei He <droxer.he@gmail.com>
Co-authored-by: ybalbert001 <120714773+ybalbert001@users.noreply.github.com>
Co-authored-by: Yuanbo Li <ybalbert@amazon.com>
Co-authored-by: douxc <7553076+douxc@users.noreply.github.com>
Co-authored-by: liuzhenghua <1090179900@qq.com>
Co-authored-by: Wu Jiayang <62842862+Wu-Jiayang@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: kimjion <45935338+kimjion@users.noreply.github.com>
Co-authored-by: AugNSo <song.tiankai@icloud.com>
Co-authored-by: llinvokerl <38915183+llinvokerl@users.noreply.github.com>
Co-authored-by: liusurong.lsr <liusurong.lsr@alibaba-inc.com>
Co-authored-by: Vasu Negi <vasu-negi@users.noreply.github.com>
Co-authored-by: Hundredwz <1808096180@qq.com>
Co-authored-by: Xiyuan Chen <52963600+GareArc@users.noreply.github.com>
12 months ago
Bowen Liang 166221d784
chore(lint): fix quotes for f-string formatting by bumping ruff to 0.9.x (#12702) 1 year ago
mbo 1859d57784
api tool support multiple env url (#12249)
Co-authored-by: mabo <mabo@aeyes.ai>
1 year ago
lotsik f230a9232e
fix: Parsing OpenAPI spec for external tools (#12518) (#12530) 1 year ago
yihong 8339d2c7c9
fix: issue #11868 bring old logic back (#12100)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
1 year ago
yihong 56e15d09a9
feat: mypy for all type check (#10921) 1 year ago
yihong 0067b16d1e
fix: refactor all 'or []' and 'or {}' logic to make code more clear (#10883)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
1 year ago
非法操作 bf4b6e5f80
feat: support custom tool upload file (#10796) 1 year ago
Zane 14f3d44c37
refactor: improve handling of leading punctuation removal (#10761) 1 year ago
Bowen Liang 51db59622c
chore(lint): cleanup repeated cause exception in logging.exception replaced by helpful message (#10425) 1 year ago
liuhaoran 570f10d91c
fix issues:Image file not deleted when a doc is removed #9541 (#10465)
Signed-off-by: root <root@localhost.localdomain>
Co-authored-by: root <root@localhost.localdomain>
1 year ago
走在修行的大街上 f562a88249
feat(Tools): add lark tools (#10117)
Co-authored-by: 黎斌 <libin.23@bytedance.com>
1 year ago
走在修行的大街上 a9db06f5e7
feat(Tools): Refactor the base table plugin (#9182)
Co-authored-by: 黎斌 <libin.23@bytedance.com>
1 year ago
-LAN- 740a723072
fix(validation): improve variable handling and validation (#9578) 1 year ago
-LAN- e61752bd3a
feat/enhance the multi-modal support (#8818) 1 year ago
AkisAya d7b8e071dd
fix:#9222 create or update custum tools error (#9228) 1 year ago
Bowen Liang 240b66d737
chore: avoid implicit optional in type annotations of method (#8727) 1 year ago
走在修行的大街上 c828a5dfdf
feat(Tools): add feishu tools (#8800)
Co-authored-by: 黎斌 <libin.23@bytedance.com>
1 year ago
走在修行的大街上 72b7f8a949
Bugfix/fix feishu plugins (#8443)
Co-authored-by: 黎斌 <libin.23@bytedance.com>
1 year ago
Bowen Liang a1104ab97e
chore: refurish python code by applying Pylint linter rules (#8322) 1 year ago
Bowen Liang 40fb4d16ef
chore: refurbish Python code by applying refurb linter rules (#8296) 1 year ago
Bowen Liang c69f5b07ba
chore: apply ruff E501 line-too-long linter rule (#8275)
Co-authored-by: -LAN- <laipz8200@outlook.com>
1 year ago
Bowen Liang 0f14873255
chore: cleanup ruff flake8-simplify linter rules (#8286)
Co-authored-by: -LAN- <laipz8200@outlook.com>
1 year ago
Bowen Liang 2cf1187b32
chore(api/core): apply ruff reformatting (#7624) 1 year ago
takatost dabfd74622
feat: Parallel Execution of Nodes in Workflows (#8192)
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
Co-authored-by: Yi <yxiaoisme@gmail.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
1 year ago
Nam Vu 2d7954c7da
Fix variable typo (#8084) 1 year ago
走在修行的大街上 92cab33b73
feat(Tools): add feishu document and message plugins (#6435)
Co-authored-by: 黎斌 <libin.23@bytedance.com>
1 year ago
Zhi b7ff98d7ff
fix: Remove useless debug information. (#7647) 1 year ago
-LAN- 8f16165f92
chore(api/core): Improve FileVar's type hint and imports. (#7290) 1 year ago
eric-0x72 98d9837fbc
fix wrong charset when decoding Chinese content (#6774)
Co-authored-by: zhangwb <zhangwb@zts.com.cn>
2 years ago
Bowen Liang cb01bf2986
chore: set logging level to debug when reading YAML files and falling back to default value in case of None (#6758) 2 years ago
Bowen Liang 20268708cc
chore: improve position map conversion and tolerate empty position yaml file (#6541) 2 years ago
Yeuoly 79cb23e8ac
security/SSRF vulns (#6682) 2 years ago
非法操作 c0ada940bd
fix: tool params not work as expected when develop a tool (#6550) 2 years ago
Weaxs 4e2fba404d
WebscraperTool bypass cloudflare site by cloudscraper (#6337) 2 years ago
Bowen Liang dcb72e0067
chore: apply flake8-comprehensions Ruff rules to improve collection comprehensions (#5652)
Co-authored-by: -LAN- <laipz8200@outlook.com>
2 years ago
-LAN- 45a3ea6fed
fix: add support for FILE type in ToolParameterConverter (#5578) 2 years ago
fishisnow 11fd4a5dcc
Fix: fix load_yaml logging, Avoid setting the log level to warning (#5019)
Co-authored-by: huangyusong <huangyusong@yingzi.com>
2 years ago