dify - ✅(Solved) Fix [Refactor/Chore] Manage vdb providers with uv workspaces [1 pull requests, 1 participants]

wylswz · 2026-04-10T05:21:38Z

[dify] Move vdb providers and tests to workspaces like this api/packages ├── pycache ├── conftest.py ├── dify-vdb-alibabacloud-mysql ... ├── dify-vdb-analyticd… Move vdb providers and tests to workspaces like this ``` api/packages ├── __pycache__ ├── conftest.py ├── dify-vdb-alibabacloud-mysql ... ├── dify-vdb-analyticdb └── dify-vdb-weaviate ``` Then in the `pyproject.toml` ``` vdb-alibabacloud-mysql = ["dify-vdb-alibabacloud-mysql"] ... vdb-couchbase = ["dify-vdb-couchbase"] ``` # PR #34900: refactor: move vdb implementations to workspaces - Repository: langgenius/dify - Author: wylswz - State: open | merged: False - Link: https://github.com/langgenius/dify/pull/34900 ## Description (problem / solution / changelog) ## Summary fixes #34899 - Moved vdb implementations to workspaces. - Added a `providers` for community service providers. ## Screenshots | Before | After | |--------|-------| | ... | ... | ## Checklist - [ ] This change requires a documentation update, included: [Dify Document](https://github.com/langgenius/dify-docs) - [x] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!) - [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change. - [x] I've updated the documentation accordingly. - [x] I ran `make lint` and `make type-check` (backend) and `cd web && pnpm exec vp staged` (frontend) to appease the lint gods ## Changed files - `.github/workflows/main-ci.yml` (modified, +1/-0) - `.github/workflows/vdb-tests-full.yml` (modified, +1/-1) - `.github/workflows/vdb-tests.yml` (modified, +5/-5) - `api/Dockerfile` (modified, +2/-1) - `api/commands/vector.py` (modified, +1/-2) - `api/configs/middleware/vdb/hologres_config.py` (modified, +3/-4) - `api/core/rag/datasource/vdb/vector_backend_registry.py` (added, +87/-0) - `api/core/rag/datasource/vdb/vector_factory.py` (modified, +2/-131) - `api/core/rag/datasource/vdb/vector_integration_test_support.py` (renamed, +13/-18) - `api/providers/conftest.py` (added, +22/-0) - `api/providers/vdb/alibabacloud-mysql/pyproject.toml` (added, +18/-0) - `api/providers/vdb/alibabacloud-mysql/src/dify_vdb_alibabacloud_mysql/__init__.py` (renamed, +0/-0) - `api/providers/vdb/alibabacloud-mysql/src/dify_vdb_alibabacloud_mysql/alibabacloud_mysql_vector.py` (renamed, +0/-0) - `api/providers/vdb/alibabacloud-mysql/tests/unit_tests/test_alibabacloud_mysql_factory.py` (renamed, +2/-3) - `api/providers/vdb/alibabacloud-mysql/tests/unit_tests/test_alibabacloud_mysql_vector.py` (renamed, +29/-79) - `api/providers/vdb/analyticdb/pyproject.toml` (added, +20/-0) - `api/providers/vdb/analyticdb/src/dify_vdb_analyticdb/__init__.py` (renamed, +0/-0) - `api/providers/vdb/analyticdb/src/dify_vdb_analyticdb/analyticdb_vector.py` (renamed, +5/-5) - `api/providers/vdb/analyticdb/src/dify_vdb_analyticdb/analyticdb_vector_openapi.py` (renamed, +0/-0) - `api/providers/vdb/analyticdb/src/dify_vdb_analyticdb/analyticdb_vector_sql.py` (renamed, +0/-0) - `api/providers/vdb/analyticdb/tests/integration_tests/test_analyticdb.py` (renamed, +4/-5) - `api/providers/vdb/analyticdb/tests/unit_tests/test_analyticdb_vector.py` (renamed, +4/-4) - `api/providers/vdb/analyticdb/tests/unit_tests/test_analyticdb_vector_openapi.py` (renamed, +3/-3) - `api/providers/vdb/analyticdb/tests/unit_tests/test_analyticdb_vector_sql.py` (renamed, +3/-3) - `api/providers/vdb/baidu/pyproject.toml` (added, +18/-0) - `api/providers/vdb/baidu/src/dify_vdb_baidu/__init__.py` (renamed, +0/-0) - `api/providers/vdb/baidu/src/dify_vdb_baidu/baidu_vector.py` (renamed, +0/-0) - `api/providers/vdb/baidu/tests/integration_tests/conftest.py` (renamed, +0/-0) - `api/providers/vdb/baidu/tests/integration_tests/test_baidu.py` (renamed, +2/-6) - `api/providers/vdb/baidu/tests/unit_tests/test_baidu_vector.py` (renamed, +1/-1) - `api/providers/vdb/chroma/pyproject.toml` (added, +18/-0) - `api/providers/vdb/chroma/src/dify_vdb_chroma/__init__.py` (renamed, +0/-0) - `api/providers/vdb/chroma/src/dify_vdb_chroma/chroma_vector.py` (renamed, +0/-0) - `api/providers/vdb/chroma/tests/integration_tests/test_chroma.py` (renamed, +2/-4) - `api/providers/vdb/chroma/tests/unit_tests/test_chroma_vector.py` (renamed, +1/-1) - `api/providers/vdb/clickzetta/README.md` (renamed, +1/-1) - `api/providers/vdb/clickzetta/pyproject.toml` (added, +18/-0) - `api/providers/vdb/clickzetta/src/dify_vdb_clickzetta/__init__.py` (renamed, +0/-0) - `api/providers/vdb/clickzetta/src/dify_vdb_clickzetta/clickzetta_vector.py` (renamed, +0/-0) - `api/providers/vdb/clickzetta/tests/README.md` (renamed, +0/-0) - `api/providers/vdb/clickzetta/tests/integration_tests/test_clickzetta.py` (renamed, +8/-11) - `api/providers/vdb/clickzetta/tests/integration_tests/test_docker_integration.py` (renamed, +45/-48) - `api/providers/vdb/clickzetta/tests/unit_tests/test_clickzetta_vector.py` (renamed, +1/-1) - `api/providers/vdb/couchbase/pyproject.toml` (added

dify2026-04-10 05:21:38

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

langgenius/dify#34899•Fetched 2026-04-11 06:32:02

View on GitHub

Comments

Participants

Timeline

Reactions

Author

wylswz

Participants

wylswz

Assignees

wylswz

Timeline (top)

assigned ×1cross-referenced ×1

Move vdb providers and tests to workspaces like this

api/packages
├── __pycache__
├── conftest.py
├── dify-vdb-alibabacloud-mysql
...
├── dify-vdb-analyticdb
└── dify-vdb-weaviate

Then in the pyproject.toml

vdb-alibabacloud-mysql = ["dify-vdb-alibabacloud-mysql"]
...
vdb-couchbase = ["dify-vdb-couchbase"]

Root Cause

Move vdb providers and tests to workspaces like this

api/packages
├── __pycache__
├── conftest.py
├── dify-vdb-alibabacloud-mysql
...
├── dify-vdb-analyticdb
└── dify-vdb-weaviate

Then in the pyproject.toml

vdb-alibabacloud-mysql = ["dify-vdb-alibabacloud-mysql"]
...
vdb-couchbase = ["dify-vdb-couchbase"]

Fix Action

Fixed

Fixed by PR: refactor: move vdb implementations to workspaces (https://github.com/langgenius/dify/pull/34900)

PR fix notes

PR #34900: refactor: move vdb implementations to workspaces

Repository: langgenius/dify
Author: wylswz
State: open | merged: False
Link: https://github.com/langgenius/dify/pull/34900

Description (problem / solution / changelog)

Summary

fixes #34899

Moved vdb implementations to workspaces.
Added a providers for community service providers.

Screenshots

Before	After
...	...

Checklist

This change requires a documentation update, included: Dify Document
I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
I've updated the documentation accordingly.
I ran make lint and make type-check (backend) and cd web && pnpm exec vp staged (frontend) to appease the lint gods

Changed files

.github/workflows/main-ci.yml (modified, +1/-0)
.github/workflows/vdb-tests-full.yml (modified, +1/-1)
.github/workflows/vdb-tests.yml (modified, +5/-5)
api/Dockerfile (modified, +2/-1)
api/commands/vector.py (modified, +1/-2)
api/configs/middleware/vdb/hologres_config.py (modified, +3/-4)
api/core/rag/datasource/vdb/vector_backend_registry.py (added, +87/-0)
api/core/rag/datasource/vdb/vector_factory.py (modified, +2/-131)
api/core/rag/datasource/vdb/vector_integration_test_support.py (renamed, +13/-18)
api/providers/conftest.py (added, +22/-0)
api/providers/vdb/alibabacloud-mysql/pyproject.toml (added, +18/-0)
api/providers/vdb/alibabacloud-mysql/src/dify_vdb_alibabacloud_mysql/__init__.py (renamed, +0/-0)
api/providers/vdb/alibabacloud-mysql/src/dify_vdb_alibabacloud_mysql/alibabacloud_mysql_vector.py (renamed, +0/-0)
api/providers/vdb/alibabacloud-mysql/tests/unit_tests/test_alibabacloud_mysql_factory.py (renamed, +2/-3)
api/providers/vdb/alibabacloud-mysql/tests/unit_tests/test_alibabacloud_mysql_vector.py (renamed, +29/-79)
api/providers/vdb/analyticdb/pyproject.toml (added, +20/-0)
api/providers/vdb/analyticdb/src/dify_vdb_analyticdb/__init__.py (renamed, +0/-0)
api/providers/vdb/analyticdb/src/dify_vdb_analyticdb/analyticdb_vector.py (renamed, +5/-5)
api/providers/vdb/analyticdb/src/dify_vdb_analyticdb/analyticdb_vector_openapi.py (renamed, +0/-0)
api/providers/vdb/analyticdb/src/dify_vdb_analyticdb/analyticdb_vector_sql.py (renamed, +0/-0)
api/providers/vdb/analyticdb/tests/integration_tests/test_analyticdb.py (renamed, +4/-5)
api/providers/vdb/analyticdb/tests/unit_tests/test_analyticdb_vector.py (renamed, +4/-4)
api/providers/vdb/analyticdb/tests/unit_tests/test_analyticdb_vector_openapi.py (renamed, +3/-3)
api/providers/vdb/analyticdb/tests/unit_tests/test_analyticdb_vector_sql.py (renamed, +3/-3)
api/providers/vdb/baidu/pyproject.toml (added, +18/-0)
api/providers/vdb/baidu/src/dify_vdb_baidu/__init__.py (renamed, +0/-0)
api/providers/vdb/baidu/src/dify_vdb_baidu/baidu_vector.py (renamed, +0/-0)
api/providers/vdb/baidu/tests/integration_tests/conftest.py (renamed, +0/-0)
api/providers/vdb/baidu/tests/integration_tests/test_baidu.py (renamed, +2/-6)
api/providers/vdb/baidu/tests/unit_tests/test_baidu_vector.py (renamed, +1/-1)
api/providers/vdb/chroma/pyproject.toml (added, +18/-0)
api/providers/vdb/chroma/src/dify_vdb_chroma/__init__.py (renamed, +0/-0)
api/providers/vdb/chroma/src/dify_vdb_chroma/chroma_vector.py (renamed, +0/-0)
api/providers/vdb/chroma/tests/integration_tests/test_chroma.py (renamed, +2/-4)
api/providers/vdb/chroma/tests/unit_tests/test_chroma_vector.py (renamed, +1/-1)
api/providers/vdb/clickzetta/README.md (renamed, +1/-1)
api/providers/vdb/clickzetta/pyproject.toml (added, +18/-0)
api/providers/vdb/clickzetta/src/dify_vdb_clickzetta/__init__.py (renamed, +0/-0)
api/providers/vdb/clickzetta/src/dify_vdb_clickzetta/clickzetta_vector.py (renamed, +0/-0)
api/providers/vdb/clickzetta/tests/README.md (renamed, +0/-0)
api/providers/vdb/clickzetta/tests/integration_tests/test_clickzetta.py (renamed, +8/-11)
api/providers/vdb/clickzetta/tests/integration_tests/test_docker_integration.py (renamed, +45/-48)
api/providers/vdb/clickzetta/tests/unit_tests/test_clickzetta_vector.py (renamed, +1/-1)
api/providers/vdb/couchbase/pyproject.toml (added, +18/-0)
api/providers/vdb/couchbase/src/dify_vdb_couchbase/__init__.py (renamed, +0/-0)
api/providers/vdb/couchbase/src/dify_vdb_couchbase/couchbase_vector.py (renamed, +0/-0)
api/providers/vdb/couchbase/tests/integration_tests/test_couchbase.py (renamed, +7/-5)
api/providers/vdb/couchbase/tests/unit_tests/test_couchbase_vector.py (renamed, +1/-1)
api/providers/vdb/elasticsearch/pyproject.toml (added, +19/-0)
api/providers/vdb/elasticsearch/src/dify_vdb_elasticsearch/__init__.py (renamed, +0/-0)
api/providers/vdb/elasticsearch/src/dify_vdb_elasticsearch/elasticsearch_ja_vector.py (renamed, +4/-4)
api/providers/vdb/elasticsearch/src/dify_vdb_elasticsearch/elasticsearch_vector.py (renamed, +0/-0)
api/providers/vdb/elasticsearch/tests/integration_tests/test_elasticsearch.py (renamed, +3/-4)
api/providers/vdb/elasticsearch/tests/unit_tests/test_elasticsearch_ja_vector.py (renamed, +2/-2)
api/providers/vdb/elasticsearch/tests/unit_tests/test_elasticsearch_vector.py (renamed, +1/-1)
api/providers/vdb/hologres/pyproject.toml (added, +18/-0)
api/providers/vdb/hologres/src/dify_vdb_hologres/__init__.py (renamed, +0/-0)
api/providers/vdb/hologres/src/dify_vdb_hologres/hologres_vector.py (renamed, +4/-4)
api/providers/vdb/hologres/tests/integration_tests/conftest.py (renamed, +0/-23)
api/providers/vdb/hologres/tests/integration_tests/test_hologres.py (renamed, +2/-7)
api/providers/vdb/hologres/tests/unit_tests/test_hologres_vector.py (renamed, +1/-1)
api/providers/vdb/huawei-cloud/pyproject.toml (added, +18/-0)
api/providers/vdb/huawei-cloud/src/dify_vdb_huawei_cloud/__init__.py (renamed, +0/-0)
api/providers/vdb/huawei-cloud/src/dify_vdb_huawei_cloud/huawei_cloud_vector.py (renamed, +0/-0)
api/providers/vdb/huawei-cloud/tests/integration_tests/conftest.py (renamed, +0/-0)
api/providers/vdb/huawei-cloud/tests/integration_tests/test_huawei_cloud.py (renamed, +2/-6)
api/providers/vdb/huawei-cloud/tests/unit_tests/test_huawei_cloud_vector.py (renamed, +1/-1)
api/providers/vdb/iris/pyproject.toml (added, +18/-0)
api/providers/vdb/iris/src/dify_vdb_iris/__init__.py (renamed, +0/-0)
api/providers/vdb/iris/src/dify_vdb_iris/iris_vector.py (renamed, +0/-0)
api/providers/vdb/iris/tests/integration_tests/test_iris.py (renamed, +3/-4)
api/providers/vdb/iris/tests/unit_tests/test_iris_vector.py (renamed, +1/-1)
api/providers/vdb/lindorm/pyproject.toml (added, +19/-0)
api/providers/vdb/lindorm/src/dify_vdb_lindorm/__init__.py (renamed, +0/-0)
api/providers/vdb/lindorm/src/dify_vdb_lindorm/lindorm_vector.py (renamed, +0/-0)
api/providers/vdb/lindorm/tests/integration_tests/test_lindorm.py (renamed, +2/-3)
api/providers/vdb/lindorm/tests/unit_tests/test_lindorm_vector.py (renamed, +1/-1)
api/providers/vdb/matrixone/pyproject.toml (added, +18/-0)
api/providers/vdb/matrixone/src/dify_vdb_matrixone/__init__.py (renamed, +0/-0)
api/providers/vdb/matrixone/src/dify_vdb_matrixone/matrixone_vector.py (renamed, +0/-0)
api/providers/vdb/matrixone/tests/integration_tests/test_matrixone.py (renamed, +3/-4)
api/providers/vdb/matrixone/tests/unit_tests/test_matrixone_vector.py (renamed, +1/-1)
api/providers/vdb/milvus/pyproject.toml (added, +18/-0)
api/providers/vdb/milvus/src/dify_vdb_milvus/__init__.py (renamed, +0/-0)
api/providers/vdb/milvus/src/dify_vdb_milvus/milvus_vector.py (renamed, +0/-0)
api/providers/vdb/milvus/tests/integration_tests/test_milvus.py (renamed, +3/-4)
api/providers/vdb/milvus/tests/unit_tests/test_milvus.py (renamed, +1/-1)
api/providers/vdb/myscale/pyproject.toml (added, +18/-0)
api/providers/vdb/myscale/src/dify_vdb_myscale/__init__.py (renamed, +0/-0)
api/providers/vdb/myscale/src/dify_vdb_myscale/myscale_vector.py (renamed, +0/-0)
api/providers/vdb/myscale/tests/integration_tests/test_myscale.py (renamed, +3/-4)
api/providers/vdb/myscale/tests/unit_tests/test_myscale_vector.py (renamed, +1/-1)
api/providers/vdb/oceanbase/pyproject.toml (added, +20/-0)
api/providers/vdb/oceanbase/src/dify_vdb_oceanbase/__init__.py (renamed, +0/-0)
api/providers/vdb/oceanbase/src/dify_vdb_oceanbase/oceanbase_vector.py (renamed, +0/-0)
api/providers/vdb/oceanbase/tests/integration_tests/bench_oceanbase.py (renamed, +23/-20)
api/providers/vdb/oceanbase/tests/integration_tests/test_oceanbase.py (renamed, +3/-5)
api/providers/vdb/oceanbase/tests/unit_tests/test_oceanbase_vector.py (renamed, +1/-1)
api/providers/vdb/opengauss/pyproject.toml (added, +16/-0)
api/providers/vdb/opengauss/src/dify_vdb_opengauss/__init__.py (renamed, +0/-0)

Code Example

api/packages
├── __pycache__
├── conftest.py
├── dify-vdb-alibabacloud-mysql
...
├── dify-vdb-analyticdb
└── dify-vdb-weaviate

---

vdb-alibabacloud-mysql = ["dify-vdb-alibabacloud-mysql"]
...
vdb-couchbase = ["dify-vdb-couchbase"]

---

uv sync \
--group vdb-weaviate \
--group vdb-pgvector

RAW_BUFFERClick to expand / collapse

Self Checks

I have read the Contributing Guide and Language Policy.
This is only for refactors or chores; if you would like to ask a question, please head to Discussions.
I have searched for existing issues search for existing issues, including closed ones.
I confirm that I am using English to submit this report, otherwise it will be closed.
【中文用户 & Non English User】请使用英语提交，否则会被关闭：）
Please do not modify this template :) and fill in all the required fields.

Description

Move vdb providers and tests to workspaces like this

api/packages
├── __pycache__
├── conftest.py
├── dify-vdb-alibabacloud-mysql
...
├── dify-vdb-analyticdb
└── dify-vdb-weaviate

Then in the pyproject.toml

vdb-alibabacloud-mysql = ["dify-vdb-alibabacloud-mysql"]
...
vdb-couchbase = ["dify-vdb-couchbase"]

Motivation

VDB providers may introduce transitive dependencies potentially with CVEs. By moving providers to workspaces, we can specify which providers to include by changing uv command

uv sync \
--group vdb-weaviate \
--group vdb-pgvector

Additional Context

No response

extent analysis

TL;DR

Move VDB providers and tests to workspaces and update pyproject.toml to specify included providers.

Guidance

Move VDB providers to separate packages under the api/packages directory, as shown in the provided example.
Update pyproject.toml to include the new package names, using the format vdb-provider-name = ["dify-vdb-provider-name"].
Use the uv sync command with the --group option to specify which providers to include, as shown in the motivation example.
Verify that the providers are correctly included or excluded by checking the dependencies and running tests.

Example

No explicit code example is provided, but the issue includes examples of the desired directory structure and pyproject.toml configuration.

Notes

This solution assumes that the uv command and pyproject.toml configuration are already set up and functional. If issues arise during the migration, additional debugging or configuration may be necessary.

Recommendation

Apply workaround: Move VDB providers to workspaces and update pyproject.toml to mitigate potential transitive dependency issues and improve dependency management.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #ssr #integration issue #index setup #retrieval issue

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

dify - ✅(Solved) Fix [Refactor/Chore] Manage vdb providers with uv workspaces [1 pull requests, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

Root Cause

Fix Action

Fixed

PR fix notes

PR #34900: refactor: move vdb implementations to workspaces

Description (problem / solution / changelog)

Summary

Screenshots

Checklist

Changed files

Code Example

Self Checks

Description

Motivation

Additional Context

extent analysis

TL;DR

Guidance

Example

Notes

Recommendation

Still need to ship something?

TRENDING

dify - ✅(Solved) Fix [Refactor/Chore] Manage vdb providers with uv workspaces [1 pull requests, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

Root Cause

Fix Action

Fixed

PR fix notes

PR #34900: refactor: move vdb implementations to workspaces

Description (problem / solution / changelog)

Summary

Screenshots

Checklist

Changed files

Code Example

Self Checks

Description

Motivation

Additional Context

extent analysis

TL;DR

Guidance

Example

Notes

Recommendation

Still need to ship something?

RELATED_DISCOVERY

TRENDING