dify - 💡(How to fix) Fix Add smoke tests for full-node Workflow and Chatflow execution paths [1 participants]

Self Checks

• I have read the Contributing Guide and Language Policy.
• I have searched for existing issues, including closed ones.
• I confirm that I am using English to submit this report, otherwise it will be closed.
• Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

Dify has many workflow and chatflow node types, and regressions in node orchestration can be hard to detect with isolated unit tests alone.

I would like to add backend integration smoke tests that verify both a Workflow app and an Advanced Chat / Chatflow app can run through representative “all-node” DSLs successfully.

The intended coverage is:

• one Workflow smoke test
• one Chatflow smoke test
• each test imports a DSL containing representative coverage of all major node types
• external-interaction nodes are mocked at their boundary, so the tests remain deterministic and CI-friendly
• each test publishes or prepares the app as needed
• each test invokes the app through the relevant API path
• each test asserts the final output with custom expected-output checks

The goal is not to test external providers, plugins, model vendors, HTTP services, or datasets themselves. The goal is to catch regressions in Dify’s own DSL import, workflow/chatflow execution orchestration, node wiring, API invocation path, and final output generation.

This would give us a stable smoke-test layer above unit tests and below full E2E tests.

2. Additional context or comments

A possible reference DSL is:

https://raw.githubusercontent.com/QuantumGhost/dify-workflows/refs/heads/main/All%20Nodes/All%20Nodes.yml

That DSL may need to be adapted into two repository-local test fixtures:

• one Workflow-mode fixture
• one Advanced Chat / Chatflow-mode fixture

The tests should not rely on downloading the DSL from GitHub during test execution. The fixture should live in the repository and be intentionally adjusted for deterministic testing.

Suggested test behavior:

1. Import the DSL.
2. Persist the draft workflow/chatflow.
3. Publish or otherwise prepare it for API execution.
4. Call the relevant backend API.
5. Mock nodes that require external interaction, such as LLM providers, agents/tools, HTTP requests, knowledge retrieval, document extraction, and similar dependencies.
6. Assert that the run succeeds and the final output matches the expected custom assertion.

This would complement existing unit-level workflow fixture tests and the current container-based backend integration tests.

3. Can you help us with this feature?

• I am interested in contributing to this feature.