Skip to content

[Benchmark] Enhance traditional Saga benchmark with a more realistic state-machine workload #8039

@WangzJi

Description

@WangzJi

Check Ahead

  • I have searched the issues of this repository and believe that this is not a duplicate.
  • I am willing to try to implement this feature myself.

Why you need it?
The current seata-benchmark-cli already supports --mode SAGA, and when --branches > 0 it uses Seata's state machine engine. This is a good first step, but the current benchmark workload is still relatively lightweight and benchmark-oriented.

Today, the benchmark can measure the state machine path itself, but it is still not ideal for evaluating more realistic Saga behavior under business-style workloads, especially when we want to compare results with AT / TCC / XA. In particular, the current implementation does not yet provide enough workload realism, reproducible failure modeling, or richer Saga-specific result reporting.

So the need here is not to add a brand new transaction mode, but to make the existing traditional Saga benchmark more representative and more useful for performance comparison and regression tracking.

How it could be?
This issue is about the existing traditional Saga path only, i.e. BranchType.SAGA and the current --mode SAGA support in seata-benchmark-cli.

A possible direction is:

  • keep the current state machine engine based implementation
  • enhance SagaModeExecutor instead of introducing another Saga mode
  • make the workload more realistic, for example by supporting optional DB-backed business actions or more realistic order / inventory / payment style steps
  • add reproducible failure injection, such as configured failure rate, timeout simulation, or per-step failure control
  • improve result reporting so the benchmark output can clearly show committed / compensated / failed / compensation-failed results
  • define clearer semantics for --branches in Saga mode, or allow selecting different predefined state machine shapes while preserving backward compatibility
  • update README with runnable examples and explanation of the Saga-specific parameters

Expected outcome:

  • existing --mode SAGA remains backward compatible
  • benchmark results become more representative of a traditional state-machine Saga workload
  • compensation scenarios can be triggered and observed more clearly
  • final benchmark report becomes more actionable for comparison and diagnosis

Other related information
Current implementation evidence:

  • test-suite/seata-benchmark-cli/src/main/java/org/apache/seata/benchmark/BenchmarkRunner.java
    • currently only dispatches AT, TCC, and SAGA
  • test-suite/seata-benchmark-cli/src/main/java/org/apache/seata/benchmark/config/BenchmarkConfig.java
    • currently validates only AT, TCC, and SAGA
  • test-suite/seata-benchmark-cli/src/main/java/org/apache/seata/benchmark/executor/SagaModeExecutor.java
    • branches == 0: simplified mock mode
    • branches > 0: state machine engine mode
  • test-suite/seata-benchmark-cli/README.md
    • documents SAGA support and state machine usage

Non-goal for this issue:

  • annotation-based Saga support (BranchType.SAGA_ANNOTATION) should be tracked separately because it is a different execution path and likely needs different bootstrap/integration logic.

Metadata

Metadata

Assignees

No fields configured for Feature.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions