vllm - 💡(How to fix) Fix [CI/Perf] Invalid JSON in serving benchmark config [1 pull requests]

Summary

.buildkite/performance-benchmarks/tests/serving-tests.json is currently invalid JSON on main.

The file contains a stray duplicate object after serving_llama8B_tp1_sharegpt, so JSON parsers fail before the serving benchmark suite can read the default config.

Repro

jq . .buildkite/performance-benchmarks/tests/serving-tests.json

Current result:

parse error: Objects must consist of key:value pairs at line 35, column 5

I also verified the same failure with Python json.load.

Why this matters

run-performance-benchmarks.sh defaults to serving-tests$ARCH.json, and the non-arch-specific default points at tests/serving-tests.json. That means the default serving benchmark config is broken on the current main branch.

Likely regression source

git blame points the malformed block to 5d041cc1fe5181daabf39943efc7b678380d57bd from PR #43262.

Expected fix

Remove the stray duplicate object and add a small validation guard so future benchmark config edits fail fast if one of these JSON files becomes unparsable again.