Issue Station
Sign In
Tools hereToolbox

gemini-cli - ✅(Solved) Fix Need a perf-memory test approach for CI and setting baseline [1 pull requests]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

Fix Action

Fixed

PR fix notes

PR #25705: test(perf): overhaul performance and memory baseline management

Description (problem / solution / changelog)

Summary

Comprehensive automation upgrades for performance and memory baselines. Includes GitHub Actions workflows for remote updates, automatic local comparisons against main, and git-ignored temporary baselines.

Details

  • Added update-baselines.yml GitHub Action to automate remote baseline upgrades efficiently in CI.
  • Created scripts/run-perf-tests.js to wrap performance executions, safely stashing dirty alterations and gathering main-branch baselines locally when run without arguments.
  • Enhanced PerfTestHarness and MemoryTestHarness to accommodate tolerance limits assertions safely.
  • Updated test files to process TEMP_BASELINES_PATH environment variables, protecting tracked files clean during local evaluations.
  • Formed docs/performance-and-memory-testing.md safely centrally detailing general strategies.
  • Obsoleted folder files perf-tests/README.md, and memory-tests/README.md deleted altogether.
  • Registered temporary baseline outputs inside .gitignore and updated scripts/clean.js safely for fast removals on npm run clean.

Related Issues

Fixes #25704

How to Validate

CI should use existing baselines. And Command to set baselines should work from github actions.

Pre-Merge Checklist

  • [ ✅ ] Updated relevant documentation and README (if needed)
  • [ ✅ ] Added/updated tests (if needed)
  • [ NA ] Noted breaking changes (if any)
  • [ ✅ ] Validated on required platforms/methods:
    • [ ✅ ] MacOS
      • [ ✅ ] npm run
      • npx
      • Docker
      • Podman
      • Seatbelt
    • Windows
      • npm run
      • npx
      • Docker
    • Linux
      • npm run
      • npx
      • Docker

Changed files

  • .github/workflows/memory-nightly.yml (modified, +14/-2)
  • .github/workflows/perf-nightly.yml (modified, +14/-2)
  • .github/workflows/update-baselines.yml (added, +243/-0)
  • .gitignore (modified, +2/-0)
  • docs/performance-and-memory-testing.md (added, +110/-0)
  • memory-tests/baselines/.gitkeep (added, +0/-0)
  • memory-tests/memory-usage.test.ts (modified, +8/-2)
  • package.json (modified, +2/-2)
  • packages/test-utils/src/index.ts (modified, +1/-1)
  • packages/test-utils/src/memory-baselines.ts (modified, +23/-0)
  • packages/test-utils/src/memory-test-harness.ts (modified, +38/-1)
  • packages/test-utils/src/perf-test-harness.ts (modified, +67/-3)
  • perf-tests/README.md (removed, +0/-121)
  • perf-tests/baselines/.gitkeep (added, +0/-0)
  • perf-tests/perf-usage.test.ts (modified, +8/-2)
  • scripts/clean.js (modified, +2/-0)
  • scripts/run-perf-tests.js (added, +118/-0)
RAW_BUFFERClick to expand / collapse

Implement the CI and dev/test approach for setting baselines for perf and mem tests

extent analysis

TL;DR

Implementing a CI/CD pipeline with automated performance and memory testing can help set baselines for future comparisons.

Guidance

  • Identify key performance and memory metrics to track, such as response times, throughput, and memory usage.
  • Research and integrate suitable testing tools, like Apache JMeter or Gatling for performance testing, and Java Mission Control or YourKit for memory profiling.
  • Develop a strategy for storing and comparing test results over time to establish a baseline.
  • Consider using a version control system to track changes in test configurations and results.

Example

No specific code example can be provided without more context, but a general approach might involve using a testing framework like JUnit or TestNG to write performance and memory tests.

Notes

The choice of testing tools and strategy will depend on the specific requirements and technologies used in the project.

Recommendation

Apply workaround: Implement a basic CI/CD pipeline with automated testing to start collecting performance and memory data, and refine the approach as needed.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#database connection#vector store#embedding generation#cache error#pipeline error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

gemini-cli - ✅(Solved) Fix Need a perf-memory test approach for CI and setting baseline [1 pull requests]