Reducing Flaky Tests Without Deleting Them

A flaky test is one that passes and fails without the code changing. One or two feel harmless. A dozen and the team learns that red means “click rerun”, which defeats the point of having tests at all.

Name it, do not tolerate it

The moment a test is known to be flaky, it needs a decision, not a shrug. Either it gets fixed this week or it moves to a quarantine group that still runs and still reports, but does not block a merge. What you must not do is leave it in the main suite failing at random. That is how a whole team stops trusting the pipeline.

The usual causes

Most flakiness comes from a small set of habits. A test that sleeps for a fixed time and hopes the work is done by then. A test that assumes it runs first, or that another test cleaned up after itself. Shared state: a database row, a file, a global that one test sets and another reads. Real time and dates, where a test written at 23:59 fails at midnight. Network calls to something outside your control.

How to find which one

Run the suspect test on its own, many times. If it is solid alone but flaky in the full run, the problem is order or shared state. Shuffle the test order deliberately and see if the failure moves. Add logging around the assertion that fails so you can see the actual value when it goes wrong, not just that it did.

The fixes

Replace fixed waits with waiting for a condition: the element is visible, the response has arrived, the queue is empty. Give each test its own data and tear it down, so nothing leaks between tests. Freeze time in tests that care about dates. Stub the network so a test never depends on a server you do not run. Where a test genuinely needs a real external service, mark it clearly and run it separately from the fast suite.

Keep it from coming back

Add a check that runs the changed tests a few times in a row on every pull request. A test that cannot pass three times running does not get merged. It is a small cost that stops the slow drift back into a suite nobody believes.

Why it is worth the effort

A test suite is only useful if a red result means something. Every flaky test chips away at that. Fixing the cause, rather than deleting the test, keeps the coverage and the trust.

Common questions

Why not just delete a flaky test?

It was written to catch something. Deleting it removes the coverage and the signal. Quarantine it, fix the cause, then bring it back.

What causes most flakiness?

Timing and order. Tests that wait a fixed number of milliseconds, or that depend on another test having run first and left state behind.

Should a flaky test fail the build?

While you investigate, move it to a separate group that is reported but does not block. Leaving it in the main run teaches everyone to rerun and ignore.