Reducing Flaky Tests

What Are Flaky Tests?

Flaky tests are tests that sometimes pass and sometimes fail without any code changes. They erode team confidence in CI and waste developer time.

Industry benchmarks suggest 2-15% of tests in a typical project are flaky.

How FixSense Helps

1. Automatic Detection

FixSense categorizes each failure and flags tests as flaky when:

• The same test fails intermittently across different PRs
• The failure pattern matches known flaky signatures (timing, network, race conditions)
• The test passes on retry without code changes

2. Top Failing Tests (Pro+)

The Top Failing Tests widget on your dashboard shows which tests fail most frequently. Focus on fixing these first for the biggest impact on CI reliability.

3. Fix Suggestions

For each flaky test, FixSense provides specific suggestions:

• Add explicit waits: await page.waitForSelector('.loaded')
• Use web-first assertions: await expect(locator).toBeVisible()
• Replace hard-coded timeouts with dynamic waits
• Mock unstable external dependencies

Best Practices

1. Fix the top 5 first — the Pareto principle applies: 20% of tests cause 80% of flaky failures
2. Use test.describe.configure({ retries: 2 }) sparingly — retries hide problems
3. Monitor trends — check your dashboard weekly for new flaky patterns
4. Set team KPIs — aim for less than 5% flaky test rate