Skip to content

e2e: Fix flaky login and redirect tests#2521

Open
jcscottiii wants to merge 1 commit into
mainfrom
jcscottiii/fix-flakiness
Open

e2e: Fix flaky login and redirect tests#2521
jcscottiii wants to merge 1 commit into
mainfrom
jcscottiii/fix-flakiness

Conversation

@jcscottiii

Copy link
Copy Markdown
Collaborator

Resolve flakiness in E2E tests by using deterministic waiting mechanisms for backend API responses.

  • Login Test (login.spec.ts):

    • Make the syncing state screenshot test deterministic by intercepting the /v1/users/me/ping API request and blocking it using a Promise until both light and dark theme screenshots are captured. This prevents the UI from prematurely transitioning to the idle state, which was causing race conditions and snapshot mismatches (especially in WebKit).
  • Redirect Tests (feature-page.spec.ts, utils.ts):

    • Synchronize E2E tests by waiting for backend API GET responses before asserting redirects, rather than relying on page load events or timing out.
    • In feature-page.spec.ts:
      • Update redirects for a moved feature test to wait for both the redirect response (old-feature) and the redirect target response (new-feature), filtering both by GET method to avoid matching CORS OPTIONS preflights. This ensures the test doesn't assert the URL before the redirect target has finished loading, resolving failures on cold start cache misses.
      • Update shows gone page for a split feature to wait for the before-split-feature GET response before asserting the 410 URL.
    • In utils.ts:
      • Update goTo404Page helper to wait for the feature's GET response before asserting the 404 URL.

This shifts network wait times to the safe 30-second default timeout of waitForResponse, preventing them from eating into the strict 5-second assertion timeout of toHaveURL in resource-constrained CI environments.

Resolve flakiness in E2E tests by using deterministic waiting mechanisms for backend API responses.

- Login Test (`login.spec.ts`):
  - Make the syncing state screenshot test deterministic by intercepting the `/v1/users/me/ping` API request and blocking it using a Promise until both light and dark theme screenshots are captured. This prevents the UI from prematurely transitioning to the idle state, which was causing race conditions and snapshot mismatches (especially in WebKit).

- Redirect Tests (`feature-page.spec.ts`, `utils.ts`):
  - Synchronize E2E tests by waiting for backend API GET responses before asserting redirects, rather than relying on page load events or timing out.
  - In `feature-page.spec.ts`:
    - Update `redirects for a moved feature` test to wait for both the redirect response (`old-feature`) and the redirect target response (`new-feature`), filtering both by `GET` method to avoid matching CORS `OPTIONS` preflights. This ensures the test doesn't assert the URL before the redirect target has finished loading, resolving failures on cold start cache misses.
    - Update `shows gone page for a split feature` to wait for the `before-split-feature` GET response before asserting the 410 URL.
  - In `utils.ts`:
    - Update `goTo404Page` helper to wait for the feature's GET response before asserting the 404 URL.

This shifts network wait times to the safe 30-second default timeout of `waitForResponse`, preventing them from eating into the strict 5-second assertion timeout of `toHaveURL` in resource-constrained CI environments.
@neilv-g

neilv-g commented Jun 6, 2026

Copy link
Copy Markdown
Collaborator

If not already done, it would be good to verify the fix by running the tests with npx's --repeat-each flag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants