Test Suites
Cypress
Debugging Tips

Debugging Tips

Replay can help you gather more information about a flaky test. There are many strategies that can help you narrow down the root cause of a test failure or flake.

Replay DevTools enable you to deep-dive into your code and print out values that would otherwise be hidden from you.

Comparing passed and failed test

Great way to narrow down the root casue of a flaky test is to open two replays - one with a passed test and one with the failed one. Usually the recording can give you enough information to notice the moment when these two tests start to diverge.

At this point, you can click Jump to code button and move from your test code into your app code. This allows you to see how did your test code impact the app itself. Since the recording will take you to an exact moment in test execution, you can examine your app’s function calls and reveal timing issues.

A good example can be found in this video (opens in a new tab), where a timing issue causes problems with adding an item to a cart.

Adding console logs in the spec file

Typically console logs are not that helpful in a spec file because the spec is run at the beginning of the test to build the test plan.

The exception for the rule is .should and other cases where the test code is run in the middle of the test and there are values that can be inspected.

Evaluating in Cypress callbacks

Most Cypress steps run at test start to build the test schedule. An exception to this rule is Cypress commands and should assertions.

In these cases you can jump to a line of code and evaluate expressions in the Console’s terminal.

  1. Hold down command and click the blue jump button
  2. Evaluate an expression in the Console’s terminal

Jump to application events

With Replay you can jump from a test cy.click() command into your React component’s onClick hander. In the example below, we jump from the test’s click command into the Todo app’s handleEdit callback.

Test burn-in

Rerunning your test repeatedly can be a helpful ways to catch a test flake. This strategy can sometimes be easier if you want to quickly record a flake locally.

cypress/e2e/login.js
describe(`Verify "Login" is visible. Test: ${i}`, () => {
  for (let i = 0; i < 3 ; i++) {
    it('finds the Login link in the header', () => {
      // Place code inside the loop that you want to repeat
    })
  }
})

To run your the test, use following command:

npx cypress run --spec cypress/e2e/login.js --browser replay-chromium
💡

Pro tip: You can use @cypress/grep package to repeat your tests without changing code in your spec. Read more about it in the package readme (opens in a new tab).

When running locally, you can use Replay CLI to manually upload your recordings to Replay Dashboard.

npx @replayio/replay upload-all

Great articles on test flakiness

ConfigurationFAQ