Test Suites
Join the waitlist
Replay for Test Suites was released in June 2023 as a beta so that we can help each team setup Replay and fix their flakes. If you’d like to start recording your tests, join our waitlist (opens in a new tab) and we’ll reach out to you as soon as possible. Also feel free joining our Discord (opens in a new tab) #testing channel or emailing us at hey@replay.io.
Integrating Replay Browser into your test suite is as easy as telling your test framework which browser to run on. A simplified example for each test framework of integration will look something like this:
npx playwright test --project replay-chromiumClick here Learn more about how to integrate into Playwright
Whenever you run your tests, the test execution will be recorded and a replay will be created. This allows you to debug failures, analyse the root cause of a flaky test and much more. With frameworks such as Playwright and Cypress, we bring you a custom sidebar that allows you to examine each test step.
Benefits of integrating Replay to your test suite
- See what happened on CI
- Tests fail on CI but pass locally? Recording a replay of your tests will give you insight you needed.
- Debug with ease
- Finding a root cause is much easier whan your test flake is recorded in a replay. No more guessing games. Examine the test flake at the very place it happened.
- Goodbye flaky tests
- With Replay DevTools you dive deeper than with any other tool. Test flakiness is no longer a mystery, but something that can be found and fixed.
- Explore both test and app code
- Flakiness can come from anywhere - both the app and the test. You cannot make them stable by ignoring one or the other. Replay allows you to take them both into consideration
Dashboard
The Test Suite Dashboard provides you with an overview of all the tests and test runs that were recorded with Replay Browser. This will serve you as a great reporter tool, but with every test result, you’ll have access to the test recording. If anything went wrong, you can start debugging with all the powers that Replay DevTools provide you with.
Each test run contains information on failed, passed and flaky tests.
You can choose to open a replay that ran on a failed test, or start debugging a flaky one.
The Test Suite Dashboard is your launching off point. If you want to see why a recent change caused several tests to fail, you can go to the dashboard, and open each of the replays and begin debugging.
Replay DevTools
Replay DevTools enable you to see what happened during your test run. This is great for flaky tests and revealing timing issues that tend to happen when running end-to-end tests. You can deep dive into your test code as well as your application code and make your app and tests work well together.
Cypress panel
Examine each command step by step, travel back in time and see what each command returned. This will bring the great experience from your local Cypress GUI mode into your CI execution as well.
For action commands, you can click "Jump to code" to see what part of your application code was actually triggered. This empowers you to fix test dlakes at the exact place they happen.
All network requests are captured as well, including static assets. With your test result you are getting all the powers of Replay DevTools that allow you to take a deep dive into your test execution and examine your code using print statements, debugger and React panel.
Ready to integrate Replay to Cypress? Learn how to setup here.
Playwright panel
Playwright panel will provide you with a trace-viewer-like experience in which you can examine not only your test code, but your application code as well. You’ll get more insight into CI execution and fight off flakes more effectively.
With every recorded test, you can examine your code using Replay DevTools. You can take a look at network calls, React components of your app, or you can debug your test using print statements and debugger.
Learn how to integrate Playwright into Pipeline here
Other testing tools
Are you using other tools? We got you covered. You can use Replay Browser to record with Puppeteer, WebdriverIO and Jest (alpha). Your recorings will be available in the same way, so you can examine your code and search for flakes with these frameworks as well.
Integrating into CI
Recording your tests is as simple as running them with the Replay browser. You can record your test suite in any CI provider. This recording happens in two steps:
- Run your tests in Playwright, Cypress, Puppeteer or WebdriverIO using Replay Browser
- Upload your recordings to Replay Dashboard using your API key
After this, you can view your test results in Replay Dashboard and debug your failed and flaky tests with Replay DevTools.
Read these guides to see how to set up your pipeline in CircleCI, GitHub Actions and other providers.
Pull request comments
For a smoother workflow, we have created a GitHub App (opens in a new tab) that will add comments to your pull request showing all your failed an passed test. Clicking on a failed test will take you directly to the Replay DevTools where you can debug your test.
Learn more
Integrating Replay with your existing test suite is pretty straightforward, but we have prepared some more detailed tutorials to walk you through the basic setup and some advanced features. Click below to choose the framework of your choice.