False positive failures are a really common issue with existing E2E testing tools, so we try to do a number of things to prevent them in Reflect tests:
- We generate multiple selectors for each element you interact with. So if in the future you change a class or id, we'll fallback to other selectors when running your test. We also only choose a selector if (1) it matches a single element in the DOM, and (2) that element is interactable (e.g. not completely hidden), and (3) if it has text associated with it then we'll only choose it if the text captured at recording time matches the text at run time. This helps prevent us from selecting the wrong element when running a test.
- For React apps, we use heuristics to ignore classes generated by CSS-in-JS classes like Styled Components and EmotionJS. We also have the ability to target elements based on React component name (it requires a one-time setup on your side to enable this)
- For Visual Testing failures (e.g. you've screenshotted an element and now that element has a different UI) we have a simple workflow to 'Accept Changes' and mark the new UI as the new baseline for tests going forward.
Certainly more to do here but this is one of the key problems we're looking to tackle.
1) You can delete steps on a recorded test but not during the recording. This is to ensure that we have an exact copy of the initial recording. Thereafter, you can click on the test step and click "Delete" on the middle pane.
2) These should be captured out of the box. Can you email me your test URL so I can investigate? fitz at Thank you!
- We generate multiple selectors for each element you interact with. So if in the future you change a class or id, we'll fallback to other selectors when running your test. We also only choose a selector if (1) it matches a single element in the DOM, and (2) that element is interactable (e.g. not completely hidden), and (3) if it has text associated with it then we'll only choose it if the text captured at recording time matches the text at run time. This helps prevent us from selecting the wrong element when running a test.
- For React apps, we use heuristics to ignore classes generated by CSS-in-JS classes like Styled Components and EmotionJS. We also have the ability to target elements based on React component name (it requires a one-time setup on your side to enable this)
- For Visual Testing failures (e.g. you've screenshotted an element and now that element has a different UI) we have a simple workflow to 'Accept Changes' and mark the new UI as the new baseline for tests going forward.
Certainly more to do here but this is one of the key problems we're looking to tackle.