I'm currently in the process of setting up Cypress for my project. Currently we're only using testing library for frontend tests. And reading the Cypress documentation has gotten me a bit confused as the two libraries seem to have opposite philosophies in regards to how you're supposed to query for elements.
Testing library basically says test what the user can see/touch and only use data-testid if all else fails. Cypress on the other hand states that best practice is that you should query elements by data-testid / data-cy attributes.
I feel conflicted between the two approaches. I get the point about we should test what the user actually sees (testing library). But I do also get that those things often change (cypress) and we need to spend time updating tests whenever we make small changes (i.e "Ok" -> "Done"). And when testing with data-cy attributes, are we not also ignoring accessibility / screen readers?
What are your thoughts on this?
React Testing library(RTL) is specifically made to write tests from a user perspective. From their Guiding Principles:
The more your tests resemble the way your software is used, the more confidence they can give you.
Meaning, RTL wants you to use accessibility queries like getByRole and only fallback to getByTestId for cases where you can't match by role or text, or it doesn't make sense (e.g. the text is dynamic).
However, thanks to the render method allowing us to specify props (compared to Cypress), we have much more flexibility and may entirely omit dynamic text.
Cypress, on the other hand, runs with all dependencies. In case of dynamic content from a C.M.S or multi-language application, things are not that easy using getByRole("heading", {name: /welcome/i }). In this case, the recommendation of testId's make sense.
My personal recommendation is to use accessibility query selectors in both Cypress and RTL, unless the text is dynamic. Then testId's in Cypress and a combination of testId's & accessibility query selectors provide the best solution.
It should also be noted that Playwright and Cypress test-generator tools select by accessibility query selectors.
I thought a lot for a few days before answering this question and I even got to do some tests, and after that time I came to the conclusion that the Cypress approach is the best.
The main reason that led me to this answer was that when the testing library says that we should test what the user actually sees, it is already applicable in Cypress even when we use a data-testid, because let's suppose you have a button that exists in the DOM, but it is not visible when you select this button, with the data-testid when you try to click in this button Cypress will return an error saying that the button is not visible and if you really want to do this action you must apply force:true. The same happens if the button is not clickable or if there is another element in front of the button.
Cypress already checks by default in click and type actions if the element:
element is into view
it is visible
it is not disabled
it is not detached
it is not readonly
it is not animating
it is not covered
fire the event at a
descendent
Also if you fetch the element by text, placeholder, or class, this does not guarantee that the element is actually visible to the user, as the element can be in the DOM and not be visible to the user for various reasons.
So the best way to make tests easier to maintain, easier to read, and avoid flaky tests is to use the data-testid and whenever possible or necessary combine the location of the element with an assertion to ensure that the element is visible. Example:
cy.get('[data-testid="button"]').should('be.visible')
I hope I had contributed to the discussion and would love to hear other people's points of view.
Related
I want to evaluate my content blocks before running my test suite but the closures' property names is in bytecode already. I'm ooking for the cleanest solution (compared with parsing source manually).
Already tried solution outlined in this post (and I'd still wind up doing some RegEx/parsing) but could only get it to work via script execution engine. It failed in IDE and GroovyConsole. Rather than embedding a Groovy script in project's code, I thought I'd try using Geb's native classes.
Is building on the suggestion about extending Geb Navigators here viable for Geb's PageContentSupport class whose contentTemplates contain a LinkedHashMap of exactly what I need? If yes, would someone provide guidance? If no, any suggestions?
It is currently not possible to get hold of all content elements for a given page/module. Feel free to create an issue for this in Geb's bug tracker, but remember that all that Geb can provide is either a list of content element names or a map from these names to closures that create these elements.
Having that information isn't a generic solution to your problem because it's possible for content elements to take parameters and there are situations where your content elements will be available on the page only after some other actions are performed (for example you have to click on button to reveal a section of a page that uses ajax to retrieve it's content). So I'm afraid that simply going over all elements and checking if they don't throw any errors will not cut it.
I'm still struggling to see what would "evaluating" all content elements prior to running the suite buy you. Are you after verifying that your content elements still work to get a faster feedback than running the whole suite? I'm pretty sure that you won't be able to fully automate detection of content definitions that don't work anymore. In my view it will be more effort than it's worth.
I want to check whether a possibly mandatory field is filled in using selenium. I would very much prefer to do this using the IDE rather than exporting to code.
To be more specific, I want to make sure that a post code field is filled in if the value of the country drop down list is 'United Kingdom'. The post code field can be left blank if the drop down list is set to anything else.
If the consensus is that I have to export and do the test in code, then I will, but I would be a lot happier if I could sort this out just through the IDE.
Unfortunately, Selenium-IDE does not offer any flow control functionality by default. However, the Flow Control plugin introduces a few new commands - while, gotoIf, label and the ...andWait variants. Using these, you should be able to check the currently selected value in the dropdown menu, and jump past a verify command if the value is not "United Kingdom".
Fair warning; it's not the easiest thing in the world to work with. For anything beyond very basic flow control, you'd probably be better off exporting to code anyway. But, it's an option for when you really want to push the IDE!
I'm automating a workflow (survey) . This has few questions on each page.
Each page has few questions and a continue button .Depending on your answers next pages load. .How can I automate this scenario.
TL;DR: Selenium should only form a part of your automated testing strategy & it should be the smallest piece. Test variations at a lower level instead.
If you want to ensure full coverage of all possibilities, you've two main options:
Test all variants through browser-based journey testing
Test variations outside of the browser & just use Selenium to check the higher-level wiring.
Option two is the way to go here — you want to ensure as much as possible is tested before the browser level.
This is often called the testing pyramid, as ideally you'll only have a small number of browser-based tests, with the majority of your testing done as unit or integration tests.
This will give you:
much better speed, as you don't have the overhead of browser load to run each possible variant of your test pages.
better consistency, i.e. with unit tests you know that they hold true for the code itself, whereas browser-based tests are dependent on a specific instance of the site being deployed (and so bring with it the other variations external to your code, e.g. environment configuration)
Create minimal tests in Selenium to check the 'wiring'.
i.e. that submitting any valid values on page 1 gives some version of page 2 (but not testing what fields in particular are displayed).
Test other elements independently at a lower level.
E.g. if you're following an MVC pattern:
Test your controller class on it's own to see that with a given
input, you get are sent to the expected destination & certain fields populated in the model.
Test the view on it's own that given a certain model, it can display all the variations of the HTML, etc.
It will be better to give if else statements and automate the same. Again it depends on how much scenarios u need to automate.
I've read the documentation and understand that this is to be expected:
Scripts are injected into the top-level page and any children with HTML sources, such as iframes. Do not assume that there is only one instance of your script per browser tab.
I'm wondering, though:
Other than iframes, what other elements have "HTML sources" (images? objects?)? The term "HTML sources" is uncomfortably vague to my ears.
Is there any way to detect which element is executing the script?
I've filtered out iframes by determining that window === window.top, as recommended, but other elements are still executing the script and it's executing a lot more than I'd like.
Thanks.
It's really my own fault that I'm answering my own question because I didn't really provide enough information in my question. In my defense, I didn't know at the time that I was providing too little information.
Anyway, while traveling down the road to a solution for this, I asked a question on Apple's dev forum and included the following bit of key info:
Everything in the script occurs on the beforeload event of the document (or was supposed to).
What I learned was that the beforeload event only fires for subresources within the document. Not for the document (or window) itself. I removed the event handler and made sure that the script was applied as a start script (it was). I'd already applied the test for the window as top window so I was covered. Now my injection script only fires once.
Other HTML sources may be frames or object tags (with HTML contents). I don't think it can be anything else. However, to the extent of my knowledge, they should also be filtered off with window === window.top. Try console.loging the document.location variable to see which URL runs your injected script, and maybe you can find what loads them.
I'm looking for a way to process a web page and associated Javascript from the command-line, so that the resulting DOM model can be outputted.
The purpose for this is to identify forms within the page without doing any nasty HTML (and Javascript) parsing with regular expressions.
Are there any command-line tools that will do this? So hypothetically speaking, a command-line web browser that downloads the content and outputs the DOM as text rather than producing a pretty page.
I don't know of any, but I wanted to highlight one difficulty with what you've suggested:
process a web page and associated Javascript
When would the output be? Many webpages have time-sensitive javascripts, or onclick/onhover scripts which would affect the DOM. Would you want these to be executed? All of them, or only some? It's not trivial to decide when the page is "finished" and ready for the DOM to be output after javascript manipulation. (Before javascript manipulation, it's an easier problem; just wait till the document.DOMReady event...)
Edit: I'm not saying that you don't need javascript execution at all: you might want to handle any document.write sections during loading, as they might write out a form... I'm saying it's hard to know when you've done "enough" javascript...
For java, I've had fairly good experiences with htmlunit.
I've also used the BeautifulSoup python library to parse forms and formdata. No need to specify regexps, as it'll let you traverse the DOM tree without much effort.