Tests using the 'currentURL()' started to randomly fail when the URL changes more than 2 times (in the same test case).
This started after the application was updated (ember-cli) from the version 1.X to the ~2.16.2.
I've tried (With no positive results):
Updating the ember-cli-qunit.
Using import { currentURL } from '#ember/test-helpers';.
Update:
Here is an example of one of those tests (Marked as 'Flaky' the problematic parts):
test('My test', async () => {
await visit('/testing-page-1');
const selectSortId = find('[data-test-select="my-select"]');
equal(selectSortId.val(), "1", "Wrong selection,...");
await selectSortId.val(3).trigger('change');
equal(currentURL(), '/testing-page-2'); // All good here
const firstEditButton = find('[data-test-button="1"]');
await click(firstEditButton);
equal(currentURL(), '/testing-page-3'); // Flaky
const secondButton = find('[data-test-button="2"]');
await click(secondButton);
equal(currentURL(), '/testing-page-4') // Flaky
});
I finally fixed this, and it was not an issue with the ember test suit...
There was an Ember.hash in the app, one of those calls had some data needed to populate our urls with the some parameters.
In production, the call with the data was noticeable slower than the others, so we luckily got the data.
In testing mode, we had this data being overwritten by the other calls.
i.e; it was a matter of a bad design needed to be refactored...
Related
I have a typical search where the user types some text in an input, async work is done and a table with the results is properly updated.
I have tests that must wait for this search step and then assert business rules regarding the results, like if the table records are eligible for edit.
Every time I ran a complete test battery (something like 80 test files), one or two of the tests involving that search inevitably fail. But if immediately after that, I run the same test alone, the test passes. It's excruciating and makes e2e testing in CI/CD pointless for the project.
I've read the Cypress documentation about flaky tests and searched questions in StackOverflow and GitHub with only complete failure. It's a drama.
Here is one of the tests:
import { searchList } from '../helpers';
import { createFluxoIniciado, randomFluxoNome } from './common';
import { fluxoSelectors } from './selectors';
describe('fluxos finish', () => {
it('can manually finish a fluxo INICIADO', () => {
// feed data to be searched
const fluxoNome = randomFluxoNome();
createFluxoIniciado({ fluxoNome });
// search
searchList(fluxoNome);
// do something with search results
fluxoSelectors.fluxos.view().click();
fluxoSelectors.finish().click();
fluxoSelectors.confirm().click();
// serach again
searchList(fluxoNome);
cy.contains('FINALIZADO');
});
});
The code in searchList is where trouble emerge sometimes. It uses the callback strategy recommended here. The code attempts to cause retries if not all rows have the searched text.
export function searchList (text) {
cy.get('#searchText')
.scrollIntoView()
.type(text)
.blur();
cy.get('tbody tr').should($trs => {
$trs.each((i, $tr) => {
expect($tr).to.contain(text);
});
}, { timeout: 15000 });
}
Here is an example of a test failure inside a run all test execution:
The problem is obviously caused by the async fetch between .blur() and testing the rows.
You are correctly trying to use Cypress retry with .should(callback), but if the callback is complex or there are multiple steps it may not retry the element that is changing (the table rows).
Ideally you want to keep the cy.get(...).should(...) as simple as possible, and start by testing that the table loading has completed.
// wait for expected number of rows
cy.get('tbody tr', {timeout: 15000}).should('have.length', 5)
cy.get('tbody tr').each($tr => {
expect($tr).to.contain(text);
})
But you have a randomizer there, so maybe it's not possible to test explicitly the row count.
Another approach, test the whole table for text (.contains() checks child text also)
// wait for text to appear somewhere
cy.get('tbody tr', {timeout: 15000}).should('contain', text)
cy.get('tbody tr').each($tr => {
expect($tr).to.contain(text);
})
You can also add an intercept between start and end of api call
export function searchList (text) {
cy.intercept('search/api/endpoint').as('search')
cy.get('#searchText')
.scrollIntoView()
.type(text)
.blur();
cy.wait('#search') // wait for api response
cy.get('tbody tr', {timeout: 15000}).should('contain', text)
cy.get('tbody tr').each($tr => {
expect($tr).to.contain(text);
})
}
I just noticed you have the {timeout} option on .should(), but that's the wrong place,
see Timeouts
cy.get('input', { timeout: 10000 }).should('have.value', '10')
// timeout here will be passed down to the '.should()'
// and it will retry for up to 10 secs
This may be successful
cy.get('tbody tr', { timeout: 15000 })
.should($trs => {
$trs.each((i, $tr) => {
expect($tr).to.contain(text);
});
})
I'm wanting to log into an app, run several searches from test data, then log out. I don't want to login and out for each item in the data set, which would be the case if I coded this way...
dataSet.forEach(data =>{
test('Search Test', async t => {......
I would like to be able to...
test('Search Test', async t => {......
foreeach(data in Data set)
call a function to search
call a function to verify search return.
Something like this...
test('Simple Search Test', async t => {
//await t
await loginPage.login(loginName, password);
await t
.expect(getURL()).contains('home')
// Check logged in user display...
.expect(pageHeader.userName.withText(data.loggedInUser).visible).ok()
dataSet.forEach(data =>{
leftSidebar.searchWithCriteria(data.criteria, 'Filename');
recordNav.verifyTotal(data.srchresult);
});
// Log out
await pageHeader.logout();
await t
.expect(loginPage.copyRight.visible).ok();
});
enter code here
I've tried everything, but can't get it to work. Is this possible or does the entire test have to be run for each data record in the set?
TestCafe allows you to loop through test code in any manner, including iterating through custom data.
To help us determine why this does not work for you, please provide an example that I can run on my machine (including the test code, page object, and the tested page's URL).
I got it to work using this...
for (var i = 0; i < dataSet.length; i++){
leftSidebar.searchWithCriteria(dataSet[i].criteria, 'Filename');
recordNav.verifyTotal(dataSet[i].srchResult);
}
I have this retarded amount of product in a collection on Shopify (over 50k products) and I would need to delete them all, is there a way I can automate that? All I can find on the internet is to use the "bulk edit tool" which is the most useless thing I've ever seen as it can only grab 50 products at a time.
I've tried automating a script to update the rows with the CSV export file, but it takes over 6 hours for 20K products to import. Plus, since there are hashtags in the title and handle, it apparently doesn't overwrite the products for some reason. So I just can't use the archive anymore...
Has anyone ran into this issue and found a solution?
Thank you!
When it comes to this kinds of tasks I usually write myself a quick dev console script that will do the job for me instead of relying on an app.
Here is a script that you can use in the dev console of your shopify admin page (just copy /paste):
let productsArray = [];
// Recursive function that will grab all products from a collection
const requestCollection = (collectionId, url = `https://${window.location.host}/admin/api/2020-10/collections/${collectionId}/products.json?limit=250`) => {
fetch(url).then(async res => {
const link = res.headers.get('link');
const data = await res.json();
productsArray = [...productsArray, ...data.products];
if(link && link.match(/<([^\s]+)>;\srel="next"/)){
const nextLink = link.match(/<([^\s]+)>;\srel="next"/)[1];
requestCollection(collectionId, nextLink)
} else {
initDelete(productsArray)
}
})
}
// Get CSRF token or the request will require password
const getCSRFToken = () => fetch('/admin/settings/files',{
headers: {
"x-requested-with": "XMLHttpRequest",
"x-shopify-web": 1,
"x-xhr-referer": `https://${window.location.host}/admin/settings/files`
}
}).then(res => res.text()).then(res => {
const parser = new DOMParser();
const doc = parser.parseFromString(res, 'text/html');
return doc.querySelector('meta[name="csrf-token"]').getAttribute('content')
})
// The function that will start the deleting process
const initDelete = async (products) => {
const csrfToken = await getCSRFToken();
products.forEach(item => {
fetch(`https://${window.location.host}/admin/api/2020-10/products/${item.id}.json`, {
method: "delete",
credentials: 'include',
headers: {
"x-csrf-token": csrfToken,
"x-requested-with": "XMLHttpRequest",
"x-shopify-web": 1,
"x-xhr-referer": `https://${window.location.host}/admin/settings/files`
}
})
})
}
And you start it by using requestCollection(ADD_YOUR_COLLECTION_ID_HERE).
To clarify the script, there are 3 main functions:
requestCollection - this handles the product grabbing from the collection. It's a recursive function since we can't grab more than 250 products at the same time.
getCSRFToken - this grabs the CSRF token since most of the post/update/delete request requires it or they will fail (I grab it from the files page)
initDelete - this function start the delete process where we stack all the request one of the other without waiting, you may want to await each request, but even if you crash your browser I think it will be still faster to repeat the process rather then wait for each request to finish.
If you plan to use this script please TEST IT BEFORE USING IT. Create a collection with a few products and run against that, in case there are issues. I've tested it on my side and it's working but it's a code I wrote in 10 minutes after midnight, there can be issues there.
Have in mind that this script will delete ALL products in the collection you specify in the requestCollection(1231254125) method.
PS: All of this can be done using a Private App as well with the products scope set to read/write, using a back-end language of your choice. The main difference will be that you won't need the CSRF token and most of the headers that I set above. But I like quick solutions that doesn't require you to pull out the big guns.
I am trying to achieve a board that enables real-time editing for cooperating users. I am running a parse server using Sashido, where I have LiveQueries enabled for among other things 'Sticky'. For the frontend I have the following code:
const query = new Parse.Query(Sticky);
... some query constraints
this.subscription = await query.subscribe();
this.subscription.on('open', () => {
console.log('SUBSCRIPTION: opened');
}
this.subscription.on('create', (sticky) => {
console.log('SUBSCRIPTION: created');
}
this.subscription.on('update', (sticky) => {
console.log('SUBSCRIPTION: updated');
}
this.subscription.on('enter', (sticky) => {
console.log('SUBSCRIPTION: entered');
}
this.stickies = await query.find();
When I open my application in two different browser tabs, I get the 'SUBSCRIPTION: opened'. When I edit or create Sticky instances, I expect to get the corresponding events and see changes in the Sashido database.
However, I always see the changes in the database, but half of the times when I create or edit Sticky instances, I do not get the update/create/enter events. Note: Sometimes they do get triggered, but I have not found a sequence of events that leads to them being triggered or not, it seems to happen at random.
Can someone see what I'm doing wrong?
After sending keys to an input field with selenium, the result is not as expected - the keys are inserted in incorrect order.
e.g. send_keys('4242424242424242') -> result is "4224242424242424"
EDIT: On some machines I observe the issue only randomly, 1 case out of 10 attempts. On another machine it is 10/10
This happens specifically with Stripe payment form + I see this problem only in Chrome version 69 (in previous versions it worked OK)
This can be easily reproduced on sample Stripe site: https://stripe.github.io/elements-examples/
Sample python code:
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('https://stripe.github.io/elements-examples/')
driver.switch_to.frame(driver.find_element_by_tag_name('iframe')) # First iframe
cc_input = driver.find_element_by_css_selector('input[name="cardnumber"]')
cc_input.send_keys('4242424242424242')
Result:
I am able to get pass this by sending the keys one by one with slight delay - but this is also not 100% reliable (plus terribly slow)
I am not sure if this is a problem with selenium (3.14.1)/chromedriver (2.41.578737) or if I am doing something wrong.
Any ideas please?
We are having the exact same problem on MacOS and Ubuntu 18.04, as well as on our CI server with protractor 5.4.1 and the same version of selenium and chromedriver. It has only started failing since Chrome 69, worse in v70.
Update - Working (for the moment)
After much further investigation, I remembered that React tends to override change/input events, and that the values in the credit card input, ccv input etc are being rendered from the React Component State, not from just the input value. So I started looking, and found What is the best way to trigger onchange event in react js
Our tests are working (for the moment):
//Example credit input
function creditCardInput (): ElementFinder {
return element(by.xpath('//input[contains(#name, "cardnumber")]'))
}
/// ... snippet of our method ...
await ensureCreditCardInputIsReady()
await stripeInput(creditCardInput, ccNumber)
await stripeInput(creditCardExpiry, ccExpiry)
await stripeInput(creditCardCvc, ccCvc)
await browser.wait(this.hasCreditCardZip(), undefined, 'Should have a credit card zip')
await stripeInput(creditCardZip, ccZip)
await browser.switchTo().defaultContent()
/// ... snip ...
async function ensureCreditCardInputIsReady (): Promise<void> {
await browser.wait(ExpectedConditions.presenceOf(paymentIFrame()), undefined, 'Should have a payment iframe')
await browser.switchTo().frame(await paymentIFrame().getWebElement())
await browser.wait(
ExpectedConditions.presenceOf(creditCardInput()),
undefined,
'Should have a credit card input'
)
}
/**
* SendKeys for the Stripe gateway was having issues in Chrome since version 69. Keys were coming in out of order,
* which resulted in failed tests.
*/
async function stripeInput (inputElement: Function, value: string): Promise<void> {
await browser.executeScript(`
var nativeInputValueSetter = Object.getOwnPropertyDescriptor(window.HTMLInputElement.prototype, "value").set;
nativeInputValueSetter.call(arguments[0], '${value}');
var inputEvent = new Event('input', { bubbles: true});
arguments[0].dispatchEvent(inputEvent);
`, inputElement()
)
await browser.sleep(100)
const typedInValue = await inputElement().getWebElement().getAttribute('value')
if (typedInValue.replace(/\s/g, '') === value) {
return
}
throw new Error(`Failed set '${typedInValue}' on ${inputElement}`)
}
Previous Idea (only worked occasionally):
I have setup a minimal repro using https://stripe.com/docs/stripe-js/elements/quickstart and it succeeds when tests are run sequentially, but not in parallel (we think due to focus/blur issues when switching to the iframes).
Our solution is similar, although we noticed from watching the tests that input.clear() wasn't work on tel inputs which are used in the iframe.
This still fails occasionally, but far less frequently.
/**
* Types a value into an input field, and checks if the value of the input
* matches the expected value. If not, it attempts for `maxAttempts` times to
* type the value into the input again.
*
* This works around an issue with ChromeDriver where sendKeys() can send keys out of order,
* so a string like "0260" gets typed as "0206" for example.
*
* It also works around an issue with IEDriver where sendKeys() can press the SHIFT key too soon
* and cause letters or numbers to be converted to their SHIFT variants, "6" gets typed as "^", for example.
*/
export async function slowlyTypeOutField (
value: string,
inputElement: Function,
maxAttempts = 20
): Promise<void> {
for (let attemptNumber = 0; attemptNumber < maxAttempts; attemptNumber++) {
if (attemptNumber > 0) {
await browser.sleep(100)
}
/*
Executing a script seems to be a lot more reliable in setting these flaky fields than using the sendKeys built-in
method. However, I struggled in finding out which JavaScript events Stripe listens to. So we send the last key to
the input field to trigger all events we need.
*/
const firstPart = value.substring(0, value.length - 1)
const secondPart = value.substring(value.length - 1, value.length)
await browser.executeScript(`
arguments[0].focus();
arguments[0].value = "${firstPart}";
`,
inputElement()
)
await inputElement().sendKeys(secondPart)
const typedInValue = await inputElement().getAttribute('value')
if (typedInValue === value) {
return
}
console.log(`Tried to set value ${value}, but instead set ${typedInValue} on ${inputElement}`)
}
throw new Error(`Failed after ${maxAttempts} attempts to set value on ${inputElement}`)
}
I faced a similar issue in ubuntu 14.04, the following trick helped me.
Have not got any issue since.
First I used the regular send_keys method.
Then I called the execute script to update the value
input_data = "someimputdata"
some_xpath = "//*[contains(#id,'input_fax.number_')]"
element = web_driver_obj.find_element_by_xpath(some_xpath)
element.clear()
element.send_keys(input_data)
web_driver_obj.execute_script("arguments[0].value = '{0}';".format(input_data), element)
Edit
Thanks a lot to #Benno - his answer was correct.
I will just add python solution that worked for me, based on his JS
driver.get('https://stripe.github.io/elements-examples/')
driver.switch_to.frame(driver.find_element_by_tag_name('iframe')) # First iframe
cc_input = driver.find_element_by_css_selector('input[name="cardnumber"]')
value = "4242424242424242"
driver.execute_script('''
input = arguments[0];
var nativeInputValueSetter = Object.getOwnPropertyDescriptor(window.HTMLInputElement.prototype, "value").set;
nativeInputValueSetter.call(input, "{}");
var eventCard = new Event("input", {{bubbles: true}});
input.dispatchEvent(eventCard);
'''.format(value), cc_input)
driver.switch_to.default_content()
driver.quit()
After couple of hours of trying, I gave up and accepted the fact that this is really a random issue and went with a workaround.
Where it is not necessary to update, I will stay with Chrome version < 69
In order to test latest Chrome, I will use React solution
What I've found out
The issue manifested itself mostly on MacOS, quite rarely on Windows (there are most probably other factors in play, this is just an observation)
I've run an experiment with 100 repetitions of filling the form.
Mac - 68 failures
Windows - 6 failures
The cookies/local history (as suggested in comments) do not seem to be the problem. The webdriver always spawned a "clean" instance of the browser, with no cookies or local storage.
Maybe my solution will help for somebody:
I used sendKeys(" 4242424242424242")
Same for cvc field
With a space before string, it actually works for selenide + chrome + java
You could make your own generic SendKeys method that takes the input element and the string you would like to send. The method would split the string into individual characters and then use the selenium sendkeys method on each character.
Adding in some backspaces worked for me for whatever reason:
from selenium.webdriver.common.keys import Keys
my_value = "123"
my_xpath="//input[#class='form-text']"
element = driver.find_element_by_xpath(my_xpath)
element.clear()
element.send_keys(Keys.BACKSPACE * 3, my_value)
I was having the same issue using RSelenium and got the idea to try adding spaces to the credit card number as they appear on the card from #Pavel's answer, since adding a space before the card number didn't work for me.
Using RSelenium this would be:
element$sendKeysToElement(list("4242 4242 4242 4242"))