I am trying to submit/callback after entering h-captcha-response and g-recaptcha-response with the solved token but I don't understand how I am supposed to submit it.
How can I submit the hCaptcha without form,button or data-callback.
Here is the entire HTML of the page containing the hCaptcha.
https://justpaste.me/57J0
You have to find in the javascript files a specific function (like "testCaptcha") who submit the answer. When you find it, you can call it like this:
captcha = yourTOKEN
driver.execute_script("""
let [captcha] = arguments
testCaptcha(captcha)
""", captcha)
Could you please precise an URL where you have this captcha ? It'll be helpful to find this specific function.
Related
I am testing a login functionality on a 3rd party website. I have this url example.com/login . When I copy and paste this into the browser (chrome), page sometimes load, but sometime does not (empty blank white page).
The problem is that I have to run a script on this page to click one of the elements (all the elements are embedded inside #shadow-root). If the page loads, no problem, script is evaluated successfully. But page sometimes does not load and it returns a 404 in response to an XHR request, and as a result, my * eval(scrip("script") step returns "js eval failed...".
So I found the solution to refresh the page, and to do that, I am considering to capture the xhr request response. If the status code is 404, then refresh the page. If not, continue with the following steps.
Now, I think this may work, but I do not know how to implement karate's Intercepting HTTP Requests. And firstly, is that something doable?
I have looked into documentation here, but could not understand the examples.
https://github.com/karatelabs/karate/tree/master/karate-netty
Meanwhile, if there is another way of refreshing the page conditionally, I will be more than happy to hear about it. Thanks anyone in advance.
First, using JavaScript you should be able to handle shadow roots: https://stackoverflow.com/a/60618233/143475
And the above answer links to advanced examples of executing JS in the context of the current page. I suggest you do some research into that, try to take the help of someone who knows JS, the DOM and HTML well - and you should be find a way to know if the XHR has been made successfully or not - for e.g. based on whether some element on the page has changed etc.
Finally here is how you can do interception: https://stackoverflow.com/a/61372471/143475
This is my scenario "i am submitting the form and once I submit its navigate into page call WebTable.I want to confirm now am incorrect page after submitting the form"
how to check this by using assertion? please help selenium /Java /TestNG
In my experience, you have basically two options available to you.
Wait for the URL to contain what you expect using driver.getCurrentUrl()
Wait for a specific element that will only appear on the next page
If you don't have a class already built for performing retries, you can use FluentWait.
Once you click the form you wait for an element to show up using Webdriverwait.
You pull the current URL and check with value and assert it.
submitBtn.click();
WebDriverWait(driver,10).until(ExpectedConditions.visibilityOfElementLocated(By.id("etc")));
expectedUrl = driver.getCurrentUrl();
Assert.assertTrue(expectedUrl.equals("something"));
Selenium provides a good practice for this type of validation, and here comes FluentWait:
FluentWait fluentWait = new FluentWait(webDriverBPO).withTimeout(Duration.ofSeconds(20)).pollingEvery(Duration.ofSeconds(1)).ignoring(NullPointerException.class);
All you have to do after your last action, is to wait until URL contains your expected value:
fluentWait.until(ExpectedConditions.urlContains("your URL or keyword from URL"));
You can try also to wait for any element from the new page, but the URL loads faster than the content, so you will save some time (as you don't need to perform any other validations).
I've been trying to figure this out using various different methods. I'm trying to create a script/bookmark or some type of quick action to open a browser tab or window with a specific URL, and automatically log me in using my credentials. I'm not all that concerned about security for this at the moment.
At first I figured I'd try to use a javascript bookmark to do this, but nothing I found in my research worked. Next I tried to create a bash script, but I couldn't figure out how to send the credentials in via the terminal. Most recently, I literally copied the source code of a site, created a local file and tried to hack together something where I could prefill the form data with credentials and use JS to submit the form, and I've gotten close with this, but for some reason when I use the JS submit function, it errors out and says that the username and password are invalid. But when i turn off the submit function and manually click "log in" on my local html page, it works as expected. I want this to be a one click process, so the idea of using onload/submit or something to that affect is really important to me.
The site I'm testing with has a Rails backend and my next attempt might be trying to use POST to do what I'm thinking, but that's currently outside of my level of knowledge on the subject.
Anyone answering: i do not want to use a password manager to accomplish this.
My requirement is that i will either be able to a) run a script or b) use a 1-click option to do this per website. Ideally i'd be able to set this up in a sort of programmatic way to do this with multiple sites, but I'd be happy with 1 at the moment.
i know similar questions have been answered before, but I haven't been able to use information from those posts (the ones I've seen anyway) to figure out a good way to do this.
Create a bookmark for the current page you have opened.
Edit the bookmark
Change the value for the URL to something like this.
(javascript:(function(){CODE_GOES_HERE_FROM_BELLOW})();
find the field for username and password on the page.
Given example for hotmail
var inputs = document.getElementsByTagName('input'); for(var i=0;i<inputs.length;i++){if(inputs[i].name === 'passwd'){inputs[i].value = 'YOUR_PASSWORD'}else if(inputs[i].name === 'loginfmt'){inputs[i].value = 'YOUR_USERNAME'}}; document.getElementById(document.getElementsByTagName('form')[0].id).submit();
OR
try out casperjs.
The proposed solution didn't work for me and rather than spending tons of time installing a testing framework that I'll never use other than for this purpose, I decided to try to do this another way.
First, I found out that the reason my JS wasn't working before is because the site did not allow a JS submit to be done, or atleast that's what it seemed to be when I got this error: "Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience"
The javascript I was using was in fact working, just not submitting. I used the following code to fill the fields (using "Class Name" elements on the page since there was no name or ID):
document.getElementsByClassName('username')[0].setAttribute('value', 'user');
document.getElementsByClassName('password')[0].setAttribute('value', 'password');
As I mentioned, the problem was when I tried to use JQuery to submit the form: document.getElementsByClassName('loginForm')[0].submit();
Which is when the above error cropped up. I can't really say for sure whether this is the root of the cause, but the page does submit, but I get an invalid username/password error when I do
I haven't figured out a great way to get around this just yet, but my short-term, "hacky" solution was to use Applescript to send a return keystroke to the browser to submit the form. I'd ideally like to figure out how to get the submission to work using JQuery, but I'm not sure how to get around it.
Hi all I have seen lots of questions regarding this. I know that javascript dynamic page will rendered using scrapyjs or webdriver like selenium or phantomjs. webdriverkit is bit slow. I want somebody to guide me in this link
Price info before view deal button. I don't know which js is executing for this to use splash, scrapyjs can someone help me for this link.
thanks in advance.
EDIT
as per andres reply i have recreated XHR request. when we enter the XHR request url in browser window since it is a GET method if first hit i got partial json output. if we hit reload next time it loads more data that seems weired. can anyone help me in this. thanks in advance
When you request this URL:
http://ar.trivago.com/?iPathId=38715&iGeoDistanceItem=47160&aDateRange%5Barr%5D=2016-01-01&aDateRange%5Bdep%5D=2016-01-02&iRoomType=7&tgs=4716002&aHotelTestClassifier=&aPriceRange%5Bfrom%5D=0&aPriceRange%5Bto%5D=0&iIncludeAll=0&iGeoDistanceLimit=20000&aPartner=&iViewType=0&bIsSeoPage=false&bIsSitemap=false&
An XHR request is made to:
http://ar.trivago.com/search/region?iPathId=38715&bDispMoreFilter=false&iSlideOutItem=47160&aDateRange%5Barr%5D=2016-01-01&aDateRange%5Bdep%5D=2016-01-02&aCategoryRange=0%2C1%2C2%2C3%2C4%2C5&iRoomType=7&sOrderBy=relevance%20desc&aPartner=&aOverallLiking=1%2C2%2C3%2C4%2C5&iGeoDistanceLimit=20000&iOffset=0&iLimit=25&iIncludeAll=0&bTopDealsOnly=false&iViewType=0&aPriceRange%5Bfrom%5D=0&aPriceRange%5Bto%5D=0&iGeoDistanceItem=47160&aGeoCode%5Blng%5D=-0.1589&aGeoCode%5Blat%5D=51.513802&bIsSeoPage=false&mgo=false&bHotelTestContext=false&th=false&aHotelTestClassifier=&bSharedRooms=false&bIsSitemap=false&rp=&sSemKeywordInfo=&tgs=4716002&bRecommendedItem=false&iFilterTab=0&&_=1446673248317
Where you can find these values (in JSON format):
Which are the ones showed here:
So I think you don't need any ScrapyJS nor PhantomJS to scrape that information. Just understand where is it getting the information from and scrape the endpoint, directly.
I am trying to retrieve the ad URLs for this website:
http://www.appledaily.com
The ad URLs are loaded using javascript so a standard crawlspider does not work. The ads also changes as you refresh the page.
I found this question here and what I gathered is that we need to first use selenium to load a page in the browser then use Scrapy to retrieve the url. I have some experiences with scrapy but none at all in using Selenium. Can anyone show/point me to resource on how I can write a script to do that?
Thank you very much!
EDIT:
I tried the following but neither works in opening the ad banner. Can anyone help?
from selenium import webdriver driver=webdriver.Firefox()
driver=webdriver.Firefox()
driver.get('http://appledaily.com')
adBannerElement = driver.find_element_by_id('adHeaderTop')
adBannerElement.click()
2nd try:
adBannerElement =driver.find_element_by_css_selector("div[#id='adHeaderTop']")
adBannerElement.click()
CSS Selector should not contain # symbol it should be 'div[id='adHeaderTop']' or a shorter way of representing the same as div#adHeaderTop
Actually on observing and analyzing the site and the event that you are trying to carry out, I find that the noscript tag is what should interest you. Just get the HTML source of this node, parse the href attribute and fire this URL.
It will be equivalent to clicking the banner.
<noscript>
"<a href="http://adclick.g.doubleclick.net/aclk%253Fsa%...</a>"
</noscript>
(This is not the complete node information, just inspect the banner in Chrome and you will find this tag).
EDIT: Here is a working snippet that gives you the URL without clicking on the Ad banner, as mentioned from the tag.
driver = new FirefoxDriver();
driver.navigate().to("http://www.appledaily.com");
WebElement objHidden = driver.findElement(By.cssSelector("div#adHeaderTop_ad_container noscript"));
if(objHidden != null){
String innerHTML = objHidden.getAttribute("innerHTML");
String adURL = innerHTML.split("\"")[1];
System.out.println("** " + adURL); ///URL when you click on the Ad
}
else{
System.out.println("<noscript> element not found...");
}
Though this is written in Java, the page source wont change.