I am trying to hit particular web page and record post its load for example:
http://serv1.project.com/page7
but on hitting above page only http://serv1.project.com gets recorded. When i play same script then http://serv1.project.com is opened without subsequent page hit.
Note: I am trying to run my scripts using RC with java as base.
Why does it matter what gets recorded? Can't you just do:
selenium.open("page7");
in your Java code?
Of course I assume you have created the selenium session with the http://serv1.project.com/ as a base URL.
Related
I have a web crawling task which the website needs the user to login and pass the Email 2FA authentication before getting to the main page. On the main page, I have a lot of detail pages(all in the same format) to visit and collect the data and store them in a CSV file.
Now I got no problem with getting to the main page. However, I'd like to optimise tasks after entering the main page, my current solution for this part is I use a for loop and visit each page and collect the data.
for each in page_id_list:
driver.get("https://...." + str(each))
driver.find_elements_by_xpath("...")
....
Functionally it works fine, but I was thinking if I can make the process faster. As selenium loads the whole web page and other packages like request + lxml only retrieve HTML maybe I should somehow use another way to visit those pages, and also I wonder if Scrapy will be useful in my case?
Thank you.
I am using phantomjs for loading some pages from a single page application,
Phantom fires up all the scripts on the page, however there is no way to tell when the page is ready to be examined. Given that some of the data is coming from external sources it is impossible to tell when all the scripts on the page finished.
Any ideas on this, I tried waitfor, however that is not a solution for my scenario since I do not know which elements I am looking for (the external source can change the data it sends)
I want to test a page.Where i want to fill up the fields like first name last name etc.and after going two pages further if i come back to the original page by using back navigation ,data entered for first name and last name remains the same.or it is filled up.
In jmeter i want to check the same if data entered for the fields remain same if i navigate back .
How can i achieve this.
I tried gving url directly in the path its not happening since it is not the way.
please help me since i'm new to jmeter.
You need to understand 2 things. How JMeter works and how your application works.
JMeter only captures data that is communicated to server. It does not matter how data is entered from UI. It does not check if data retains in the fields or not. It only records the request that is sent by your application to server-side.
So, if you understand above, you also need to understand how your application sends data to server. Does it sends the request as you move from first page to second. Or does it send (Submit) data on final page.
Either way, JMeter is not a tool to test if your form fields are retaining data in them as you navigate between pages. As mentioned earlier it only monitors data requests/responses.
Selenium seems a better option for your test requirement.
Please read the apache documentation carefully:
JMeter is not a browser. As far as web-services and remote services are concerned, JMeter looks like a browser (or rather, multiple browsers); however JMeter does not perform all the actions supported by browsers. In particular, JMeter does not execute the Javascript found in HTML pages. Nor does it render the HTML pages as a browser does (it's possible to view the response as HTML etc., but the timings are not included in any samples, and only one sample in one thread is ever viewed at a time).
First, you have to understand how JMeter works!!! To do the Functional Testing, Selenium would be a good choice.
Thanks
How does one write a script to download one's Google web history?
I know about
https://www.google.com/history/
https://www.google.com/history/lookup?hl=en&authuser=0&max=1326122791634447
feed:https://www.google.com/history/lookup?month=1&day=9&yr=2011&output=rss
but they fail when called programmatically rather than through a browser.
I wrote up a blog post on how to download your entire Google Web History using a script I put together.
It all works directly within your web browser on the client side (i.e. no data is transmitted to a third-party), and you can download it to a CSV file. You can view the source code here:
http://geeklad.com/tools/google-history/google-history.js
My blog post has a bookmarklet you can use to easily launch the script. It works by accessing the same feed, but performs the iteration of reading the entire history 1000 records at a time, converting it into a CSV string, and making the data downloadable at the touch of a button.
I ran it against my own history, and successfully downloaded over 130K records, which came out to around 30MB when exported to CSV.
EDIT: It seems that number of foks that have used my script have run into problems, likely due to some oddities in their history data. Unfortunately, since the script does everything within the browser, I cannot debug it when it encounters histories that break it. If you're a JavaScript developer, use my script, and it appears your history has caused it to break; please feel free to help me fix it and send me any updates to the code.
I tried GeekLad's system, unfortunately two breaking changes have occurred #1 URL has changed ( I modified and hosted my own copy which led to #2 type=rss arguments no longer works.
I only needed the timestamps... so began the best/worst hack I've written in a while.
Step 1 - https://stackoverflow.com/a/3177718/9908 - Using chrome disable ALL security protocols.
Step 2 - https://gist.github.com/devdave/22b578d562a0dc1a8303
Using contentscript.js and manifest.json, make a chrome extension, host ransack.js locally to whatever service you want ( PHP, Ruby, Python, etc ). Goto https://history.google.com/history/ after installing your contentscript extension in developer mode ( unpacked ). It will automatically inject ransack.js + jQuery into the dom, harvest the data, and then move on to the next "Later" link.
Every 60 seconds, Google will force you to re-login randomly so this is not a start and walk away process BUT it does work and if they up the obfustication ante, you can always resort to chaining Ajax calls and send the page back to the backend for post processing. At full tilt, my abomination script collected 1 page a second of data.
On moral grounds I will not help anyone modify this script to get search terms and results as this process is not sanctioned by Google ( though not blocked apparently ) and recommend it only to sufficiently motivated individuals to make it work for them. By my estimates it took me 3-4 hours to get all 9 years of data ( 90K records ) # 1 page every 900ms or faster.
While this thing is going, DO NOT browse the rest of the web because Chrome is running with no safeguards in place, most of them exist for a reason.
One can download her search logs directly from Google (In case downloading it using a script is not the primary purpose),
Steps:
1) Login and Go to https://history.google.com/history/
2) Just below your profile picture logo, towards the right side, you can find an icon for settings. See the second option called "Download". Click on that.
3) Then click on "Create Archive", then Google will mail you the log within minutes.
maybe before issuing a request to get the feed the script shuld add a User-Agent HTTP header of well known browser, for Google to decide that the request came from that browser.
I'm a newbie to jMeter, so please bear with me.
I've been assigned the task of testing how an e-commerce website responds under load. I've managed to set up basic tests in jMeter that basically just repeatedly visit the home page, but I'd like to simulate something a bit more realistic:
User arrives on home page
User goes to catalogue page
User views product
User adds product to cart
User returns to catalogue, selects another product, adds to cart
User removes first product from cart
User proceeds to checkout
User completes checkout process.
I'm having trouble finding adequate documentation to explain how to do this. I figured out that I need a cookie manager in my test so that the user session will be maintained, but I haven't figured out how to get the user to traverse the site in a realistic use pattern (such as the one described above). Can anyone help out with this, give me some pointers as where to look for good examples, etc?
This should be no problem, record or manually create the necessary steps as HTTP Samplers, then add them into a Runtime Controller for example to execute them iteratively.
The individual steps will be executed in the order they are in the tree and, in case Cookies are used to handle session state, you might need to add the Cookie Manager to the top of the tree which will handle cookie headers for each user.
Add some timers to simulate user's think time and scale up by increasing the number of virtual users in the thread group.
Use some listener like the Aggregate Report to view the response times for every step.
Try to read http://jmeter.apache.org/usermanual/index.html at first.
Also you'll encounter the problem that Jmeter can't process dynamic pages:
http://wiki.apache.org/jmeter/JMeterFAQ#Does_JMeter_process_dynamic_pages_.28e.g._Javascript_and_applets.29
Does JMeter process dynamic pages (e.g. Javascript and applets)?
No. JMeter does not process Javascript or applets embedded in HTML pages.
JMeter can download the relevant resources (some embedded resources are downloaded automatically if the correct options are set), but it does not process the HTML and execute any Javascript functions.
If the page uses Javascript to build up a URL or submit a form, you can use the Proxy Recording facility to create the necessary sampler. If this is not possible, then manual inspection of the code may be needed to determine what the Javascript is doing.