I can't found any question/answer about that (probably I don't know how to find it...)
Could somebody give me a global idea to execute +200 Selenium webdriver tests (Python) from cloud servers/tools?
Thanks!!
rgzl
Another way is Saucelabs, using this service you'll be able to just send your Selenium
Java/Python tests
to their Cloud infrastructure for execution. The benefits of such testing are obvious – no need to waste time and resources setting up and maintaining your own VM farm, and additionally you can run your test suite in various browsers in parallel. Also no need to share any sensitive data, source code and databases.
As said in this acticle:
Of course inserting this roundtrip across the Internet is not without cost. The penalty of running Selenium tests this way is that they run quite slowly, typically about 3 times slower in my experience. This means that this is not something that individual developers are going to do from their workstations.
To ease the integration of this service into your projects, maybe you'll have to write a some kind of saucelabs-adapter, that will do the necessary SSH tunnel setup/teardown and Selenium configuration, automatically as part of a test.
And for a better visualization:
Here's a global idea:
Use Amazon Web Services.
Using AWS, you can have a setup like this:
1 Selenium Grid. IP: X.X.X.X
100 Selenium nodes connecting to X.X.X.X:4444/wd/register
Each Selenium node has a node config, running 2 maxSessions at once. (depending on size of course)
Have also, a Continuous integration server like Jenkins, run your Python tests Against X.X.X.X grid.
Related
Good morning to everyone,
i would like to understand if it is possible and how to import macros with .jar extension produced by Selenium into webinspect (version 21.2) and then use them to conduct a scan.let me try to explain, on our machines we only have webinspect, the tests with selenium are run by other people on other systems. we wanted to understand if by simply passing us these files webinspect would be able to read and execute them, or if it is necessary to put the webinspect proxy in the selenium scripts while this other team records the macros. Can anyone help me? Thank you
I need to speed up a process by taking advantage of macros already registered to do other tests prior to mine, so as to avoid a new registration
ps: I have already read the documentation, but it does not explain whether it is actually possible to do what I asked, it explains other procedures.
Unfortunately, WebInspect's integration with Selenium is essentially a replay of the Selenium scripts in real-time, used as the Crawl phase of the scan. WebInspect cannot simply consume your JAR file. It will require you set up a listener/proxy of some sort, so when the script replays, WebInspect can capture the traffic, and then it performs an Audit-Only of what it saw. There are two methods to insert this proxy technology into the process, as detailed in the WebInspect Help. The user must configure some features so when WebInspect replays the Selenium script everything connects automatically.
e.g. from WI 22.10:
file:///C:/ProgramData/HP/HP%20WebInspect/Help/WebInspect/index.htm#Selenium_WD_1.htm?TocPath=Using%2520WebInspect%2520Features%257CIntegrating%2520with%2520Selenium%2520WebDriver%257C_____0
Besides Selenium, there are several other alternatives when it comes to Functional Testing driven WebInspect scans. You had asked about requiring the dev staff to record something to provide to your for WebInspect.
Have the QA team capture their Selenium test runs using BURP Proxy. Have them save that captured proxy traffic as an artifact for the security team, e.g. "macro1.burpcap". Use the Workflow-driven Scan wizard options in WebInspect and simply Import that BURP capture as a native Workflow Macro. I like this option since BURP is easy to acquire and run, and supports multiple OS as a Java app.
WebInspect's Web Proxy could also be used, as BURP was used above. However, this complicates things for your dev team, as they do not have access to WebInspect. There are other free options for WebInspect customers which your dev team could install, including the Standalone WebInspect Toolkit, the Web Proxy standalone tool, or the Web Proxy API tool (REST service). One annoyance with all of these today is that they (currently) require Windows, and it requires an authorized WebInspect user (you) to download and deploy these installers inside your network for the dev staff to get.
The WebInspect REST API offers several endpoints for Proxy listeners. This means that remote users (i.e. your Dev) could spawn a proxy listener, run their functional test script through that proxy, then have the captured data saved as a Workflow Macro, and kill the listener. By itself, this combination could produce the artifacts your appsec team will want to use later in their Workflow-driven Scan.
To support this with further automation ("developer-driven DAST"), you could have those same proxy API calls add on a New Scan endpoint call at the end, to go ahead and trigger a Workflow-driven Scan using the Macro that was just recorded in the prior calls. Good for putting in a cicd pipeline, provided you have a dedicated WebInspect machine sitting on the network with its API available.
The challenge with using the WebInspect API as a "poor man's pipeline scanning tool" is that WebInspect is simplistic and has no resource management features in and by itself. This means that your pipelines could trigger lots of scans quickly, and the WebInspect machine would fall over after 4+ scans got started. What a mess! You would have to design API checks into your pipelines to monitor the number of Running scans on the WebInspect machine, and then Pause/Poll the pipeline until the WebInspect machine was free and the pipeline could then submit its new scan order.
Our solution for this sort of enterprise automation would be to use Fortify ScanCentral DAST instead of just WebInspect standalone. SCDAST affords a central web GUI for your appsec staff to configure/operate/review scans, with multiple headless "WebInspect" scan machines managed in resource pools. Scan orders coming in (REST API calls) would be queued and prioritized automatically, and the remote scan machines would be brought on-line/shutdown as needed (think as headless WebInspect API on Docker). So now your cicd pipelines can simply trigger the DAST scan and not worry about the scanner machine resources.
This brings me around to another great option for your Selenium needs, the Fortify FAST Proxy. This solution only operates with ScanCentral DAST, which is why I had to go on that side tangent above. With FAST Proxy, your dev would only start up the FAST Proxy (includes authentication details on the ScanCentral API), run their Selenium scripts through that proxy, and then kill the FAST Proxy when done. That completes their Functional Testing with Selenium. Meanwhile, on shutdown, the FAST Proxy automatically delivers the captured traffic to ScanCentral as a New Workflow-driven Scan order. In a little while, the DAST scan of their Selenium script traffic has completed. If you configured Notifications, dev now receives a link to their appsec results.
I am new to Appium/Selenium parallel testing and I was wondering if one could run different tests concurrently across multiple devices? My team needs to reduce the total runtime of our UI tests and are not concerned with different OS versions affecting the behaviour of the application for these specific tests. I have been reading through many posts and trying to search for answers but all I can seem to find on the internet are articles, tutorials and forums on how to run the same test in parallel on multiple devices.
Can I run different tests concurrently on multiple devices without kicking off different tests manually, or is that a limitation of Appium? Ideally this would be implemented using an open source solution.
(Right now we are trying to use a JUnit approach for testing due to specific limitations of other tools. All tests are being written in Java.)
Thanks for your time.
Depending on your setup, you can accomplish this. However, a lot of your build automation and device management will need to be set up by you or your team custom, so you will not be able to use an out-of-the-box solution to do this.
I've accomplished the same with both Selenium and Appium -- you will need a test framework that allows for test execution with parameters, and your devices will need to be connected to separate USB hubs that each have their own virtual server attached.
Using NUnit, here's my approach:
Generate .txt files for each different set of tests I want to run -- test_list_1.txt, test_list_2.txt, etc. Each list contains a different group of test cases to run.
Write a build script to clean & build your project from scratch -- for C#, I use Cake.
Set up a job in Jenkins that executes your build script and calls NUnit's console runner, which takes a test_list as a parameter. This initiates a test execution against a list of test cases
You should be able to build your Jenkins job against any test list you want, so you now have the ability to run your automation against different tests, as mentioned in your problem description.
Connect your virtual machines (which connect to your Appium devices) to Jenkins and add them as executors on your job. Now you have multiple machines to run your job against.
With this set up, you can run as many jobs as you have machines -- 4 VM's means 4 jobs, which means you can run 4 different sets of test cases concurrently.
Setting this up on my end was completely custom -- I used certain tools to accomplish individual steps, but it worked for our needs and we did accomplish concurrent execution against different sets of test cases.
What you are asking , basically it's not possible.
You can't run different test cases on different devices.
Though you can run same test cases on wide range of devices using Hive or Browserstack or AWS device farm.
Hope this helps.
You can run your tests locally on multiple devices by creating multiple instances of appium server. Every Appium server should be running on different ip and proxy address. So you should set your capabilities for each instance accordingly.
But there is another solution as well but that's bit costly. That is AWS Device Farm. AWS provides multiple real devices hosted at there servers which you can use for executions of your customised test suits. They give initially 1000 free test minutes. You have to create a maven project for your test scripts. I prefer using testng rather than Junit.
Details -
Can we run the multiple instances of selenium script at same time
while running multiple instances of selenium script at same time my script failed connection refused
AnyOne has the solution for it .
You can try something called Selenium Grid.
Selenium-Grid allows you run your tests on different machines against
different browsers in parallel. That is, running multiple tests at the
same time against different machines running different browsers and
operating systems. Essentially, Selenium-Grid support distributed test
execution. It allows for running your tests in a distributed test
execution environment.
You can find more on the following website= https://www.seleniumhq.org/docs/07_selenium_grid.jsp.
Also, check out this response as well. It was for a similar question= Running multiple Selenium tests at the same time
I've been working with Selenium for a few years already.
I started with some little stuff in Java and in my previous job I did a project using C# bindings with SpecFlow framework, page objects model, I dealt with complex locators, some JavascriptExecutor even some browsermobproxy work, bottom line I have some experience with Selenium.
Still there is something that is not clear to me.
Is Selenium expected to work properly when there is no "interactive session" into the machine that is running the code ?
Let's say that I connect into remote desktop to a machine in the cloud (Let's say Amazon or Azure), I develop a script and schedule a windows job to run it on the next 10 minutes, then I disconnect from the remote desktop session, the machine is on but no user is connected to it in remote desktop.
Will the script work ? Or depends what does it do ? Might some of the actions not work (Script might include changing the window size, sending keystrokes both through selenium sendkeys and by OS level actions) ? Can we ensure that any script that we developed and works OK while we are connected to the machine will also work when I'm not ?
I hope the question is clear, if not I can maybe explain further
Thanks !!!
Yes, Selenium can run on the cloud machine even though you are not connected to it. It should work without any issue.
As per my knowledge cloud machines are protected by firewall which blocks almost all the ports. You may need to get permission to use the default selenium ports like 4444 or 5555.
Also, may need to increase wait time because the cloud machines are slow in performance compared to normal physical desktop.
Usually cloud machines are linux based, you need to consider the environment as well.
I am looking for ways to set up like a central 'hub' for Selenium in my work, allowing anyone to access in within the company. For example, Tester A writes test scripts, the Person B can run without having to manually copy over the test scripts to their local workstation)
So far, I've only thought of installing Selenium in a VM which will then execute as per normal. But if I run Selenium Grid, it will run VMs within VM (?). My only concern with VMs is that it'd run slowly.
If anyone can think of a better solution or recommendation please do give me some advice. Thank you in advance.
One idea. You can create an infrastructure combining Jenkins/Selenium/Amazon.
The following is my solution from another post.
You can do it with a grid.
First of all you need to create a Selenium hub with an EC2 ubuntu 14.04 AMI without UI and link it as a jenkins slave to your Jenkins master. Or as directly a master. What you want. Only command line. Download Selenium Server standalone. (be careful on downloading the version. If you Download the Selenium3Beta, things could change). Here you can configure the HUB. You can also add the Selenium Hub as a service and configure to run automatically at server start. its important that you open the Selenium default port (or the one that you configured) so the nodes can connect to it. You can do that on the Amazon EC2 console when you have created your instance. You just need to add a security group with an inbound rule for TCP in the port you want for the IPs you want.
Then, you can create a Windows server 2012 instance server (for example, that's what I did), and do the same process. Download the same version for Selenium and the chromedriver (there is no need to download any firefoxdriver for Selenium versions before Selenium3). Generate a txt file and prepare the Selenium command to link to the HUB as a NODE. And convert it to *.bat in order to execute it. If you want to run the bat at start you can create a service with the task scheduler or use NSSM (https://nssm.cc/). Don't forget to add the rules to the security groups for this machine too!
Next, create the Jenkins server. You can use the Selenium Hub as the Jenkins master or as a slave.
Last step is configuring a job to be run in the Jenkins-Selenium machine. This job needs to be linked to your code repository (git, mercurial...) Using the parametrized build plugion for jenkins you can tell that job to pull the revision you want (where every developer can pull the revision with the new changes and new tests) and run the Selenium tests in that build with the current breanch/revision and against one unique selenium. You can use ANT or Maven to run the Selenium tests in Jenkins.
May be it's complicated to understand because there are so many concepts here but it's robust and it works fine!
If you have doubts, tell me!
If Internet Explorer is not one of the browsers on which you must run your automation tests, I would recommend that you consider docker selenium.
Selenium is providing pre-configured docker images for both Selenium Hub and Node ( refer here for more information ). For making use of docker selenium all you need to do is find a machine (preferably unix machine), install docker on it by following instructions detailed here and then start the hub and node by starting off those containers. In the case of docker you can literally transform a VM (or) a physical machine into a VM farm and yet not have to worry about slowness etc., because I believe docker is optimised for these and it runs your VM as a process.
Resorting to using Amazon cloud for running your selenium nodes is all fine, but if you have corporate policies that prevent in-coming traffic from the internet into your intranet region, then I am not sure how far Amazon cloud would be useful.
Also remember that Jenkins is not something that is absolutely required but is more of a good to have part in the setup because it would let anyone run their tests from a web UI. This will however require that all your tests are checked-in and made available in a central version control system in your organization.
PS : The reason why called out Internet Explorer as an exception is because IE runs only on windows and there are no docker images (yet) for windows. All the docker images are UNIX based images.