I am trying to execute a feature with 3 scenarios in it using:
Results results = Runner.parallel(tagQuery, featurePaths, null, new ArrayList<>(), 3, karateOutputPath);
With #parallel=false works fine however when I removed, it fails with the following error:
com.intuit.karate.exception.KarateException: test_input.feature:50 - driver config / start failed:
org.apache.http.conn.HttpHostConnectException: Connect to localhost:9222 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused: connect, options: {type=chrome, target=null}
This occurs at the * driver <url> phase. This is an intermittent failure which most of the time 2/3 scenarios pass and one fails with this error.
Version: 0.9.6.RC3
I recommend using Zalenium like Remote Browser Grid to achieve parallelism and make sure that your scenario are self-contained or atleast keep feature independent.
Sorry, running browser tests in parallel is non-trivial which is why we have the Docker option.
EDIT: and in case you landed here because you wanted karate.callSingle() to work for UI tests, sorry that is not possible as well. But you are encouraged to perform an API sign-in via karate.callSingle() and then speed up your UI tests: https://github.com/intuit/karate/tree/develop/karate-core#hybrid-tests
Consider that this is not supported on a single node. It can be made to work if you know what you are doing, but you need to figure this out depending on whether you are using Chrome or WebDriver.
Please refer the docs: https://twitter.com/ptrthomas/status/1159295560794308609 | https://github.com/intuit/karate/tree/master/karate-core#configure-drivertarget
EDIT - also see this answer: https://stackoverflow.com/a/60387907/143475
Related
we are trying to run Selenium-Tests on Browserstack against an AWS-Vaadin-App on several Jenkins slaves in parallel.
Companies-Jenkins -> Browserstack -> AWS-Vaadin-App
Our test framework uses the Vaadin Testbench with a valid license key.
All tests start as expected with a login (at the app) and the business workflow. But after a while there is a connection closed on all tests, the Vaadin framework shows "server connection lost".
T0 -> T1
-> T2
-> Tn
If we run the same on just one Jenkins slave in a sequence, it runs ok (also here we can see sometimes "server connection lost" but the selenium tests tries to wait and go on when the warning disappears, usually that works. In parallel it never works).
T0 -> T1 -> T2 -> Tn
Do you have an idea, why this happens? Could it be a problem with our Vaadin license?
It sounds like a problem with the server, not the tests. Is your server running out of memory by doing too much in parallel? Check the server logs, that is where you should find the reason. You can most likely see the same "server connection lost" if you manually open a session in your browser while the tests are running
We got help from the browserstack support:
eg. Initiate a Binary connection as:
./BrowserStackLocal --key $BROWSERSTACK_ACCESS_KEY --local-identifier test123
This initiates a unique Binary connection with a unique modifier "test123". This can then be used when executing the tests by setting the below capabilities:
caps.setCapability("browserstack.local","true");
caps.setCapability("browserstack.localIdentifier", "test123");
The same details are also mentioned in the link: https://www.browserstack.com/local-testing/app-automate#multiple-local-testing-connections
I am trying to run testcafe script through saucelabs. But getting the following error message.
MAIN failed to create listener on port 4445
MAIN possible socket error: libevent connection error
Failed to start Selenium listener. Please make sure there are no other applications using the Selenium port, or specify an alternative port for Sauce Connect via the --se-port option (exit code 2).
It looks like another application already used the 4445 port (see a similar discussion here).
Also, you can use the official Saucelabs Troubleshooting Guide to fix this problem.
I am trying to mock Appium’s behavior using WireMock.
Problem I am facing is: my POST stub is able to respond me with the response code I have set (which I have monitored using Wireshark), but it is throwing exceptions related to session creation.
POST stubs I have tried for creating session for reference:
- Trial 1:
wireMockRule.stubFor(post(urlEqualTo("/wd/hub/session”)).
withRequestBody(matching(“JSON representation of set of desired capabilities specific to application"))
.willReturn(aResponse().
withStatus(200).
withHeader("Content-Type", "application/json”).
withBody("{\"sessionId\":\"e523dc70-c068-46e7-b966-8462401119b4\",\"status\":0,\"value\":\"Created Session\"}")));
Outcome: “It is impossible to create a new session because 'createSession' which takes HttpClient, InputStream and long was not found or it is not accessible” exception.
Trial 2: Just removed request body from above stub
wireMockRule.stubFor(post(urlEqualTo("/wd/hub/session”))
.willReturn(aResponse().
withStatus(200).
withHeader("Content-Type", "application/json”).
withBody("{\"sessionId\":\"e523dc70-c068-46e7-b966-8462401119b4\",\"status\":0,\"value\":\"Created Session\"}")));
Outcome:
org.openqa.selenium.SessionNotCreatedException: Unable to create new remote session” exception.
Environment reference:
Java-client of Appium: 6.0.0
Selenium standalone server: 3.12.0
Selenium client of Java: 3.12.0
Please help.
This does not really seem to related to WireMock?
Maybe you can switch to the .NET version from WireMock to simulate your calls?
I want to sync automation result from jenkins to testlink. I tried with Testlink -jenking plugin and testlink-api-client but not worked getting error.
Pre-setup :
$tlCfg-> api-> enabled
$tlCfg-> exec_cfg-> enable_test_automation
From Testlink UI enable automation for the project.
Test code :
TestLinkAPIClient testlinkAPIClient = new TestLinkAPIClient(APIKEY, "http://localhost/testlink/lib/api/xmlrpc/v1/xmlrpc.php");
testlinkAPIClient.reportTestCaseResult(Project, TestPlan, TEST_CASE, Build, notes/comments, teststatus);
output :
"testlink.api.java.client.TestLinkAPIException: The call to the xml-rpc client failed.".
References used :satishjohn.wordpress.com
2. softwaretestinghelp.com
and other stackoverflow threads.
I browsed and try out defined steps from some of the blogs but still facing same issue?. Can anyone help me to resolve this issue or other approach on sync result with testlink ?.
I believe you should follow the documentation(1) written by kino who wrote the plugin.We recently managed to sync automation results from Jenkins to Testlink by following above doc.Our auto tests were written based on testng framework, Hence we used "testng-results.xml" and TestNg method name based result seeking strategy.
We didn't come across an issue as you mentioned. From (2) and (3) you can get the plugin source .My advice is to debug the code after enabling the debug on Jenkins hosted tomcat server. So you can find the actual cause of the issue by yourself.
Reference:
(1) https://wiki.jenkins-ci.org/download/attachments/753702/jenkins.pdf
(2) https://github.com/jenkinsci/testlink-plugin
(3) https://github.com/kinow/testlink-java-apienter code here
You can run wireshark and filter on port "tcp port http" to see exact error you get from the server. When it was not working for us we were getting 200 OK with text "XML-RPC server accepts POST requests only."
You can also check /var/log/apache2/error.log for testlink errors.
We fixed the issue by setting following config in config.inc.php and restarting apache.
$tlCfg->api->enabled = TRUE;
$tlCfg->exec_cfg->enable_test_automation = ENABLED;
I run Selenium tests on Saucelabs by creating a job in Jenkins. Some of the tests timeout on Saucelabs due to network issue or auth challenge. The result of the job in Jenkins is shown as successfully passed. How do I get the error to be shown on Jenkins job ? Is there a way to make the job fail when tests timeout ? Thanks in advance for the help.
If there is a string that you can search for in the console log output that indicates when the tests have timed out, you can use the Text Finder plugin to look for that string and fail the build.
If your test loses connection to the grid while its running, then Sauce will timeout (after 90 seconds of inactivity in JSON packeets) and because there is no connection from your tests, there is no way for your tests to know that it happened. Unless you set a timeout on your Junit or TestNG test so that if the long timeout is reached you are certain it must be a failure and then there could be a #AfterTest step that tries to ping the grid status with the standard JSON /status request, which if it fails, you could throw a Assert.fail . I haven't tried this but it seems like it might work.
Also, you might be able to use Saucerest to query test status and possibly get a timeout reply?