I'm using the selenium IDE for Chrome on Mac Big Sur. I want to add a puse in between commands to see why something isn't executing properly. This is in my ".side" file
}, {
"id": "32f35ed7-1a28-4540-a93d-3cb8ba0e012a",
"comment": "",
"command": "pause",
"target": "",
"targets": [],
"value": "100000"
}, {
I have put a really high value but when I play back my test, it just breezes through it without pausing at all, although it tells me the command was successfully run
What's the right way to pause my test?
Use Target instead of Value, Like below
Target: 100000
instead of
value: 100000
Also,
Set speed to fastest (Actions --> Fastest), otherwise it won't work.
Related
I'm trying to create a CI workflow which requires me to do both code coverage and ordinary testing. We run both at this time, because the coverage test instrumented for grcov can fail, but the normal test could pass. I want to automate comparing the test that failed in coverage instrumentation to the tests run without it. So for each test that failed in the grcov instrumentation I want to run a similar test without instrumentation.
Currently, the way to do it, is to hardcode the test paths, and/or use unreliable text-based testing detection. It's both labour intensive and not very reliable once people start adding more tests (which they will).
So my question is
Is there a way to get a list of all tests that are to be run from a multi-crate project.
Is there a way to attach actions to cargo -test if it fails, without stopping cargo test.
Is there a way to ask cargo test to only mention the names of failed tests (if any).
Your best bet for acting on cargo test results would be to use the unstable --format json option. It will output events in a JSON Lines format:
cargo test -- -Zunstable-options --format json
{ "type": "suite", "event": "started", "test_count": 2 }
{ "type": "test", "event": "started", "name": "tests::my_other_test" }
{ "type": "test", "event": "started", "name": "tests::my_test" }
{ "type": "test", "name": "tests::my_test", "event": "ok" }
{ "type": "test", "name": "tests::my_other_test", "event": "failed", "stdout": "thread 'tests::my_other_test' panicked at 'assertion failed: false', src/lib.rs:20:9\nnote: run with `RUST_BACKTRACE=1` environment variable to display a backtrace\n" }
{ "type": "suite", "event": "failed", "passed": 1, "failed": 1, "ignored": 0, "measured": 0, "filtered_out": 0, "exec_time": 0.003150667 }
You can save that to a file and use another utility like jq to parse out the relevant data. Here are some examples that might be helpful to you.
Get all the tests that were ran:
jq -c 'select(.type=="test" and .event=="started") | .name' <test-output.json
"tests::my_other_test"
"tests::my_test"
Get all the tests that failed:
jq -c 'select(.type=="test" and .event=="failed") | .name' <test-output.json
"tests::my_other_test"
I have a setup where I make simple tests with the Selenium IDE, then save the project as a *.side file to a repository, which a server running Selenium Grid then pulls, runs all tests contained and then emails me the results. For most of the tests I get the same results as locally, only a few cookie banners caused trouble because it seems that locally it's not an issue if you don't click them away but when the tests run on the Selenium Grid they will fail because the cookie banner blocks all clicks. Now I have a similar issue but there is nothing like a cookie banner. I am testing a Swagger UI and whenever I run the test on Selenium Grid I get failures like this:
FAIL DefaultSuite/click-a-couple-headers.test.js (7.275s)
✕ click-a-couple-headers (5319ms)
● click-a-couple-headers
ElementClickInterceptedError: element click intercepted: Element <div class="opblock-summary opblock-summary-get">...</div> is not clickable at point (476, 18). Other element would receive the click: <header>...</header>
(Session info: chrome=80.0.3987.106)
at Object.throwDecodedError (../../../node_modules/selenium-side-runner/node_modules/selenium-webdriver/lib/error.js:550:15)
at parseHttpResponse (../../../node_modules/selenium-side-runner/node_modules/selenium-webdriver/lib/http.js:560:13)
at Executor.execute (../../../node_modules/selenium-side-runner/node_modules/selenium-webdriver/lib/http.js:486:26)
Test Suites: 1 failed, 1 total
Tests: 1 failed, 1 total
Snapshots: 0 total
Time: 7.58s, estimated 8s
Searching for the CSS selector in the Selenium IDE test this seems to be the part where it fails:
{
"id": "12345678-1111-5555-8888-1234567890ab",
"comment": "",
"command": "setWindowSize",
"target": "968x648",
"targets": [],
"value": ""
}, {
"id": "12345678-2222-6666-8888-1234567890ab",
"comment": "",
"command": "runScript",
"target": "window.scrollTo(0,413)",
"targets": [],
"value": ""
}, {
"id": "12345678-4444-7777-8888-1234567890ab",
"comment": "",
"command": "click",
"target": "css=#operations-category-method_path_to_endpoint > .opblock-summary",
"targets": [
["css=#operations-category-method_path_to_endpoint > .opblock-summary", "css:finder"],
["xpath=//div[#id='operations-category-method_path_to_endpoint']/div", "xpath:idRelative"],
["xpath=//span[2]/div/div/span/div/div", "xpath:position"]
],
"value": ""
}
So I thought it might be a timing issue with the scrolling operation, so I added a whatForElementVisible command before all clicks:
/* scroll command */
{
"id": "12345678-3333-7777-8888-1234567890ab",
"comment": "",
"command": "waitForElementVisible",
"target": "css=#operations-category-method_path_to_endpoint > .opblock-summary",
"targets": [
["css=#operations-category-method_path_to_endpoint > .opblock-summary", "css:finder"],
["xpath=//div[#id='operations-category-method_path_to_endpoint']/div", "xpath:idRelative"],
["xpath=//span[2]/div/div/span/div/div", "xpath:position"]
],
"value": "30000"
},
/* click command */
Locally the test still worked fine in Firefox and Chrome (which the Grid runs) but on the Grid it fails with the same error as before. Does someone know how to fix this issue or how to properly debug it?
Turns out it wasn't an issue with timing at all, but rather a problem with an information bar (about the browser being controlled with automation software) at the top of the browser window that I didn't get locally. The video recording feature of Selenium Grid Extras proved to be really useful to track this down. Useful to know because the README of the repo doesn't mention it: the node info (and videos) are exposed on the ports 3100 (Chrome browser) and 3200 (FireFox browser) respectively.
My fixtures are set up like so
{
"fixtures": [
{
"name": "login",
"pageUrl": "http:\/\/localhost:3000\/",
"tests": [
{
"name": "type name",
"commands": [
{
"type": "type-text",
"studio": {
},
"callsite": "0",
"selector": {
"type": "js-expr",
"value": "input[type=email]"
},
"options": {
},
"text": "example#email.com"
}
]
}
]
}
]
}
with one simple test to find the input and type some text but when running the command I get
testcafe chrome login.testcafe
ERROR Unable to establish one or more of the specified browser connections. This can be caused by network issues or remote device failure.
Type "testcafe -h" for help.
I've seen this issue a couple of times on their issues board one relating to CI integration on a Linux server and another which seems like a similar issue of trying to connect to localhost
https://github.com/DevExpress/testcafe-browser-provider-electron/issues/20
https://github.com/DevExpress/testcafe/issues/1133
New to testcafe any help would be appreciated!
I've found the solution some network policies don't allow access to your machine on some ports in my example it's 57501.
testcafe chrome login.testcafe --hostname localhost
adding --hostname resolves the issue
documentation
https://devexpress.github.io/testcafe/documentation/using-testcafe/command-line-interface.html#--hostname-name
I still don't know how to launch from the IDE but this resolves my main issue.
TestCafe Studio Preview does not support setting command line options (hostname in your case). The TestCafe team is going to implement this functionality in the official release.
So, for now, it is only possible to run tests via a command line.
UPDATE:
You can set the hostname option in the TestCafe Studio Settings dialog:
Using Selenium Builder, I've created the following json file:
{
"type": "script",
"seleniumVersion": "2",
"formatVersion": 2,
"steps": [
{
"type": "get",
"url": "http://stackoverflow.com/"
},
{
"type": "saveScreenshot",
"file": foo.png"
}
],
"data": {
"configs": {},
"source": "none"
},
"inputs": [],
"timeoutSeconds": 60
}
I tried running it on Windows 7 and two different Ubuntu machines. Instead of the filepath "foo.png" I had also inserted "E:\foo.png" / "/home/swege/foo.png". However, I always get the "exception":
Could not take screenshot of current page - [object Object]
At least I would like to be able to read the "error object", but every system just puts out that the error is a JavaScript object. Any idea how to fix the issue or read the full error message?
Try following :
http://www.ontestautomation.com/how-to-create-screenshots-in-your-selenium-webdriver-tests/
Above can help you to get exact code that how you can take screenshots using selenium.
So I pulled down and looked at the source for Selenium Builder 2 and found two files that I think show the origin of the error, command_processor.js and driver_component.js.
I can't make out why the error is occurring - maybe someone here can build on this?
Try using a full file path, not just foo.png. On Mac, these worked for me: ~/foo.png and ~/Downloads/screenshots/foo.png.
It's also important that the folder exists and is writable by the account running the web browser.
I've got both Leiningen & Clojure working on Windows 8 separately from Sublime Text (e.g. I can get a repl working in Windows PowerShell).
My problem is that I can't get the SublimeREPL working in SublimeText (the REPL loads up but doesn't then do anything). Are there any simple traps that I might be missing or, failing that, are there a series of steps I could follow to troubleshoot?
Please see this SublimeREPL issue for instructions on how I got a Clojure REPL to work, at least on XP (I haven't tried it on Win7 or 8 yet). Basically, I edited the menu file for Clojure, and changed the command from lein repl to lein trampoline run -m clojure.main, which for some reason did the trick. I also changed the path to $file so you can open up a REPL while your project.clj is the current tab in Sublime, and the REPL should inherit the project's settings.
For reference, the complete Packages/User/SublimeREPL/config/Clojure/Main.sublime-menu file (Packages is accessible via Preferences -> Browse Packages...) is as follows:
[
{
"id": "tools",
"children":
[{
"caption": "SublimeREPL",
"mnemonic": "r",
"id": "SublimeREPL",
"children":
[
{"caption": "Clojure",
"id": "Clojure",
"children":[
{"command": "repl_open",
"caption": "Clojure Trampoline",
"id": "repl_clojure",
"args": {
"type": "subprocess",
"encoding": "utf8",
"cmd": {"windows": ["lein.bat", "trampoline", "run", "-m", "clojure.main"],
"linux": ["lein", "repl"],
"osx": ["lein", "repl"]},
"soft_quit": "\n(. System exit 0)\n",
"cwd": {"windows":"$file_path",
"linux": "$file_path",
"osx": "$file_path"},
"syntax": "Packages/Clojure/Clojure.tmLanguage",
"external_id": "clojure",
"extend_env": {"INSIDE_EMACS": "1"}
}
},
{"command": "clojure_auto_telnet_repl",
"id": "repl_clojure_telnet",
"caption": "Clojure-Telnet"}]}
]
}]
}
]
I solved this problem with Git Bash Shell. I have used the shell script version of leiningen instead of lein.bat
This is the command I use:
["C:\\Program Files\\Git\\bin\\sh.exe", "-l", "--", "/d/lein.sh", "repl"]
assuming lein.sh is in d:\
lein trampoline command sometimes behaves differently from lein repl and could fail due to unknown reasons.