I'm evaluating Botium for chatbot testing. Could you please confirm if Botium supports below operations ?
UI testing - Send request from ChatBot UI and capture the response from UI (Black box testing)
How Botium handles dynamic requests and dynamic responses (Not static content requests/responses in the excel sheet). Dynamic requests/responses are prepared from difference sources like database, APIs etc.,
Please let me know if more details are needed in this regard.
Question 1: Yes. Botium includes a Webdriver connector based on Selenium, so you can drive your test cases against the web user interface of your chatbot. More info in the repository. A simple example for a botium.json doing UI tests:
{
"botium": {
"Capabilities": {
"PROJECTNAME": "WebdriverIO Plugin Sample",
"CONTAINERMODE": "webdriverio",
"WEBDRIVERIO_OPTIONS": {
"desiredCapabilities": {
"browserName": "chrome"
}
},
"WEBDRIVERIO_URL": "secret",
"WEBDRIVERIO_INPUT_ELEMENT": "#textInput",
"WEBDRIVERIO_INPUT_ELEMENT_SENDBUTTON": ".btn-weiter",
"WEBDRIVERIO_OUTPUT_ELEMENT": ".from-watson"
}
}
}
Question 2: Botium supports severall "matching modes" to assert if you consider the test case as success or not. Substring matching is the most simple one, then there are regular expression, synonym lists, and you can even plug in your own assertion logic ... furthermore, there is something called a "scripting memory" which makes it possible to re-use dynamic content later in the conversation. Some examples are available in this blog post. A simple example for a convo file using the scripting memory:
...
#bot
Super! I've found $count locations for you. Which one would you like to drive to?
#me
$count
#bot
Sure! Restaurant $count on the list gets great reviews.
...
Related
Does Karate UI supports multi remote or any web based Chat application? Is it possible to initiate two drivers? Can someone suggest here?
Background:
* configure driver1 = { type: 'chrome' }
* configure driver2 = { type: 'chrome' }
Scenario: try to login to github
and then do a google search
Given driver1 "<chat-url>"
And input("#user1", 'Hello User')
Given driver2 '<chat -url>'
And match html("#user2) contains "Hello User"
Karate needs changes to support 2 browsers in the same scenario at the same time. I suggest you try the Java API here: https://github.com/intuit/karate/blob/master/karate-demo/src/test/java/driver/demo/Demo01JavaRunner.java
This is not a priority for us right now. you can try find others who need this or contribute code changes if needed. you will need to watch out for this: https://stackoverflow.com/a/61331058/143475
I am trying to write a script that launches an app if not running or activates the window if already visible in the current activity.
Using xdotool or wmctrl I am able to get the list of windows and activate them. If they are not open, then I can launch them. But the problem comes with KDE Activities. These tools list windows from all the activities even if they are not visible in current activity.
I am going through various qdbus methods but not finding anything close.
have anyone created such scripts? how one could get the windows visibility with respect to the activities?
Edits:
as shown in the picture below , I was able to see the activity IDs that the window is attached to. But I am not able to find any way to get it programmatically.
An alternative aproach was given in kde forum. But it is not completely clear if it can help solve your issue.
The recommendation is as follows:
On the activity level you can make use of URI > Activity relations and
query dbus for further scripting. For example:
Link a directory to an activity in dolphin.
Add an application "dolphin-directive" to application launcher and make it run a custom script to conditionally start dolphin instances.
Set "dolphin-directive" as default filemanager
A similar workflow is possible for each filetype via File Association
Settings
As far as I could figure out through experiments it is not possible to link windows to activities and query relations via ActivityManager. I guess the multiple-screen-workspace-uri-activity-window-rule architecture is supposed to setup workflows to solve the problem in a more holistic manner. But hopefully someone can give a better answer here.
I wrote a script to regex inspect the whole session bus tree for related and usefull methods. You can simply use it by ./query-dbus.py --pattern "^.*activit.*$". So the answer is work in progress.
EDIT: some services do have the method isMonitorActivity, isOnActivity
"org.kde.konsole": {
"/Sessions/1": {
"org.kde.konsole.Session": {
"method": [
"setMonitorActivity",
"isMonitorActivity"
]
}
}
}
"org.kde.kate": {
"/MainApplication": {
"org.kde.Kate.Application": {
"method": [
"isOnActivity"
]
}
}
}
}
Did you already file a feature request?
According to this article, Selenium 4 alpha has a sendDevToolsCommand that sends an arbitrary DevTools command to the browser and returns a promise that will be resolved when the command has finished:
Added “sendDevToolsCommand” and “setDownloadPath” for chrome.Driver.
But I can't seem to find how to use it. It sounds a bit like using JavaScript executor in Selenium.
Can someone provide an example usage? I'm using Selenium + Java.
The command to call the devtool api was added a few years back in the Chrome driver.
You can already use it with Selenium even if the method is not yet present:
Take full page screenshot
Print To PDF
Inject some Javascript before the page loads
Block a network URL
Save/restore the cookies for all domains
Get transparent screenshot
This command gives you access to the devtools api, which is used by ChromeDriver internally to drive the browser.
The method takes the name of the command as first argument and a dictionary of parameters as second argument. To figure out how to call a command, add puppeteer in your searches. For instance puppeteer set download location.
Note that executeCdpCommand is implemented in the Java master branch, so it should be available in the next release.
I couldn't find the sendDevToolsCommand in the Selenium documentation yet, but the source actually has the setDownloadPath that you also mentioned above defined right below, which actually uses the sendDevToolsCommand. Based on that usage, it seems like you could do something like:
const { Builder } = require("selenium-webdriver");
const driverInstance = await new Builder()
.withCapabilities({ browserName: "chrome" })
.build();
driverInstance.sendDevToolsCommand('Page.setDownloadBehavior', {
behavior: 'allow',
downloadPath: path
})
or for a visually obvious example:
await driverInstance.sendDevToolsCommand("Emulation.setDefaultBackgroundColorOverride", {
color: { r: 0, g: 255, b: 0, a: 1 } // watch out, it's bright!
});
where the first argument is a Chrome Devtools Protocol Domain method (e.g. or Page.setDownloadBehavior or Emulation.setCPUThrottlingRate) and the second argument is an object containing the options for that Domain method (as described in the same protocol docs).
Edit: just tested and the above works :)
I'm excited that this was added because it means that, in addition to network throttling, it should be pretty trivial to add cpu throttling to Selenium tests now! Something like:
driverInstance.sendDevToolsCommand('Emulation.setCPUThrottlingRate', {
rate: 4 // throttle cpu 4x
}
Selenium 4 release will have a user friendly API for Chrome DevTools protocol.
I just finished implementing Network and Performance domains for the Selenium Java client.
https://github.com/SeleniumHQ/selenium/pull/7212
In addition, there is a generic API for all domains in Java client that was merged a while ago.
All those new features will be released probably in the next Alpha release.
After performing a search using POST /session/{session id}/element, I get this from the Chrome webdriver:
{ sessionId: '3241e7da289f4feb19c1f55dfc87024b',
status: 0,
value: { ELEMENT: '0.12239552668870868-1' } }
Is this what the specs demand?
I am asking because I couldn't find anywhere a spot where it said clearly "ELEMENT" in capital letters. All I can find in the specs is that a key called value is set (which it is: it's set as { ELEMENT: '0.12239552668870868-1' }
Can I always always expect this response, from other browsers' webdrivers? That is, are status and sessionId always returned?
Is that { ELEMENT: '0.12239552668870868-1' } the way chromium makes up the object? Or is this true for any webdrivers? Of not, what do other webdrivers return?
The WebDriver-W3C Candidate Recommendation clearly mentions the following points:
The Find Element command is used to find an element in the current browsing context that can be used for future commands.
Let location strategy be the result of getting a property called "using".
Let selector be the result of getting a property called "value".
The result of getting a property with argument name is defined as being the same as the result of calling Object.GetOwnProperty(propertyName).
GetOwnProperty(propertyName) in ECMAScript® Language Specification is defined as :
String objects use a variation of the GetOwnProperty internal method used for other native ECMAScript objects. This special internal method provides access to named properties corresponding to the individual characters of String objects.
So invoking GetOwnProperty are internal method used for other native ECMAScript objects and are resolved within the internal scope of the Browser Drivers and Browser Clients.
Mozilla have well documented Object.getOwnPropertyNames() and getOwnPropertyDescriptors().
Browser Specific Implementation
I did a small test with all the info you have provided with Search Box of Google Home Page i.e. https://www.google.co.in with all the major variants of WebDrivers and here is the result :
ChromeDriver - OSS :
[[ChromeDriver: chrome on XP (0d24fd038bde751b1e411711271c3e69)] -> name: q]
[[ChromeDriver: chrome on XP (0d24fd038bde751b1e411711271c3e69)] -> name: q]
FirefoxDriver - W3C :
[[FirefoxDriver: firefox on XP (e7a56813-97c5-466e-9c35-24c9f89af6ed)] -> name: q]
[[FirefoxDriver: firefox on XP (e7a56813-97c5-466e-9c35-24c9f89af6ed)] -> name: q]
InternetExplorerDriver - W3C :
[[InternetExplorerDriver: internet explorer on WINDOWS (367257db-cdbc-4be7-aeac-805a21ad9d2d)] -> name: q]
[[InternetExplorerDriver: internet explorer on WINDOWS (367257db-cdbc-4be7-aeac-805a21ad9d2d)] -> name: q]
So as you can observe from the field details of the concerned value field returned is in similar pattern and till the WebDriver variant passes the correct reference to user it shouldn't be a obstacle.
Finally, it is worth to mention at this point of time that like FirefoxDriver and InternetExplorerDriver (both being W3C compliant), ChromeDriver is almost W3C compliant and may vary from a few functional aspects.
Update A
As per your question and update, you are pretty right about ChromeDriver and Chrome communication protocol. Getting more granular we can find some difference in the webdriver call as follows :
Firefox :
1516626575533 webdriver::server DEBUG <- 200 OK {"value":{"element-6066-11e4-a52e-4f735466cecf":"6e35faa4-233f-400c-a6c7-6a66b54a69e5"}}
So, Firefox Browser returns :
"value":{"element-6066-11e4-a52e-4f735466cecf":"6e35faa4-233f-400c-a6c7-6a66b54a69e5"}
Chrome :
[14.921][DEBUG]: DEVTOOLS RESPONSE Runtime.evaluate (id=25) {
"result": {
"type": "object",
"value": {
"status": 0,
"value": {
"ELEMENT": "0.7086986861512812-1"
}
}
}
}
So, Chrome Browser returns :
"value": {"ELEMENT": "0.7086986861512812-1"}
What matters most to we user is the value of the element returned by the browser object which is always referred by an user and correctly identified by the webdriver instance. All these inner logic becomes abstract to the end user.
Update B
Adding some significant bytes from #FlorentB. 's comments :
The earlier versions of Selenium i.e. Selenium v2.x used the keyword ELEMENT to store the reference of a DOM Tree element. This key was changed within the recent versions of Selenium i.e. Selenium v3.x to element-6066-11e4-a52e-4f735466ce. Most of the implementation of the current ChromeDriver is still from the Selenium 2.x specifications.
I have just encountered this same issue, and found the change was made around 3.5 of the Selenium server and related images.
I found this comment the most specific to understand the change and identify which version it changed in:
https://github.com/SeleniumHQ/selenium/issues/4773#issuecomment-333092149
I am using Docker images like selenium/node-firefox:3.4.0-actinium, and have found v3.4.0 returns the ELEMENT key from the older JSonWire spec, whereas v3.9 returns the format element-6066-11e4-a52e-4f735466cecf from the new WebDriver spec. (I haven't checked any other versions in between).
It's part of their gradual migration to WebDriver, but it is a bit confusing that they did this breaking change at 3.5 (or thereabouts) and not v3.0.0 which I think everyone would have been OK with.
Also there's a mix of implementations in the "native" drivers like Gecko which is produced by the Firefox team now, and Chrome, as they will have different development roadmaps.
Furthermore, I've found the client-side library I'm using hasn't even implemented the new response yet, so I'll have to hang back for a while (or patch and PR it myself). I've seen similar conversations in other clients (like the Java client 2 years ago).
You can see the differences between the two protocols' definitions of the Element response:
https://github.com/SeleniumHQ/selenium/wiki/JsonWireProtocol#webelement-json-object
https://www.w3.org/TR/webdriver/#elements
I'm designing a web service running on Google App Engine that scrapes a number of websites and presents their data via a RESTful interface. Based on some background reading, I think I'd like to attempt Test Driven Development (TDD) and develop my tests before I write any business code.
My problem is caused by the fact that my list of scraped elements includes timetables and other records that change quite frequently. The limit of my knowledge on TDD is that you write tests that examine the results of code execution and compare these results to a hardcoded result set. Seeing as the data set changes frequently, this method seems impossible. Assuming that this is true, what would be the best approach to test such an API? How would a large-scale web API be tested (Twitter, Google, Netflix etc.)?
You have to choose the type of test:
Unit tests just test proper operation of your modules (units). You provide input data and test that code outputs proper results. If there are system dependent classes you try to mock them or in case of GAE services, you use google provided local services. Unit tests can be run locally on your machine or on CI servers. There are two popular unit test libs for java: Junit & TestNG.
Integration tests check that various modules (internal & external) work together - they basically check that APIs between modules are working. They are usually run on real servers and call real external services. They are technology specific and are harder to run.
In your case, I'd go with unit tests and provide sets of different input data which you logic should parse and act upon. Since your flow is pretty simple (load data from fixed Url, parse it) you could also embed loading of real data into unit tests (we do this when we parse external sources).
From what you are describing you could easily find yourself writing integration tests. If your aim is to test the logic for processing what is returned from the scraped data (e.g. you know that you are going to get a timetable in a specific format coming in and you now have logic to process that data) you will need to create a SEAM between your web services logic and your processing logic. Once you have done this you should be able to mock the data that is returned from the web service call to always return the same table data and then you can write consistent unit tests against it.
public class ScrapingService : IScrapingService
{
public string Scrape(string url)
{// scraping logic}
}
public interface IScrapingService
{
string Scrape(string url);
}
public class ScrapingProcessor
{
private IScrapingService _scrapingService
// inject the dependency
pubilc ScrapingProcessor(IScrapingService scrapingService)
{
_scrapingService = scrapingService;
}
public void Process(string url)
{
var scrapedData = _scrapingService.Scrape(url)
// now process the scrapedData
}
}
To test you can now create a FakeScrapingService that implements the IScrapingService interface and then return whatever data you like from the Scrape method. There are some very good Mocking frameworks out there that make this type of thing easy. My personal favorite is NSubstitue.
I hope this explanation helps.