Test case:
User books a hotel room. The user status changes to booked_room. This room will become unavailable for other users logged in at the same time. After booking the room, user performs other tasks based on the booked room.
The problem is:
If we run this test on multiple devices simultaneously with the same user account, the test will pass only on the first device. When test runs on second device, the app will request the user status and won't show the rooms list because user already selected one on previous device, causing the test to fail because the REPL couldn't find any room on the elements tree.
Every single test of my suite starts with the login process.
Have you faced to this problem using Xamarin Test Cloud? Should I implement a server-side solution to create a database instance every time the test suite is about to run?
Thank you :)
This is a problem that is not only related to the testcloud. The repeatability of tests is always a problem when a database / shared resource is involved ^^
The answer depends on what you want to achieve with these UI Tests.
A possibility would be to distinguish between integration test and UI test.
Integration:
Test the Web API and the consumer
Services that consume your API are executed as Unit tests
UI:
Test the visual appearance (and App logic)
Mock the services for the UI tests (use BookingServiceMock instead of BookingService for IBookingService) I hope you are using dependency injection :) The Advantage of this approach is, that you can use Backdoors to control your mock (e.g. simulate error cases).
The disadvantage are:
You have no system test, because the App never talks with the actual backend.
Your App under test differs from the App in the App store (on iOS it is normal, because you link the test runner library into the app)
You could solve this with a separate Testsuite that is executed after the integration and UI tests just on one Android device and then on one iOS device.
For a relatively simple approach (depending on your app and test code), you can have each concurrent test (device) use a different set of user credentials and book different rooms. In your test code you would then have some collection of user credentials, room numbers and other variables that should vary per concurrent test.
Xamarin Test Cloud has an environment variable XTC_DEVICE_INDEX:
This is a string in the range of 0 to N-1, where N is the number of
devices the test is run on. This variable is useful in situations
where the same test is being run in parallel on multiple devices. The
value of XTC_DEVICE_INDEX is unique for each test script executing on
a specific device.
Xamarin Test Cloud Environment Variables
This works if you only have one test run executing (on multiple devices) at a time. If a second set of tests start before the first finished it would try to use the same set of variables as the first set of tests. If you need to support this you could provide a web service that allows your test code to check-out the user credentials (and other associated variables) or a key or index to use to pick the right values from those embedded in your test code.
Related
My requirement is to test when four to six different users accessing the web application from different locations and doing the same functionality at the same time.In such scenario which testing tool i should use?To perform such testing can i use selenium webdriver using testNG(session handling)?For example if 6-10 users are creating gmail account from different locations at the same time(complete the functionality of account creation),then i want to see the performance of the web application wheather it has been performed smoothly without any delay or hiccups .
Selenium WebDriver with TestNG, you can write automated tests to simulate your scenario
JMeter or LoadRunner can provide metrics such as response times and resource utilization.
We have a web project and the main business flow is going through the chain of processes (conveyor). We don't have any automatic tests for front. So, to test we use our "hands".
What I want to do is to create some auto test that goes through every step and every window that I do always using hands from my browser.
Any suggestions?
P.S. I don't have an access to frontend app. All I have is the running app on my browser :)
In my experience with quizzes/wizards, such using gamification logic and transitioning from screen to screen - I had to properly plan my efforts.
test that goes through every step and every window
Is easy to go way overboard with all possible paths, resulting in combinatorial explosion. As general advice, try limit the really critical scenarios and cover first those. It is quite likely that the conveyor engine, follows same basic (business) rules and just combines them to produce the flows. In other words, bugs are most likely to be found in the codified domain logic, than the mechanics behind them.
Since this is an Angular app and we all know how pesky Selenium is with it. I would go with Protractor as it is built to handle
native events and browser-specific drivers to interact with your application as a user would
and
supports Angular-specific locator strategies
I would like to know the difference between end-to-end and website test in java web application,If we are testing the features of an website then it could be stated as end-to-end test right?
My second question is if i want to do end-to-end test which the best tool ?
selenium or cucumber ?
I am not able to conclude my self which tool to use.My requirement is as follows
1)Test webapplication automatically by giving the required fields,and also it should interact with my service and database for retrieving information in my UI.
Please help me.I am sorry if its a stupid question !!!
Answering all of your questions one by one :
End-To-End Testing : End-to-End Testing is a testing methodology which is used to validate whether the flow of an application is performing as designed from start to finish.
Website Testing : Web Testing is the name given to the software testing methodology that focuses on web applications. Website Testing can be catagorized as Basic Functionality Testing of the site, Security Aspects of the Web Application, Accessibility to intended users and Performance Aspects as well.
Best Automation Tool for End-To-End Testing : Each and every automation tool have its advantages and disadvantages. Selenium-Cucumber is one of the widely used combination for Behavior Driven Development (BDD). But as a pure End-To-End Automation candidate Protractor is widely used.
Overall using Selenium you can perform Web Application Testing by giving the required fields and also interact with the services and data retrieved from database on your UI.
You're quesiton will probably be removed from here since it doesn't fit the standards stack overflow requires but here's an answer in case it sticks around.
End to end testing concerns program flow from start to finish. If you have a system that is solely a website then they could be considered equivalent. If your project has a website and runs batch processes in the background, sends emails, etc. based on data entered into the website or actions users of the site take, then they're not really equivalent because testing your website isn't going to test that the mail server is configured properly or that your nightly batch runs properly.
There is no single tool to handle end to end testing. At best you'll have an amalgamation of various tools to handle testing but more than likely there will be some manual testing involved - it's all going to depend on your application.
You could for example: script selenium to enter orders into a site, then have a scheduled console application run overnight that verifies the end of day accounting handles the orders properly, you would manually check outlook to verify you received your order confirmation, and then come up with a way to verify the orders get sent to the fulfillment system.
As you can see by the example for that use case you're unlikely to find a general purpose tool that can handle all of that out of the box.
I have started to use Specflow, Selenium and PageObject pattern for testing. I would like to make scenarios independent of each other, but when I start running my test features, I see that my user is not anonymous after the first scenario.
When should I log out the test user?
Before each scenario?
After each scenario?
Can specflow and selenium drop state after each scenario?
Must I call page.Logout() every time?
I would say that you should log out, when it's relevant to the test case. Say you log in to SO and test posting a new question, then write an answer and add two comments to each. Logging in and out between each step would be a big hassle and no user would do that in the real world. Thus just log in at the start of the test scenario and log out when you're finished with it.
Another example would be doing each of the above steps as different users, then it would be a necessity to log in/log out at every step.
This applies in the bigger picture as well, if you have multiple scenarios that require a logged in user, but don't depend on any of that user's information, may as well just log in once before running all of them and then log out.
Specflow and Selenium don't hold the state, it's the system you're testing that does it. For instance if your session is stored in cookies, you could clear said cookies and that would in effect log you out. But this is not testing the system as the end user is meant to use it, so it's just cutting corners and testing different area of system (authorization of unauthenticated users) and thus doesn't really correlate very well to the real world case. Just use the log out button/link on the page, since you're writing browser based tests.
I think you should make your test feature wise or module wise.
if you consider all the processes in a feature and according to the flow, you will only need to logout in the end.
If you make module wise, after a process you can logout each time and login again at the new module.
Now I hava a requirement about Web GUI TA
I want to simulate some users(20-30) click a button at the same time and evaluate the performance of Web GUI at that time.
I use RobotFrameWork + Selenium library to do the Web Gui TA before, but as far as I know. selenium library only can handle one broswer at one time, so i dont know how to do now.
Can you give me some advice? need use another library or framework?
Like mentioned by other, what you want to do in this case is not UI testing but rather stress/load testing. You should be able to try easily Gatling. First you record the http request associated with the click on your button. Then, you write a simple scenario that launches this request 20 times at once. Something like:
setUp(scn.inject(atOnce(20 users)))
.protocols(httpConf)
Selenium has a "grid" option you can use to configure many instances running many browsers.
http://www.seleniumhq.org/projects/grid/
http://www.seleniumhq.org/docs/07_selenium_grid.jsp
Grid allows you to :
scale by distributing tests on several machines ( parallel execution )
manage multiple environments from a central point, making it easy to
run the tests against a vast combination of browsers / OS. minimize
the maintenance time for the grid by allowing you to implement custom
hooks to leverage virtual infrastructure for instance.
In short, you create a "hub" that manages things, then each "node" can perform tests as required by the hub.
Consider, however, this may not be the best route to go down. Something like multi-mechanize might be more useful: http://testutils.org/multi-mechanize/
That will allow you to have many users "clicking" the button, but not via a browser but via direct HTTP calls. That might be more suitable for multi user simultaneous "headless" load testing, which is what I think you are attempting to do.
I'm slightly confused at this question:
Are you wanting to test the GUI? If it's something like "This button makes a dropdown menu appear", then it doesn't matter how many users do it at the same time, it'll either always work, or never work.
Are you wanting to test the server under load? If so, then Selenium will work, but there are better tools. I have used JMeter with success, but there is a really good listing of all of them here: http://performance-testing.org/
Finally, are you wanting to press 20 different buttons on the same page on the same browser at the same time? If so, this isn't possible with selenium, and it isn't a standard use case either.