Let's say I'm testing a web service and I have a couple of scenarios requires user to be authenticated:
Scenario #1: Customer sign-up
Scenario #2: Customer sign-in
Scenario #3: Customer change name
Scenario #4: Customer update image
Should all the tests go through all login steps like:
1) Go to register page
2) Enter new user information
3) Activate account
4) Go to login page
5) Enter login and password
6) Press the Login button
7) Check if I authenticated as a customer
Or I can just test it once and implement endpoint which quickly creates a user and log it in.
So if I have that kind of endpoint that means I can skip retesting the same things all the time and just have short scenarios #3 and #4 implementation. But in this case, I have a less natural environment.
Please tell me about the best practices that you use in real projects.
Few best practices:
use the testing pyramid integration > ui (tests are much slower on UI, automate in the UI only the necessary things to have main flows covered)
for the UI use fast methods for the setup (so yes, web services, test login only once)
if possible keep some test data trough the builds (for example to make sure a new build that might change data structure does not affect basic functionality, e.g. login)
tests should be atomic (not depend on each other)
do some cleanup from time to time to remove duplicate test code and to improve the framework(speed, stability)
You shouldn't copy and paste the "log in" scenario to all of the other scenarios, but having an account and being logged in are prerequisites for the other use cases. From a behavior driven development perspective this will translate to one or more Given steps that simulate or actually perform those steps:
Scenario: Customer change name
# Calls web service or database to create new user
Given "Bob" is a registered user
# Calls web service or database to make account active
And "Bob" has an active account
# Opens browser, navigates to login page, fills out login form and submits it
And the user is logged in as "Bob"
# Steps specific to changing name and asserting it has changed
When the user changes their name to "Samuel"
Then the user's name is "Samuel"
Related
I created concurrency Thread Group with 10 Users in 1 Min, but when i run it ,i am getting 409 conflict in Login Authorization request.
is it possible to run 10 Concurrent Users with single login details?
enter image description here
It depends on how the system under test is configured, in your case it seems to be impossible
In general you need to configure JMeter to behave exactly like a real browser, to wit each JMeter's thread (virtual user) must represent a real user using a real browser and a real user:
has its own credentials
cache
cookies
think time
business actions
so ideally you need to create as many "login details" as virtual users you need to simulate
I am doing load tests to a apply option of career fair section. But I have only one user's login informations and I want to do apply tests more than one time with same user. For example when I create 10 users with Thread Group, the http request for apply is going to be fall 9 times. I think that's why we use only one user and the apply button is going to be applied button and not clickable. How can I do this test?
There is one golden rule: 1 virtual user == 1 real user with all its stuff like credentials, permissions, business actions, cookies, cache, etc. See How to make JMeter behave more like a real browser article for more details.
So ideally you need to have as many credentials as threads (virtual users) you set in JMeter's Thread Group
Test need to be repeatable, to wit leave the system in unmodified state so you could re-run the test once again. If the application process is irreversible you need to either cancel the application somewhere in the tearDown Thread Group or modify the database using JDBC Test Elements to restore the system state
If your system doesn't allow multiple logins from one user I don't think you will be able to conduct your test with a single user for multiple sessions.
A coworker and I are having an argument about the proper way to test. I'll lay them out as neutrally as I can, and won't give my opinion in this post.
This is written in gherkin. Substitute whatever pseudocode you want to.
Given I am a registered user
When I submit my credentials
Then I can login
Case one:
Given I am a registered user
(instantiates the user,
stores the user to scenario-global memory,
adds the user to the db with an API endpoint
stores the API endpoint result to scenario-global memory [200, "Success message"])
When I submit my credentials
(test the result of the previous step [200],
fills the credential field(s),
clicks submit, stores the result to scenario-global memory [200, "Success message"])
Then I can login
(tests the results of the previous step)
Case two:
Given I am a registered user
(instantiates the user,
stores the user in scenario-global memory,
adds the user to the db, tests the result of the db command)
When I submit my credentials
(fills the UI credential field(s),
clicks submit)
Then I can login
(perform some operation that only a logged-in user could do [see the my profile button/call some api endpoint])
The contention is whether or not the Then step can rely on the result from a previous operation.
It depends. First of all, as a tester, i would never directly write something to DB, instead i register the user in UI for UI testing or call endpoints to cover API testing scenarios.
Also, IMO, not every step is responsible for testing its own step, otherwise there is no need for 'Then' Gherkin syntax. This way of writing user stories makes it more understandable to non-technical but business people very clearly about the scenario under test.
For both cases above, i vote for case one and your co-worker's suggestion.
Overall, one should aim for declarative BDD scenarios, when trying to cover the features from end-user perspective. Pushing your how down is vital part of it. Employing Back door manipulation is great, but ask youself this - does your end user (or stakeholder) really care about technicalities?
adds the user to the db with an API endpoint
is something that is best used, when the register user flow is covered at least once through the UI. Mixing APIs and UI in the scenarios, can lead to a mess and maintenance issues. A proper DSL should handle that for you.
As to
each step is responsible for testing its own results. It is not responsible for what happened in the past or what is about to happen in the future
Cucumber-ish frameworks do use DI of context/state, exactly for this - share steps execution run in the current World instance.
I need some clarification in the testing process, specifically when multiple users (100 Users) login to a web application through JMeter.
I can log in with a single valid user but if there are 100 users and 1 is a valid user and 99 are invalid users, the 99 users cannot log in.
The problem is creating 100 is a difficult process.
Now, is testing login as mentioned above the same as testing with 100 valid users?
If not, is there any better process to test login with multi-users?
There is only one obvious requirement: each JMeter thread (virtual user) should use different credentials, in other words JMeter user must represent real user using real browser as close as possible, otherwise your load testing will not make sense.
So ideally you should have 100 different credentials so each virtual user could use its own username/password combination and have its own session. It particularly matters when your test scenario assumes some business processes, i.e. one user starts workflow, another one approves, third one finishes, etc.
If each load test iteration assumes "clean" system you could consider automating user creation process via setUp Thread Group where you can create the prerequisites (users, content, whatever). Ask around, it might be the case you can create the user using a single REST API or Database call, or it could be possible to import users from LDAP or using a shell command
As a last resort you can use single credentials with multiple JMeter virtual users, however in this case you may run into issues with your application so try avoiding CRUD operations so your test would represent just browsing.
Here is my project architecture components:
Front-end (Drupal crm)
Symfony 2 services (API layer)
CRM (external service to store users)
I would like to setup functional testing with codeception to test subscription part of my application.
Now, here is the process for subscription:
1/ Display form in front-end
2/ When form is submitted data is post to Services
3/ Services establish a connexion with the CRM and return a Json response to front-end.
4/ Front-end redirect and display a success message (or errors).
I need to clean up my data before run my test (test is get on /subscription, fill form, submit and test response). In fact, if a user exist in CRM, I need to delete it if I want test a success case.
In my case, I don't have a way to delete a specific user before running my test (no route in API to delete a user).
How can I achieve this? I'm not supposed to have a detached database to test this case.
You need to write background in feature file to delete data for each scenario. I quickly visited codeception and it is BDD testing framework.
Feature: Test Subscription
As a system admin
I want to test subscription system of "APPLICATION NAME"
Background:
Given delete all test data from system
And test data is delete successfully
Scenario: Create User in CRM
Given I am logged in as system admin
When I try to click to new customer
And Enter ...
And Enter......
When I click on customer list
Then I should see created customer in list
Background will run after each scenario and do job for you.