I've been asked to carry out some testing to assess the response time under varying load on an API that is being developed in my current job. I’ve got a very basic knowledge of jMeter so I’m looking at this as my chosen tool to use.
Here’s quick Back ground info of what I’m tying to achieve:
I want to test the API by simulating a number of unique Json calls each made by different users. Each unique Json call contains a unique user ID and item number.
Lets say for example I want to test, 10 different users, all making calls for different Items to the API at once.
I’m a little confused with the terminology and difference between thread groups and users and loop count (which are found inside thread groups) and how best to use them. I couldn't find an answer on the jMeter site or on here so have come up with what I think may be a possible approach....
From my understanding of how jMeter works. In order for me to achieve my aim would require me to create 10 separate thread groups within the test plan, each of these having their unique Json body data configured.
For example : Thread group 1 containing request 1, Thread group 2 containing request 2, Thread group 3 containing request 3, Thread group 4 containing request 4, Thread group 5 containing request 5.... and so on.
( note, I would uncheck the option ‘run thread groups consecutively so that all requests get sent at once)
If the number of users and the loop count in each thread group is set to 1 then each call is made only once. Meaning a total of unique 10 calls would be made during a single test run?
I guess, what I’m kind of asking is in my scenario, does the term thread group actually represent unique users, and if so what does the number of threads(users) actually represent
I think I've got this right but it would be good if someone more experienced might be able to add some confirmation or advice to point me in the right direction if I've got this wrong.
Thread Group is an element in JMeter test plan which is responsible for executing the scenario/business work flow for the given set of users & for the given duration(or iteration/loop count).
Lets say I have a business flow like this.
users login
search for product
select a product
order the product
Another business flow for the same application would be
Admin users login
Search for users
modify the order/cancel the order/issue refund
I would create 2 thread groups for 2 different work flows.
In production/live environment, I might have 1000 users searching for products and 500 users might be placing the order.
I might have only 10 admin users to provide support for 1000 users - to provide refund/modify the order etc. [some might do refund / some cancel / some modify]
So,
Thread Group 1 [users1=1000, rampup=3600,loop count=forever, duration=7200(2 hrs)]
users login
search for product
If Controller [let only 500 users to order product]
select a product
order product
Thread Group 2 [users1=10, rampup=600,loop count=forever, duration=7200(2 hrs)]
Admin users login
Search for users
Throughput controller [40%]
modify the order
Throughput controller [30%]
cancel the order
Throughput controller [30%]
some other support
As you mention you are new to JMeter, I would suggest you to check this.
JMeter Tips/Tricks for beginners / Best Practices
JMeter - REST API Testing
I believe using Synchronizing Timer to ensure all 10 threads are running at exactly the same moment will be easier and smarter choice.
And this is some general information regarding how JMeter works:
JMeter starts the specified amount of threads during ramp-up period
Each thread starts executing samplers upside-down (or according to the Logic Controllers)
When there are no samplers to execute and loops to iterate - thread is being shut down. Elsewise it starts over until desired test duration or loops count is met/exceeded.
Hopefully it clarifies everything
Related
Hi i'm actually working on a report system for my social network, i want to achieve something like : if a user is reported 5 times it will update a field on the user table to deactivate the user.
So if 5 row of reports is created i want to update the "activated" field
The thing is i'm working with GraphQL and Vue for the first time, my backend is in Symfony with ApiPlatform.
Should i try to query all the reports with the userId who is reported to count if there is 5 row when a user is reported and then mutate the user object to deactivate it or can i achieve this more easily on the backend side ?
So anything of importance should be done on the backend. The frontend is easily hacked and while it's good to stop normal (good) users from seeing something they're not supposed to, it's not a blocker to anyone who has even rudimentary web coding ability.
Seeing as you're talking about someone being reported, I'd recommend doing it on the backend and then make sure the frontend reflects that gracefully.
Hum it seems that the answer is that i need a reportListener and i need to listen the postPersist event, then inside this function i can check how many report there is for a user using the ReportRepository->findBy function, then i can count the row of the array return by the function and if the count is > 5 then i edit my user, persist and flush.
We use Camunda to run business processes but my question is more about general BPM patterns than a specific tool that implement it.
Let suppose the following very simple use case and a homemade application on top of it that use BPM engine API to query tasks and complete them.
Let suppose that Action 1 will be auto assigned to John and Action 2 to Bob. From my comprehension of BPM, when the process start, Action 1 is created and assigned to John. When John complete the Action 1, the Action 2 is created and assigned to Bob. And that's what we want because Bob could not start working on Action 2 if Action 1 is not yet completed. Fine...
Now suppose that as the team manager, I want to schedule activities of John and Bob. I'm thinking about introducing an initial task handled by me to configure a planning.
When the process now start, a new task is created and is assigned to me. I want in my homemade application to query the BPM engine in order to get all tasks (actives and inactives) in order to add planning information (planned start date and due date). For example, I want to specify that:
Action 1 should start on 01/01/2021 and should be closed on 03/01/2020.
Action 2 should start on 04/01/2021 and should be closed on 05/01/2020.
With this kind of information, John and Bob could connect to the task list and see that a "not yet started" task is waiting for them on a specific time range. The main goal of this is to manage a team calendar of activities.
I don't see how to implement that in BPM due to the fact that, in my comprehension, tasks are created when the process reach them so we could not have information about "Not Yet Started" tasks.
Any ideas? Suggestions? Bests practices?
Tx
I asked a question above, but if the answer is "Action 2 still requires Action 1, and I don't want people to see the tasks before their start date." Then the model would look like this -
The timers would be set to the start dates the manager picked. I have a strong opinion that the tasks shouldn't be in the list if the user can't work them. So if they should not work it before the start date, we shouldn't have it in their task list. This is different than say, a report on the status of the process, which certainly can show that we are waiting until date X to issue Action 1. There is also a way to model it so that someone can say "Yeah, you know what, I want to start this now" and trigger the action early from that dashboard.
To have the tasks for John and Bob active at the same time, you could use a parallel gateway ... and then filter the tasks in a custom task list based on the configurations you made ...
If you want to keep the sequence of tasks (which is the better option imho) you can still set variables in your planning task up front and then read these values when the second task is started by registering a TaskListener on your model.
Is it possible to anonymously programmatically verify that a transaction has reached n number of validations without running a full node? If so, what is the best means to do this?
Basically I want it to build a payment system where after the transfer is initially detected, the customer sees a message thanking them and telling them that their purchase will be processed within 24 hrs and that they'll receive an email once confirmation is complete. Then throughout the day maybe run a cron job that checks that each transaction reaches the desired number of validations and if so divide the money between two wallets and mark the product to be sent. I also don't want it to be with a service like Coinbase or Bitpay where they have control of your coins.
So far I've been experimenting with Blocktrail and mycelium gear. Both have some elements I like but still not everything that I need. With mycelium you can set the number of verifications but for instance if I want to set it for 6 verifications the customer would have to sit there possibly an hour before they see the next screen. Blocktrail allows me to query that a transaction is validated but it only has the ability to check that 1 validation was completed as far as I can tell. Can anyone suggest an API or widget that can accomplish these things? Preferably PHP or if not JQuery.
Blockchain.info has a simple Query API for querying how much bitcoin an address has received. You can add a confirmations=n parameter that will only include bitcoin that has been confirmed 'n' times. It returns a simple value in satoshis.
For example to check how much bitcoin was received with at least 2 confirmations at a specific address you could have your code query the API like this:
https://blockchain.info/q/getreceivedbyaddress/1PFtyX9nQvjP8U2N3iUk2oNorzPfpjX9sK?confirmations=2
I'd like to use mturk to have 10 workers visit my website, log in with a test account, and enter some information on their profile. I don't want them to see each other's entries, so each worker should get login information for a different test account when they view the HIT.
This almost looks like what mturk's template feature is for -- I could upload a CSV with the information for each test account. But if I understand correctly, that will make 10 separate HITs, and allow one worker to do all 10 of them. Is there any way to have mturk put information that varies between workers into a single HIT?
Here are the solutions I'm currently aware of:
Use the CLI to automate creation of a bunch of different HITs. This would be a lot of work, and also make approving and retrieving the results cumbersome.
Direct workers to a survey website that's capable of doing what I want, and have them get the login information there.
Dynamically fill in part of the HIT using an AJAX request to an external website and database. That seems like crazy overkill for something so simple.
Are there other options?
I have set up SpecFlow to execute a number of Selenium tests to test a website. This all works fine!
I am just considering if there might be room for some optimization on the structuring of the SpecFlow features.
All the scenarios in a specific feature should use the same login. This is something I have hardcoded in the StepDefinition using the [BeforeScenario()] hook at the moment, since I don't really wan't pollute the scenarios with login info. It isn't relevant for the test.
But at the same time, I would like to remove the hardcoded part, and move this into my feature.
My question is two part.
Can I specify login credentials in my feature description. Sort of like a Given:
Feature: As a user I want to be able to see my ongoing orders, and interact with them.
Given I am logged in with username abc and password xyz
Scenario: See list of ongoing order
Given I place an order
When I navigate to MyOrders page
Then I can see at least one order in the list
Is this good practice?
Does it make sense to do it like this, on the feature level I mean. The scenarios are not dependent on a specific order, and they would execute faster if I didn't need to log in for each scenario.
Thanks for your input.
For steps that are common to all scenarios in a feature you can use Backgrounds:
Feature: As a user I want to be able to see my ongoing orders, and interact with them.
Background:
Given I am logged in with username abc and password xyz
Scenario: See list of ongoing order
Given I place an order
When I navigate to MyOrders page
Then I can see at least one order in the list
Too much abuse of background steps can be a bad practice because it introduces coupling between your scenarios.
Another solution is to put the login part directly into your "I Place an order" step.
This will remove all the noise about login stuff since it's implicit that you need to log in to place an order.
I'd also suggest to call it "I've placed an order" instead of "I place an order".
Given steps are usually pre-condition that describes what happened prior using the functionnality (when steps)
When I first read your post, I thought of Dan North: Who's domain is it anyway? and that influences me to think that we should be trying to write our tests so that they stick to a single knowledge domain. As you talk about specific users and navigation and orders in a list, well its very like the example he gives in that your specification is crossing domains. This kind of leads me towards almost making your specification less specific, i.e. its not about navigating and checking a list, do you have an order or not!
Scenario: See list of ongoing order
Given I am logged in with username abc and password xyz
And I place an order
Then should have at least one order
I wanted to give you another example link, to a post I cant find, which talks about the benefits that a team has got by actually defining some example users and what value that gives their testing. So in your business domain you might have a user called Jack who will have placed a large order in the system, and another prospective client Jill with no orders. In this way tests such as
Given I am Jack
When I search for my orders
Then I will find 1
Given I am Jill
When I search for my orders
Then I will find 0
can guarantee that you are only testing your Search functionality, and less about getting the setup in place.
I'd also suggest that you have a look at Liz Keogh: Acceptance Criteria vs Scenarios who asserts that more general broad definitions are less valuable than very specific examples. (It is Specification By Example after all :-) )
So to answer your question, I think having the specified user is a good thing, but doing it how you are doing it is heading a very complicated route.
The scenario examples that are given are completely not in line with the Feature.
Feature: As a user I want to be able to see my ongoing orders, and interact with them.
Background:
Given I am logged in as John Doe
Scenario: This is taking to long
And I have placed an order more than 29 days ago
When I navigate to MyOrders page
Then I can see my order in the list
and I am able to cancel it
Scenario: Track package
And I have been notified my order has been send
When I navigate to MyOrders page
Then I can see my order in the list
and I am able to navigate to the postman's site and see its status
Scenario: Review item
And I have received my package
When I navigate to MyOrders page
Then I can see my order in the list
and I am able to review the item
Scenario: Contact Seller
And I have placed an order two weeks ago
When I navigate to MyOrders page
Then I can see my order in the list
and I am able to ask the seller what is going on
As you can see the When and the Then is starting to duplicate itself and you might want to consider tucking those away.