I am reading book Embedded Software Development for software critical systems about combinatorial testing.
Suppose if we have 11 parameters that we can configure the operating system, and each parameter had 5 possible values (for a total of 48,828,125 possible tests, were the system to be tested exhaustively).
Note: Here t is the strength" of the testing (all combinations of t of the parameters).
With t = 2 (i.e., all possible combinations of every pair of parameters), there must be at least 5 * 4 = 20 test cases and, For t = 3, there must be at least 5 * 4 * 4 = 80 test cases.
My question is how we author got 5 *4 for t =2 and 5 *4 *4 for t=3? Is there any generic formula present? Kindly help me to understand.
Thanks for your time and help
Related
I'm using BigQuery at my new position, and I'm totally new to SQL/BigQuery.
I'm testing a machine learning model and monitoring an A/B test with a different ratio, e.g., 3 vs. 10. To compare the A/B results, e.g., # of page view, I want to make the ratios equal first so that I can compare easily. For example, say we have a table with 13 records (3 are from A and 10 are from B). In addition, each row contains an id field that is identical. What I want to do is to extract only 3 samples out of 10 for B to match the sample number to A.
I'm trying to use the FARM_FINGERPRINT function to map fields to integers. Then I'm taking ABS and then calculating MOD to convert the integer numbers to a specific range, e.g., [0, 10). Eventually, I would like to get 3 in 10 items using the following line:
MOD(ABS(FARM_FINGERPRINT(field)), 10) < 3
However, I found that even if I run A/B with exactly the same ML model with different A/B ratio, the result is different between A and B (The results should be same because A and B are running the same ML model with just the different ratio). This made me doubt that the above implementation may bring some biased data sampling. I also read this post and confirmed the FARM_FINGERPRINT might not bring a randomly distributed result.
*There's a critical reason why I cannot simply multiply 3/10 to B, which is confidential and cannot disclose here.
Is there a better way to accomplish the equally distributed sampling?
Thank you in advance. (I'm sorry if the question is vague, as I'm hiding the confidential parts.)
I'm doing the research for my QA project and I'm wondering if Karate is able to handle certain use cases. Basically I need to run tests for different environments (local, staging, production). What I understood from the documentation, it is not a problem because of karate-config.js and karate-config-env.js.
The problem starts with the execution itself. Each environment has different urls for 3 different countries, so actually there are 9 urls in total. Moreover, because of the development process, certain features are deployed not at the same time for all countries. So I want to be able to run tests against:
1 - staging for one country (one url)
2 - staging for all countries (the same request with 3 urls, I guess I can use parallel execution)
The json structure is the same for all environments and countries and I want to execute one request with different configurations. I was thinking about TDD but I'm not sure if I can skipp some rows from Scenario Outline table if I'm executing tests for only one country. Is it possible? or is there any other way? Any advice appreciated.
You can "tagify" Scenario Outline rows. See the docs: https://github.com/intuit/karate#tags-and-examples
Scenario Outline: examples partitioned by tag
* def vals = karate.tagValues
* match vals.region[0] == expected
#region=US
Examples:
| expected |
| US |
#region=GB
Examples:
| expected |
| GB |
Karate can handle pretty much any data-driven challenge you have, once you understand how JSON, manipulating JSON and data-driven testing works. Here are some answers that will give you further ideas to consider:
https://stackoverflow.com/a/61685169/143475
https://stackoverflow.com/a/59162760/143475
Assume you are a test analyst working on a banking project to upgrade an existing automated teller machine system to allow customers to obtain cash advances from supported credit cards. The system should allow cash advances from 20 dollars to 500 dollars, inclusively, for all supported credit cards. The correct list of supported credit cards is American Express, Visa, Japan Credit Bank, Eurocard, and MasterCard. The user interface starts with a default amount of 100 dollars for advances, and the ATM keypad is used to increase or decrease that amount in 20-dollar increments.
Consider the decision table shown in table 1.0 that describes the handling of these transactions.
Table 1.0. Cash advance decision table
Check the table in attached image
Assume that you want to design a set of test cases where the following coverage is achieved:
Decision table coverage
Boundary values for allowed and disallowed advance amounts
Successful advance for each supported card
Design a set of test cases that achieves this level of coverage with the minimum possible number of test cases. Assume each test case consists of a single combination of conditions to create and a single combination of actions to check. How many test cases do you need?
Can someone help me understanding this problem and solution?
Thanks in Advance :-)
Decision table coverage Boundary values for allowed and disallowed advance amounts ->
Boundary values for your example will be: less than 0; 0; 20-500; 500+.
Equivalence partitioning, boundary value testing and decision table described here: http://www.maniuk.net/search/label/test%20design%20technique
Successful advance for each supported card ->
Set of instruction number 5 (in decision table) should be applied for all types of provided cars. Depends on risks #4 should be tested too.
Design a set of test cases that achieves this level of coverage with the minimum possible number of test cases. -->
a. If we can assume that cards work totally the same with the same limits and processing procedures so 9 test cases needed, during boundaries testing you can use different cards, so each card can be used. b. If we assume that some specific still exists in processing so 13 test cases needed (9 from previous test + 4 other cards to test instruction #5.
c. If cards has different limits by themselves addition verification will be needed.
I have n things that are going to be tested by t testers. Each tester is subject to order bias (that is, they may give higher ratings to things they test earlier - or the reverse), so I want to eliminate that. One way to do that would be to generate a random testing sequence for each tester.
For example, with n=5, m=2:
Tester 1 Tester 2
4 2
2 5
5 4
1 3
3 1
Notice that in this generated sequence however, both testers test thing 5 immediately after thing 2. Also, things 3 and 1 both appear towards the end of the testing, for both testers. This is sub-optimal.
My question: how can I generate an optimal set of sequences, maximising the chance of each thing appearing at every different position, and minimising the repetition of individual sequences.
Harder question: how much more optimal would that be than the naive (pseudo-)random generation? Can that be quantified?
This isn't a homework question, although it may sound like it :) (It's for a wine tasting...)
I really wasn't sure whether this should go in math.stackexchange, cs.stackexchange, or here. Ultimately I actually want to implement this, so...
I am working on a system to display information about real estate. It runs in angular with the data stored as a json file on the server, which is updated once a day.
I have filters on number of bedrooms, bathrooms, price and a free text field for the address. It's all very snappy, but the problem is the load time of the app. This is why I am looking at Redis. Trouble is, I just can't get my head round how to get data with several different filters running.
Let's say I have some data like this: (missing off lots of fields for simplicity)
id beds price
0 3 270000
1 2 130000
2 4 420000
etc...
I am thinking I could set up three sets, one to hold the whole dataset, one to create an index on bedrooms and another for price:
beds id
2 1
3 0
4 2
and the same for price:
price id
130000 1
270000 0
420000 2
Then I was thinking I could use SINTER to return the overlapping sets.
Let's say I looking for a house with more than 2 bedrooms that is less than 300000.
From the bedrooms set I get IDs 0,2 for beds > 2.
From the prices set I get IDs 0,1 for price < 300000
So the common id is 0, which I would then lookup in the main dataset.
It all sounds good in theory, but being a Redis newbie, I have no clue how to go about achieving it!
Any advice would be gratefully received!
You're on the right track; sets + sorted sets is the right answer.
Two sources for all of the information that you could ever want:
Chapter 7 of my book, Redis in Action - http://bitly.com/redis-in-action
My Python/Redis object mapper - https://github.com/josiahcarlson/rom (it uses ideas directly from chapter 7 of my book to implement sql-like indices)
Both of those resources use Python as the programming language, though chapter 7 has been translated into Java: https://github.com/josiahcarlson/redis-in-action/ (go to the java path to see the code).
... That said, a normal relational database (especially one with built-in Geo handling like Postgres) should handle this data with ease. Have you considered a relational database?