Is it ok to run more than one experiment at the same time?
Is it valid to emit more than one line for experiment.variant in JS code targeting Google Optimize?
For example:
-- first test
ga('set', 'exp', 'ExperimentA.Variant1');
-- second test
ga('set', 'exp', 'ExperimentB.Variant1');
Clearly, I can't assign exp twice.
At the same time calling
ga('send', 'pageview');
after each set would be wrong too since it would result in GA counting the page twice.
Can someone help?
Related
Do we have an API in Qtest that can provide summary of test cycle execution ?
E.g. Passed: 23 Failed: 7 Unexecuted: 10 Running: 2
Need this data for generating report in our consolidated reporting tool along with data from some other sources.
Nothing that gives exactly what you ask for, but you could use the API calls below to create it yourself.
You can get the status of all test runs in a project using
GET /api/v3/projects/{projectId}/test-runs/execution-statuses
Or, to get results from a specific test cycle, first find all the test runs in that cycle using
/api/v3/projects/{projectId}/test-runs?parentId={testCycleID}&parentType=test-cycle
(append &expand=descendants to find test runs in containers under the test cycle)
and then get the results of each run individually using
/api/v3/projects/{projectId}/test-runs/{testRunId}/test-logs/last-run
See https://qtest.dev.tricentis.com/
In normal applescript, the script is executed down the page, and so any code in loops for every 5 seconds will only run while the loop is running - there is no way to have a single function run every few second regardless of what the script is currently doing or where it is in the script (that I know of). In cocoa-applescript, however, is there a way to run a handler every 5 seconds, at all times, no matter what it is currently doing? Here is what it should be doing in my cocoa-applescript app:
on checkInternetStrength()
do shell script "/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport -I | grep 'agrCtlRSSI:'" -- this being the script which returns the line containing the signal strength
set SignalStrength to result
set RSSIcount to (count of characters in SignalStrength)
set SignalStrength to ((characters 18 thru RSSIcount of SignalStrength) as string) as integer -- this to turn SignalStrength into just the number and not the whole output line
set SignalStrength to (100 + SignalStrength) as integer
set SignalBar's setIntValue_(SignalStrength) -- SignalBar being the Level Indicator described below
end checkInternetStrength
Summed up, it runs the airport command to check internet connection, turns this into a number from 1 to 100 and uses this on an NSLevelIndicator (100 maximum) to show current signal strength graphically. Now, there is no point having this run once or when you hit a button - that is an option, but it would be nice if it updated itself every, say, 5 seconds with the realtime value. So is there any way to have a process which runs every 5 seconds to do this, while still enabling full functionality of the rest of the script and interface - i.e. as a background process? Comment if you need more extracts from the script.
Example
In Unity-C# scripting, the 'void Update() {code}' will run the code within it every frame while doing everything else simultaneously, so a cocoa-applescript version of this might be an answer, if anyone knows.
I Dont believe this is possible but what I had a similar problem before, what i do, I have an external applescript applicaion that is hidden the repeats the commands, the only problem is, it wont send it back to the app, you'll have to make the external applescript app do it, like
display notification, etc..., in the applescript apps "Info.plist" you can add this:
<key>LSUIElement</key>
<string>1</string>
To make the app run invisibly, but sorry i dont think you can run a handler in the app its self
How does one use the flag options for benchmarks with the gocheck testing framework? In the link that I provided it seems to be that the only example they provide is by running go test -check.b, however, they do not provide additional comments on how it works so its hard to use it. I could not even find the -check in the go documentation when I did go help test nor when I did go help testflag. In particular I want to know how to use the benchmark testing framework better and control how long it runs for or for how many iterations it runs for etc etc. For example in the example they provide:
func (s *MySuite) BenchmarkLogic(c *C) {
for i := 0; i < c.N; i++ {
// Logic to benchmark
}
}
There is the variable c.N. How does one specify that variable? Is it through the actual program itself or is it through go test and its flags or the command line?
On the side note, the documentation from go help testflag did talk about -bench regex, benchmem and benchtime t options, however, it does not talk about the -check.b option. However I did try to run these options as described there but it didn't really do anything I could notice. Does gocheck work with the original options for go test?
The main problem I see is that there is no clear documentation for how to use the gocheck tool or its commands. I accidentally gave it a wrong flag and it threw me a error message suggesting useful commands that I need (which limited description):
-check.b=false: Run benchmarks
-check.btime=1s: approximate run time for each benchmark
-check.f="": Regular expression selecting which tests and/or suites to run
-check.list=false: List the names of all tests that will be run
-check.v=false: Verbose mode
-check.vv=false: Super verbose mode (disables output caching)
-check.work=false: Display and do not remove the test working directory
-gocheck.b=false: Run benchmarks
-gocheck.btime=1s: approximate run time for each benchmark
-gocheck.f="": Regular expression selecting which tests and/or suites to run
-gocheck.list=false: List the names of all tests that will be run
-gocheck.v=false: Verbose mode
-gocheck.vv=false: Super verbose mode (disables output caching)
-gocheck.work=false: Display and do not remove the test working directory
-test.bench="": regular expression to select benchmarks to run
-test.benchmem=false: print memory allocations for benchmarks
-test.benchtime=1s: approximate run time for each benchmark
-test.blockprofile="": write a goroutine blocking profile to the named file after execution
-test.blockprofilerate=1: if >= 0, calls runtime.SetBlockProfileRate()
-test.coverprofile="": write a coverage profile to the named file after execution
-test.cpu="": comma-separated list of number of CPUs to use for each test
-test.cpuprofile="": write a cpu profile to the named file during execution
-test.memprofile="": write a memory profile to the named file after execution
-test.memprofilerate=0: if >=0, sets runtime.MemProfileRate
-test.outputdir="": directory in which to write profiles
-test.parallel=1: maximum test parallelism
-test.run="": regular expression to select tests and examples to run
-test.short=false: run smaller test suite to save time
-test.timeout=0: if positive, sets an aggregate time limit for all tests
-test.v=false: verbose: print additional output
is writing wrong commands the only way to get some help with this tool? it doesn't have a help flag or something?
I'm 5 years late, but to specify how many N times to run. Use the option -benchtime Nx.
Example:
go test -bench=. -benchtime 100x
BenchmarkTest 100 ... ns/op
Please read more about all go testing flags here.
see the Description_of_testing_flags:
-bench regexp
Run benchmarks matching the regular expression.
By default, no benchmarks run. To run all benchmarks,
use '-bench .' or '-bench=.'.
-check.b works the same way as -test.bench.
E.g. to run all benchmarks:
go test -check.b=.
to run a specific benchmark:
go test -check.b=BenchmarkLogic
more information about testing in Go can be found here
I am facing the following problem:
I have multiple HTTP Requests in my testplan.
I want every request to be repeated 4 times if they fail.
I realized that with a BeanShell Assertion, and its already working fine.
My problem is, that I don't want requests to be executed if a previous Request failed 5 times,
BUT I also dont want the thread to end.
I just want the current thread iteration to end,
so that the next iteration of the thread can start again with the 1st request (if the thread is meant to be repeated).
How do I realize that within the BeanShell Assertion?
Here is just a short extract of my code where i want the solution to have
badResponseCounter is being increased for every failed try of the request, this seems to work so far. Afterwards, the variable gets resetted.
if (badResponseCounter = 5) {
badResponseCounter = 0;
// Stop current iteration
}
I already checked the API, methods like setStopTest() or setStopThread() are given, but nothing for quitting the current iteration. I also need the preference "continue" in the thread group, as otherwise the entire test will stop after 1 single request failed.
Any ideas of how to do this?
In my opinion the easiest way is using following combination:
If Controller to check ${JMeterThread.last_sample_ok} and badResponseCounter variables
Test Action Sampler as a child of If Controller configured to "Go to next loop iteration"
Try this.
ctx.setRestartNextLoop(true);
if the thread number is 2, i tried to skip. I get the below result as i expected (it does not call b-2). It does not kill the thread either.
Let's say I have a test case with some steps in it. Now, let's say that step 3 needs to be repeated after you complete steps 4 and 5 ... so that when you do step 6 you are in the right place.
Is it good practice to tell the tester to repeat a step? Or would it be better copy-and-paste the repeated step into the step where you would need to repeat it?
I am hearing arguments that it is not industry standard to tell the tester to repeat steps and that one might not pass certain certifications if test cases are written in this manner.
Example:
*Step 1: Click the View Event Log button; Expected Results: Event Log window appears
Step 2: Close the event log window (X) or OK; Expected Results: The Event Log window disappears
Step 3: Repeat Step 1; Expected Results: Expected Results from Step 1
Step 4: Click the Cancel button; Expected Results: The Event Log window closes and any changes (such as clearing the log) are not applied
Step 5: Repeat Step 1; Expected Results: Expected Results from Step 1
Step 6: Click the Clear button and hit apply; Expected Results: The log is cleared
...*
Some people think that I should be copying-and-pasting what is in Step 1 each time I need to repeat that step rather than just simply saying that the tester should repeat the step. Any input as to industry standards, potential downfalls, etc ... would be greatly appreciated.
Test case design does not really follow an industry standard, if you were trying to get certified, listing to repeat a step is a no-no. I personally think that's crap. I see no problem asking a tester to repeat a step. As a believer in agile methodology, I prefer much simpler test cases so a tester has more time to test scenarios rather than design test cases (or a developer more time to develop if you are in a cross functional team). If your looking for more input from a larger testing community try http://www.qaforums.com/
Test cases should be as independent as possible and not verifying two outcomes in a single test case. The test cases should not be designed in a way where the tester has to repeat any previous step. In this case, a new test case should be written because it is a new path. The prons for this approach is that at the end of execution you'll have a clear picture of the test coverage and the pass/fail % of the requirements because all the test cases are independent.