Is it possible and if so, how to have TFS display jasmine test results? I am successfully creating jasmine test reports using the plato npm package on the server during a build although I'm yet to link the report to the TFS build success page in TFS. Can someone please advise how this is possible?
Many thanks!!
Update
Initial comments refer to looking at the following link although I think this is a general link in TFS as opposed to a link relative to the "current" build.
https://www.visualstudio.com/en-us/docs/integrate/extensions/get-started/node
I have a build definition that cleans, compiles and runs jasmine tests and using plato (npm install plato) creates a html report on the server. I want developers looking at the build to be able to open the report for the build that they're looking at. Please confirm is the suggested extension link is still applicable because I've tried it just now without success.
Please advise.
Update
I've successfully included NUnit tests to be executed in our build, shown in the screen shot. I want to have a similar feature for Jasmine. The second screen shot shows the report page the Plato package produces for Jasmine tests. I would like this available within the TFS results page.
Screenshot #1 - NUnit test results (as seen in TFS)
Screenshot #2 - Plato produced jasmine results report
This reported is generated on the server in a build definition step.
You can add a custom section in build result through extension, with this way, you can add the link in that custom section.
There is the sample about build result extension:
vsts-extension-samples
More information about how to build extension, you can refer to this article (apply to on premise TFS)
Create your first extension for Visual Studio Team Services
You can use tampermonkey to inject a link into TFS. The following will work. (It's horrible but I haven't achieved the same result with any "proper" approach).
Example script
(function() {
'use strict';
setTimeout(function() {
$(".pivot-view.pivot-tabs").append("<li><a href='plato-report.html' target='_blank'>Jasmine tests</a></li>");
}, 3000);
})();
https://tampermonkey.net (for chrome)
https://addons.mozilla.org/en-gb/firefox/addon/greasemonkey/ (for firefox)
Related
I have recently upgraded to version 1.0.0 from 0.9.6 and noticed that the generated karate-summary.html file, it doesn't display all the tested feature files in the JUnit 5 Runner unlike in 0.9.6.
What it displays instead was the last tested feature file only.
The below screenshots are from the provided SampleTest.java sample code (excluding other Tests for simplicity).
package karate;
import com.intuit.karate.junit5.Karate;
class SampleTest {
#Karate.Test
Karate testSample() {
return Karate.run("sample").relativeTo(getClass());
}
#Karate.Test
Karate testTags() {
return Karate.run("tags").relativeTo(getClass());
}
}
This is from Version 0.9.6.
And this one is from Version 1.0.0
However, when running the test below in 1.0.0, all the features are displayed in the summary correctly.
#Karate.Test
Karate testAll() {
return Karate.run().relativeTo(getClass());
}
Would anyone be kind to confirm if they are getting the similar result? It would be very much appreciated.
What it displays instead was the last tested feature file only.
This is because for each time you run a JUnit method, the reports directory is backed up by default. Look for other directories called target/karate-reports-<timestamp> and you may find your reports there. So maybe what is happening is that you have multiple JUnit tests that are all running, so you see this behavior. You may be able to over-ride this behavior by calling the method: .backupReportDir(false) on the builder. But I think it may not still work - because the JUnit runner has changed a little bit. It is designed to run one method at a time, when you are in local / dev-mode.
So the JUnit runner is just a convenience. You should use the Runner class / builder for CI execution, and when you want to run multiple tests and see them in one report: https://stackoverflow.com/a/65578167/143475
Here is an example: ExamplesTest.java
But in case there is a bug in the JUnit runner (which is quite possible) please follow the process and help the project developers replicate and then fix the issue to release as soon as possible.
I am evaluating Karate UI and really like it. I use it as standalone jar (as the team do not want to support Java project but is fine with JS) and am wondering how can I attach screenshots of the web app generated right when a step fails?
I have seen this question - Attaching screenshots to json report and hope it would be possible to doing something with the standalone version as well.
If you look at the section on "hooks": https://github.com/intuit/karate#hooks
You should be able to use an afterScenario hook like this:
* configure afterScenario = function(){ if (karate.info.errorMessage) driver.screenshot() }
Works with with 0.9.5.RC4. With latest official release 0.9.4 it does not.
Edit2 - works globally with the following line in karate-config.js
karate.configure('afterScenario', read('afterScenarioScreenshot.js'))
Where afterScenarioScreenshot.js contains the JS function from above
After checking out the source from: https://github.com/RallyApps/app-catalog
I attempted to build the Portfolio Item cumulative flow diagram src/apps/charts/rpm/cfd locally using
rally-app-builder build which shows no errors
Launching App-debug.html allows me to input the relevant settings, but once I click 'save', nothing renders and there are no errors logged to the JS console either, making debugging challenging.
I attempted to copy the build output into a custom html app inside rally, which displayed the same behaviour.
Is this a bug with the example app? Or have I missed some crucial configuration step?
EDIT:
Extra info - I tried running via the rally-app-builder to no avail (think it's related to this issue: https://github.com/RallyApps/rally-app-builder/issues/45). I attempted to flatten the structure and the result of that was a cross-origin request error.
Running from file resulted in a 403 for the SDK
Sorry about that- that Rally App Builder issue is definitely the problem you're running into. Just as a test I downloaded the built html output for that app from the app catalog release: https://github.com/RallyApps/app-catalog/releases/download/2.1/2.1.zip
It worked when installed on a custom html app on my dashboard once I edited the app settings and configured a portfolio item.
If you're looking to tweak this app from source you'll probably have to copy all those files referenced in config.json that live in parent directories into the main app directory and fix the paths in config.json. Then you should be able to build and run the app normally using Rally App Builder.
It seems this app was really not designed to be run externally, so I added a little code to ease the issues I ran into:
In PortfolioChartAppBase.js, _loadSavedPortfolioItem function:
scope: this,
fetch: ['Name', 'ObjectID'] //this is new
And I added some default settings as a top level config object in CumulativeFlowChartApp.js (since the code to force into settings mode doesn't seem to work correctly when running externally):
config: {
defaultSettings: {
portfolioItemPicker: '/portfolioitem/feature/52725935318', //insert a valid oid here
startDate: 'actualstartdate',
endDate: 'actualenddate',
chartAggregationType: 'storycount'
}
},
I'm trying to follow this article to match Cucumber specs with step definitions in IntelliJ.
When I press Alt+Enter, I see Inspection 'Undefined Step' options. However, I should see the intention action Create Step Definition.
I thought I had the Cucumber IntelliJ plugin installed, so that shouldn't be a problem. Any help is greatly appreciated.
Turns out I had the Gherkin plugin but not the Cucumber for Java plugin.
I wasted around an hour to solve this. My issue was, Idea was able to navigate from feature to step file. But when I wanted to execute one cucumber test from feature file (Right click and Run Scenario), it was giving error as undefined steps.
Solution: In the Edit Configuration -> provide the Glue for the cucumber which should be absolute path till steps folder. Please see below screen shot
This fixed my problem of running feature file from Idea.
Hope this helps others.
Most probably you need to install the cucumber for java plugin, if already installed then you need to enable from File>>Settings>>pugins.
I had to uncheck the "Create separate module per source set" checkbox under the "Build, Execution, Deployment" -> "Build Tools" -> "Gradle" settings, and then rebuild the project.
"Undefined" step error message would appear if you import a new BDD project.
This error could appear due to two reasons.
If you have not installed the "Cucumber for Java" plugin.
If you import any BDD projects then it will not detect step definition file.
Solution:
1. If the plugin is not found then you need to install from the below location.
File->Settings->Plugins->MarketPlace->Cucumber for Java
2. After Importing the project disable the plugin and enable once again in the Installed section under Installed.
For me there was a collision between Sidesteps plugin and Cucumber plugin in Intellij and as a result *.feature file extension was taken over by the Sidesteps plugin and was expecting Sidesteps step definitions ignoring Cucumber step definitions. No clue what Sidesteps actually is. So went to IntelliJ settings and reassigned the *.feature extension to Cucumber Scenario type and then everything worked fine and Cucumber steps are recognized by Intellij now.
I had the same issue where all of a sudden my feature to step definition glue was missing. All i did was goto Run->Edit Configurations->and removed the cucumber java
configuration and restarted IntelliJ. it worked fine.
I found that even with the Cucumber for Java plugin installed it was still generating only one step. I eventually uninstalled the Cucumber for Java plugin and reinstalled it and all step definitions were generated.
The issue was fixed after updating the Intelij to the latest version and after updating the cucumber and gherkin intelij plugins
Me not help not one of suggestions above.
But i find if you start one test from runner the problem goes on (it is worked if you have runner for some tests( Runner is class that have line #CucumberOptions(
features = "src/test/resources/stability_*****_features/",
glue = "steps"
)
And may be the next line in config helped you^
in configurations i put line: --plugin org.jetbrains.plugins.cucumber.java.run.CucumberJvm4SMFormatter
in Programm arguments line - it help me
If still not working, you can add runner class
add -> runner package -> Main Runner class
-test
-runner - Create this package
-stepPackage
-resources
-features
#CucumberOptions(features = {"classpath:features"}, glue = {"stepDefinition"},
monochrome = false,dryRun = false)
public class MainRunner extends AbstractTestNGCucumberTests {
}
That is it. Just run this class first. Right mouse click and Run'MainRunner'
Then it will work if you just go back and run Scenarios as well
I had the same issue and was resolved by going to Run> Edit configuration> Before Launch then click on the add option "+" and add Build Project option.
enter image description here
I am trying to set up some AngularJS e2e tests with Karma Scenario Test Runner. I did some modifications to the source files, but Karma doesn't seem to use these latest versions when testing.
In the source files, I added ids to some elements. Karma still couldn't find them, so I added a pause in the e2e test, so that I can mark and "Inspect elements" (using Chrome) on the current page in the test runner. The source code seems correct, except the latest changes are missing, i e, the ids aren't there. So what's happening here? Do I need to explicitly tell Karma the files have been updated somehow?
You can fix this issue by forcing angularjs to clear the application cache:
app.run(function($rootScope, $templateCache) {
$rootScope.$on('$viewContentLoaded', function() {
$templateCache.removeAll();
});
});
In Chrome developer tools settings, check "Disable cache (while DevTools is open)".
Obviously, this is a much more general issue than Angular's e2e test runner, but I decided to leave it here for now, in case somebody else has the same question.