I am using SonarQube 5.2 integrated with Team Build being executed using an on prem agent connected to Visual Studio Online. The build completes and shows a summary of SonarQube results in build summary, screen shot below. However, when I navigate to it the results are not available in SonarQube dashboard.
I am trying to understand why the analysis results are not uploading into SonarQube dashboard?
Indeed, as pointed out by Bogdan Gavril, since SonarQube 5.2, the analysis results are not immediately available on the server: There is a background task running on the server that does some further analysis - the project only becomes visible when that background task is completed.
You can see the status of the backgrounds tasks by navigating to: Administrator -> Projects -> Background Tasks
Related
How can I execute a test via command in a way that the test run is shown with all its results in TestCafé Studio afterwards?
I'm using:
npx testcafe [browser] [TestCafe file]
The test is executed but the results are only visible in the console. Is there a way to fully integrate it in TestCafé Studio?
BTW: Why isn't there a tag for [testcafestudio] – the product is not that new ;-)
EDIT: I'll take a little further out: We would like to switch from TestCafé to TestCafé Studio to increase the number of people who can maintain and create individual tests in QA. Some tasks could then also be handled by employees with somewhat lower HTML skills. In addition, we would like to keep the connections we are used to, so that the tests are still triggered at certain times or manually via a Jenkins pipeline (Jenkins-->VIX-->CMD-->TestCafé Studio). Depending on the configuration of the respective test run, different branches would be used for the TestCafé Studio project via Git. The test results are read, parsed, and written to a database after the test run is complete. In addition, I would like to see the atomatically triggered calls available in TestCafé Studio, as it is very convenient to navigate directly to the failed tests.
Is it not yet possible to start tests in TestCafé Studio via CMD?
TestCafe Studio stores reports in its own format while TestCafe stores reports in various different formats that are inconsistent with the IDE format.
You can run tests in the TestCafe Studio itself. Are you running your tests in CI? If so, what CI are you using and why does not its reporting system meet your requirements? If not, could you please clarify why you are required to run tests outside the TestCafe Studio IDE?
Our web application codes are stored on SVN instead with TFS. We are trying to set up our automated UI testing.
These are following tools our team are considered to use.
- Microsoft Test Manager (Create Test cases associate with User stories)
- Coded UI (Connect to TFS and Microsoft Test Manager)
- Selenium (Automated UI coded)
- Team Foundation Server 2015 (Test cases and User stories)
Web Application with URL
We only want to configure automated UI testing for our website. Is there any way to run the set up the automated testing without build through TFS?
Thank you and any feedback is appreciated.
So here is the thing,
Coded Ui is not a connector to TFS & MTM it by itself an automation
framework/tool like selenium
You don't need both Selenium & Coded UI for your automation. You only need either of it
If all you want is to get started with automation for your application, you can do it just with a version of visual studio which has Coded Ui. (The latest supporting version is Visual Studio 2015 Enterprise edition)
See this like to know How to create Coded Ui Tests , see this link to know how to Run a Coded Ui Test
It's based on your workflow. Whether your app/code is on-premises or in the cloud, you can automate build-deploy-test workflows and choose the technologies and frameworks, then test your changes continuously in a fast, scalable, and efficient manner. And just as Prageeth said, Coded Ui is an automation framework/tool just like selenium.
In TFS either code ui or selenium test more like continuous testing. The workflow is such as bleow:
First make sure that your app still works after every check-in and
build using TFS. Find problems earlier by running tests
automatically with each build. When your build is done, review your
test results to start resolving the problems that you find.
Add some related tutorials for your reference:
How to run Code UI in TFS: Executing Automated tests in Build vNext
using Test Plan, Test Suites
How to run Selenium in TFS: Get started with Selenium testing in a
continuous integration pipeline
If you insist on without building from TFS, you could also use the Code UI function with VS on local.
You can setup Test controller and Test Agent VMs in MTM test lab. This will allow you to execute your automated tests on Test Agent directly in MTM. Otherwise, you can execute Coded UI tests locally in Visual Studio.
I am using Visual Studio Team Services to carry out an automated build, and using SonarQube to display Code Quality, Coverage, etc. I am also using a privately hosted build agent.
The build steps all work successfully with data being processed and populated through to SonarQube which is great. However, no code coverage is being displayed in SonarQube. After looking through the logs in VSTS I found that SonarQube is looking for the .trx file (which contains code coverage) in a different directory to the one that VSTS publishes the .trx file to.
So when VSTS builds the solution it creates the test results file here: C:\agent_work\3\s\TestResults
But SonarQube is trying to use test results from here: C:\agent_work\3\TestResults
On the build server, if I manually copy the .trx file into the correct location and then run the build again, the code coverage all works fine and processes through to SonarQube. So the issue is definitely the mismatch of the locations where the .trx is published and where it is to be picked up form.
I can't find a way of changing the publish location, or the SonarQube source location.
Please help!
This is a known issue, and will be fixed by the next release. See: https://jira.sonarsource.com/browse/SONARMSBRU-262.
We just announced the RC versions of the products that have this fix, you can give them a try. See this thread.
I just installed the latest version of sonarqube on my server and did a test run on my project with the sonar-scanner executable. This works.
In the next step, i'm creating a new VSTS Build Definition with the existing SonarQube build steps.
When I trigger this build, it fails in the last step, after the report has been uploaded to my SonarQube server:
VSTS build log
SonarQube server log
I don't know why this happens, because the sonar-scanner is working just fine. I'm using the same parameters in my MS Build step.
Looks like your VSTS task is pending completion of the Background Task on SonarQube server side, as per this log in your first screenshot:
Waiting on the sonarqube server to finish processing in order to determine the quality gate status
(note: this is a log from the VSTS task itself, not from the SonarQube Scanner which did finish successfully)
The problem is that the Background Task actually fails with an error as per your second screenshot. The failure corresponds to bug SONAR-8013 (fix to be released in upcoming SonarQube 6.1).
We are building apps for iOS and Android. Currently we are experimenting with TeamCity for CI.
Currently we have various tools to help us in the Test automation.
1. Selenium: For testing the web interface
2. SOAPUI: For testing the web services
3. Calabash-iOS: For testing iOS app
This is the scenario that we have done successfully with TeamCity:
1. Developer commits code to BitBucket
2. TeamCity runs calabash
3. If calabash pass, TeamCity will build and give the artifacts
This is the scenario that I am trying to do so that I can achieve an end-to-end automated testing:
When developer commits his code to our Bitbucket git repository,
TeamCity will trigger Calabash to run and check for the UI and the output of the details. On end,
TeamCity will trigger Selenium to add data in the Back Office web. On end,
TeamCity will trigger SOAP UI to check newly added data. On end,
TeamCity will trigger Calabash to check the newly added data
Has anyone tried this before or can give me an idea on how I can approach this?
You can setup different builds for each step since you can trigger builds via simple GET requests triggered by a script. TeamCity Accessing Server by HTTP
We run a similar setup. Basically we have one build kick off the Selenium/SoapUI scripts. That build doesn't wait for the job to finish, it just kicks it off. Then in our Selenium/SoapUI tests as the last step we hit the URL for the next build that picks up reports etc.