I have a web project structured the following way:
WebSite
WebSite.test
In the test project, I have all the unit tests (using Microsoft test framework). In the WebSite project I have a target to deploy the site using MSDeploy, the MSDeployPublish target, and DeployOnBuild=true. I then run this build using Jenkins. The problem is that using this setup, the site will get deployed before it's tested.
I could add another build step in Jenkins, so that the entire solution is built first, then the deploy projects are run. However, I would like for it all to be in one step. Is this at all possible?
This is probably a bug in Jenkins. See this bug (or this one). See comments for possible workarounds.
Related
I have a big web project with a separate backend and a front-end (webpack). I'm going to use Cypress to create end-to-end tests.
What is not clear is where I should add the Cypress tests and Cypress itself. The documentation says to add it right to the testee project and it shows how to run the tests on the production website (which URL is different from the local, dev project). This means that I'm not able to run the tests on the development project because Cypress testing IDE and the testee project can't be run simultaneously because they share the same terminal.
If so, the best solution is probably to organize one more project, only for testing purposes, and having only Cypress installed and tests themselves? Is it a good practice and if so, which project should it be?
We have the same setup at work. We include the Cypress folder in the front-end repo. I'd agree with keeping it right next to the project because you have access to that code easily i.e. accessing utility functions, selectors, etc. As far as the terminal issue, you should be able to run your project locally in one terminal tab and the cypress test runner in another.
So this is a question of concept more than technical,
Say I have two projects project A is the project that I'm testing, it's a website that is built with angular 8, and project B is the selenium testing framework that has all my testing scripts that will run on project A,
NOW both projects are on GitLab in separate repositories.
Question 1:
on a merge, for project A with new code, a build will happen with the new code and if successful then I want to trigger the build of project B and run the tests on project A's new build
( is it possible ?)
Question 2 :
if it is possible how do I access the build of project A when I have already defined the URLs that selenium will run the scripts on?
thank you in advance for whoever is willing to help and I'm sorry if this question is considered a dumb question
You can use a CI/CD tool to achieve this (ex: Jenkins, Bamboo, Teamcity,CircleCI, Gitlab etc)
You will need to configure a job to deploy app A, (you can provide any source and configure the deployment) then you can do the same with app B. Even if you are building the app on localhost, or you deploy it in another place, you can keep environment variables in your automation framework.
I am having an issue getting selenium end to end tests to work after an automated deployment using visual studio team services (VSTS).
I have a build working that generates a build artefact. This is triggered from VSTS but runs on an on premises build server. I have a deployment working that deploys to an on premises development web server. All this works including unit tests running after the build.
When I try to add testing after the deployment is when I run into the problem. The tests are to be run on the build server and point to the dev server website. The deployment has two phases. A deploy and then an agent phase that runs a test assemblies task using the build agent on the build server. The problem seems to be that the test dll's are not being included in the build artifact and so are never found when the test process runs. Deploy setup us as follows.
I have a copy files before the publish artifact in the build definition that seems to copy the files in to the right place but they are not included in the zip file artefact. I've looked at several websites and posts on here but I still seem to be missing a vital bit of knowledge that will get this working.
Use of the log did help as recommended so thanks for that.
I have managed to get this working. I separated the selenium tests out into a separate solution and built that separately creating it's own build artefact. I then added this to the agent task in the deploy. This worked. The only thing I need to get sorted now is the correct search path to find the test DLL's. It's not quite as dynamic as I would like at the moment. I can play tunes on that until I get it right though.
I accept this is working around rather than solving the original problem but needs and timescales must. I think moving the end to end UI tests out of the main solution makes sense anyway but no doubt others may think differently.
Thanks for your help everyone
As I'm using Selenium/MSTest for UI tests, I got a problem: I need to deploy an ASP.NET site to a staging server just before the automated tests are executed during a TFS build (TFS 2012).
Although I thought I could do this configuration in the TFS Build process template (DefaultTemplate.xaml), I can't figure out how to change the order to execute a Build->Deploy->Test flow.
Note I've found some how-tos aiding in this goal when using TFS Lab Environment, but this isn't my case.
Default Template Unit Testing is different from UI Testing. Default template is designed to run Build -> Unit Test -> Deploy.
You are now in Build -> unit Test -> Deploy -> UI Test.
For UI Testing, You need separate Test Agent and set up build agent in interactive process.
These links may help
http://www.deliveron.com/blog/post/Configuring-a-TFS-2010-Team-Build-Server-to-Run-Coded-UI-Tests.aspx
http://blogs.infosupport.com/testing-your-web-application-with-selenium-and-mtm-part-1/
http://blogs.infosupport.com/testing-your-web-application-with-selenium-and-mtm-part-2/
Finally I've solved the problem chaining builds.
For example, when I queue a production build, this is the sequence:
Build, deploy to staging without UI tests
Build, no deploy, staging UI tests only
Build, deploy to production without UI tests
Build, no deploy, production UI tests only
I got the solution from this blog post:
http://blog.stangroome.com/2011/09/06/queue-another-team-build-when-one-team-build-succeeds/
Post's author posted the modified DefaultTemplate.xaml (build definition) in GitHub Gist:
https://gist.github.com/jstangroome/1196590/
Credits for the whole blogger!
One simple solution is to have an intermediate msbuild project file that is specified in your build definition and from there, call the actual solution file.
You can then do whatever is required pre- or post- build to ensure that the environment is up and running prior to TFS executing the tests. We do things like compile and deploy databases prior to TFS running our integration tests.
This is one solution that avoids having to manipulate the build template.
Our department uses Visual Studio 2008 Team System, and we have a build server that integrates with our TFS source control server. It pulls the source code, builds the solution and runs the unit tests, just as we might do from within VS, and emails a report. The build server is setup using MSBuild and MSTest as the primary tools. All very sweet.
On our development machines we also run a set of selenium unit tests, and I want to include this in the test suite on the build server. I have been told that 'this is not possible using MSBuild/MSTest', but I am at a loss to understand why.
Does any one have experience of running selenium tests (they are just conventional test methods written in C#) who might be able to advise me on whether this is possible and what the gotchas are? Thinking about it, apart from giving the browser access to the desktop when the server is not logged in, once MSBuild has handed off a test list to MSTest it's exactly the same process as on our develop machines.
TIA
I know it's 3 years on, but someone might drop in on this post and not see an answer. This is possible todo.
In a similar fashion how you'd run unit test, in the build definition using default template you need to specify the name of the UnitTest DLL and ensure Run Unit Test is not disabled. Also ensure that the build is building your Automated UI Test solution.
Simples.