I have to test the project code using Jenkins. In Jenkins, I am using automation testing throw selenium. then my question is I have test scripts on my laptop and using Google Cloud virtual machine SHH i have to set up and test in every push on git. Here is my 2 demand-
1. on demand-whenever we want all the test run we should just trigger the test run.
2. Whenever we deploy something on staging then test all the test cases
Regarding (1), you are always able to trigger a job manually through the Jenkins UI. No special configurations there.
Regarding (2), you can install a plugin that will integrate webhooks functionality into Jenkins. In my case, I like to use Generic Webhook Trigger for this purpose, as it has the flexibility that I need on my setups.
In order to trigger the job on every deploy to staging, and assuming that your deploys are automated, you will need to add a final step on the deploy script, to make an HTTP request to the webhook URL (eg. JENKINS_URL/generic-webhook-trigger/invoke?token=<your-token>
I don't fully understand your setup with your machine and the VM on GCloud, in any case, I believe that the test code should be available to the machine running the tests, and not be stored in a location that might be unavailable when the tests need to be run (as your laptop might be).
Related
I currently have an Angular frontend and a dockerized Django backend. Both are deployed separately. Angular via Netlify and Django via Heroku. Both are triggered to deploy any publishes to the master branch.
I've written end to end tests using Cypress and integrated it to the CI using GitHub actions. At the moment I ended up deploying a test backend server which is a replica of the production on a separate Heroku instance and am using that link as the backend for the cypress test suite that lives inside the UI repo and runs every time before pushing to master. To be more specific, whenever I need to publish changes, I push to staging and on push to staging, the CI workflow is triggered and runs the test and if the tests pass, the changes to staging get merged with master and from there changes are automatically deployed. The backend gets published to production without any prior testing.
There are two problems with this setup. First is that for the CI to rely on an external API is time consuming. Second thing is that for me to push changes to the API without the e2e suite running and having it only be run when corresponding UI changes get published is sort of problematic because breaking changes to the API could get published unchecked.
Is there a more elegant way to do this within GitHub actions? Because I find it convenient. If not, what other solution do I have to address these 2 issues?
We have a CI setup using bamboo where one job was to run our automation, it checks out our Automation Solution from a repository, and run it using NUnit runner where it contains tests using selenium webdriver and some unit tests. It runs unit tests(not using webdriver commands) just fine, but not the webdriver tests.
We are sure that the solution is being checkout correctly because we can run the test using the command line referencing the created local repository, including tests that uses webdriver using the command below:
C:\pathToNUnitConsole\nunit-console C:\pathToCreatedLocalRepositoryInTheServer\NUnitTest.dll -include=FirefoxBrowserTest
We can see that the bamboo job never launched a browser, and we are getting test failure for not seeing browser elements. Also, we are getting error as we see in the logs:
10-Apr-2015 13:19:49 http://10.215.12.28:15871/cgi-bin/blockpage.cgi?ws-session=587410860
10-Apr-2015 13:20:42 JavaScript error: resource://gre/modules/CertUtils.jsm, line 168: NS_ERROR_ABORT: Certificate issuer is not built-in.
I don't know if this is your problem, but it sounds very similar to an issue I had trying to get NUnit runner to run tests built in SpecsFor.MVC, which is a wrapper that deals with instantiating an IIS Express instance, deploying a target web application to known location, then using Selenium/Chrome Driver to run the tests against the web app.
The problem I had was trying to run the Bamboo remote agent as a windows service. The Local Service account was not able to access the interactive desktop. Also, changing the service to run as a specific user account still failed, as the service was not allowed access to the user's isolated Session0.
I could login as that test user and run the test from the command line no trouble, but I couldn't get it to run when initiated by the remote agent windows service.
I eventually fixed it by using AlwaysUp (not a plug. I'm sure other similar solutions are available). This allowed me to set the machine running the tests to always log in as the test user on startup, would automatically restart the bamboo remote agent if it was closed for some reason and most importantly, to give the service access to the user's isolated session0.
Hope that might give you a few clues...
I'm a QA Engineer where I work and I want to enable all the other
team members to run the tests without me- i.e - even when I'm not here- they can for example get to some url, enter the url of the environment they want to test and play the tests, without installing anything on their computers.
what would be the best practice?
thanks.
Sounds like Jenkins will do what you want.
In that situation I would:
Install Jenkins on a dedicated machine (onsite or in the cloud), or try out a commercial Jenkins cloud provider
Configure a Jenkins job that executes your tests (you can configure the tests to run on the same machine where Jenkins is installed, or if you would rather have Jenkins run the tests on a second machine, that's possible too)
Show your team members how to access Jenkins and launch the test job (or, you can also configure your test job to run automatically at a certain time of day, or even better, when changes are made in your version control system)
We are new to test automation, and finally we finally automated our setup (black box testing). A brief overview of our setup:
Each setup consists of 2 linux PCs and 1 windows PC - the PCs communicate to one another via an embedded board.
Before we used to manually run test cases from the linux machine - this would require both typing in the Linux machine and also some operations in the windows PC. Now instead we have written a C wrapper from which you can trigger any number of test cases and thanks to AutoIT the windows PC operations are now automated as well.
Now we have multiple such setups - I want to have a central test controller that,
Given a set of test cases (and the corresponding executable to be run on the embedded board) can distribute and trigger it in parallel across setups
During overnight tests it can keep track of which test cases have been executed and which ones are pending
Quarantine test cases
Continuous integration - we use CVS
etc etc
Basically a powerful test harness software running on a PC - this PC is connected to all the setups using a router.
Any suggestions from open source (free) projects for such a software, more than having all the features mentioned above, i want something that does most of it, I can code and add additional functionalities as need
I tried browsing online and seem to find some but they all seem to be for testing websites, not sure if it would suit my use case. Would really appreciate inputs in this regard.
Thanks
ok i am going to ignore most of the stuff that you described which looks like you have automated already, and correct me if i am wrong but i think you are looking for is controller for all these automated jobs.
I would say that Jenkins CI is the ideal solution for you.
In jenkins everything is controlled by a master machine, this machine then in turn controls slaves (being your other pcs or linux machines) via java. so this gives you overview of the system.
you can then create jobs and ristrict where they can run, these jos can pretty much do anything. Including taking parameters for what to run, you can also create matrix configuration jobs which allow for one setup to be run simlutansily on however many slaves you need.
You can set this process or timer, or trigger , build externaly or internaly or etc etc ...
Also I am pretty sure Jenkins has some dedicated plugins for working with CVS some of which are built into the OOTB setup.
Jenkins is the way to go here .
We're working on a Rails 3 project and testing using Capybara/RSpec. The problem is that the staging and production environment differ somewhat. Sometimes, the tests will run fine and there will be no problem on staging, but will break in production.
An example is when we added a middleware that uses Rack::File to send files. The application sent the header 'X-Sendfile' which works under Apache but Nginx expects 'X-Accel-Redirect'.
I'm looking for the best way to run a battery of tests when we push to production. Has anyone done this? Ideally the tests should not be run on the production server itself.
The tests would basically cover the core features of our product and would be different from the tests we are currently running.
Thanks a lot
What I ended up doing is have another set of RSpec test in a production_test environment that has read-only access to the database. I use the capybara-webkit driver and each test starts by visiting the complete URL for that test.