I've been playing around with Angular the past week or so learning its ins and outs. I am now at a point where I'm trying to figure out the best way to restructure my app to do E2E testing, given that it will be embedded into a preexisting site, rather than stand on its own.
The problem comes in how to handle index.html from the angular-seed project, since my understanding is that Karma needs to point itself via scenarios.js at something in order to load and bootstrap the app before testing.
The markup currently in index.html will need to be inserted into the larger HTML structure of my page when it is being put together by our servers. There is no further preprocessing that must be done to the top-level Angular template, but this seems to be enough to complicate the testing process considerably.
This leaves me with three options as far as I can tell for enabling E2E testing:
1) duplicate the index.html code in its entirety to keep around as a complete mockup for testing
2) use a single angular route and ngView to access my top level view template (reducing the test purposes index.html to little more than <ng-view></ng-view> and <script> tags)
3) fold and just use Selenium for high-level testing
The first option introduces a maintainability problem, and the second is undesirable because it limits the testability of any aspect of the app outside the <ng-view>. Not an issue now, but could be as the app grows. And three, well, I've heard Selenium creates a lot of headaches with Angular apps that are best avoided by using the purpose-built Angular E2E testing tools.
So how is this situation typically handled?
Related
I currently have an application with my backend in PHP (Codeigniter) and Frontend in Vuejs. I know that the test culture is super important to ensure the quality of a software, but I have a lot of doubts on how to start:
Where do I start? Do I write the tests for the backend or frontend first?
Before deploying the application, should I run all tests? Backend and frontend?
I know that there are test frameworks for PHP (PHPUnit) and for Vue as well (Jest) I wonder if there is any way to test the styling part, too, of CSS. Because sometimes i often create a class in my style sheet that breaks some other element of the application.
It depends on you, in a perfect world you should test BE and FE too.
It's tricky because testing a BE is essential, because all of your business logic there and it can cause security problems.
FE testing maybe a bit more important for your users / customers because they only see that result but if you have a shinny FE but your BE have a lot of issues your customers will be frustrated.
I think if you don't have enough resources or a testing team you can testing your BE via API, because that's how your Vue FE will be communicate with your PHP and it's easier to coverage your functionality than write unit test all of your PHP code.
Anyway you mentioned that sometimes i often create a class in my style sheet that breaks some other element of the application. if you use scoped style in your components it won't be problem anymore:
https://vue-loader.vuejs.org/guide/scoped-css.html
First off, I'm brand new to Dojo.
I'm integrating it into our existing web app.
We initially only need the Calendar widget functionality.
I'm looking to keep the number and size of files as small as possible.
I don't believe downloading just the base code file will be sufficent?
http://dojotoolkit.org/download/
Additionally, the Dojo toolkit download is a huge zip (Even if I was to only use compressed files)
Am I left with downloading the toolkit and manually removing everything I don't need?
Is there no custom download builder like jquery ui?
Well, the dojo library is much larger than jquery ui and I don't know of an equivalent to the download builder. If you are just interested in using dojo for a single widget, you might consider exploring a different library.
To use dojox/Calendar, you are still going to need the many dependencies it has on other dojo modules. You can do this manually, but it will be tedious.
One thing you can do is run dojo's build system to package dojox/Calendar and all of its dependencies into a single file. This isn't a trivial task and requires a good understanding of dojo's AMD loader and package system.
If you want to go down this route, I would clone the dojo-boilerplate project on github. It contains everything you need to do this out of the box. Then follow the build system tutorial to understand how you set this up. From there you can have your app depend on dojox/Calendar to produce the file you include on your page to consume it.
I suggest that you put the whole thing (yes, it's a lot of tiny files) to your server.
Dojo 1.9 is written so that when users visit, their computers will only download the individual pieces on an as-needed basis. This is possible because every piece (AMD modules) is explicit about what it needs.
Once you have something that works, you can choose speed-up loading times by using the build system. Basically, this involves going: "If the user wants this thing, they'll probably want all this other stuff, so create a big minified lump and give it to them whenever they start asking." Best of all, it doesn't have to be perfect: If you miss including something, the users browser will still request it a la carte.
At work we're using the Dojo Boilerplate starting application which helps give some initial organization to the build process.
I am converting a good old ASP.Net website to a single page application using Ember.js in a ASP.NET Web API project.
All the devs of my team and myself are pretty new to javascript. We spent the last 2 weeks learning the basis and comparing SPA frameworks. I apologize in advance if my question sounds stupid :)
All the Ember tutorials I have found so far included all Handlebars templates into one single file. I assumed it would be pretty obvious to split them into separates files (*.hbs) when the time would come, but it's not. I might be totally missing something here, but I found about 4 ways to get my templates back when I need them. I'd like to know which method you would recommend:
Concatenate and then inject all the template files when the app loads. I could write some C# code on the server-side that concatenates all the templates files into a single one when the app loads (i.e. each time a visitor enter the app). It seems odd to me, in terms of processing, but also because the generated HTML file will be pretty heavy.
Load each template dynamically via Ajax when I need it. Pretty much what is done here. I kinda like this solution even though I haven't tried it yet. It makes sense to me to get asynchronously a template when I need it instead of loading the entire app on the first load.
Use the Bundling mechanism of Asp.Net MVC. I found stuff like csharp-ember-handlebars to precompile the templates on the server-side and return them as a single javascript file. It works-ish but I feel like the precompiled file will become pretty heavy as I add new templates.
Use Grunt with the plugin grunt-ember-handlebars to precompile the templates. I'm not familiar with Grunt but if I understand well all the devs working on the project will have to install Node.js + Grunt + learn how to use a command prompt + remember to run the command before each commit (if they modified a template). This is not obvious for the web designers. And adding grunt to the build actions will require the entire dev team (working on other projects) to have grunt on their machine (not acceptable).
I need to find a simple and elegant solution to address this issue. My project is in a solution with 35 other projects and I cannot add too much complexity to the build, neither depend on unstable libraries. Maybe I have been too optimistic when I thought I could use Ember for my project. Any suggestion would be welcome!
Your #3 is the most ideal (and common) way that I've seen applications handle templates. With a compiled and minified template file you really don't have to worry to much about performance problems in regards to adding new templates, especially if you take advantage of caching.
One benefit to having the templates compiled and available off-the-bat is that users only need to Download Your Resources Once™, as apposed to downloading resources for each subsequent page load. This leads to a fantastic user experience.
I have been writing tests for my Ruby code for a while, but as a frontend developer I am obviously interested in bring this into the code I write for my frontend code. There is quite a few different options which I have been playing around with:
CasperJS
Capybara & Rspec
Jasmine
Cucumber or just Rspec
What are people using for testing? And further than that what do people test? Just JavaScript? Links? Forms? Hardcoded content?
Any thoughts would be greatly appreciated.
I had the same questions a few months ago and, after talking to many developers and doing a lot of research, this is what I found out. You should unit test your JavaScript, write a small set of UI integration tests and avoid record and playback testing tools. Let me explain that in more detail.
First, consider the test pyramid. This is a interesting analogy created by Mike Cohn that will help you decide which kind of testing you should be doing. At the bottom of the pyramid are the unit tests, which are solid and provide fast feedback. These should be the foundation of your test strategy and thus occupy the largest part of the pyramid. At the top, you have the UI tests. Those are the tests that interact with your UI directly, like Selenium does for example. Although these tests might help you find bugs, they are more expensive and provide very slow feedback. Also, depending on the tool you use, they become very brittle and you will end up spending more time maintaining these tests than writing actual production code. The service layer, in the middle, includes integration tests that do not require an UI. In Rails, for instance, you would test your REST interface directly instead of interacting with the DOM elements.
Now, back to your question. I found out that I could greatly reduce the number of bugs in my project, which is a web application written in Spring Roo (Java) with tons of JavaScript, simply by writing enough unit tests for JS. In my application, there is a lot of logic written in JS and that is the kind of thing that I am testing here. I am not concerned about how the page will actually look or if the animations plays as they should. I test if the modules I write in JS will execute the expected logic, if element classes are correctly assigned and if error conditions are well handled. For these tests, I've been using Jasmine. This is a great tool. It is very easy to learn and has nice mocking capabilities, which are called spies. Jasmine-jQuery adds more great functionality if you are using jQuery. In particular, it allows you to specify fixtures, which are snippets of the HTML code, so you don't have to manually mock the DOM. I have integrated this tool with maven and these tests are part of my CI strategy.
You have to be careful with UI tests, specially if you rely on record/playback tools like Selenium. Since the UI changes often, these tests keep breaking and you will spend a lot of time finding out if the tests really failed or if they are just outdated. Also, they don't add as much value as unit tests. Since they need an integrated environment to run, you will mostly like run them only after you finished developing, when the cost of fixing things is higher.
For smoke/regression tests, however, UI tests are very useful. If you need to automate these, then you should watch out for some dangers. Write your tests, don't record them. Recorded tests usually rely on automatically generated xpaths that break for every little change you do on your code. I believe Cucumber is a good framework for writing these tests and you can use it along with WebDriver to automate the browser interaction. Code thinking about tests. In UI tests, you will have to make elements easier to find so you don't have to rely on complex xpaths. Adding class and id elements where you usually wouldn't will be frequent. Don't write tests for every small corner case. These tests are expensive to write and take too long to run. You should focus on the cases that explore most of your functionality. If you write too many tests at this level you will probably test the same functionality that you have previously tested on your unit tests (supposing you have written them).
In my current project I am using Spock and Geb to write the UI tests. I find these tools amazing. They are written in Groovy, which suits better my Java project.
There are lots of options and tools for that. But their choice depends on whether you have a web UI or it's a desktop app?
Supposing from the tools you've mentioned it's Web UI. I would suggest Selenium (aka WebDriver): http://seleniumhq.org/docs/
There is a variety of languages it supports (Ruby is in the list). It can be run against a variety of browsers, ad it's quite easy to use with lots of tutorials and tips available.
Oh, and it's free, of course :)
I though as this post gets a lot of likes, I would post my answer to my question as I do write lots of tests now and how you test front end has moved on a lot now.
So in terms of FE testing I spent lot of time using karma with Jasmine, although karma will work nicely with other test suites like mocha & qunit. While these are great and karma allows you to interface directly with browsers to run your tests. The downside is as your test suite gets large it can become quite slow.
So recently I have moved to Jest which is much faster and if your writing react app, using enzyme with snap shot testing give you really good coverage. Talking of coverage Jest has Istanbul coverage built in and set up and mocking is really easy simple to use. The downside it doesn't test in browser and it using something called jsdom which is fast, but does have a few nuisances. Personally I don't find this a big deal particularly when I compile my code through webpack/babel which means the cross browser bugs are fairly few and far between, so generally isn't an issue if you manually test anyway (and imo you should).
In terms of working within the rails stack, this much easy now that the webpacker gem is now available and using npm and node is generally much more excepted. I would recommend using nvm to manage your node versions
While this isn't strictly testing, I would also recommend using linting as this also picks up a lot of issues in your code. For JS I use eslint with prettier and scss/css I use stylelint
In terms on what to test, I think as Carlos talks about the test pyramid is still relevant, after all the theory doesn't change, just the tools. I would also add to be practical about tests, I would always test, but to what level and coverage will depend on the project. It is important to manage your time and spending hours/days testing a short lifecycle project. Larger/longer term projects the benefits of a larger test suite is obviously greater.
Anyway I hope that helps people that look at the question.
A similar question was already asked (
Performing a Stress Test on Web Application?), but I'd like to test a web application that prevents double-submits and takes some counter-XSRF actions and therefore REQUIRES JavaScripts to be evaluated.
Has anybody done stress tests with web apps that require (and use) JS and any experience to share?
jMeter wouldn't work I guess...
Thanks!
Watir?
Watir is a simple open-source library for automating web browsers.
Watir drives browsers the same way people do. It clicks links, fills in forms, presses buttons. Watir also checks results, such as whether expected text appears on the page.
It drives Internet Explorer, but is also functional with Firefox (and Safari to some extent).
The problem with Watir and Selenium RC or any other full browser solution is that they need a full browser :P
Browsers are very expensive to run, often requiring 300MB or more of RAM. Multiply those requirements by even 100 and you need massive hardware. Fortunately, there is a solution: I recently started a company that does exactly what you're looking for.
Check out http://browsermob.com and you can run a limited test (up to 25 users) to get a feel for the app. Feel free to contact us if you have any questions at all!
One solution that may be worth pursuing is to run Selenium on Amazon EC2 to provide the scalability you need. There is a tutorial over at Selenium using a sample that ships with Selenium grid. Windows machines are 12.5 cents an hour for a small machine meaning that a 500 machine test is going to cost $62.50 an hour.
PROS:
Selenium runs in a real browser meaning that your Javascript is executing as it would on a client
Low cost - trying to do this on your own hardware would cost significantly more
CONS:
You would have to establish network connectivity from Amazon to your application
The testers I work with use Bad Boy for load testing. I'm fairly certain you can test interactions that use javascript, so you should be able to test stuff like double-submits.
As far as your backend is concerned, it doesn't matter what triggers a request whether it's from JavaScript or a load testing tool as long as the request is valid.
You can create a bunch of fake requests that do lots of different things (hopefully representative of actual usage patterns) and slam your webserver with a load testing tool.
There's a bunch out there:
jMeter
http_load
Grinder
httpperf
Because JMeter is not a browser, it won't interpret the JavaScript code on the page you GETting:
JMeter does not process Javascript or applets embedded in HTML pages.
[JMeter Wiki]
So, what can you do? You can add WebDriver to JMeter test and by this, evaluate the web pages.
Web Driver Sampler automates the execution and collection of
Performance metrics on the Browser (client-side). A large part of
performance testing, up to this point, has been on the server side of
things. However, with the advancement of technology, HTML5, JS and CSS
improvements, more and more logic and behaviour have been pushed down
to the client. This adds to the overall perceived performance of
website/webapp, but this metric is not available in JMeter. Things
that add to the overall browser execution time may include:
Client-side Javascript execution - eg. AJAX, JS templates
CSS transforms - eg. 3D matrix transforms, animations
3rd party plugins - eg. Facebook like, Double click ads, site analytics, etc
All these things add to the overall browser execution time, and this
project aims to measure the time it takes to complete rendering all
this content.
Official guide: https://jmeter-plugins.org/wiki/WebDriverTutorial/
I've tried Badboy which is OK. The big, fat, heavy tool is SilkTest. It requires a lot of programming to get up and running, but you can get something very solid done!
If you only need to stress test request from e.g. IIS log files, I have a custom build tool. I will publish it at CodePlex very soon.
Selenium RC is another alternative.
Also related, check out my recent article on Ajaxian. I think it does a good job of explaining why real browsers do matter and why executing JavaScript is becoming more important for load testing.
http://ajaxian.com/archives/why-load-testing-ajax-is-hard
There's a new tool in this area called k6
it has a way to access the DOM, and I'm planning to try it in a project.
background story:
You can visit this and this blog.
maybe it will help.