I find it not so easy as one might think to start with TDD in Rails.
I've searched for some examples but no-one really explains how to do TDD/BDD with Rspec AND cucumber. They usually use an old version of rspec, or prefer one on the other (so you find cucumber tests and rspec tests separated). And by the way the say what to do, but the real process is often not explained at all (I've read a lot on make them work togheter with Capybara, but only once on how to proceed on bdd with them).
I'd like to know how to use them togheter, how to avoid test duplication (disabling view specs and other tests that should be done with cucumber), and maybe an explanation of what not to test (for example I find it useless to test the HTML output in most cases, it can be useful to test if you're logging in to match what you're seeing, for example a flash message).
Can you please provide a step-by-step list of how do you proceed with BDD using this two tools togheter? It would be nice a fresh tutorial (using latest versions) or maybe a screencast.
Thanks.
I highly recommend you the Rspec book which is exactly what you are looking for :
It explains how to use Rspec and Cucumber together with a simple 2 loops process : The outer big loop is a Test - Code - Refactor process you do with cucumber and each step is made of multiple iteration of a Test - Code - Refactor process you do with Rspec.
That books explains at the same time how and when to use each of the two tools.
If you want a broader vision of TDD-BDD, I also suggest you the GOOS book which is more language/tool agnostic and more process oriented.
Related
My software company never did BDD or even TDD before. Testing before meant to simply try out the new software some days before deployment.
Our recent project is about 70% done. We also use it as a playground for new technologies, tools and ways of developement. My boss wanted that I switch to it to "test testing".
I tried out Selenium 2 and RSpec. Both are promising, but how to catch up months of developement? Further problems are:
new language
never wrote a line of code by myself
huge parts are written by freelancer
lots of fancy hacking
no documentation at all besides some source comments and flow charts
All I was able to was to do was to cover a whole process with Selenium. This appeared to be quite painfully (but still possible), since the software was obivously never meant to be testet this way. We have lots of dynamically generated id ´s, fancy jQuery and much more. Dont even know how to get started with RSpec.
So, is it still possible to apply BDD to this project? Or should I run far away and never come back?
Before you start - have you asked your boss what he values from the testing? I'd clarify this with your boss first. The main benefits of system-level BDD are in the conversations with business stakeholders before the code is written. You won't be able to get this if all you're doing is wrapping existing code. At a unit level, the main benefits are in questioning the responsibilities of classes and their behavior, which, again, you won't be able to get. It might be useful for helping you understand the value of the application and each level of code, though.
If your boss just wants to try out BDD or TDD it may be simpler to start a new project, or even grab an existing project from someone else to wrap some tests around. If he genuinely wants to experiment with BDD over legacy code, then it may be worth persisting with what you have - #Esko's book suggestion rocks. Putting higher-level system tests around the existing functionality can help you to avoid breaking things when you refactor the lower-level code (and you will need to do so, if you want to put tests in place). I additionally recommend Martin Fowler's "Refactoring".
RSpec is best suited to applying BDD at a unit level, as a variant of TDD. If you're looking to wrap automated tests around your whole environment, take a look at Cucumber. It makes reusing steps a lot simpler. You can then call Selenium directly from your steps.
I put together a page of links on BDD here, which I hope will help newcomers to understand it better. Best of luck.
Reading the book Working Effectively with Legacy Code might be helpful. There is also a short version in PDF form.
I'm just learning Cucumber and BDD way to create applications.
When I'm start learning rails by http://guides.rubyonrails.org/ I'm use scaffolding to generate CRUD functionality and skeleton to my controllers and views. This way give me large speed up versus PHP custom writing code. But when I'm looking to cucumber screencasts, reading comments about BDD, or looking to https://github.com/diaspora/diaspora source code - all of them doesn't use scaffold. When I try to write tests I'm spent long time to compose and test, but I know that the basic code which generated by scaffold isn't crashable. That is my question: How give compromise between BDD and scaffold speed?
The scaffold serves as a good starting point whilst you are getting to know the TDD/BDD cycle. I found that when I first read the RSpec Book that it was confusing with what to use when and why ! Then along came the Cucumber Book which helps a little more since it takes you through the steps a little slower (although the book is still in beta, but a fantastic resource).
One other great resource that helped was a blog post by Sarah Mei called "Outside-in BDD: How?". What is nice about this post is the discussion of the flow and style that you use as a developer. This was useful since it puts some context around the style of doing BDD and not just a re-hash of a basic example.
Of course there is the usual debate that 'real programmers' shouldn't use the scaffold. That may be true for a large scale, production application. The reality is that we all have to learn and start somewhere and Rails is no exception; it's a large framework and once you add in RSpec + Cucumber the breadth and depth grows very fast.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
As a long time php dev and a rails newb I've been trying to get a grasp on tdd/bdd. There are so many tools and methodologies out there that this can be a pretty daunting task. After a lot of reading I came to the conclusion that this would be my preferred method of testing:
No cucumber, it seems to me that it's just an unnecessary complexity.
RSpec for unit tests
RSpec/Capybara for integration/acceptance tests.
No testing for controllers/views.
Factory Girl or Machinist for factories.
Don't mock my own code.
Being a total BDD newb, this could be way off base so feel free to offer suggestions on this testing 'stack'.
What I'm really looking for though is an open source project using these tools that I can look to for inspiration.
Tutorials would be appreciated too, but I think seeing a real world project implement all these ideas would be very helpful.
Everyone seems to have their own recipe for testing that works for them. My background is like yours -- long-time PHP developer using no tests (but doing plenty of manual white-box testing while developing). I recently started writing my first production-level, getting-paid-for-playing-with-Rails app. Here are my thoughts after the first three months.
I too found Cucumber to be too complex for me to start with. I struggled with it for far too long before finally realizing that I was just too far removed from the actual testing framework. Once I dropped down to pure RSpec things became much clearer. That said, now that I've written my first app with a robust test suite I think I could probably drop Cucumber into my next Rails app recipe and feel more comfortable using it for integration testing while still using RSpec for unit testing.
I've developed an unhealthy infatuation with RSpec once I finally understood the process. It really is beautiful to run those tests and see all green. Actually, it's even satisfying to see red because it tells me what to work on next.
I'm using the default Webrat for integration tests. I haven't run up against anything that Webrat couldn't solve, but I haven't given Capybara a try either so maybe I just don't know how bad I have it.
I started writing my tests by following along with the Rails Tutorial Book, and in it the author says "Since I’m not partial to separate tests for views or helpers, which I’ve found to be either brittle or redundant, our first step is to remove them." (Page 93, paragraph 2) After writing a bunch of helper methods, I decided to go back and write separate helper tests for all of them. I'm still rolling my view tests into my controller tests, but I could see separating them in my next project. FWIW, I think you really should be writing tests for everything at some level, even if you're just relying on your controller tests to exercise your views, and/or your view tests to exercise your helper methods.
I'm using Factory Girl with this project and I have found it to be very useful and easy to use. I haven't tried Machinist but it seems to be equally popular. One thing I will mention is that when I first started writing my factories I also used the FFaker gem to generate random content into every factory object that I created. This seemed nice at first as the variable content sometimes caused me to find things that I had missed in development (can't think of a good example at the moment, though), but it also meant I had to worry about truncating all of the returned FFaker data to fit wihtin the length constraints in the model. This proved to be too unwieldy in the long run so I removed all of that code and went with using static strings for all my factories, and only for the required fields. Then when I wanted to exercise a specific field I would do so in my actual specs. IMO, your factories should contain only enough data to create a valid object and no more.
As for mocking, I've only found a few places where I needed to mock something, but perhaps I'm too new to Rspec to know that I'm doing it wrong.
And finally, as for specific examples, just look on GitHub for example Rails apps or open source Rails projects that contain tests. Or follow along with the Rails Tutorial Book as I did. Whatever you decide to do, start as close to the metal as possible and then try different gems/plugins/etc, always making sure you can back off your changes if you decide you don't like the results. I found that trying to start with a complete recipe (i.e. suspenders, blue-light-special, etc.) was too overwhelming. By starting with a raw Rails app I was able to try out different solutions (paperclip versus carrierwave, typus versus rails_admin, clearance versus devise, etc.) to see which ones worked for my personal preferences. It's clear that there is no one perfect recipe for a Rails app, and everyone's preferred recipe is just an opinion.
I am new to Cucumber testing and I am trying to understand when to use Cucumber and when to use RSpec. For my models, I know I should be testing them with RSpec, and I know that I don't need to write RSpec request specs if I write Cucumber stories instead. The thing I don't know when to use what is with views and controllers. Should I be using RSpec to test my views and controllers, or can I just skip them because I am using Cucumber for my higher level testing? Any advice would be greatly appreciated! Thanks!
I use RSpec to drive the development and testing of my models and to some extent my controllers, and Cucumber to drive the development and testing of my views (and subsequently their integration with the controllers).
I don't feel writing Cucumber tests allows me to "skip" writing RSpec tests because a lot of development takes place in the model layer before any views are created.
If you keep writing Cucumber scenarios instead of unit tests, then you will probably find that your tests become very slow to run. If the tests are slow, you probably won't run them very often, so your development pace will slow down.
You can use RSpec for integration tests operating at the same level as Cucumber. If you need to share and discuss features with non-developers, then Cucumber is a huge help. But even if you're a solo developer, Cucumber helps by acting as a 'mental guard' between the behaviour you need and the underlying implementation.
The standard practice most people follow is to write extensive unit tests for the model, minimal tests for the controller, and very rarely any unit tests for the view. Then you would use Cucumber to check everything is interacting properly (be sure that you scenarios cover any loops/conditions in the controllers and views).
I'm not sure if it's OK to recommend a book here, but The RSpec Book by David Chelimsky (ISBN 9781934356371) does provide a nice practical example of how Cucumber and RSpec can be interleaved to behaviour-drive the development of a small command-line game in Ruby.
In a nutshell: writing your Cucumber features adopting the point of view of your end-user, and your RSpec specs using your own point of view as a developer is a good balance between execution speed of the test suite, testing granularity (how precise are the error messages when your tests fail) and documentation coverage (the Cucumber features can be seen as an always up-to-date documentation - the very own RSpec documentation is a great example of that).
Just a small addition that might be helpful to you. Many people think (i also do) that cucumber is more of a hassle than a practical test framework for the programmer. As long as you do not have to cooperate by discussing scenarios with clients that do not know much about programming, i would highly recommend that you do not use cucumber. I'm with DHH on this one. But RSpec, use and even abuse :)
I've started a new role in my life. I was a front end web developer, but I've now been moved to testing web software, or more so, automating the testing of the software. I believe I am to pursue a BDD (Behavior Driven Development) methodology. I am fairly lost as to what to use, and how to piece it together.
The code that is being used/written is in Java to write a web interface for the application to test. I have documentation of the tests to run, but I've been curious how to go about automating it.
I've been directed to Cucumber as one of the "languages" to help with the automation. I have done some research and come across a web site for a synopsis of BDD Tools/Frame works,
8 Best Behavior Driven Development (BDD) Tools and Testing Frameworks. This helped a little but then I got a little confused of how to implement it. It seems that Selenium is a common denominator in a lot of the BDD frameworks for testing a GUI, but it still doesn't seem to help describe what to do.
I then came across the term Functional Testing tool, and I think that confused me even more. Do they all test a GUI?
I think the one that looked like it was all one package was SmartBear TestComplete, and then there is, what seems to be, another similar application by SmartBear called, SmartBear TestLeft, but I think I saw that they still used Cucumber for BDDing it. There a few others that looked like they might work as well, but I guess the other question is what's the cheapest route?
I guess the biggest problem I have is how to make these tests more dynamic, as the UI/browser dimensions can easily change from system to system, and how do I go about writing automation that can handle this, and tie into a BDD methodology?
Does anyone have any suggestions here? Does anybody out there do this?
Thanks in advance.
BDD Architecture
BDD automation typically consists of a few layers:
The natural language steps
The wiring that ties the steps to their definition
The step definitions, which usually access page objects
Page objects, which provide all the capabilities of a page or widget
Automation over the actual code being exercised, often through the GUI.
The wiring between natural language steps and the step definitions is normally done by the BDD tool (Cucumber).
The automation is normally done using the automation tool (Selenium). Sometimes people do skip the GUI, perhaps targeting an API or the MVC layer instead. It depends how complex the functionality in your web page is. If in doubt, give Selenium a try. I've written automation frameworks for desktop apps; the principle's the same regardless.
Keeping it maintainable
To make the steps easy to maintain and change, keep the steps at a fairly high level. This is frequently referred to as "declarative" as opposed to "imperative". For instance, this is too detailed:
When Fred provides his receipt
And his receipt is scanned
And the cashier clicks "Refund to original card"
And the card is inserted...
Think about what the user is trying to achieve:
When Fred gets a refund to his original card
Generally a scenario will have a few Givens or Thens, but typically only one When (unless you have something like users interacting or time passing, where both events are needed to illustrate the behaviour).
Your page objects in this scenario might well be a "RefundPageObject" or perhaps, if that's too large, a "RefundToCardPageObject". This pattern allows multiple scenario steps to access the same capabilities without duplication, which means that if the way the capabilities are exercised changes, you only need to change them in one place.
Different page objects could also be used for different systems.
Getting started
If you're attacking this for the first time, start by getting an empty scenario that just runs and passes without doing anything (make the steps empty). When you've done this, you'll have successfully wired up Cucumber.
Write the production code that would make the scenario run. (This is the other way round from the way you'd normally do it; normally you'd write the scenario code first. I've found this is a good way to get started though.)
When you can run your scenario manually, add the automation directly to the steps (you've only got one scenario at this point). Use your favourite assertion package (JUnit) to get the outcome you're after. You'll probably need to change your code so that you can automate over it easily, by e.g.: giving relevant test ids to elements in your webpage.
Once you've got one scenario running, try to write any subsequent scenarios first; this helps you think about your design and the testability of what you're about to do. When you start adding more scenarios, start extracting that automation out into page objects too.
Once you've got a few scenarios, have a think about how you might want to address different systems. Avoid using lots of "if" statements if you can; those are hard to maintain. Injecting different implementations of page objects is probably better (the frameworks may well support this by now; I haven't used them in a while).
Keep refactoring as you add more scenarios. If the steps are too big, split them up. If the page objects are too big, divide them into widgets. I like to organize my scenarios by user / stakeholder capabilities (normally related to the "when" but sometimes to the "then") then by different contexts.
So to summarize:
Write an empty scenario
Write the code to make that pass manually
Wire up the scenario using your automation tool; it should now run!
Write another scenario, this time writing the automation before the production code
Refactor the automation, moving it out of the steps into page objects
Keep refactoring as you add more scenarios.
Now you've got a fully wired BDD framework, and you're in a good place to keep going while making it maintainable.
A final hint
Think of this as living documentation, rather than tests. BDD scenarios hardly ever pick up bugs in good teams; anything they catch is usually a code design issue, so address it at that level. It helps people work out what the code does and doesn't do yet, and why it's valuable.
The most important part of BDD is having the conversations about how the code works. If you're automating tests for code that already exists, see if you can find someone to talk to about the complicated bits, at least, and verify your understanding with them. This will also help you to use the right language in the scenarios.
See my post on using BDD with legacy systems for more. There are lots of hints for beginners on that blog too.
Since you feel lost as to where to start, I will hint you about some blogs I have written that talks a bit about your problem.
Some categories that may help you:
http://www.thinkcode.se/blog/category/Cucumber
http://www.thinkcode.se/blog/category/Selenium
This, rather long and old post, might give you hints as well:
http://www.thinkcode.se/blog/2012/11/01/cucumberjvm-not-just-for-testing-guis
Notice that versions are dated, but hopefully it can give some ideas as what too look for.
I am not an expert on the test automation but I am currently working on this part. So let me share some idea and hope it will help you at the current stage.
We have used selenium+cucumber+intellij for testing web application. We have used testcomplete+cucumber+intellij for testing java desktop application.
As to the test of web application, we have provided a test mode in our web application, which allows us to get some useful details of the product and the environment; and also allows us to easily trigger events through clicking the button and inputting text into the test panel under test mode.
I hope these are helpful for you.